Is It the Model or the Harness? | Episode 54

And so from a security perspective, I think the the idea here is that we hear a lot of things about defenders that, you know, they they keep blaming the model for things such, you know, like that it leaked data through prompt injection or something like that. But in reality, if you look at what's happened in, you know, 2025 and 2026, the the locus of agent security has shifted to the harness. But there's still a little bit of a a debate on is it the model or the harness. And before we get in to the debate, I'll take a moment to remind everyone that this show is being brought to you by Black Hills Information Security and Anti Siphon Training. BHIS helps organizations identify and close real world security gaps through pen testing, adversary, adversary emulation and simulation, and Purple Team engagements, manage detection and response, pretty much if it's, security.

We can help you with it. Anti Syphon delivers hands on practitioner led training built around real attacks and real defenses and real tools so you can apply what you learn immediately. Learn more at blackhillsinfosec.com and antisiphontraining.com. And so, what do you all think? What do you think?

Is the is the harness the more important part for security or is it the model? And, you know, before we talk about that, maybe, Brian, do you wanna take a stab at kind of, you know, the definitions that matter when we say model versus harness?

So when we're talking about a model, we are talking, in in the case of LLM, usually some kind of deep, learning neural network, that has been gone through a training process in which data has been fed in, and there's expected outcomes for that data. And based upon the difference between, input and expected output, there are series of weights that are tuned to try to get a proper alignment such that when you give the model certain input, you get closer to the output that is expected. And so when we're talking about the model, that's really the core principle there. And so there are a lot of other features that can go along with the model, especially when we talk about, how we can better tune it for alignment purposes, things like reinforcement learning, human feedback. So that's something that a lot of people probably encountered where you get a respond you get maybe two responses when you're interacting with, Claude or ChatGPT and it says, which one do you like more?

And you'll pick, you know, answer one or answer two. And it'll take that information, and it'll use it to adjust weights in future iterations, to try to get that that better alignment. Also baked into the model is a certain level of refusal refusal behavior that goes along with both safety and security principles. Safety meaning that it's likely not gonna tell you how to make a bomb right off the bat. Security being that it might not provide its system prompt or other it might refuse other behaviors that it sees as overtly malicious.

Right? Like So that's kind of the model.

The harness allows it to go, in additional additional steps, to where it's not just a single step of generation or reasoning. What it can actually do is now it can start writing code. It can iterate through that code, testing it, making changes to it as it goes to find errors, or you can hook it up to be able to run, additional tasks, such as summarizing your email and sending out, an an email from that or sending out an automated Slack message or updating user accounts. So the harness is really all of that, you know, kind of the capabilities that we build around the model that that kind of runs on on top of it. Does that sound about right?

Does everyone agree with those general definitions?

So let's just pick one Golang or something. Right? Model could be kind of analogous to the to the code to the coding language. So you have a certain set of capabilities that are in there. They add features and everything, but it's really what you do with that code, how you piece it together to perform certain actions that can really make a difference in how it functions.

And so in the case of, to your point of companies wanting to keep this more secret, I mean, that's kind of like their their closed source source code if you will, that all these capabilities that they build around it that make it much more that make it useful, much more useful than it is as as it just sits normally.

But I think, you know, looking at stuff throughout 2025, like a a argument for the harness is more important than a model when it comes to security is that most of the, when agent went rogue kind of thing traced to, like, over permission tools and not really, like, model alignment or, you know, some kind of, like, supply chain attack and not model alignment. Right? And so not saying model alignment's not important. I'm just saying, like, from a practical security perspective, I think there might be, you know, a little bit more importance on the harness. But I would like to see what what what you all think about specifically security.

I don't know. Your mileage may vary. But it means that then by swapping out these helpers and changing what they can and can't do, that drastically alters the capabilities both in terms of defending the the model itself, but also in defending anything else in the ecosystem. What if you've got rag involved? What if you've got got things that are baked in?

The harness, I think, is going to have there's going to be more opportunity to tweak the settings, I think, with the harness. And so I would think that that attacking the harness is probably gonna get more bang for your buck, especially if you wanna get into other things like documents and whatever that may have been attached to the AI ecosystem in this implementation.

Or if that prompt injection is then, like, leading to a tool call, it has no allow list on a tool call. Like, I I've spent a lot of time in the last, like, a couple months building agents. And one of the things that I build in from, you know, the ground up is, like, let's log all the tool calls. Right? And let's make sure that we know what's happening.

I mean, because, well, that's just, you know, kind of a defense in-depth kind of thing. Right? And you you look at something like Claude code, like, at least maybe I'm wrong, but at the moment, I don't think, like, built in to Claude code, which is a type of harness, I don't think they have, like, logging built in. Now I think you can do things to, like, you know, beef that up. Right?

Like, you know, put something like personal AI infrastructure or change the configuration yourself, but they don't have logging. They do have sandboxing, and they do have the ability to allow list or deny list tools, which out of the box, it doesn't come that way, but you can make those changes, which kinda leads me to the the next kind of case where an agent loop or the clawed code is able to essentially gain access to something that it shouldn't have access to because it's able to change its configuration, like, by turning the sandbox off. I can tell you from experience many times, I have the sandbox on a Claude code. It will say, oh, I can't reach that or I can't do that because it's sandboxed. Let me just go turn that off, and then it'll go do it, which I guess you could argue if that's a sandbox or not.

At least I I see that it's doing it because I'm paying attention. Right? But I guess what I'm saying is I think we're still trying to figure out this. This is an interesting debate in my opinion because I think it's still in its infancy in information security.

I I think that's a that that's a great one, especially if you're dealing with a model that you want to keep local. I've got a wasp that's gonna come harass me here. And you want the actions to keep local, well, did you set up a a policy that disallows it from talking out to the to the Internet? And when when I say model, I really mean I'm talking about, like, open code, something that has a harness around it that has the ability to say perform web searches or take some other action. Well, you know, it's it's not that the model is misbehaving, it's that it has the capability to perform certain actions and it doesn't necessarily know whether or not it should do those.

So it's important to put in to, like, hey, no. Just block off all Internet access if if that's what you want. Those types of scenarios to to consider and think about.

That's in the tool underscore or tool..log. Right? Right. And so it was every agent has everything logged. And so and and there's a couple other ideas, like especially, like, for something like clawed code, a hook layer for deterministic policy. So every time an agent starts or stops or makes a decision, there's gates in there, kill switches, like network ACLs. And so so the concept of hooks is something that's in quad code already. What else?

It's been a while, a long time. And that's defense in-depth. Right? You need a lot of different things layered on in, you know, security to be able to detect or prevent some kind of attack for happening, and AI agents are no different.