A Practical Guide, Part 8 of 15: Guardrails, Permissions and Safety

Host: Which is the perfect setup, because the more agents you have, the more things there are that can be talked into doing something silly. Today we design the defences.

Host: Picture an assistant reading a web page for you. Somewhere in the page, in white text on a white background, is a sentence addressed not to you but to the model. That's the whole problem in one image.

Co-host: Right, direct injection is the user typing something nasty into the chat box. Indirect is when the nasty thing arrives through content the agent fetches. Tool outputs, emails, documents.

Host: The original indirect injection paper put it neatly: applications built on language models blur the line between data and instructions.

Co-host: And as an engineer that's the part that stings. In a classic system we'd parameterise the query and the data could never execute. Here everything is just tokens in the same context window.

Host: The same authors, Greshake and colleagues, reported that back in 2023 effective mitigations for these threats were lacking.

Co-host: Has that changed? Steelman it for me. Surely years of training against this has fixed it.

Host: Training has helped, but OWASP, the security project, says it is unclear whether any fool-proof prevention exists, because models work stochastically.

Co-host: So the honest framing is: injection resistance is a probability, not a property. You can push the rate down, you can't prove it's zero.

Host: Exactly. So in practice, design as if the model will sometimes obey the attacker, and ask what it could do when it does.

Host: Here's OWASP's own cautionary tale. An assistant summarises your mailbox. Its plugin can also send messages. A malicious email arrives, and the assistant is tricked into forwarding sensitive information to the attacker.

Co-host: Classic. Nobody asked for sending. It came along for free because the email library had it.

Host: OWASP calls this Excessive Agency, and names three usual root causes: too much functionality, too many permissions, and too much autonomy.

Host: And the fix OWASP stresses most: enforce authorization in the downstream systems, rather than trusting the model to decide whether an action is allowed.

Co-host: Which is a mindset shift. The system prompt saying please don't email strangers isn't a control. The API refusing the call is.

Host: Asking the model to police itself is a bit like asking the burglar to double-check the locks. Polite, but optimistic.

Co-host: Practically, I'd have the agent act with the user's own scoped credentials, so it can never do more than the person could anyway, and the backend checks every call.

Host: So the takeaway: strip each tool to the narrowest verb and scope the task needs, and put the yes or no in code the model can't argue with.

Co-host: Okay, but if I lock everything down the agent becomes useless. How do the big platforms think about the balance?

Host: Google's framework rests on three principles. Agents need well-defined human controllers, carefully limited powers, and actions and planning that can be observed.

Host: And on mechanism, Google argues for a hybrid, defence in depth: traditional deterministic security controls combined with dynamic, reasoning-based defences.

Co-host: So deterministic is your hard floor, policy engine, allowlists, rate limits. Reasoning-based is a classifier or model that tries to spot that something smells off.

Host: Precisely. The deterministic layer is dumb but unbribable. The reasoning layer is clever but, being a model, can itself be fooled.

Co-host: Where does the human controller fit concretely?

Host: In my view, at the high-impact actions: payments, deleting data, sending things outside the organisation. The trade-off is approval fatigue. Ask for every click and people approve by reflex, which is consent theatre.

Host: So the practical move: let deterministic rules carry the weight, use model-based checks as an extra net, and spend human attention only where the blast radius is large.

Host: Back to that mailbox. Simon Willison has a name for what went wrong. The lethal trifecta: access to private data, exposure to untrusted content, and the ability to communicate externally in a way that could leak it.

Co-host: And the mailbox agent ticked all three. Your inbox is private, incoming email is untrusted, and send is outbound.

Host: Here's the argument I find convincing. Suppose a guardrail catches ninety-five percent of attacks. An attacker gets to try again and again, so the remaining slice is the one that matters. Remove one leg of the trifecta and the exfiltration path simply isn't there.

Co-host: Let me steelman the guardrails though. There's new research here, right?

Host: There is. A paper submitted on 29 October 2026 introduces SEAL, which learns a soft prompt and a scoring readout offline to sanitise tool outputs, using a frozen task model.

Co-host: And the target is local agents, where untrusted tool outputs are the attack surface.

Host: On the AgentDojo benchmark, they report attack success falling from 34.6 percent to 0.1 percent, while task completion without a successful attack rises from 51.4 to 77.5 percent.

Co-host: That second number is the interesting one. Lots of defences win on security by breaking the task. And it needs no extra defence tokens or a separate verifier model, so latency stays sane.

Host: Now test it. It's a fresh research result, the authors' own evaluation on one benchmark, not deployment guidance. And adaptive attackers rarely stay on the benchmark.

Host: So use filters like this as a layer, but first ask which leg of the trifecta you can cut, because a missing capability can't be talked into anything.

Co-host: Let me make it concrete. I've got a coding agent on my laptop. How do I configure it?

Host: Think in order: deny, then ask, then allow. Deny rules win first, for things like reading secrets files or pushing to main. Anything risky but legitimate asks a human. Only boring, reversible actions are pre-allowed.

Co-host: And what can't the rules see? Because I've been burned here.

Host: Rules usually match on the command string. Allow running the tests, and the test script itself can do anything. Allow a harmless tool, and a pipe or a script can smuggle a network call through it.

Co-host: Right, the rule checks the label on the box, not what's inside.

Host: Which is why the last layer is operating-system isolation: a sandbox or container limiting which directories are writable and which network hosts are reachable. That's enforced by the kernel, so it still holds when the model has been fooled.

Co-host: And a network allowlist cuts the outbound leg of the trifecta, which is a lovely double win.

Host: The open problem is usability. Tight sandboxes break real workflows, so people disable them. My advice: make the safe path the easy path, then you'll keep it on.

Host: The one takeaway: assume the model will sometimes be fooled, and design so that being fooled doesn't matter much. Narrow tools, authorisation outside the model, a broken trifecta, and a sandbox underneath.