A Practical Guide, Part 10 of 15: Humans in the Loop and Designing Agentic Systems End to End

Host: And those checkpoints come back today with a new job. Not just surviving crashes, but waiting politely for a person to say yes.

Host: Picture a team on a Monday morning. Someone has drawn a whiteboard diagram of an agent that triages tickets, writes fixes and ships them. Arrows everywhere. The first question isn't which model, it's whether this needs to be an agent at all.

Host: Anthropic draws the line cleanly. A workflow orchestrates models and tools through code paths you wrote in advance. An agent lets the model steer its own process and decide which tools to use.

Co-host: Which, as someone who's built both, is really a question of who owns the if statements. You, or the model.

Host: And their advice is to find the simplest thing that works and add complexity only when it pays. Agentic systems often buy better task performance with extra latency and cost.

Co-host: Let me steelman the whiteboard, though. Ticket triage is messy. Fixed code paths break on the weird ticket. Isn't that exactly where an agent earns its keep?

Host: Stopping conditions, human checkpoints, and transparency about what it's doing. Those three are the skeleton the rest of today hangs on.

Host: So in practice: default to a workflow, write down why you need an agent, and decide where it stops and where a human checks in before you write a single prompt.

Co-host: Okay, so say the agent does exist. It's decided to deploy the backend. What happens next is where I've seen teams get burned.

Host: The 12-factor agents guide makes a precise point. You need to be able to interrupt the agent after it has chosen a tool but before that tool runs, so a person can review and approve the call.

Co-host: That gap is everything. If the framework fuses choose and execute into one step, there's nowhere to stand.

Host: And their worked example is exactly your scenario. A high-stakes request like deploying a backend breaks out of the loop and waits for human approval.

Host: And then nothing is running. No process holding its breath for three hours while the on-call engineer finishes lunch.

Co-host: Right. When the approval arrives, say as a webhook, you load the saved state, append the human's decision as the tool result, and the loop continues as if it never paused.

Host: The trade-off is that you now own more code. A framework that hides the loop is quicker on day one and harder on the day something goes wrong.

Co-host: What still goes wrong? Stale state. The world moves while you wait. The approved deploy might target a branch that changed an hour ago.

Host: Which is why, in my view, an approval should be re-validated against current state when it resumes, not trusted blindly. So: separate choosing from doing, make every pause a saved state, and treat resume as a fresh check.

Co-host: Here's what surprised me most from that guide. It flips the direction. The agent asks the human, through a tool.

Host: The guide models a human request as a structured tool call. It carries an urgency of low, medium or high, and a response format of free text, yes or no, or multiple choice.

Co-host: As an engineer, I love that. A yes or no request can go to a phone notification. A high urgency one can page somebody. Free text goes to a ticket. The format tells your routing what to build.

Host: And it frees agents from the chat window. The guide notes this lets agents be kicked off by a cron job or an event, rather than someone typing into a box.

Co-host: So the nightly agent wakes at two in the morning, finds a failing dependency, and asks: approve upgrade, yes or no, medium urgency. Nobody was chatting with it.

Host: The failure mode is the obvious one. Make everything high urgency and you've built a very expensive alarm clock that people learn to ignore.

Co-host: Approval fatigue. People click yes to make it stop.

Host: So in practice: give the human a narrow, structured question, use urgency honestly, and design for agents that start without anyone watching.

Host: Now the uncomfortable part. You've built a lovely approval step. Can the agent simply walk around it?

Host: In the Claude Agent SDK, a deny rule wins. It blocks a tool call even in the bypass permissions mode, and even when a hook has returned allow.

Co-host: Good. So deny is the floor nobody falls through. But there's a catch, right?

Host: There is. The SDK warns that in that bypass mode, tool calls are approved before your approval callback gets consulted, so the callback only runs for a few calls.

Co-host: That's the trap. You write a careful callback, test it in normal mode, someone flips to bypass for speed, and your callback is mostly decorative.

Co-host: And the OpenAI side has a subtler one. Durable approvals.

Host: In the OpenAI Agents SDK, sticky always approve decisions for local shell and file edit tools match on tool type and name, not on which agent owns the tool. So after a handoff, an approval can carry over to another agent's tool.

Co-host: So I approve the shell for my careful research agent, it hands off, and the next agent inherits my yes. That's a convenience feature, not a security boundary.

Host: Exactly. Remembered approvals are state, not a security check. So: put hard denies in config, never rely on a callback alone, and scope stored approvals as narrowly as you can.

Co-host: So how do you decide what to automate versus supervise across a whole organisation? Gut feel doesn't scale.

Host: The outline points to NIST's voluntary AI Risk Management Framework and its generative AI profile as shared vocabulary. Our checked sources today don't detail them, so I'll just say: use a common risk language so security, legal and engineers argue about the same thing.

Co-host: And there's fresh research making the gate itself more formal.

Host: A paper submitted on 31 October 2026 proposes Closed-Loop Dependency Assurance, adding dependency-aware validation to agentic network digital twins.

Host: Its key insight: a network change can hit its goal while breaking the monitoring or recovery you need for the next decision.

Co-host: Oh, that's real. You fix the outage and quietly take down the alerting. Success, technically.

Host: And unlike the language model planner, its assurance step uses explicit, deterministic rules to decide and repair.

Co-host: Model proposes, rules dispose. And the second paper?

Host: Submitted on 22 October 2026, it treats the harness between agent and tools as a Petri-net supervisory control problem, where tokens stand for capabilities, approvals and budgets.

Host: On an autonomous release agent, it reports safe deployment needs a step the supervisor can still refuse after a rollout has begun.

Co-host: Which matches my experience. Approval isn't one gate at the start. These are proposals, though, not proven in broad production yet.

Host: Fair caveat. So: keep the deterministic checks outside the model, and keep the right to refuse mid-action.

Host: The one takeaway: a human in the loop is only real if it's a step in your control flow, saved as state, with permissions that no mode or remembered yes can skip.

Host: That was part of our series. The show notes list every source, and this episode was made with AI voices. Follow AI Shadow for the rest of the series. Goodbye.