A Practical Guide, Part 7 of 15: Multi-Agent Orchestration

Co-host: Which, as anyone who has worked in an office knows, can go either way.

Co-host: And we closed the loop with checks the agent can run on itself. Which matters today, because more agents means more places to be wrong.

Host: Exactly. So the question running through this episode is simple: when does a task actually deserve several agents, and what do you pay for them?

Host: Picture an analyst asked a sprawling question, say, every company working on a niche technology, their funding, their products and their partnerships. One person would open forty browser tabs and slowly lose the will to live.

Co-host: Or hand the tabs out to a team and collect the notes. That's the instinct behind multi-agent systems.

Co-host: And orchestrator-workers sits in between, right? A central model breaks the job down at runtime and hands pieces to workers.

Host: Right, and the key word is runtime. With plain parallelization, you know the subtasks in advance and just run them side by side. With an orchestrator, the subtasks are discovered as the problem unfolds.

Host: The strongest case for it is that some questions are wide, not deep. Many independent threads, each worth exploring, none depending much on the others.

Co-host: And the weakest case is when the threads are tangled. Then the workers keep needing each other and your orchestrator becomes a very expensive switchboard operator.

Host: So in practice: reach for parallelization when you can list the subtasks yourself, and for an orchestrator only when the breakdown genuinely depends on what you find.

Host: Our case study is Anthropic's Research feature, which does exactly that sprawling-question job. A lead agent coordinates and delegates to specialised subagents running in parallel.

Co-host: What's the actual mechanism? Like, what does a subagent see?

Host: Each subagent has its own context window. It explores one aspect of the question, then condenses the most important tokens and passes them back to the lead.

Host: It's compression by delegation. The lead never sees the forty tabs, only the notes.

Co-host: Okay, but does it work? Give me a number.

Host: Anthropic reported that a system with Claude Opus 4 as lead and Claude Sonnet 4 subagents beat single-agent Claude Opus 4 by 90.2 percent on its internal research eval.

Host: Quite. It's a measured result, which beats a demo, but it's their benchmark on their kind of task. Treat it as strong evidence for wide research questions, not a law of nature.

Co-host: And the bill?

Host: They say agents typically use about four times the tokens of a chat interaction, and multi-agent systems about fifteen times.

Co-host: Fifteen. So part of that gain is just spending more compute. Which is fine, if the answer is worth it.

Host: That's the honest framing. You're buying parallel thinking time. For a high-value research report, sensible. For what's the capital of France, a little extravagant.

Co-host: Did they hit early failures?

Host: They did. Early versions made coordination errors, like spawning 50 subagents for simple queries.

Co-host: I love that. Asked to boil an egg, it hired a catering company.

Host: The fix in practice is to tell the lead how to scale effort to the question. The takeaway: multi-agent pays off when the value of the answer justifies roughly an order of magnitude more tokens, and you must teach the orchestrator restraint.

Co-host: So say I'm building this. Who decides which agent runs next, the model or me?

Host: That's precisely the split OpenAI's Agents SDK draws: orchestration by the LLM, or orchestration by your code.

Co-host: Walk me through the LLM side.

Host: Two shapes. In a handoff, one agent passes the conversation to another, who takes over. In agents as tools, a manager agent keeps control of the conversation and calls specialists through a method called as tool.

Co-host: The difference matters a lot in production. With a handoff the specialist owns the user now. With agents as tools, the manager stays the single voice and just treats experts like function calls.

Host: And agents as tools is much closer to the Anthropic pattern: one lead, workers returning results.

Co-host: And orchestrating by code?

Host: Your program does the routing: classify, then send to a fixed agent, chain outputs, loop until a check passes, or run agents in parallel and merge. The docs say this makes things more deterministic and more predictable in speed, cost and performance.

Host: The trade-off is flexibility. Code can't improvise a subtask it never anticipated. The model can, and occasionally improvises fifty of them.

Co-host: So, a hybrid: code for the skeleton, model decisions only at the joints where you really need judgement.

Host: That's my view too. Default to code, and promote a decision to the model only when your evals show it earns its unpredictability.

Host: The research literature has been playing with this for a while. AutoGen frames agents as conversable, chatting with each other to solve a task. MetaGPT borrows standard operating procedures and assembly lines, to stop one agent's mistake cascading down the chain.

Co-host: The cascading-error thing is real. A bad spec from agent one becomes confident wrong code from agent three.

Host: Then there's multi-agent debate. Several model instances propose answers, argue over multiple rounds, and converge on a common answer, and the paper reports better factual validity.

Co-host: Steelman: different samples make different mistakes, and cross-examination filters them. Stress test: they can also just talk each other into the same wrong answer.

Host: And every round multiplies the tokens. Debate is an expensive committee.

Co-host: Which brings us to coding. Why not just throw a swarm at my repo?

Host: Anthropic's own view is that most coding tasks have fewer truly parallel pieces than research, and that agents aren't yet great at coordinating and delegating to each other in real time.

Co-host: Makes sense. Two agents editing the same interface is a merge conflict with extra steps.

Host: There's also the synchronous bottleneck. If the lead waits for every worker before moving on, the slowest subagent sets the pace, and nobody can steer mid-flight.

Co-host: And going asynchronous fixes latency but makes state and error handling much harder. That's an open problem, not a solved one.

Host: So in practice: split work only along seams where pieces genuinely don't depend on each other, and keep tightly coupled work like most coding in one agent's head.

Host: If you keep one thing: more agents buy breadth, not wisdom. Use them for wide, parallel work worth many times the tokens, orchestrate in code by default, and teach your lead agent when not to delegate.

Host: Which matters more now our agents have colleagues. This was part of a series; the show notes list every source, and this episode was made with AI voices. Follow AI Shadow for the rest of the series. Goodbye.