A Practical Guide, Part 5 of 15: Planning and Task Decomposition

Co-host: Quick rewind first. Last time, in Context and Memory, we said the context window is a budget, not a bucket.

Host: Right. And for an agent that runs long or comes back later, you choose between compaction, just-in-time retrieval, note-taking, memory tiers and a proper memory store. Keep that in your pocket, because planning leans on it today.

Co-host: So the question for this episode is simple to ask and annoyingly hard to answer. When should an agent stop and make a plan, and when is planning just procrastination with extra steps?

Host: Procrastination with extra steps. I feel personally seen. Let's find out.

Host: Picture a maths word problem. A train leaves a station, some apples get shared out, and the model has to land on one number. Early language models would often just blurt an answer, and the answer was often wrong.

Co-host: The classic fix was getting the model to show its working. That's chain-of-thought, right?

Host: Exactly. Chain-of-thought prompting means you show the model a few worked examples, called exemplars, where the reasoning is written out step by step, and it copies that style. In the original work, a model with 540 billion parameters was given just eight such exemplars and hit state of the art accuracy on a benchmark of maths word problems called GSM8K.

Co-host: Parameters being the internal numbers a model learns during training, so 540 billion is a very big brain. And a benchmark is just a standard test set everyone scores against.

Host: Nicely put. But writing your reasoning as you go isn't quite planning. Plan-and-Solve made it explicit. First, devise a plan that splits the task into smaller subtasks. Second, carry out those subtasks according to the plan.

Co-host: Why bother with the split, though? If the model reasons step by step anyway?

Host: Because step by step reasoning without examples still skipped steps. Plan-and-Solve was designed to tackle exactly those missing-step errors that zero-shot chain-of-thought kept making.

Co-host: Zero-shot meaning no worked examples at all, just 'think step by step' and good luck. Okay, that makes sense. Writing the checklist first means you notice the gap before you fall into it.

Host: Then Tree of Thoughts went further. Instead of one line of reasoning, the model explores several paths, judges its own choices, and can look ahead or back up when a path is going nowhere.

Co-host: Like a chess player trying moves in their head instead of committing to the first one. Did it actually help?

Host: On a puzzle called Game of 24, where you combine four numbers to make twenty-four, GPT-4 with chain-of-thought solved four percent of tasks. With Tree of Thoughts it reached seventy-four percent.

Co-host: That's a huge jump. Let me play the practitioner here, though. Searching many paths means many more model calls. That's not free.

Host: So what it means for you: explicit plans catch skipped steps, and search catches dead ends, but only reach for the expensive version when a wrong turn early actually costs you.

Co-host: Okay, puzzles are cute. What happens when a real product plans?

Co-host: And I'm guessing it didn't go perfectly the first time.

Host: It did not. Anthropic says its early research agents made mistakes like spawning fifty subagents for simple queries.

Co-host: Fifty! For a simple question. That's hiring a whole consultancy to find out what time the shop closes.

Host: Which is why the lesson is to scale effort to the difficulty of the question. A simple fact-find needs one worker and a few tool calls. A sprawling comparison might deserve several. The planner has to judge size before it judges steps.

Co-host: And where does the plan live? If the lead agent runs for ages, the plan could fall out of its memory.

Co-host: Tokens being the chunks of text a model reads, roughly word pieces, and the context window being how many it can hold at once. So the plan gets written down somewhere safe before it gets cut off.

Host: Exactly. And the bill matters. Anthropic says multi-agent systems use around fifteen times more tokens than a chat, so the task has to be valuable enough to pay for the better performance.

Co-host: That's the number I'd tape to my monitor. Worth noting it's the company's own estimate about its own system, so treat it as a rough guide, not a law of physics.

Host: Fair. So, in practice: let the planner size the job before it splits the job, keep the plan somewhere it can't be truncated, and only go multi-agent when the answer is worth many times the cost.

Co-host: Let's get concrete. Say I'm the lead agent and I have to brief a subagent. What does a good brief look like?

Host: Let's do a worked example. Imagine the brief just says, research the semiconductor shortage. What could go wrong?

Co-host: Honestly? Everything. Which shortage, which years, cars or phones? One subagent might dig into a past crisis while another does the exact same digging. You'd get duplicated work and gaps at the same time.

Host: Right, a vague brief is like telling a contractor to 'do the kitchen.' So a good subtask has four parts. First, an objective: what question, specifically, are you answering. Second, an output format: a short list of sources with dates, say, or a table.

Co-host: Because the lead agent has to stitch the results back together, so they should arrive in a shape it can use.

Host: Exactly. Third, which tools to use: web search, an internal database, whatever fits. Fourth, boundaries: what's out of scope, and when to stop.

Co-host: Boundaries are the one people skip. In my experience the stop condition is what saves you. Without it, a diligent subagent just keeps searching forever.

Host: So the improved version reads something like: find recent causes of chip supply delays affecting car makers, return five sources with dates and a one-line summary each, use web search only, ignore consumer electronics, stop after five good sources.

Co-host: Way better. Though there's a trade-off. Too tight, and the subagent can't follow an interesting lead it wasn't told about.

Host: Quite. You're trading flexibility for predictability. My view: tight for routine fact-finding, looser for genuinely open exploration. What it means for you: before you hand anything to an agent, write the objective, the format, the tools and the boundaries, and if you can't, the plan isn't ready.

Co-host: Here's what bugs me. Even a good plan goes stale. The agent hits a broken website, or the plan was slightly wrong from the start.

Host: Which is why some systems put a human in before anything happens. Microsoft's Magentic-UI lets you edit its plan, through a plan editor or by typing feedback, before it takes any actions.

Co-host: I love that. It's the cheapest moment to catch a mistake. Fixing a plan costs a sentence; fixing an action, like a booked flight, costs a phone call.

Host: And researchers are now measuring planning on its own. A new Agent Planning Benchmark paper, first submitted in June 2026 and revised to version four in October, tries to separate how good a plan is from whether the whole run succeeds.

Co-host: That's useful, because normally a failed run tells you nothing about why. Bad plan, or bad execution?

Host: It has 4,209 cases with images and text, across 22 domains and five settings. Testing twelve multimodal models, it found consistent weaknesses in long-horizon planning, coping with noisy tools, refusing sensibly when a task can't be done, and improving a plan at run time.

Co-host: Calibrated refusal is a big one. An agent that confidently plans an impossible task is worse than one that says no.

Host: There's a hopeful bit too. On two hundred tasks each from two tool-use test suites, refining plans using the benchmark's guidance consistently improved plan correctness, plan grades and the downstream execution, across three representative models.

Co-host: Three models is a modest sample, and it's the authors measuring their own method. Promising, not settled.

Host: Agreed. So in practice: review the plan before anything irreversible runs, expect the plan to need re-planning when tools misbehave, and judge the plan separately from the outcome.

Co-host: Now the flip side. I've watched coding agents write a three-page plan to rename one variable.

Host: The fifty subagents problem in miniature. A pattern many developers find useful for coding agents is four steps: explore the code, plan, implement, then commit.

Co-host: Explore first is key. A plan written before reading the code is basically fan fiction.

Host: Lovely. And here's a rule of thumb I'd offer: if you can describe the change in one sentence, skip the plan and just do it.

Co-host: So 'fix the typo in the header' goes straight through, but 'add login to the app' gets explored and planned.

Host: So the one takeaway: plan when mistakes are expensive, brief subtasks with an objective, a format, tools and boundaries, let a human edit the plan before it acts, and skip planning when the job fits in one sentence.