A Practical Guide, Part 3 of 15: Tools and Function Calling

Host: And reflection let it notice a bad step and re-plan. Which leaves one obvious question. When the loop says act, what exactly does it act with? That's today.

Host: Picture a language model asked to multiply two long numbers. It's brilliant with words and quietly hopeless at arithmetic, a bit like some journalists I know. The obvious fix is to hand it a calculator. The hard part is teaching it when to reach for one.

Co-host: That's the problem the Toolformer paper went after, right?

Host: Yes. It describes a model trained to decide which outside service to call, when to call it, what to pass in, and how to fold the answer back into what it writes next.

Co-host: Quick definitions for listeners. An API is just a doorway one program offers another, like a calculator or a search engine you can ask questions of. And a token is a small chunk of text, the unit a model reads and writes one at a time.

Host: The striking part is how little hand-holding it needed. The training was self-supervised, meaning the model largely taught itself, with only a handful of examples for each API.

Co-host: As an engineer, that's the part I'd push on. A handful of examples per tool sounds amazing, but that's the paper's own setup, not a promise that any model learns any tool that cheaply.

Host: Fair test. What it does show is the idea that matters for us: deciding to use a tool can be part of the model's own judgement, not something bolted on by hand.

Co-host: So jump to today. I've built these. The single most misunderstood thing is who actually runs the tool.

Host: And the answer is: not the model. Anthropic's docs are blunt about it. The model only writes a structured request, and either your code or Anthropic's servers carry out the work and send the result back.

Co-host: Let's walk a worked example. Say you've built a travel assistant, and a user asks whether their flight tomorrow is delayed.

Host: Step one, you send the model the question plus a list of tools, each with a description. Step two, the model reads the descriptions and picks one, say a flight status tool. Step three, instead of a normal answer, it hands back a neat structured request: this tool, this flight number, this date.

Co-host: Step four, my code sees that request, calls the airline's system, and gets real data. Step five, I send that result back, and the model writes the human answer. Other vendors, OpenAI included, describe a similar step-by-step flow.

Host: Why the detour? Because the model can't see live data and shouldn't hold the keys to your systems. Your code stays the gatekeeper.

Host: Then there's the other kind, server tools. For things like web search, fetching a web page, running code, and searching for tools, Anthropic runs the code itself, and you never have to package up the result.

Co-host: Less plumbing, less control. That's the trade.

Host: So in practice: if the tool touches your data, your money or your customers, run it on your side and check every request before you act on it.

Co-host: So what does a tool actually look like when you hand it to the model?

Host: Think of it as a form with a few boxes. A name. A description in plain words. And the parameters, which are the inputs the tool expects, written as a schema, a strict template saying which fields exist and what type each one is.

Co-host: Vendors label that box differently, parameters in some APIs, input schema in others, and there's usually a setting for whether the model must call a tool, may call one, or should call a specific one. Same idea everywhere.

Host: Then the newer piece, strict mode. That's the JSON schema contract in the title. The question is whether the model's request is guaranteed to match your template, or just very likely to.

Co-host: Here's a fresh one. The open source serving engine vLLM has developer-preview docs saying the strict field works across the Chat Completions, Responses, and Anthropic Messages styles of API.

Host: And the catch, which I rather like: a tool's argument template is only locked down if that tool sets strict to true, or the server is configured to enforce it.

Co-host: Which means if you forget the flag, you've got a polite suggestion, not a contract. I've debugged that exact situation at two in the morning.

Host: I can't confirm whether those docs are brand new or just newly noticed, so treat that as current guidance for one engine, not an industry standard.

Host: The practical point: if a malformed request would break something downstream, turn strict mode on and confirm it's really being enforced, rather than assuming.

Co-host: Now the part people skip. They spend a week on the code behind the tool and thirty seconds on the description.

Host: Which is backwards. Anthropic's guide says extremely detailed descriptions are by far the most important factor in how well a tool performs.

Co-host: Makes sense once you remember step two. The model picks the tool from the description. It's the only thing it can read.

Host: A test I find useful: imagine handing the description to a bright new colleague on their first day. Would they know what it does, when to use it, what each input means, and what comes back? If not, the model won't either.

Co-host: So for our flight tool, not just check flight. More like: returns live status for one flight, use when the user asks about delays or gates, flight number with airline code, date of departure, and what it returns if the flight isn't found.

Host: And when a tool misbehaves, read the description before you touch the code. Quite often the fix is a better sentence, not a better program.

Host: Last chapter, and a shift in mindset. Anthropic's engineering team frames tools as a contract between predictable software and an unpredictable agent, so they should be built for agents, not as APIs for other developers.

Co-host: That one hit me. My instinct as a developer is one neat function per action. Create, update, delete, list.

Host: And the advice runs the other way: group related actions into fewer tools with an action parameter, rather than one tool for each.

Co-host: Fewer choices, fewer wrong choices. And naming matters too.

Host: Prefix names with the service, so something like github underscore list underscore p r s, which keeps the choice unambiguous as your toolbox grows.

Co-host: vLLM's docs take namespacing a step further. In its Responses API you can pick a function in a namespace by its full name or just its short local name.

Host: Then what comes back. Return what the agent needs, not a raw dump. Every word in a result lands in the context window, the model's limited working memory for the conversation, so bloated responses crowd out the useful stuff.

Co-host: Some teams even have an agent test their tools and suggest description fixes. I like it, though I'd want to see the measured result on my own tasks before trusting anyone's headline number.

Host: And sometimes the best tool is no tool. Anthropic lists latency as a reason to skip one, since every call adds at least one extra round trip.

Co-host: If the model already knows the answer, making it phone a friend just slows everyone down.

Host: So, before adding a tool, ask: does it need live data or a real action? If yes, build one consolidated, clearly named, clearly described tool. If no, leave it out.

Host: The one takeaway: the model never touches your systems, it only asks. So the quality of what it asks for depends on how clearly you've described the tool and how firmly you enforce the contract.

Co-host: Write the description like it's the product. Because it kind of is.

Host: This was part of our series Agentic AI: A Practical Guide. The show notes list every source, and this episode was made with AI voices. Follow AI Shadow for the rest of the series. Goodbye for now.