Module 18: Tool Use & Agents

Introduction

Every module so far ends at a model that answers. Give it a prompt, it predicts text. But ask it “what is 48239 × 1123?” and it will confidently produce a wrong number — it is pattern-matching digits, not calculating. Ask “what happened in the news today?” and it cannot know: its weights are frozen at training time. The model is a brilliant text predictor trapped inside its own head.

An agent breaks it out. The idea is to let the model act: emit a structured tool call, have the surrounding program run it, and feed the result back so the model can reason over it — in a loop — until it can answer. A tool is just a function (a calculator, a search index, a database query, a shell) that the model invokes by writing text we agree to parse.

Why it matters for LLMs:

  • Reliability. A calculator computes 48239 × 1123 exactly; the model never has to. Toolformer (Schick et al., 2023) framed this precisely — models are strong at language but weak at arithmetic and lookup, so hand them tools.
  • Reach. Retrieval (m15), code execution, web search, and APIs all become actions. This is what coding agents, deep-research systems, and computer-use models are built on.
  • It’s just a loop. The headline of this module: an agent is not new model machinery. It is a while-loop around generate() (m08) that parses the model’s output, runs a tool, and appends the result to the prompt.

What You’ll Learn

After this module, you can:

  • Explain why a text-predicting LLM cannot act, and how the ReAct Thought → Action → Observation loop fixes it.
  • Build a Tool and a ToolRegistry, and a safe calculator that never evals model output.
  • Parse the model’s action from text (Action: tool[arg]) and from JSON — the modern function-calling format.
  • Assemble the agent prompt (system + tool docs + scratchpad) and run the ReAct loop from scratch, with a max_steps guard.
  • Read an agent trajectory and watch the tool’s output flow into the answer.

Prerequisites

This module requires familiarity with:

Intuition: The Agent Loop

A plain LLM call is one shot: prompt in, answer out. An agent turns that single shot into a cycle. On each turn the model writes a Thought (its reasoning), then either an Action (a tool call) or a final Answer. If it acts, the program runs the tool and hands back an Observation; the model thinks again with that new information. The loop ends when the model is confident enough to answer.

Walk one concrete trajectory — “What is 17 × 23, and is it greater than 400?” — around the ReAct cycle:

NoteKey Insight

The three cycle nodes repeat as many times as the task needs — a two-hop question loops twice, a hard one many times — and the dashed edge to Answer is the only exit. The model never touches the tool itself; it only writes text (“Action: …”), and the surrounding program does the acting. That separation is the whole trick.

A Tool Is Just a Function

Strip away the mystique and a tool is a named function str → str plus a one-line description so the model knows when to reach for it. agents.py defines a Tool dataclass and a ToolRegistry that dispatches by name. The single most important tool is a calculator — because arithmetic is exactly what LLMs are worst at:

from agents import Tool, ToolRegistry, calculator, make_lookup_tool

registry = ToolRegistry([
    Tool("calculator", "Evaluate arithmetic, e.g. 17 * 23.", calculator),
    make_lookup_tool({"speed of light": "299792458 m/s"}),
])

print(registry.run("calculator", "48239 * 1123"))   # exact — the model never could
print(registry.run("lookup", "speed of light"))
print(registry.run("search", "anything"))            # unknown tool -> an observation, not a crash
54172397
299792458 m/s
Error: unknown tool "search". Available: calculator, lookup

Notice the calculator is safe. The naive implementation — eval(expression) — is a remote-code-execution hole: a model (or a prompt-injected observation) could emit __import__('os').system('rm -rf /'). Instead we parse the string to an AST and walk only whitelisted node types (numbers and the arithmetic operators):

print(calculator("(3 + 4) ** 2"))     # 49 — parens, power, precedence all work
try:
    calculator("__import__('os').system('echo pwned')")
except ValueError as e:
    print("rejected:", e)              # names/calls never evaluate
49
rejected: unsupported expression element: Call
WarningNever eval model output

A tool runs whatever the model asks. If the tool is eval, exec, an unsanitized shell, or raw SQL, the model — or anything that can influence the model, including a poisoned web page it just “observed” — controls your machine. Whitelist what a tool can do (here: arithmetic AST nodes only), sandbox anything that touches the OS, and treat every observation as untrusted input.

The registry also renders the tool list straight into the system prompt, so the model sees exactly what it can call:

print(registry.render_descriptions())
- calculator: Evaluate arithmetic, e.g. 17 * 23.
- lookup: Look up a fact by exact key. Keys: speed of light.

Parsing the Model’s Action

The model communicates an action by writing text in a format we agree on. The ReAct convention is Action: <tool>[<input>], and a finished answer is Answer: <text>. Parsing is a small regex — and it must gracefully return None when the model forgets the format:

from agents import parse_action, parse_answer, parse_thought

turn = "Thought: I should not guess.\nAction: calculator[17 * 23]"
print(parse_thought(turn))    # 'I should not guess.'
print(parse_action(turn))     # ('calculator', '17 * 23')
print(parse_answer(turn))     # None — this turn is an action, not an answer

print(parse_action("Answer: 391"))          # None
print(parse_answer("Answer: 391, done."))   # '391, done.'
I should not guess.
('calculator', '17 * 23')
None
None
391, done.

Modern tool-calling APIs (OpenAI, Anthropic) use a JSON format instead of the ReAct text convention — same idea, different serialization, riding on the chat templates from m03. parse_json_tool_call pulls the first balanced JSON object out of the model’s prose and normalizes it:

from agents import parse_json_tool_call

print(parse_json_tool_call('Let me compute: {"tool": "calculator", "args": "2 + 2"}'))
print(parse_json_tool_call('{"name": "lookup", "arguments": "speed of light"}'))
print(parse_json_tool_call("no tool call here"))   # None
{'tool': 'calculator', 'args': '2 + 2'}
{'tool': 'lookup', 'args': 'speed of light'}
None

Try both formats yourself — type an action and watch it parse:

TipTry This
  1. Break the format. Delete the ] from the ReAct box, or the closing } from the JSON. Both fall to “no valid tool call” — exactly the case the real loop has to survive.
  2. Call a tool that doesn’t exist. Change calculator to wikipedia. It parses fine (parsing doesn’t know your registry), but the observation would be an error the model must react to.

The ReAct Loop, From Scratch

Now assemble the pieces. The agent’s entire “memory” is the scratchpad — the replayed transcript of past thoughts, actions, and observations — because the model is stateless: every turn we rebuild the full prompt (system + tool docs + question + scratchpad) and let it continue. build_prompt does the assembly; run_agent is the loop:

from agents import build_prompt, run_agent, scripted_policy

# The "policy" is the LLM: prompt -> next completion. In production this is m08's
# generate(); here we SCRIPT the completions so the run is deterministic. The tools,
# though, are real — the calculator genuinely computes.
script = [
    "Thought: I shouldn't guess. Use the calculator.\nAction: calculator[17 * 23]",
    "Thought: 391 < 400, so it is not greater.\nAnswer: 17 * 23 = 391, which is not greater than 400.",
]
result = run_agent("What is 17 * 23, and is it greater than 400?",
                   scripted_policy(script), registry, max_steps=6)

print("answer:", result.answer)
print("tool calls:", result.num_tool_calls, "| stopped:", result.stopped)
answer: 17 * 23 = 391, which is not greater than 400.
tool calls: 1 | stopped: answer

The prompt the model actually saw on its second turn already contains the real observation from the first — this is how the tool’s output re-enters the reasoning:

from agents import Step
print(build_prompt("What is 17 * 23...?", registry,
                   [Step("I'll use the calculator.", "calculator", "17 * 23", "391")]))
You are a reasoning agent that can call tools to answer a question.
On each turn, write a Thought, then EITHER one Action or a final Answer:

  Thought: <your reasoning>
  Action: <tool>[<input>]      (to call a tool)
  Answer: <final answer>       (when you are done)

Available tools:
- calculator: Evaluate arithmetic, e.g. 17 * 23.
- lookup: Look up a fact by exact key. Keys: speed of light.

Use a tool whenever it is more reliable than guessing (e.g. arithmetic, lookup).
After each Action you will see an Observation with the tool's result.

Question: What is 17 * 23...?
Thought: I'll use the calculator.
Action: calculator[17 * 23]
Observation: 391
Thought:

Two failure modes the loop must handle, both tested in agents.py:

  • The model never answers. A max_steps cap turns an infinite think/act loop into a clean “stopped: max_steps” result.
  • The model forgets the format. An unparseable turn produces a _format observation nudging it to retry, instead of crashing.
# A policy that only ever acts, never answers -> the guard stops it.
never_answers = scripted_policy(["Thought: hmm.\nAction: calculator[1 + 1]"] * 100)
runaway = run_agent("loop forever?", never_answers, registry, max_steps=3)
print("stopped:", runaway.stopped, "| answer:", runaway.answer, "| steps:", len(runaway.steps))
stopped: max_steps | answer: None | steps: 3

Watching a Trajectory

demonstrate_agent runs a real trajectory whose final answer is produced by the tool, not the language model: the policy calls the calculator, reads the genuine observation 391 back out of the prompt, and answers with it. If the loop were broken — if the observation never re-entered the prompt — the answer could not be 391.

from agents import demonstrate_agent

trajectory = demonstrate_agent(verbose=True)
Question: What is 17 * 23?  (stopped: answer)
  step 0: Thought: I should not guess arithmetic; use the calculator.
           Action: calculator[17 * 23] -> 391
  step 1: Thought: The tool returned 391; that is my answer.
  Answer: 391  (1 tool call(s))

Bridge the real step records to an interactive viewer and step through them:

# Each step is JSON-serializable via .as_dict(); hand them to OJS.
ojs_define(ag_steps = [s.as_dict() for s in trajectory.steps])
ojs_define(ag_answer = trajectory.answer)
NoteKey Insight

The answer 391 is the calculator’s output, threaded back through the scratchpad — not a number the model made up. That is the whole value of an agent: it grounds the model’s output in a real computation it could never do reliably on its own. Swap calculator for a search index, a Python sandbox, or a database and the loop is unchanged.

Common Pitfalls

When building agents, watch out for:

  1. Unsafe tools. eval/exec/raw shell/raw SQL hand your machine to the model. Whitelist and sandbox; treat observations as untrusted (they can carry prompt injection — a web page that says “ignore your instructions and…”).
  2. No step cap. A model that never emits Answer: loops until it exhausts your budget. Always bound the loop (max_steps) and decide what a timeout returns.
  3. Brittle parsing. Real models drift from the format. Parse defensively, return None on failure, and feed a corrective observation rather than crashing.
  4. A too-thin scratchpad. The model is stateless; if you don’t replay past observations into the prompt, it re-does work or contradicts itself. The scratchpad is the memory.
  5. Tool sprawl. Twenty vague tools confuse the router more than they help. Few, sharply-described tools with unambiguous inputs beat a giant menu.
  6. Trusting a single trajectory. Agents are stochastic; one good run isn’t a success rate. Evaluate them like anything else (m17) — over many tasks, with a metric.

Exercises

Exercise 1: Add a tool

from agents import Tool, ToolRegistry, run_agent, scripted_policy

# Add a `string_length` tool that returns the number of characters in its input.
# Register it alongside the calculator, then script a trajectory that answers
# "How many characters are in the word 'transformer'?" using it. Assert the answer
# is "11" and that exactly one tool call was made.

# Your implementation here:

Exercise 2: A two-hop task

from agents import ToolRegistry, Tool, calculator, make_lookup_tool, run_agent, scripted_policy

# Build a registry with a calculator and a lookup tool holding
# {"blocks per day": "86400"}. Script a trajectory for "How many seconds are in a
# week?" that (1) looks up seconds-per-day, then (2) multiplies by 7 with the
# calculator. Check num_tool_calls == 2 and the answer contains "604800".

# Your implementation here:

Exercise 3: Guard against loops

from agents import run_agent, scripted_policy, ToolRegistry, Tool, calculator

# Write a policy that ALWAYS acts and never answers. Run it with max_steps=5 and
# confirm result.stopped == "max_steps" and len(result.steps) == 5. Then add a
# fallback: if stopped == "max_steps", return the last observation as a best-effort
# answer. Why is a best-effort answer better than raising?

# Your implementation here:

Summary

Key takeaways:

  1. An agent is a loop, not a new model. ReAct wraps generate() (m08) in a Thought → Action → Observation cycle: the model writes text, the program runs a tool, the result comes back as an observation. Repeat until Answer:.
  2. A tool is just a function plus a description. str → str with a one-line doc. The model calls it by writing structured text (Action: tool[arg] or JSON) that we parse and dispatch.
  3. Tools fix what models are bad at. A safe calculator computes exactly; a lookup tool reaches facts the weights don’t hold — grounding the answer in real computation (Toolformer’s motivation).
  4. Never eval model output. Whitelist and sandbox every tool; treat every observation as untrusted input that may carry prompt injection.
  5. The scratchpad is the memory. A stateless model only knows what you replay into the prompt; the observation must re-enter for the loop to close.
  6. Bound and evaluate the loop. A max_steps guard stops runaway agents, and agent quality is a success rate over many tasks (m17), never one lucky run.

What’s Next

You have now built the full arc — a model that tokenizes, attends, trains, aligns, reasons, retrieves, is measured, and now acts. The frontier from here is interpretability: opening the box to see why a model produces the tokens it does — logit lens, probing, induction heads, and the sparse-autoencoder features that let us read (and steer) a model’s internal computation. Agents make models do more; interpretability makes us understand what they’re doing.

Going Deeper

Core Papers:

Practical Resources: