{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Module 18: Tool Use & Agents"
      ],
      "id": "7e055b25-47fd-4ae5-944e-c94591f92b13"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [],
      "id": "32c20198-013a-4609-b8d2-10069ac57934"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {}
        }
      ],
      "source": [],
      "id": "91a2a03b-fc2e-4a50-8170-ca64511ad108"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Introduction\n",
        "\n",
        "Every module so far ends at a model that **answers**. Give it a prompt,\n",
        "it predicts text. But ask it “what is 48239 × 1123?” and it will\n",
        "confidently produce a *wrong* number — it is pattern-matching digits,\n",
        "not calculating. Ask “what happened in the news today?” and it cannot\n",
        "know: its weights are frozen at training time. The model is a brilliant\n",
        "text predictor trapped inside its own head.\n",
        "\n",
        "An **agent** breaks it out. The idea is to let the model *act*: emit a\n",
        "structured **tool call**, have the surrounding program run it, and feed\n",
        "the result back so the model can reason over it — in a loop — until it\n",
        "can answer. A **tool** is just a function (a calculator, a search index,\n",
        "a database query, a shell) that the model invokes by writing text we\n",
        "agree to parse.\n",
        "\n",
        "Why it matters for LLMs:\n",
        "\n",
        "- **Reliability.** A calculator computes `48239 × 1123` exactly; the\n",
        "  model never has to. Toolformer (Schick et al., 2023) framed this\n",
        "  precisely — models are strong at language but weak at arithmetic and\n",
        "  lookup, so hand them tools.\n",
        "- **Reach.** Retrieval (m15), code execution, web search, and APIs all\n",
        "  become *actions*. This is what coding agents, deep-research systems,\n",
        "  and computer-use models are built on.\n",
        "- **It’s just a loop.** The headline of this module: an agent is not new\n",
        "  model machinery. It is a `while`-loop around `generate()` (m08) that\n",
        "  parses the model’s output, runs a tool, and appends the result to the\n",
        "  prompt.\n",
        "\n",
        "### What You’ll Learn\n",
        "\n",
        "After this module, you can:\n",
        "\n",
        "- Explain why a text-predicting LLM cannot act, and how the **ReAct**\n",
        "  Thought → Action → Observation loop fixes it.\n",
        "- Build a `Tool` and a `ToolRegistry`, and a **safe** calculator that\n",
        "  never `eval`s model output.\n",
        "- Parse the model’s action from text (`Action: tool[arg]`) *and* from\n",
        "  JSON — the modern function-calling format.\n",
        "- Assemble the agent prompt (system + tool docs + scratchpad) and run\n",
        "  the ReAct loop from scratch, with a `max_steps` guard.\n",
        "- Read an agent **trajectory** and watch the tool’s output flow into the\n",
        "  answer.\n",
        "\n",
        "### Prerequisites\n",
        "\n",
        "This module requires familiarity with:\n",
        "\n",
        "- [Module 08: Generation](../m08_generation/lesson.qmd) — the `generate`\n",
        "  loop; an agent wraps a control loop around it.\n",
        "- [Module 13: Reasoning](../m13_reasoning/lesson.qmd) —\n",
        "  chain-of-thought; the “Thought” in ReAct *is* that reasoning, now\n",
        "  interleaved with actions.\n",
        "- [Module 15: Retrieval-Augmented Generation](../m15_rag/lesson.qmd) —\n",
        "  retrieval as a capability; here it becomes one tool the agent can\n",
        "  choose to call.\n",
        "\n",
        "## Intuition: The Agent Loop\n",
        "\n",
        "A plain LLM call is one shot: prompt in, answer out. An agent turns that\n",
        "single shot into a **cycle**. On each turn the model writes a\n",
        "**Thought** (its reasoning), then either an **Action** (a tool call) or\n",
        "a final **Answer**. If it acts, the program runs the tool and hands back\n",
        "an **Observation**; the model thinks again with that new information.\n",
        "The loop ends when the model is confident enough to answer.\n",
        "\n",
        "Walk one concrete trajectory — *“What is 17 × 23, and is it greater than\n",
        "400?”* — around the ReAct cycle:"
      ],
      "id": "11edead0-8dbb-4d2e-9032-30b2c025a7b4"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {}
        }
      ],
      "source": [],
      "id": "8a1b18f7-1092-42a1-8ae6-1a262c5d82ab"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {}
        }
      ],
      "source": [],
      "id": "5ae9c05d-023c-419c-ae6b-a8a5659af6c9"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {}
        }
      ],
      "source": [],
      "id": "39507f73-2352-4983-a8c0-8f8540a5623a"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "> **Key Insight**\n",
        ">\n",
        "> The three cycle nodes repeat as many times as the task needs — a\n",
        "> two-hop question loops twice, a hard one many times — and the dashed\n",
        "> edge to **Answer** is the only exit. The model never touches the tool\n",
        "> itself; it only *writes text* (“Action: …”), and the surrounding\n",
        "> program does the acting. That separation is the whole trick.\n",
        "\n",
        "## A Tool Is Just a Function\n",
        "\n",
        "Strip away the mystique and a **tool** is a named function `str → str`\n",
        "plus a one-line description so the model knows when to reach for it.\n",
        "`agents.py` defines a `Tool` dataclass and a `ToolRegistry` that\n",
        "dispatches by name. The single most important tool is a **calculator** —\n",
        "because arithmetic is exactly what LLMs are worst at:"
      ],
      "id": "f100386c-bdb0-4601-8aaa-9b0b755ef13a"
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {},
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "54172397\n",
            "299792458 m/s\n",
            "Error: unknown tool \"search\". Available: calculator, lookup"
          ]
        }
      ],
      "source": [
        "from agents import Tool, ToolRegistry, calculator, make_lookup_tool\n",
        "\n",
        "registry = ToolRegistry([\n",
        "    Tool(\"calculator\", \"Evaluate arithmetic, e.g. 17 * 23.\", calculator),\n",
        "    make_lookup_tool({\"speed of light\": \"299792458 m/s\"}),\n",
        "])\n",
        "\n",
        "print(registry.run(\"calculator\", \"48239 * 1123\"))   # exact — the model never could\n",
        "print(registry.run(\"lookup\", \"speed of light\"))\n",
        "print(registry.run(\"search\", \"anything\"))            # unknown tool -> an observation, not a crash"
      ],
      "id": "745c898f"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Notice the calculator is **safe**. The naive implementation —\n",
        "`eval(expression)` — is a remote-code-execution hole: a model (or a\n",
        "prompt-injected observation) could emit\n",
        "`__import__('os').system('rm -rf /')`. Instead we parse the string to an\n",
        "AST and walk only whitelisted node types (numbers and the arithmetic\n",
        "operators):"
      ],
      "id": "3a87fdd4-8778-4186-808d-f5e7fe3be3af"
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {},
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "49\n",
            "rejected: unsupported expression element: Call"
          ]
        }
      ],
      "source": [
        "print(calculator(\"(3 + 4) ** 2\"))     # 49 — parens, power, precedence all work\n",
        "try:\n",
        "    calculator(\"__import__('os').system('echo pwned')\")\n",
        "except ValueError as e:\n",
        "    print(\"rejected:\", e)              # names/calls never evaluate"
      ],
      "id": "4acd79e7"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "> **Never `eval` model output**\n",
        ">\n",
        "> A tool runs whatever the model asks. If the tool is `eval`, `exec`, an\n",
        "> unsanitized shell, or raw SQL, the model — or anything that can\n",
        "> influence the model, including a poisoned web page it just “observed”\n",
        "> — controls your machine. Whitelist what a tool can do (here:\n",
        "> arithmetic AST nodes only), sandbox anything that touches the OS, and\n",
        "> treat every observation as untrusted input.\n",
        "\n",
        "The registry also renders the tool list straight into the system prompt,\n",
        "so the model sees exactly what it can call:"
      ],
      "id": "90eb48a7-cea3-46c4-b8ca-929821c80992"
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {},
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "- calculator: Evaluate arithmetic, e.g. 17 * 23.\n",
            "- lookup: Look up a fact by exact key. Keys: speed of light."
          ]
        }
      ],
      "source": [
        "print(registry.render_descriptions())"
      ],
      "id": "27c214cc"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Parsing the Model’s Action\n",
        "\n",
        "The model communicates an action by *writing text in a format we agree\n",
        "on*. The ReAct convention is `Action: <tool>[<input>]`, and a finished\n",
        "answer is `Answer: <text>`. Parsing is a small regex — and it must\n",
        "gracefully return `None` when the model forgets the format:"
      ],
      "id": "b1e64002-a304-4d91-95cb-aa4a48a1adf7"
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {},
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "I should not guess.\n",
            "('calculator', '17 * 23')\n",
            "None\n",
            "None\n",
            "391, done."
          ]
        }
      ],
      "source": [
        "from agents import parse_action, parse_answer, parse_thought\n",
        "\n",
        "turn = \"Thought: I should not guess.\\nAction: calculator[17 * 23]\"\n",
        "print(parse_thought(turn))    # 'I should not guess.'\n",
        "print(parse_action(turn))     # ('calculator', '17 * 23')\n",
        "print(parse_answer(turn))     # None — this turn is an action, not an answer\n",
        "\n",
        "print(parse_action(\"Answer: 391\"))          # None\n",
        "print(parse_answer(\"Answer: 391, done.\"))   # '391, done.'"
      ],
      "id": "ec5c7af7"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Modern tool-calling APIs (OpenAI, Anthropic) use a **JSON** format\n",
        "instead of the ReAct text convention — same idea, different\n",
        "serialization, riding on the chat templates from m03.\n",
        "`parse_json_tool_call` pulls the first balanced JSON object out of the\n",
        "model’s prose and normalizes it:"
      ],
      "id": "d7654373-5f7b-42ef-bd56-b735ef5f48df"
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {},
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'tool': 'calculator', 'args': '2 + 2'}\n",
            "{'tool': 'lookup', 'args': 'speed of light'}\n",
            "None"
          ]
        }
      ],
      "source": [
        "from agents import parse_json_tool_call\n",
        "\n",
        "print(parse_json_tool_call('Let me compute: {\"tool\": \"calculator\", \"args\": \"2 + 2\"}'))\n",
        "print(parse_json_tool_call('{\"name\": \"lookup\", \"arguments\": \"speed of light\"}'))\n",
        "print(parse_json_tool_call(\"no tool call here\"))   # None"
      ],
      "id": "1e4cdf76"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Try both formats yourself — type an action and watch it parse:"
      ],
      "id": "c3266757-714b-457d-b4ed-09fde1e34580"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {}
        },
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {}
        }
      ],
      "source": [],
      "id": "8a9fb767-01f7-4950-a12c-f7156fbae97d"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {}
        },
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {}
        },
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {}
        },
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {}
        }
      ],
      "source": [],
      "id": "ec7c78ce-a208-4aeb-8f72-d4ec9a2a5943"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {}
        }
      ],
      "source": [],
      "id": "92cc3fee-fb72-4941-a90d-4cd80d898c4d"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "> **Try This**\n",
        ">\n",
        "> 1.  **Break the format.** Delete the `]` from the ReAct box, or the\n",
        ">     closing `}` from the JSON. Both fall to “no valid tool call” —\n",
        ">     exactly the case the real loop has to survive.\n",
        "> 2.  **Call a tool that doesn’t exist.** Change `calculator` to\n",
        ">     `wikipedia`. It parses fine (parsing doesn’t know your registry),\n",
        ">     but the observation would be an error the model must react to.\n",
        "\n",
        "## The ReAct Loop, From Scratch\n",
        "\n",
        "Now assemble the pieces. The agent’s entire “memory” is the\n",
        "**scratchpad** — the replayed transcript of past thoughts, actions, and\n",
        "observations — because the model is stateless: every turn we rebuild the\n",
        "full prompt (system + tool docs + question + scratchpad) and let it\n",
        "continue. `build_prompt` does the assembly; `run_agent` is the loop:"
      ],
      "id": "c65e42e2-1487-4b21-9fa3-d46f1a42631b"
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {},
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "answer: 17 * 23 = 391, which is not greater than 400.\n",
            "tool calls: 1 | stopped: answer"
          ]
        }
      ],
      "source": [
        "from agents import build_prompt, run_agent, scripted_policy\n",
        "\n",
        "# The \"policy\" is the LLM: prompt -> next completion. In production this is m08's\n",
        "# generate(); here we SCRIPT the completions so the run is deterministic. The tools,\n",
        "# though, are real — the calculator genuinely computes.\n",
        "script = [\n",
        "    \"Thought: I shouldn't guess. Use the calculator.\\nAction: calculator[17 * 23]\",\n",
        "    \"Thought: 391 < 400, so it is not greater.\\nAnswer: 17 * 23 = 391, which is not greater than 400.\",\n",
        "]\n",
        "result = run_agent(\"What is 17 * 23, and is it greater than 400?\",\n",
        "                   scripted_policy(script), registry, max_steps=6)\n",
        "\n",
        "print(\"answer:\", result.answer)\n",
        "print(\"tool calls:\", result.num_tool_calls, \"| stopped:\", result.stopped)"
      ],
      "id": "784502e9"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The prompt the model actually saw on its *second* turn already contains\n",
        "the real observation from the first — this is how the tool’s output\n",
        "re-enters the reasoning:"
      ],
      "id": "96134e68-b65a-4a2e-a378-1f0bee8582f2"
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {},
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "You are a reasoning agent that can call tools to answer a question.\n",
            "On each turn, write a Thought, then EITHER one Action or a final Answer:\n",
            "\n",
            "  Thought: <your reasoning>\n",
            "  Action: <tool>[<input>]      (to call a tool)\n",
            "  Answer: <final answer>       (when you are done)\n",
            "\n",
            "Available tools:\n",
            "- calculator: Evaluate arithmetic, e.g. 17 * 23.\n",
            "- lookup: Look up a fact by exact key. Keys: speed of light.\n",
            "\n",
            "Use a tool whenever it is more reliable than guessing (e.g. arithmetic, lookup).\n",
            "After each Action you will see an Observation with the tool's result.\n",
            "\n",
            "Question: What is 17 * 23...?\n",
            "Thought: I'll use the calculator.\n",
            "Action: calculator[17 * 23]\n",
            "Observation: 391\n",
            "Thought:"
          ]
        }
      ],
      "source": [
        "from agents import Step\n",
        "print(build_prompt(\"What is 17 * 23...?\", registry,\n",
        "                   [Step(\"I'll use the calculator.\", \"calculator\", \"17 * 23\", \"391\")]))"
      ],
      "id": "91de541d"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Two failure modes the loop must handle, both tested in `agents.py`:\n",
        "\n",
        "- **The model never answers.** A `max_steps` cap turns an infinite\n",
        "  think/act loop into a clean “stopped: max_steps” result.\n",
        "- **The model forgets the format.** An unparseable turn produces a\n",
        "  `_format` observation nudging it to retry, instead of crashing."
      ],
      "id": "37180362-8f0a-4c31-a67f-c8a0877a16bc"
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {},
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "stopped: max_steps | answer: None | steps: 3"
          ]
        }
      ],
      "source": [
        "# A policy that only ever acts, never answers -> the guard stops it.\n",
        "never_answers = scripted_policy([\"Thought: hmm.\\nAction: calculator[1 + 1]\"] * 100)\n",
        "runaway = run_agent(\"loop forever?\", never_answers, registry, max_steps=3)\n",
        "print(\"stopped:\", runaway.stopped, \"| answer:\", runaway.answer, \"| steps:\", len(runaway.steps))"
      ],
      "id": "2c7ca42c"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Watching a Trajectory\n",
        "\n",
        "`demonstrate_agent` runs a *real* trajectory whose final answer is\n",
        "produced by the **tool**, not the language model: the policy calls the\n",
        "calculator, reads the genuine observation `391` back out of the prompt,\n",
        "and answers with it. If the loop were broken — if the observation never\n",
        "re-entered the prompt — the answer could not be `391`."
      ],
      "id": "0008a386-ada4-451c-a66f-5d8b468d1f66"
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {},
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Question: What is 17 * 23?  (stopped: answer)\n",
            "  step 0: Thought: I should not guess arithmetic; use the calculator.\n",
            "           Action: calculator[17 * 23] -> 391\n",
            "  step 1: Thought: The tool returned 391; that is my answer.\n",
            "  Answer: 391  (1 tool call(s))"
          ]
        }
      ],
      "source": [
        "from agents import demonstrate_agent\n",
        "\n",
        "trajectory = demonstrate_agent(verbose=True)"
      ],
      "id": "5e9765a9"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Bridge the real step records to an interactive viewer and step through\n",
        "them:"
      ],
      "id": "16590b66-0137-4b94-92a9-54d4cd705179"
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Each step is JSON-serializable via .as_dict(); hand them to OJS.\n",
        "ojs_define(ag_steps = [s.as_dict() for s in trajectory.steps])\n",
        "ojs_define(ag_answer = trajectory.answer)"
      ],
      "id": "8a859797"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {}
        }
      ],
      "source": [],
      "id": "8c66a160-cfaf-4f4c-b58d-cc2e3669e4a8"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {}
        }
      ],
      "source": [],
      "id": "7beec883-117a-457b-a3ae-27a11519ce20"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "> **Key Insight**\n",
        ">\n",
        "> The answer `391` is the calculator’s output, threaded back through the\n",
        "> scratchpad — not a number the model made up. That is the whole value\n",
        "> of an agent: it grounds the model’s output in a *real computation* it\n",
        "> could never do reliably on its own. Swap `calculator` for a search\n",
        "> index, a Python sandbox, or a database and the loop is unchanged.\n",
        "\n",
        "## Common Pitfalls\n",
        "\n",
        "When building agents, watch out for:\n",
        "\n",
        "1.  **Unsafe tools.** `eval`/`exec`/raw shell/raw SQL hand your machine\n",
        "    to the model. Whitelist and sandbox; treat observations as untrusted\n",
        "    (they can carry **prompt injection** — a web page that says “ignore\n",
        "    your instructions and…”).\n",
        "2.  **No step cap.** A model that never emits `Answer:` loops until it\n",
        "    exhausts your budget. Always bound the loop (`max_steps`) and decide\n",
        "    what a timeout returns.\n",
        "3.  **Brittle parsing.** Real models drift from the format. Parse\n",
        "    defensively, return `None` on failure, and feed a corrective\n",
        "    observation rather than crashing.\n",
        "4.  **A too-thin scratchpad.** The model is stateless; if you don’t\n",
        "    replay past observations into the prompt, it re-does work or\n",
        "    contradicts itself. The scratchpad *is* the memory.\n",
        "5.  **Tool sprawl.** Twenty vague tools confuse the router more than\n",
        "    they help. Few, sharply-described tools with unambiguous inputs beat\n",
        "    a giant menu.\n",
        "6.  **Trusting a single trajectory.** Agents are stochastic; one good\n",
        "    run isn’t a success rate. Evaluate them like anything else (m17) —\n",
        "    over many tasks, with a metric.\n",
        "\n",
        "## Exercises\n",
        "\n",
        "### Exercise 1: Add a tool"
      ],
      "id": "95b65ae3-4c7d-44c9-973a-5c9810fb5cf0"
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {},
      "outputs": [],
      "source": [
        "from agents import Tool, ToolRegistry, run_agent, scripted_policy\n",
        "\n",
        "# Add a `string_length` tool that returns the number of characters in its input.\n",
        "# Register it alongside the calculator, then script a trajectory that answers\n",
        "# \"How many characters are in the word 'transformer'?\" using it. Assert the answer\n",
        "# is \"11\" and that exactly one tool call was made.\n",
        "\n",
        "# Your implementation here:"
      ],
      "id": "bfb53a58"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Exercise 2: A two-hop task"
      ],
      "id": "f7d820e5-4769-4e28-a44c-d2fe0021698e"
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {},
      "outputs": [],
      "source": [
        "from agents import ToolRegistry, Tool, calculator, make_lookup_tool, run_agent, scripted_policy\n",
        "\n",
        "# Build a registry with a calculator and a lookup tool holding\n",
        "# {\"blocks per day\": \"86400\"}. Script a trajectory for \"How many seconds are in a\n",
        "# week?\" that (1) looks up seconds-per-day, then (2) multiplies by 7 with the\n",
        "# calculator. Check num_tool_calls == 2 and the answer contains \"604800\".\n",
        "\n",
        "# Your implementation here:"
      ],
      "id": "93df8221"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Exercise 3: Guard against loops"
      ],
      "id": "4a4fd84c-5ebe-4512-b685-e417c6b978b3"
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {},
      "outputs": [],
      "source": [
        "from agents import run_agent, scripted_policy, ToolRegistry, Tool, calculator\n",
        "\n",
        "# Write a policy that ALWAYS acts and never answers. Run it with max_steps=5 and\n",
        "# confirm result.stopped == \"max_steps\" and len(result.steps) == 5. Then add a\n",
        "# fallback: if stopped == \"max_steps\", return the last observation as a best-effort\n",
        "# answer. Why is a best-effort answer better than raising?\n",
        "\n",
        "# Your implementation here:"
      ],
      "id": "bffadd59"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Summary\n",
        "\n",
        "Key takeaways:\n",
        "\n",
        "1.  **An agent is a loop, not a new model.** ReAct wraps `generate()`\n",
        "    (m08) in a Thought → Action → Observation cycle: the model writes\n",
        "    text, the program runs a tool, the result comes back as an\n",
        "    observation. Repeat until `Answer:`.\n",
        "2.  **A tool is just a function plus a description.** `str → str` with a\n",
        "    one-line doc. The model calls it by writing structured text\n",
        "    (`Action: tool[arg]` or JSON) that we parse and dispatch.\n",
        "3.  **Tools fix what models are bad at.** A safe calculator computes\n",
        "    exactly; a lookup tool reaches facts the weights don’t hold —\n",
        "    grounding the answer in real computation (Toolformer’s motivation).\n",
        "4.  **Never `eval` model output.** Whitelist and sandbox every tool;\n",
        "    treat every observation as untrusted input that may carry prompt\n",
        "    injection.\n",
        "5.  **The scratchpad is the memory.** A stateless model only knows what\n",
        "    you replay into the prompt; the observation must re-enter for the\n",
        "    loop to close.\n",
        "6.  **Bound and evaluate the loop.** A `max_steps` guard stops runaway\n",
        "    agents, and agent quality is a *success rate over many tasks* (m17),\n",
        "    never one lucky run.\n",
        "\n",
        "## What’s Next\n",
        "\n",
        "You have now built the full arc — a model that tokenizes, attends,\n",
        "trains, aligns, reasons, retrieves, is measured, and now **acts**. The\n",
        "frontier from here is **interpretability**: opening the box to see *why*\n",
        "a model produces the tokens it does — logit lens, probing, induction\n",
        "heads, and the sparse-autoencoder features that let us read (and steer)\n",
        "a model’s internal computation. Agents make models do more;\n",
        "interpretability makes us understand what they’re doing.\n",
        "\n",
        "### Going Deeper\n",
        "\n",
        "**Core Papers:**\n",
        "\n",
        "- [ReAct: Synergizing Reasoning and Acting in Language\n",
        "  Models](https://arxiv.org/abs/2210.03629) — Yao et al. (2022), the\n",
        "  Thought → Action → Observation loop built here.\n",
        "- [Toolformer: Language Models Can Teach Themselves to Use\n",
        "  Tools](https://arxiv.org/abs/2302.04761) — Schick et al. (2023),\n",
        "  self-supervised API use; the arithmetic/lookup motivation.\n",
        "- [MRKL Systems](https://arxiv.org/abs/2205.00445) — Karpas et\n",
        "  al. (2022), modular reasoning + tools, an early “neuro-symbolic” agent\n",
        "  framing.\n",
        "- [Gorilla: Large Language Model Connected with Massive\n",
        "  APIs](https://arxiv.org/abs/2305.15334) — Patil et al. (2023),\n",
        "  teaching a model to call thousands of real APIs.\n",
        "\n",
        "**Practical Resources:**\n",
        "\n",
        "- [ToolLLM: Facilitating LLMs to Master 16000+ Real-World\n",
        "  APIs](https://arxiv.org/abs/2307.16789) — Qin et al. (2023), scaling\n",
        "  tool use to a large API zoo.\n",
        "- [Function calling\n",
        "  (OpenAI)](https://platform.openai.com/docs/guides/function-calling) —\n",
        "  the JSON tool-call format `parse_json_tool_call` mirrors."
      ],
      "id": "c60b10f9-0e0a-46d8-b483-375bf4b6dbca"
    }
  ],
  "nbformat": 4,
  "nbformat_minor": 5,
  "metadata": {
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "path": "/opt/buildhome/.asdf/installs/python/3.11.14/share/jupyter/kernels/python3"
    },
    "language_info": {
      "name": "python",
      "codemirror_mode": {
        "name": "ipython",
        "version": "3"
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.14"
    }
  }
}