Ch 1 — AI Agent Fundamentals¶

Volume 8 · Chapter 1 | Estimated reading time: 60 minutes

Learning Objectives¶

By the end of this chapter you will be able to:

1. Define an AI agent and distinguish it from a static LLM prompt-response system.
2. Describe the Sense–Think–Act loop and map each phase to concrete software components.
3. Implement tool-use (function calling) with JSON schema definitions and route tool calls correctly within an agent loop.
4. Explain the ReAct framework, trace through a multi-step ReAct execution, and identify the conditions under which it fails.
5. Classify memory types available to agents (in-context, external, episodic, semantic) and design a memory strategy for a given task horizon.

1. What is an AI Agent?¶

A static LLM call takes a prompt and returns a completion — one pass, no autonomy, no side effects beyond text generation. An AI agent is a system that uses an LLM as its reasoning core but extends it with:

• The ability to perceive an environment (read files, query APIs, receive user messages).
• The ability to act on that environment (call tools, write files, send messages).
• A loop that continues until a goal is achieved or a stopping condition is met.

The canonical formulation is the Sense → Think → Act loop, which maps directly onto how agent frameworks are structured:

flowchart TD
    ENV["Environment\n(tools, APIs, files, databases)"]
    SENSE["Sense\nCollect observations,\nbuild context window"]
    THINK["Think\nLLM reasons over context,\ndecides next action"]
    ACT["Act\nExecute tool call or\nproduce final answer"]
    STOP{{"Goal met or\nmax steps reached?"}}

    ENV -->|"Observation"| SENSE
    SENSE --> THINK
    THINK --> ACT
    ACT -->|"Tool result / new observation"| SENSE
    ACT --> STOP
    STOP -->|"No"| SENSE
    STOP -->|"Yes"| FINAL["Return answer\nto user"]

    style ENV fill:#1e293b,color:#94a3b8
    style SENSE fill:#7c3aed,color:#fff
    style THINK fill:#7c3aed,color:#fff
    style ACT fill:#7c3aed,color:#fff
    style STOP fill:#d97706,color:#fff
    style FINAL fill:#059669,color:#fff

Each iteration of the loop narrows the agent toward its goal. The LLM is never given the complete solution in advance — it constructs a plan incrementally as evidence accumulates.

2. Agent Components¶

Every production-quality agent is composed of four architectural building blocks.

2.1 LLM Backbone¶

The LLM is the reasoning engine. It must support:

• Tool-use / function calling: structured output declaring which tool to invoke and with what arguments.
• Long-context: agents accumulate observations across multiple steps; 32k–200k token windows matter.
• Instruction following: reliable adherence to system prompts that define agent persona and constraints.

Frontier models from Anthropic (Claude), OpenAI (GPT-4o), and Google (Gemini 1.5 Pro) all support native tool use.

2.2 Tools and Actions¶

Tools are the agent's effectors — the mechanisms through which it interacts with the world. Categories include:

2.3 Memory¶

Without memory, each iteration of the agent loop starts from scratch. Memory systems provide continuity:

2.4 Planning¶

Planning determines how the agent breaks a goal into sub-tasks. We cover planning strategies in depth in Section 5.

3. Tool Use: Function Calling in LLMs¶

Modern LLMs expose tool use through a structured protocol: the developer declares a set of tool schemas in JSON; the model returns a structured response indicating which tool to call and with what arguments.

3.1 JSON Schema for Tool Definitions¶

A tool schema follows JSON Schema (draft 7) conventions and must include at minimum a name, description, and input_schema:

TOOLS = [
    {
        "name": "search_web",
        "description": (
            "Search the web for current information. "
            "Use when the user asks about recent events or facts "
            "not in your training data."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "The search query string",
                },
                "num_results": {
                    "type": "integer",
                    "description": "Number of results to return (1–10)",
                    "default": 5,
                },
            },
            "required": ["query"],
        },
    },
    {
        "name": "run_python",
        "description": "Execute a Python snippet and return stdout/stderr.",
        "input_schema": {
            "type": "object",
            "properties": {
                "code": {
                    "type": "string",
                    "description": "Valid Python 3 code to execute",
                },
            },
            "required": ["code"],
        },
    },
]

3.2 Tool Routing¶

After the model returns a tool_use content block, the agent must route the call to the correct handler and return the result as a tool_result block. The routing table is typically a dictionary:

from typing import Any

TOOL_REGISTRY: dict[str, Any] = {
    "search_web": search_web_handler,
    "run_python": run_python_handler,
}

def dispatch_tool(tool_name: str, tool_input: dict[str, Any]) -> str:
    handler = TOOL_REGISTRY.get(tool_name)
    if handler is None:
        return f"Error: unknown tool '{tool_name}'"
    try:
        return handler(**tool_input)
    except Exception as exc:
        return f"Error executing tool: {exc}"

4. The ReAct Framework¶

ReAct (Reason + Act) is the dominant pattern for structuring agent loops. Introduced by Yao et al. (2022), it interleaves explicit reasoning traces (Thought) with tool invocations (Action) and their results (Observation):

Thought: The user wants the current price of AAPL. I should search for it.
Action: search_web(query="AAPL stock price today")
Observation: AAPL is trading at $213.45 as of 14:32 EST.
Thought: I have the answer. I can now respond to the user.
Answer: Apple (AAPL) is currently trading at $213.45 (as of 14:32 EST today).

4.1 The ReAct Loop¶

sequenceDiagram
    participant U as User
    participant A as Agent Loop
    participant L as LLM
    participant T as Tool

    U->>A: "What is the current price of AAPL?"
    loop Until answer or max_steps
        A->>L: messages + tool_schemas
        L-->>A: tool_use block OR text answer
        alt Tool call
            A->>T: dispatch(tool_name, args)
            T-->>A: tool result
            A->>A: append tool_result to messages
        else Final answer
            A-->>U: answer text
        end
    end

4.2 When ReAct Fails¶

ReAct is not a silver bullet. Common failure modes include:

5. Planning Strategies¶

Different task complexities call for different planning approaches.

5.1 Chain-of-Thought (CoT)¶

The model reasons step-by-step in natural language before acting. Simple to implement; works well for single-step tasks. Fails on tasks requiring parallel or hierarchical decomposition.

5.2 Plan-and-Execute¶

A separate Planner LLM call generates a structured plan (list of steps), then an Executor carries out each step in order. Benefits: the plan can be inspected and revised; failures in one step do not corrupt earlier results.

from dataclasses import dataclass

@dataclass
class PlanStep:
    step_number: int
    description: str
    tool: str | None
    completed: bool = False
    result: str | None = None

5.3 Self-Refinement¶

After producing an initial plan or answer, the agent is prompted to critique its own output and produce an improved version. Useful when output quality (not just task completion) is the primary metric. Often used with a separate Critic prompt.

6. Memory Types in Practice¶

6.1 In-Context Memory (Short-Term)¶

The context window is the agent's working memory. All observations, tool results, and prior reasoning reside here during a session. Advantages: zero latency, no additional infrastructure. Disadvantages: bounded size, ephemeral across sessions.

Practical tip: Compress or summarise intermediate observations before they consume the context budget. Use a rolling summary rather than an ever-growing transcript.

6.2 External Storage (Long-Term)¶

Persist information beyond the context window using a database. Two common patterns:

• Exact-match key-value store (Redis, DynamoDB): stores structured facts indexed by entity ID.
• Vector store (Pinecone, Qdrant, ChromaDB): stores dense embeddings of observations; retrieved by semantic similarity.

The agent uses a retrieval tool to pull relevant memories into context when needed.

6.3 Episodic vs Semantic Memory¶

Episodic memory captures what happened during past sessions — task logs, decision traces, outcomes. The agent can learn from prior successes and failures by retrieving similar episodes.

Semantic memory captures general knowledge — facts, entity relationships, domain rules. For LLM-based agents, semantic memory is largely encoded in the model weights. Supplementing it with a knowledge graph or retrieval layer allows updating world knowledge without retraining.

7. Evaluation¶

Evaluating agents is harder than evaluating static LLM outputs because correctness must be judged across a multi-step trajectory, not a single response.

7.1 Key Metrics¶

7.2 Evaluation Datasets¶

• ToolBench: 16,000+ tool-use tasks across 49 categories.
• GAIA: real-world questions requiring multi-step web and tool use.
• HotpotQA: multi-hop reasoning for retrieval-augmented agents.

8. Agent Failure Modes¶

Beyond the ReAct-specific failures covered in Section 4, agents exhibit a broader failure taxonomy:

9. Implementing a ReAct Agent with the Anthropic API¶

The following is a complete, production-ready ReAct agent implementation using Claude's native tool_use API. It includes loop termination, error handling, and type annotations throughout.

"""
react_agent.py — A minimal ReAct agent using the Anthropic tool_use API.

Requirements:
    pip install anthropic>=0.28.0
"""

from __future__ import annotations

import json
import os
from typing import Any

import anthropic

# ---------------------------------------------------------------------------
# Tool implementations
# ---------------------------------------------------------------------------

def search_web(query: str, num_results: int = 5) -> str:
    """Stub — replace with a real search API (Brave, Serper, Tavily)."""
    # In production, call e.g. the Brave Search API here.
    return json.dumps(
        [
            {
                "title": f"Result {i+1} for '{query}'",
                "snippet": f"Placeholder snippet {i+1}.",
                "url": f"https://example.com/{i+1}",
            }
            for i in range(min(num_results, 3))
        ]
    )


def run_python(code: str) -> str:
    """Execute Python code in a subprocess and return stdout/stderr."""
    import subprocess
    import sys

    result = subprocess.run(
        [sys.executable, "-c", code],
        capture_output=True,
        text=True,
        timeout=10,
    )
    output = result.stdout
    if result.returncode != 0:
        output += f"\nSTDERR:\n{result.stderr}"
    return output or "(no output)"


TOOL_REGISTRY: dict[str, Any] = {
    "search_web": search_web,
    "run_python": run_python,
}

# ---------------------------------------------------------------------------
# Tool schemas
# ---------------------------------------------------------------------------

TOOL_SCHEMAS: list[dict[str, Any]] = [
    {
        "name": "search_web",
        "description": (
            "Search the web for current information. "
            "Use when the user asks about recent events, "
            "current data, or facts not in your training data."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "The search query string.",
                },
                "num_results": {
                    "type": "integer",
                    "description": "Number of results to return (1–10).",
                    "default": 5,
                },
            },
            "required": ["query"],
        },
    },
    {
        "name": "run_python",
        "description": (
            "Execute a Python 3 code snippet and return stdout/stderr. "
            "Use for calculations, data transformations, and analysis."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "code": {
                    "type": "string",
                    "description": "Valid Python 3 code to execute.",
                },
            },
            "required": ["code"],
        },
    },
]

# ---------------------------------------------------------------------------
# Agent loop
# ---------------------------------------------------------------------------

MAX_STEPS = 10
SYSTEM_PROMPT = """\
You are a helpful AI assistant with access to tools.
Use the tools when you need external information or computation.
Always think step by step before acting.
When you have enough information to answer the user's question, \
provide a clear, concise final answer without invoking any tools.
"""


def dispatch_tool(tool_name: str, tool_input: dict[str, Any]) -> str:
    """Route a tool_use request to the correct handler."""
    handler = TOOL_REGISTRY.get(tool_name)
    if handler is None:
        return f"Error: unknown tool '{tool_name}'. Available tools: {list(TOOL_REGISTRY)}"
    try:
        return str(handler(**tool_input))
    except Exception as exc:  # noqa: BLE001
        return f"Error executing '{tool_name}': {exc}"


def run_agent(user_message: str, verbose: bool = True) -> str:
    """
    Run a ReAct agent loop until a final answer is produced
    or MAX_STEPS is reached.

    Args:
        user_message: The user's natural-language request.
        verbose: If True, print each step to stdout.

    Returns:
        The agent's final text answer.
    """
    client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

    messages: list[dict[str, Any]] = [
        {"role": "user", "content": user_message}
    ]

    for step in range(1, MAX_STEPS + 1):
        if verbose:
            print(f"\n--- Step {step} ---")

        response = client.messages.create(
            model="claude-opus-4-5",
            max_tokens=4096,
            system=SYSTEM_PROMPT,
            tools=TOOL_SCHEMAS,
            messages=messages,
        )

        # Append assistant turn to messages
        messages.append({"role": "assistant", "content": response.content})

        if verbose:
            for block in response.content:
                if hasattr(block, "text"):
                    print(f"[Thought] {block.text[:300]}")
                elif block.type == "tool_use":
                    print(f"[Action] {block.name}({json.dumps(block.input)[:200]})")

        # Check stop reason
        if response.stop_reason == "end_turn":
            # Extract the final text answer
            final_text = next(
                (block.text for block in response.content if hasattr(block, "text")),
                "(no text in final response)",
            )
            if verbose:
                print(f"\n[Final Answer] {final_text}")
            return final_text

        if response.stop_reason == "tool_use":
            # Execute each tool call and collect results
            tool_results: list[dict[str, Any]] = []
            for block in response.content:
                if block.type == "tool_use":
                    result = dispatch_tool(block.name, block.input)
                    if verbose:
                        print(f"[Observation] {result[:300]}")
                    tool_results.append(
                        {
                            "type": "tool_result",
                            "tool_use_id": block.id,
                            "content": result,
                        }
                    )
            # Append tool results as a user turn
            messages.append({"role": "user", "content": tool_results})
            continue

        # Unexpected stop reason
        break

    return "Agent reached maximum steps without producing a final answer."


# ---------------------------------------------------------------------------
# Entry point
# ---------------------------------------------------------------------------

if __name__ == "__main__":
    answer = run_agent(
        "What is 2 raised to the power of 32? Verify by computing it in Python."
    )
    print(f"\nResult: {answer}")

9.1 Key Design Decisions¶

10. Exercises¶

1. Extend the tool set: Add a read_file(path: str) -> str tool to the react_agent.py implementation. What safety constraints should you enforce on the path argument?

2. Context budget: Modify run_agent to print the total number of input tokens consumed after each step. At what point does a conversation with many tool calls risk exceeding a 100k-token context window?

3. Loop detection: Implement a loop-detection heuristic that compares the last three (tool_name, tool_input) pairs. If the same tool is called with identical arguments twice in a row, abort and return a graceful error.

4. Episodic memory: Design a schema for storing agent episodes in a SQLite database. Each episode should record: task description, list of (action, observation) pairs, final answer, success flag, and total steps used. Write the SQLAlchemy model.

5. Evaluation harness: Write a Python function evaluate_agent(tasks: list[dict]) -> dict that runs run_agent on a list of tasks, compares the output to a ground-truth answer, and returns task-completion rate, mean steps-to-completion, and mean input tokens per task.

Summary¶

• An AI agent extends an LLM with tools, memory, and a loop — the Sense → Think → Act cycle — to accomplish multi-step tasks autonomously.
• Tool use is implemented via JSON schema declarations that the LLM uses to produce structured tool_use blocks; the agent loop dispatches these to registered handlers.
• The ReAct framework interleaves reasoning traces with action invocations, providing transparency and debuggability, but it fails under tool hallucination, context overflow, and infinite loops.
• Planning strategies (CoT, Plan-and-Execute, Self-Refinement) should be chosen based on task complexity and the importance of inspectability.
• Agents have access to four memory types: in-context (fast, bounded), external storage (persistent, unlimited), episodic (experiential), and semantic (factual).
• Evaluation requires trajectory-level metrics — task completion rate, steps-to-completion, tool call accuracy — not just output quality.

Next: Ch 2 — Agent Frameworks