Think, Plan, Act
A Tool-Using agent extends an LLM by invoking a predefined set of external functions. It runs a Think → Plan → Act cycle and decouples decision logic from execution mechanics through four components:
- Reasoning core
- The “brain” — parses intent (Think) and formulates a step-by-step plan (Plan).
- Tool registry
- A catalog of tools with metadata & input/output schemas that act as explicit contracts.
- Execution engine
- The “foreman” (Act) — manages state between steps, retries, and error propagation.
- Tool chest
- Modular single-responsibility functions, each wrapped with its own safety logic (timeouts, exceptions).
The selection funnel
An agent with many tools is useless if it can't pick the right one. A multi-stage “selection funnel” narrows candidates using several strategies, often combined:
Key concept · Five selection strategies
- Intent classification / template matching — rules map keywords to tools. Fast & reliable in constrained domains; scales poorly.
- Embedding-based similarity search — match query and tool metadata by semantic meaning, not keywords.
- Constraint-based filtering — a safety layer that prunes incompatible/unsafe tools (e.g. no permission to
delete_vm_instance). - Plan-driven tool assignment — generate a plan, then select tools per step so one output feeds the next.
- Dynamic reranking with feedback — on failure/low confidence, add a constraint to avoid that tool and re-enter the funnel for a fallback.
Failure is inevitable — recover gracefully
Four failure modes: input validation errors (bad schema), runtime failures (timeouts, I/O), semantic mismatches (runs fine but wrong result), and tool unavailability. The layered defense:
Key concept · Recovery strategies
- Safe invocation wrappers — every call in a try/except with targeted retries (exponential backoff) & logging
- Fallback tool chains — a secondary tool accomplishes the same goal when the primary fails
- Confidence-based switching — discard low-confidence results, retry with a more suitable tool/model
- Failure memory — a circuit breaker: mark a repeatedly-failing tool temporarily unavailable
- Escalation paths — hand full context to a human when automated recovery is exhausted
A manager coordinating specialists
Complex tasks exceed any single agent. A manager agent coordinates specialists (PlannerAgent, DataFetcherAgent, ValidatorAgent…) via a cooperation protocol built on four pillars: clearly defined roles & capabilities, a common communication infrastructure, shared context/memory, and execution orchestration. Concretely, the protocol defines a message format, role declaration, task-delegation scheme, and status signaling — the practical expression of what MCP and A2A standardize at the wire level.
Key concept · Memory-augmented multi-agent systems
Shared memory turns independent specialists into a team. A layered design pairs working memory (the active scratchpad — recent query & tool results) with long-term memory in a vector DB, split into episodic (logbook of events & conversations) and semantic (durable factual knowledge). New tasks load relevant long-term memory into working memory so every agent shares the same facts.
Turning disagreement into reliability
When specialists disagree, an arbitration workflow resolves it:
Conflict detection
Compute semantic similarity between outputs; below a threshold (e.g. 0.7) a conflict is flagged.
Automated arbitration
An impartial arbiter agent consults a trusted knowledge base / source data, or synthesizes a merged output.
Confidence-based consensus
The arbiter emits a calibrated confidence score; above a policy threshold (e.g. 95%) its decision is accepted.
Human escalation
Low confidence → escalate to a human. Human review is a planned first-class branch, not a failure.
Long-running, stateful processes
Workflows extend orchestration into persistent business processes modeled as state machines or graphs, requiring persistence, branching, error recovery, and human-in-the-loop (HITL) checkpoints. The e-commerce order case study wires deterministic steps (inventory, payment) with a non-deterministic LLM node (fraud risk) that falls back to rules if the model is unavailable — and a HITL gate that pauses the workflow for explicit approve/reject on medium/high-risk orders. Every decision, score, and rationale is logged for a full audit trail.
Chapter 7 quiz
Fifteen questions across tool use, selection, error handling, orchestration, and workflows. Answer first, then expand Show answer.
Part A · Multiple choice
Which cycle does a Tool-Using agent follow?
- A Sense–Model–Plan–Act
- B Think, Plan, Act
- C Perceive–Reason–Learn
- D Map–Measure–Manage
Show answer
B — Think, Plan, Act. Interpret the goal, formulate tool calls, then execute them.
In a Tool-Using agent, what holds each tool's metadata and input/output schemas as explicit contracts?
- A The reasoning core
- B The execution engine
- C The tool registry
- D The tool chest
Show answer
C — the tool registry. Its schemas let the agent know exactly what parameters to supply and what to expect back.
Which selection strategy acts primarily as a safety/validation layer, pruning unsafe candidates (e.g. lacking permissions)?
- A Embedding-based similarity search
- B Constraint-based filtering
- C Intent classification
- D Dynamic reranking
Show answer
B — constraint-based filtering. After candidates are shortlisted, it removes incompatible or unsafe tools before execution.
A tool runs successfully but produces output that doesn't match the user's intent (e.g. sorted alphabetically instead of by spend). This is a:
- A Runtime failure
- B Input validation error
- C Semantic mismatch
- D Tool unavailability
Show answer
C — semantic mismatch. The subtle, dangerous mode: execution succeeds but the result is wrong for the goal.
“Failure memory” that temporarily marks a repeatedly-failing tool unavailable acts like what pattern?
- A A bulkhead
- B A circuit breaker
- C A load balancer
- D A cache
Show answer
B — a circuit breaker. It prevents the agent from wasting cycles re-selecting a clearly failing tool.
Part B · True or false
The chain-of-agents cooperation protocol is the practical expression of what MCP and A2A standardize.
Show answer
True. The message-format and role-declaration layers of the protocol map directly onto MCP/A2A specifications.
In the conflict-resolution workflow, human review is treated as a failure of the system.
Show answer
False. It's a planned, first-class branch in the control flow — invoked when the arbiter's confidence is too low.
Long-term memory in a multi-agent system splits into episodic (event log) and semantic (durable facts).
Show answer
True. Both typically live in a vector database; working memory is the separate short-term scratchpad.
Agentic workflow systems are modeled as state machines or graphs with checkpoints for human oversight.
Show answer
True. They add persistence, branching, error recovery, and HITL checkpoints to sustain long-running processes.
Intent classification / template matching scales well to large, open-ended tool sets and novel requests.
Show answer
False. It's fast and reliable for known intents in constrained domains but scales poorly and can't handle ambiguous or novel requests — that's where embedding search helps.
Part C · Short answer
Name the four components of a Tool-Using agent and each one's job.
Show answer
Reasoning core (think/plan), tool registry (schemas/contracts), execution engine (state, retries, errors — the act stage), tool chest (guarded modular functions).
Name the four pillars of a chain-of-agents cooperation protocol.
Show answer
Clearly defined roles & capabilities; a common communication infrastructure; shared context/memory; execution orchestration (a manager delegating and sequencing).
List the four steps of the conflict-resolution (arbitration) workflow.
Show answer
Conflict detection (similarity below threshold) → automated arbitration (arbiter agent) → confidence-based consensus (accept if above policy threshold) → human escalation (if confidence low).
Name the four common tool-failure modes.
Show answer
Input validation errors, runtime failures, semantic mismatches, and tool unavailability.
In the e-commerce workflow, why is fraud-risk assessment a good candidate for an embedded LLM node with a rule-based fallback and a HITL gate?
Show answer
Risk assessment is non-deterministic and hard to encode with simple rules, so an LLM adds nuanced judgment; a rule-based fallback keeps it working if the model is unavailable; and a HITL gate pauses medium/high-risk orders for human approve/reject — auto-rejecting could lose a customer, auto-approving could enable fraud.