Agent development frameworks
Frameworks provide structure, enforce patterns, and encode best practices — but they also add abstraction layers that reduce low-level control. Choosing one is a strategic decision affecting extensibility, maintainability, and performance. The chapter profiles six:
The “elder statesman.” Modular orchestration built on core abstractions — Chains, Agents, Tools, Memory, Retrievers, Embeddings — with 100+ integrations. Implements the ReAct (Reasoning + Acting) pattern. Limit: no native multi-agent support; abstraction overhead.
LangChain evolved into stateful, cyclical workflows modeled as a directed graph: Nodes, Edges, State, conditional routing. Enables loops, human-in-the-loop approval gates, and multi-agent coordination. The book's primary framework for later chapters.
“Where LangChain orchestrates, LlamaIndex remembers.” Knowledge-centric: an index → query engine → response synthesizer pipeline. The “memory cortex” for document-heavy systems. Limit: needs external orchestration.
Pioneered recursive self-prompting — autonomous goal decomposition with minimal human input. Limit: low reliability, fragile control; best for research prototypes.
Role-based multi-agent collaboration: each agent has a role, goal, and backstory and delegates via built-in messaging. Since v0.4 it leans on LangChain's Tool abstraction.
Microsoft's conversational programming paradigm — treats LLMs as conversation participants (user proxy, code executor, planner) exchanging messages. Strong at fine-grained turn-taking and stop conditions.
Key concept · Compose over build
Modern agent engineering favors a compose-over-build philosophy: assemble specialized components rather than construct a monolith. Teams typically start with LangChain for orchestration, then swap in production pieces (e.g. a Pinecone/Chroma vector store for ConversationBufferMemory) as performance or security needs emerge.
Choosing & orchestrating LLMs
The model is the reasoning engine sitting inside the cognition core from Chapter 1. Selection shapes what an agent can perceive, understand, and generate — and turns on five axes:
- Capability
- Lightweight/fast (Mistral 7B) vs. heavyweight reasoners (GPT-4, Claude 3, Gemini).
- Specialization
- Some models excel at coding, others at creative or multi-turn reasoning.
- Context window
- Ranges from 8K to 1M+ tokens.
- Performance & cost
- Hosting, pricing, and rate limits vary widely; watch token normalization.
- Licensing
- Open-weight (local deploy & fine-tune) vs. closed-source (API-managed).
Key concept · Hybrid model architecture
Rather than one model, route each query to the model best suited to it — a “cognitive division of labor.” The chapter's example classifies queries and routes: factual → Mistral 7B (fast/cheap), creative → Claude, analytical → GPT-4. An orchestration layer acts as a traffic director, optimizing cost and capability together.
Supporting infrastructure: memory & tools
Beyond frameworks and models lies the infrastructure that turns prototypes into scalable systems: memory, tool integration, evaluation, and monitoring.
The memory revolution: vector databases
Keyword search matches words; vector search matches meaning. Text is embedded into high-dimensional vectors (e.g. 768/1024/1536 dims), and the database finds the nearest ones by cosine similarity or dot product. Approximate nearest neighbor (ANN) algorithms — HNSW and IVF — make billion-vector search possible in milliseconds. Think of the vector store as the agent's hippocampus.
- Pinecone
- The specialist — purpose-built, cloud-native scaling, real-time upserts.
- Weaviate
- The hybrid — vector search + metadata filtering via GraphQL.
- Chroma
- The developer's friend — lightweight, open-source, great for prototyping.
- Milvus
- The enterprise foundation — billions of records, distributed scale.
- Qdrant
- Open-source, production-ready, strong payload filtering.
Key concept · The RAG pipeline (5 steps)
- Chunk — break documents into pieces (~500–1,000 tokens)
- Embed — transform chunks into embedding vectors
- Store — save vectors with source metadata & timestamps
- Retrieve — find relevant vectors on demand
- Inject — feed the retrieved context into the prompt before the LLM responds
Retrieval mastery adds: the Goldilocks zone of chunk size (and hierarchical chunking), rerankers (Cohere Rerank, cross-encoders) to sharpen precision, metadata filtering (recency, source authority), and observability via LangSmith. RAG makes an agent's knowledge dynamic — new info is added without retraining.
Tool integration frameworks
In the Perception–Reasoning–Action (PRA) loop, tools are where thought becomes consequence. Two foundational patterns:
LangChain Tool abstraction
Wraps ordinary Python functions into agent-compatible instruments with a name, function, and description — handling validation, error handling, and formatting.
OpenAI function calling
Language-agnostic JSON schemas (name, description, parameters) the model interprets to generate correctly formatted calls. LangChain can wrap these via its StructuredTool class.
Managed platforms: AWS, Azure, Google Cloud
Managed platforms abstract infrastructure and bundle multi-agent collaboration, RAG, memory, and guardrails. The trend is hybrid — managed services for foundations plus open-source frameworks for custom logic.
AWS
Bedrock Agents (multi-agent via a central supervisor), Bedrock Knowledge Bases for RAG, SageMaker for custom models, plus Strands Agents and MCP support. Best for teams already invested in AWS.
Azure
AI Foundry Agent Service — an “Agent Factory” with Semantic Kernel, AutoGen, agent-to-agent messaging, and strong identity/governance (Entra). Best when GPT-4 or a Microsoft-centric stack is required.
Google Cloud
Vertex AI Agent Builder/Engine, the open-source ADK, Agentspace, and an open A2A protocol. Best for cost-efficient scaling and open-standards interoperability that reduces lock-in.
Takeaway
There is no single “best” cloud — the choice follows existing organizational infrastructure and priorities. Your toolkit decisions set the boundaries of where your systems fall on the Agentic AI Progression Framework from Chapter 1. Keep adaptability a core design principle as the landscape shifts.
Chapter 2 quiz
Fourteen questions across frameworks, models, memory infrastructure, and cloud platforms. Answer first, then expand Show answer.
Part A · Multiple choice
Which framework is described as “where LangChain orchestrates, this framework remembers,” built around an index → query engine → response synthesizer pipeline?
- A CrewAI
- B AutoGen
- C LlamaIndex
- D AutoGPT
Show answer
C — LlamaIndex. It's the knowledge/memory-centric framework, ideal as the “memory cortex” for document-heavy systems.
What key limitation does LangChain have that makes LangGraph necessary for complex systems?
- A It cannot call external APIs
- B It has no native multi-agent support and is linear rather than stateful/cyclical
- C It does not support Python
- D It cannot connect to vector databases
Show answer
B. LangChain chains are typically linear; LangGraph adds stateful, cyclical, branching workflows (nodes/edges/state) and multi-agent coordination.
In the hybrid model architecture example, which model handles fast, cost-sensitive factual queries?
- A GPT-4
- B Claude
- C Mistral 7B
- D Gemini
Show answer
C — Mistral 7B. Factual → Mistral (speed/cost), creative → Claude, analytical → GPT-4.
HNSW and IVF are examples of what?
- A Embedding models
- B Approximate nearest neighbor (ANN) search algorithms
- C Agent frameworks
- D Cloud deployment architectures
Show answer
B. ANN algorithms — Hierarchical Navigable Small World and Inverted File Index — enable millisecond search across billions of vectors.
Which vector database is characterized as “the developer's friend” — lightweight, open-source, ideal for quick prototyping?
- A Pinecone
- B Milvus
- C Weaviate
- D Chroma
Show answer
D — Chroma. Pinecone is the cloud specialist, Milvus the enterprise-scale option, Weaviate the hybrid GraphQL powerhouse.
Which framework pioneered “recursive self-prompting” but is noted for low reliability and fragile control?
- A AutoGPT
- B LangGraph
- C CrewAI
- D LlamaIndex
Show answer
A — AutoGPT. It decomposes high-level goals autonomously but its control mechanisms remain fragile — best for research prototypes.
Part B · True or false
Vector search finds results by matching exact keywords in the query.
Show answer
False. Vector search matches meaning (semantic similarity), not exact words — two differently-worded texts expressing the same idea sit near each other in vector space.
RAG lets an agent's knowledge stay dynamic — new information can be added to the vector store without retraining the model.
Show answer
True. That's the revolutionary part: knowledge becomes dynamic rather than frozen at training time.
CrewAI defines each agent with a role, goal, and backstory to enable role-based collaboration.
Show answer
True. These attributes create distinct personas within a “crew,” coordinated by a central orchestrator.
The book recommends building every agent component from scratch rather than composing existing frameworks.
Show answer
False. It advocates a compose-over-build philosophy — integrate specialized components, then selectively replace them with custom implementations as constraints demand.
Part C · Short answer
List the five steps of a basic RAG pipeline in order.
Show answer
Chunk → Embed → Store (with metadata) → Retrieve → Inject into prompt. The agent chunks documents, embeds them as vectors, stores them with source metadata, retrieves relevant vectors on demand, and injects that context before the LLM responds.
Name the two foundational tool-integration patterns the chapter highlights and one difference between them.
Show answer
LangChain's Tool abstraction (wraps Python functions — Python-centric) and OpenAI function calling (JSON schemas — language-agnostic, works across platforms). They aren't mutually exclusive: LangChain can wrap OpenAI schemas via StructuredTool.
Match each cloud platform to its one-line identity: AWS, Azure, Google Cloud.
Show answer
AWS — the flexible ecosystem (Bedrock Agents, SageMaker), best if already on AWS. Azure — the enterprise powerhouse (AI Foundry “Agent Factory,” Semantic Kernel), best for GPT-4 / Microsoft-centric orgs. Google Cloud — the AI innovation hub (Vertex AI, ADK, A2A protocol), best for cost-efficient scaling and open standards.
Name three of the six core LangChain abstractions.
Show answer
Any three of: Chains, Agents, Tools, Memory, Retrievers, Embeddings. Chains are sequential pipelines; Agents are autonomous decision-makers; Tools interface to external systems; Memory maintains context; Retrievers access relevant info; Embeddings convert text to vectors.