Grounding LLMs in live, verifiable data
The retrieval agent is the assistant fetching the latest journals for a brilliant-but-dated scholar. By linking to live sources, it directly attacks two LLM weaknesses: knowledge cutoff and hallucination, anchoring outputs in verifiable evidence (typically a Level 2 tool-using agent, edging toward Level 3 planning for literature reviews). Guiding principles: address LLM limits, implement RAG, support structured and unstructured retrieval, and ground every answer in cited provenance.
Key concept · The retrieval loop (as the cognitive loop)
- 1 · Query understanding (perception/reasoning) — parse intent, clarify ambiguity, reformulate into a precise search query
- 2 · Retrieval (planning/action) — pick sources & method (lexical/semantic/hybrid); the Retriever Module hits APIs or vector DBs
- 3 · Preprocessing — chunk large docs, embed, filter irrelevant results
- 4 · Synthesis (learning) — inject retrieved content into the prompt; answer only from provided sources
Provenance runs in parallel throughout — collecting citations, metadata, and confidence so the answer is auditable, not a final afterthought.
Retrieval strategies & chunking
- Single-stage
- One query to one authoritative source. Narrow, well-defined, low-latency. Trade-off: limited recall.
- Multi-stage
- Broad search refined by targeted filters/sub-queries. Open-ended, aggregating. Trade-off: higher latency.
- Hybrid
- Lexical (keyword/BM25) + semantic (vector). Best recall on mixed corpora. Trade-off: more tuning.
Chunking is the single most consequential RAG configuration. Three strategies: fixed-size (simplest), recursive (splits on natural boundaries — the recommended default), and semantic (splits on embedding-detected topic shifts — highest fidelity, costliest). The chapter's default: chunk_size=1000, chunk_overlap=200 — overlap ensures a sentence spanning a boundary is captured whole.
Diagnosing retrieval failures
When an answer is vague or unsupported, the fault is usually one of three: retrieved chunks had low semantic similarity; chunks were relevant but lacked the needed info; or provenance was missing/mismatched. Inspect source_documents. Uniformly low similarity signals a vocabulary mismatch — fix by adding keyword (BM25) search via hybrid retrieval.
Turning messy documents into structured data
These orchestrate a sequence of tools to convert visually complex documents into decision-ready data. The pipeline:
Classification & routing
Identify document type by MIME/schema and route it — a PDF triggers OCR, a CSV goes straight to extraction, an HTML email is stripped first.
Preprocessing & OCR
Convert images to text; the OCR engine emits confidence scores so low-confidence regions get re-OCR or fallbacks.
Structural segmentation & layout parsing
Reconstruct tables and reading order from the visual layout.
Information extraction
Schema-driven key-value/entity extraction (e.g. Invoice Number, Total), with validators and provenance for each field.
Best practices: cascading extraction for low-confidence fields, hybrid ML approaches, human-in-the-loop (HITL) review for uncertain outputs, and full provenance — always link back to page number and bounding box. They take the baton where retrieval leaves off, enforcing schemas inside documents.
From retrieval to synthesis
The most sophisticated type — thinking partners that navigate an ocean of evolving literature. They run the cognitive loop over a multi-phase objective, in three phases:
Broad literature scanning
Semantic search across PubMed, arXiv, IEEE Xplore, Scopus — capturing conceptually relevant studies, not just keyword matches.
Thematic clustering & summarization
Group papers by shared themes (methodology, findings, domain) to reveal patterns; use citation graph traversal to discover related work.
Synthesis & insight generation
Produce comparative tables, evidence maps, and summaries highlighting consensus, divergence, and gaps.
Key concept · Hard limits & challenges
Fundamental limits: no true understanding (statistical correlation, not comprehension), hallucination risk (can fabricate citations even with RAG), can't generate genuinely new knowledge (no experiments), and context-window constraints. Operational challenges: data access/licensing, publication bias, expert verification needs, and scalability in fast-moving fields.
A complete knowledge pipeline
The three types form a progression of increasing sophistication that together make one pipeline: find relevant information (Retrieval), extract & structure it (Document Intelligence), and synthesize insight (Scientific Research). Mapped onto the Agentic AI Progression Framework, they move from simple tool-using systems toward planning and learning agents — complementary, not competing.
Chapter 6 quiz
Fifteen questions on retrieval, document intelligence, and research synthesis. Answer first, then expand Show answer.
Part A · Multiple choice
Which two LLM weaknesses do Knowledge Retrieval agents primarily address?
- A Latency and cost
- B Knowledge cutoff and hallucination
- C Tokenization and context length
- D Bias and toxicity
Show answer
B. By linking to live sources, they overcome the static training cutoff and reduce hallucination by grounding answers in verifiable evidence.
Which chunking strategy is the recommended default for mixed-content corpora?
- A Fixed-size chunking
- B Recursive chunking
- C Semantic chunking
- D No chunking
Show answer
B — recursive chunking. It splits on natural boundaries (paragraphs, sentences, words), producing semantically coherent chunks. Semantic chunking has the highest fidelity but is costliest.
Uniformly low similarity scores across all retrieved chunks most likely indicate:
- A The LLM context window is too small
- B A vocabulary mismatch between query and corpus
- C The provenance metadata is corrupted
- D Too much chunk overlap
Show answer
B — vocabulary mismatch. The query uses terms absent from the embedded corpus. Fix: add keyword (BM25) search via hybrid retrieval so lexical matches compensate.
In a Document Intelligence pipeline, what do OCR engines emit to flag uncertain text regions?
- A Bounding boxes only
- B Confidence scores
- C Embeddings
- D Citations
Show answer
B — confidence scores. They let the system weigh low-confidence regions cautiously and trigger fallbacks like re-OCR or alternative models.
Which is NOT one of the three phases of a Scientific Research agent?
- A Broad literature scanning
- B Thematic clustering & summarization
- C Synthesis & insight generation
- D Running physical experiments
Show answer
D. These agents explicitly cannot design experiments or generate genuinely new knowledge — they scan, cluster, and synthesize existing literature.
Part B · True or false
In the retrieval architecture, provenance tracking is only a final step performed after the answer is generated.
Show answer
False. Provenance is a parallel, continuous process collecting citations, metadata, and confidence throughout the pipeline — that's what makes the answer auditable.
Hybrid retrieval combines keyword (lexical) search with vector (semantic) search.
Show answer
True. It's best when a corpus mixes structured terminology (codes, clauses) with free-form text, delivering the best recall — at the cost of more tuning.
RAG grounding completely eliminates the risk of a Scientific Research agent fabricating citations.
Show answer
False. Even with RAG, LLMs can fabricate citations, misattribute findings, or produce plausible-but-wrong syntheses — a critical risk requiring expert verification.
A 200-character overlap on 1,000-character chunks helps capture sentences that span a chunk boundary.
Show answer
True. Overlap mitigates boundary loss; insufficient overlap creates artifacts where key facts fall between chunks.
Document Intelligence agents and Knowledge Retrieval agents are competitors that do the same job.
Show answer
False. They're complementary — Document Intelligence agents “take the baton where retrieval leaves off,” operating inside documents to enforce schemas and produce structured data.
Part C · Short answer
List the four steps of the Knowledge Retrieval loop.
Show answer
Query understanding → Retrieval → Preprocessing → Synthesis, with provenance tracking running in parallel throughout.
Name the three chunking strategies and the trade-off of each.
Show answer
Fixed-size — simplest, but ignores meaning/structure. Recursive — splits on natural boundaries; the recommended default. Semantic — splits on embedding-detected topic shifts; highest fidelity but most compute-intensive.
What is citation graph traversal, and why is it useful?
Show answer
Following the links between papers via their references and citations (papers = nodes, citations = edges). It lets a research agent identify influential works, discover clusters of related research, and track how ideas evolve — surfacing foundational and validating studies from a single seed paper.
Give three of the fundamental limitations of Scientific Research agents.
Show answer
Any three of: no true understanding (statistical, not comprehension), hallucination risk (fabricated citations even with RAG), can't generate new knowledge (no experiments), context-window constraints (breadth vs. depth trade-off).
How do the three knowledge-agent types combine into one pipeline?
Show answer
Knowledge Retrieval finds relevant information → Document Intelligence extracts and structures it → Scientific Research synthesizes insight — a complete find → structure → synthesize pipeline of increasing sophistication.