Anthropic publishes Contextual Retrieval guidance
Anthropic published an engineering discussion of Contextual Retrieval, a retrieval-augmented generation approach that adds context to chunks before embedding and indexing them. The post compared retrieval strategies on its cited benchmark and discussed combining contextual embeddings with reranking, while noting accuracy remains dependent on corpus and evaluation design.
By Hypler editorial team ยท Source: Anthropic.
Original source date: .
Hypler publication: . Hypler updated: .
Topics: RAG, retrieval, evaluation
Engineering relevance
Retrieval improvements must be measured against real documents and questions. Retain representative corpora, relevance judgments, source display checks, and regression cases instead of relying on generalized benchmark results.
Source record: contextualizing chunks before retrieval
On September 19, 2024, Anthropic published Contextual Retrieval guidance for retrieval-augmented generation systems. The article describes adding concise, chunk-specific context from the source document before creating embeddings and a BM25 index. Anthropic reported results from its cited evaluations across multiple knowledge domains: contextual embeddings plus contextual BM25 reduced its top-20 retrieval failure rate by 49 percent, and adding reranking reduced that rate by 67 percent. Those measurements describe the article's methodology, corpus choices, retrieval settings, and evaluated configurations.
Engineering guidance: preserve document identity in every chunk
A retrieval pipeline should keep each chunk attached to durable document identity. Store the document revision, title, publisher, access class, chunk ordinal, section heading, extraction timestamp, and contextual prefix beside the embedding and lexical index entry. Context then becomes queryable system data instead of generated prose with uncertain origin. Chunking policy also deserves versioning: boundary rules, overlap, token limits, parsers, and normalizers can materially change retrieval behavior. A document update should create a new ingest revision, rebuild affected chunks, and retire older index entries according to a defined retention policy.
Hybrid retrieval needs measurable ranking decisions
Contextual embeddings help semantic matching, while BM25 helps exact terms such as error codes, product identifiers, legal citations, and names. Combine their candidate sets through a documented fusion strategy, deduplicate by stable chunk identity, and record the score components used to select the final context window. Reranking introduces another model and another versioned decision point, so log its model identifier, prompt or scoring configuration, candidate count, and cutoff. These records allow an evaluator to distinguish a missed document, a weak initial rank, a reranker rejection, and a generation error after relevant evidence was retrieved.
Hypothetical implementation: a change-record knowledge base
Consider a hypothetical engineering knowledge base containing versioned change records. Ingestion splits each record by section, then generates a compact context prefix containing the project, release identifier, affected subsystem, and change date. The pipeline writes the contextualized text to embedding and BM25 indexes with the same chunk identifier. At query time, the service retrieves semantic and lexical candidates, fuses them, reranks the top set, and returns excerpts with their release identifiers. A reviewer can open the original record from every excerpt before using the answer in a decision. The example demonstrates traceable retrieval rather than a claim about a deployed knowledge service.
Evaluation must use representative questions and judgments
Build an evaluation set from actual question shapes: identifier lookups, time-bounded questions, cross-document comparisons, ambiguous terms, and questions whose answer should be unavailable. For each query, record relevant documents or chunks, access class, expected citations, and acceptance criteria for the generated response. Measure retrieval recall and precision separately from answer completeness, citation correctness, latency, and cost. Include regressions for renamed services, duplicate documents, conflicting revisions, scanned PDFs, and source material containing instructions that should remain data. A retrieval improvement matters when it improves these declared outcomes across the corpus that users will query.
Operate retrieval as a versioned data product
Retrieval quality changes with document additions, embedding replacements, parser upgrades, index settings, and access-policy updates. Publish an ingest manifest with corpus revision, document count, chunk count, rejected documents, index build identifier, and evaluation result. Monitor zero-result queries, low-confidence result sets, citation-open failures, latency percentiles, and access denials. Route each signal to an owner who can repair source metadata, adjust parsing, update evaluations, or change ranking policy. This operating model keeps retrieval work grounded in observable corpus behavior instead of treating an embedding index as a static feature.
Context generation itself needs quality controls. The contextualizer receives a whole document and one chunk, so its output can introduce a mistaken date, product name, or document relationship that changes retrieval ranking. Keep the generated prefix short, label it as derived metadata, retain the prompt revision, and sample outputs against the source document for factual accuracy. Rebuild prefixes when the document parser or contextualizer prompt changes, then compare retrieval metrics and spot-check citations before promoting the new index. A corpus with accurate source text can still produce weak retrieval when its derived context is stale or misleading.
Use access filtering before ranking, not as a post-processing cleanup. Candidate generation should receive the caller's authorization domain and exclude unavailable documents before semantic search, lexical search, fusion, and reranking. Log the number of candidates removed by policy without disclosing their titles. This prevents relevance scoring from becoming a side channel for content that the caller cannot open.
Original source
Related links
Historical source briefing with Hypler engineering analysis. Source details reflect the original publication date; implementation examples are hypothetical.