RAG Fundamentals
How RAG evolved from naive vector search to agentic and graph-based retrieval. When to choose RAG vs. long context, and the three retrieval gaps that cause production failures.
Chunking Strategies
Chunking is the process of splitting a document into discrete segments for retrieval. Production pipelines have moved beyond blind fixed-size splits to structure-aware and semantic segments, with newer techniques like late chunking and contextual prepending now in the mainstream toolkit.
Embedding Models
Embedding models convert text into high-dimensional vectors. The frontier has moved past static single-vector representations to multi-resolution, late-interaction, and multimodal embeddings.
Vector Databases
Vector databases are purpose-built systems for storing, indexing, and searching high-dimensional embeddings. The market has split into Managed Serverless and Specialized High-Performance engines. We no longer ask "Does it support vector search?" (Postgres, Redis, and Mongo all do). We ask "Does it scale to 100M+ vectors with sub-100ms P99 and full metadata filtering?"
Hybrid Search
Hybrid search combines dense (semantic) and sparse (keyword) retrieval to get the benefits of both. It is the baseline for production RAG: Elasticsearch's rrf retriever, OpenSearch hybrid search, Weaviate, Qdrant, and Azure AI Search all ship native hybrid pipelines out of the box.
Reranking Strategies
Reranking is the second stage of retrieval that re-scores a small set of candidates (Top 50-100) using a high-precision model. It is the bridge between "efficient search" and "perfect grounding": first-stage retrieval optimizes for recall, reranking optimizes for precision. Three rerankers dominate production today (BGE-Reranker-v2-m3, Cohere Rerank 3, Voyage rerank-2), with the choice driven by cost model, latency tail, language coverage, and whether you need self-hostable weights.
GraphRAG
GraphRAG is the combination of Knowledge Graphs (KG) and Retrieval-Augmented Generation. While vector RAG is good at "finding a specific chunk," GraphRAG is designed for Global Reasoning across an entire dataset.
Agentic RAG
Agentic RAG moves from a "Linear Pipeline" to a "Reasoning Loop." Instead of retrieving once, an agent decides when and what to retrieve to resolve a query. The dominant production patterns are Self-RAG (model emits reflection tokens), Corrective RAG (retrieval evaluator with corrective routing), Adaptive RAG (classifier picks pipeline depth), ReAct over documents, and multi-hop query decomposition. LangGraph is the most common control-flow runtime for stateful loops; LlamaIndex Workflows is common for single-pipeline retrieval-heavy variants.
Advanced Retrieval Patterns
Beyond the basics, production RAG systems use specialized patterns to handle complex query-document gaps. These patterns are the "secret sauce" of high-precision search and are increasingly bundled into managed RAG offerings.
Contextual Retrieval
Contextual Retrieval is an ingestion-time technique that solves the #1 cause of RAG failure: chunks that lose meaning when separated from their source document. Pioneered by Anthropic in late 2024, it is now a production standard for high-precision retrieval. Anthropic's own measurements show a 49% reduction in retrieval failures with hybrid search alone, and 67% reduction when combined with reranking.
Late Interaction & ColBERT
Late Interaction is a retrieval paradigm that sits between fast-but-imprecise bi-encoders and accurate-but-slow cross-encoders. ColBERT (Contextualized Late Interaction over BERT) is the defining model in this space, delivering cross-encoder-level accuracy at bi-encoder-level speed. The late-interaction family has matured into a production-ready alternative for high-precision search, with multimodal extensions (ColPali, ColQwen2.5, ColNomic, and unified retrievers like Wholembed v3) now in the same toolkit.
Multi-Modal RAG
Multi-modal RAG extends retrieval-augmented generation beyond plain text to handle images, tables, charts, audio, and mixed-layout documents. Production systems now routinely ingest PDFs with diagrams, slide decks, scanned invoices, and research papers where the visual layout is the meaning. Three architectures dominate: caption-and-index, unified vision-text embeddings (Cohere Embed v4, Voyage-Multimodal-3.5, Gemini Embedding 001), and page-as-image with late interaction (ColPali, ColQwen2.5, ColNomic).
RAG Evaluation Patterns
Evaluation is the hardest unsolved problem in RAG. You can build a retrieval pipeline in a day; knowing whether it actually works takes weeks. The industry has converged on a layered evaluation strategy: the RAG Triad for correctness, component-level metrics for debugging, and automated regression testing for production safety. Langfuse, LangWatch, Braintrust, and Arize Phoenix all ship native RAG eval recipes; pick by deployment model (self-hosted vs SaaS) and whether you need eval-gated CI/CD blocking.
Production RAG at Scale
Production RAG is no longer a weekend project. It is a distributed system with retrieval pipelines, caching layers, routing logic, self-correction loops, multi-tenant isolation, and cost controls, all operating under strict latency SLAs. When RAG fails in production, the failure is in retrieval roughly 73% of the time, not generation, so the enterprise deployments that succeed treat the knowledge source (not the model) as the primary investment.
Data Engineering for AI
Models are only as good as the data fed to them, and most of the work in any serious AI system is upstream of the model. This chapter covers the data layer that feeds both RAG and fine-tuning the same pipeline that prepares documents for a vector store also prepares examples for fine-tuning. Treating data engineering as "step two of RAG" understates it; it is a standalone discipline both consumers depend on.