Retrieval-Augmented Generation: Keep Answers Grounded in Documents

Retrieval-augmented generation (RAG) combines document retrieval with language-model generation. Instead of asking a model to rely only on its learned knowledge, the application retrieves relevant passages and supplies them as context.

A typical pipeline extracts text, splits it into chunks, generates embeddings, and stores them in a searchable index. At query time, the system retrieves candidate chunks, optionally reranks them, and sends the best evidence with the user's question to the model. Chunk size, overlap, metadata filters, and retrieval strategy all affect relevance.

Measure retrieval quality separately from answer quality. If the right passage is not retrieved, prompt wording alone may not fix the result. Use representative questions, check whether supporting passages appear in the top results, and evaluate whether answers cite the right source sections. Documents may be stale, duplicated, or contradictory, so versioning and source metadata matter.

Treat retrieved text as untrusted data. A document can contain instructions that should not override system rules or authorization boundaries. Filter access before retrieval, and never assume that a model's answer is grounded simply because a context window was provided.