Connect AI answers to current, permitted reference material. Retrieval-augmented generation is a system design problem involving content quality, access control, retrieval, and answer evaluation.

Content reviewed: October 25, 2026 · Practical guide
What retrieval adds—and what it does not
Retrieval-augmented generation, or RAG, supplies selected external material to a model at request time. It can connect an application to documents that are private, specialized, or newer than the model’s training data. It does not automatically make the resulting answer correct.
The system can fail at several stages: the right document may be absent, parsing may lose a table, retrieval may return the wrong passage, or the model may misinterpret good evidence. Diagnose those stages separately.
Two pipelines, one security boundary
| Ingestion | Request handling |
|---|---|
| Identify approved sources and owners. | Authenticate the user and determine permitted scope. |
| Parse and divide content into meaningful units. | Retrieve candidates using appropriate lexical or semantic methods. |
| Keep provenance, version, and access metadata. | Filter and rank evidence while enforcing permissions. |
| Build and refresh the searchable index. | Generate an answer tied to sources, then validate and present it. |
Chunking should preserve meaning, headings, and source identity. Hybrid retrieval can combine lexical matching with semantic similarity; reranking can help order candidates. The best choices depend on the documents and questions, so compare them using a representative test set.
Worked example: an internal policy assistant
Illustrative design: an employee asks how an exception process works. The assistant searches only policies the employee may access, includes relevant passages and their versions in context, and returns a concise answer with links to the source. A conflicting or incomplete source set should produce a qualified answer or escalation.
Keep document authorization in the application or retrieval layer. A prompt that says “do not reveal restricted policies” is not a substitute for preventing those policies from entering the request context.
Evaluate evidence before judging prose
- Coverage: is the required source available and current?
- Retrieval: did the system find the passages needed to answer?
- Grounding: does each important claim follow from supplied evidence?
- Citation quality: does the linked passage actually support the claim?
- Authorization: were inaccessible documents excluded?
- Abstention: does the assistant avoid answering when evidence is insufficient?
Refresh, delete, and investigate
Define how additions, corrections, revoked access, and deletions reach the index. Track document identifiers and versions so an answer can be investigated later. Treat retrieved content as untrusted input, especially when documents can contain instructions aimed at an AI reader.
Improve a RAG system by changing one stage at a time. A retrieval problem may need better parsing or ranking; changing the generation prompt alone cannot repair a missing source.
Sources & further reading
Primary references reviewed for this guide. Worked examples and delivery checklists are illustrative design guidance.