Use this guide to separate ingestion from runtime, see how the stages connect and identify the update, permission and deletion paths. Agentic RAG is one possible pattern; choose it only when it fits the task and evidence.
What is a RAG pipeline?
A RAG pipeline covers the end-to-end flow from source documents to generated answers. It includes both the offline work of preparing and indexing content (ingestion) and the online work of retrieving and generating (runtime).
The original RAG paper by Lewis et al. (2020) introduced a particular architecture combining a sequence-to-sequence model's parametric memory with retrieved passages from a non-parametric Wikipedia index. It motivated retrieval partly through provenance and knowledge-updating limitations in parametric-only models. That paper is one influential design, not a universal blueprint, security control or guarantee that RAG improves an answer.
Current implementation guidance, such as the Azure RAG solution design guide, treats a practical RAG system as separate ingestion and runtime stages. The guide covers document preparation, chunking, metadata, embeddings and persistence as ingestion concerns, and query handling, retrieval and orchestration as runtime concerns. It also recommends evaluating stages and the system end to end with representative content and queries. The Azure topology is not universally optimal.
RAG pipeline versus RAG architecture
"RAG pipeline" and "RAG architecture" are often used interchangeably, but they emphasise different things:
- Pipeline emphasises the flow: the sequence of stages through which information moves, from source to answer.
- Architecture emphasises the structure: the components, their interfaces, their deployment and their control points.
For a top-level design, you need both views. The flow tells you what happens in what order; the architecture shows which components exist, how they connect and where the design decisions sit. This page keeps them together so you can understand the whole system before making those choices.
The complete ingestion and runtime flow
The two-lane map below is an editorial synthesis. It is an illustrative control map, not a reference implementation.
Ingestion lane:
- Source approval: a named owner confirms which sources may enter the system.
- Acquisition: documents are retrieved from approved source systems.
- Preparation: documents are parsed, normalised, enriched with metadata and versioned.
- Chunking: documents are divided into retrieval units.
- Metadata projection: permissions, provenance and lifecycle metadata are attached to chunks.
- Embeddings: chunks are converted to vector representations.
- Indexing: embeddings and metadata are written to the index.
- Versioning: the exact version of each source is recorded.
- Deletion path: a mechanism exists to remove or supersede content.
Runtime lane:
- Identity: the requesting subject is authenticated.
- Query handling: the user query is parsed and prepared for retrieval.
- Retrieval: relevant passages are retrieved from the index.
- Permission filtering: retrieved passages are filtered by the subject's permissions.
- Reranking (where used): retrieved passages are reranked for relevance.
- Context assembly: selected passages are combined with instructions and system context.
- Generation: the model produces an answer from the assembled context.
- Attribution: the answer is attributed to its source passages.
- Evaluation: the answer is checked against evaluation criteria.
- Logs: the request, retrieval, generation and evaluation are recorded.
The map exposes dependencies rather than a required product stack. For example, retrieval quality is not enough if permission filtering is missing, and a sound runtime cannot correct a stale or wrongly approved corpus. The NCSC's secure design guidance supports threat modelling and limiting permissions and privileges, but it is high-level lifecycle guidance rather than a complete RAG design. Use the RAG security guide to threat-model the complete system, including the joins between stages where identity, content or control metadata can be dropped.
The stage table below is also an editorial synthesis. It summarises design questions and observable failures rather than prescribing a reference implementation.
| Stage | Owner | Input | Output | Control question | Observable failure |
|---|---|---|---|---|---|
| Source approval | Data owner | Source list, use case | Approved source list | Is this source eligible for the task? | Unapproved source enters the pipeline |
| Acquisition | Engineering lead | Approved sources | Raw documents | Are documents retrieved completely and securely? | Missing or corrupted documents |
| Preparation | Data owner | Raw documents | Parsed, normalised, versioned documents | Is the conversion faithful? Is the version recorded? | Information loss, stale version |
| Chunking | Engineering lead | Prepared documents | Chunks with metadata | Are boundaries appropriate for the task? | Overly large or fragmented chunks |
| Metadata projection | Engineering lead | Chunks, source permissions | Chunks with permission and provenance metadata | Are permissions complete and current? | Missing or stale permission metadata |
| Embeddings | Engineering lead | Chunks | Vector representations | Is the embedding model appropriate for the task? | Poor retrieval quality |
| Indexing | Engineering lead | Embeddings, metadata | Populated index | Is the index consistent and queryable? | Index corruption, stale entries |
| Versioning | Data owner | Prepared document, source record | Traceable version record | Can an indexed chunk be traced to an exact source version? | Old and current versions cannot be distinguished |
| Deletion | Data owner and engineering lead | Withdrawal or supersession event | Removed or suppressed derived records | Can every affected chunk and serving copy be found? | Withdrawn material remains retrievable |
| Identity | Security lead | Request | Authenticated subject | Is the subject correctly identified? | Unauthenticated request, impersonation |
| Query handling | Engineering lead | User query | Prepared query | Is the query correctly interpreted? | Misinterpreted query |
| Retrieval | Engineering lead | Prepared query, index | Retrieved passages | Are the right passages retrieved? | Irrelevant or missing passages |
| Permission filtering | Security lead | Retrieved passages, subject permissions | Filtered passages | Are restricted passages excluded? | Restricted passage in context |
| Reranking, where used | Engineering lead | Candidate passages | Reordered candidate set | Does reranking improve the task without hiding permission failures? | Relevant evidence is demoted or latency breaches its budget |
| Context assembly | Engineering lead | Filtered passages, instructions | Assembled context | Is the context within budget and relevant? | Context overflow, irrelevant context |
| Generation | Engineering lead | Assembled context | Generated answer | Does the answer follow the context? | Hallucination, irrelevant answer |
| Attribution | Product owner | Answer, source references | Answer with traceable references | Can a reviewer inspect the evidence actually supplied? | Citation points to the wrong passage or version |
| Evaluation | Evaluation owner | Answer, test criteria | Scores, flags | Does the answer meet release criteria? | Regression, threshold breach |
| Logs | Operations owner | Stage events and versions | Reviewable operational record | Can a failure be reconstructed without exposing unnecessary content? | Missing lineage or excessive sensitive logging |
Source approval, acquisition and preparation
Source approval is a governance decision, not a technical one. A named owner confirms which sources may enter the system for the specific use case. This decision considers the task, the sensitivity of the content, the permissions that must be preserved and the risk of including unapproved material.
Acquisition retrieves documents from approved source systems. The method (API, file transfer, database query) depends on the source. The key requirement is that acquisition is complete, secure and traceable.
Preparation is the stage where documents are parsed, normalised, enriched with metadata and versioned. This is a distinct stage with its own decisions and failure modes. The document preparation guide covers the full pre-ingestion checklist.
The Azure RAG chunking phase guide supports the idea that keeping loading and chunking separable makes intermediate output easier to inspect and experiment with. Format conversion can lose information. A successful parse is not proof that the source is authorised, current or complete.
Chunking, metadata, embeddings and indexing
Chunking divides prepared documents into retrieval units. The choice of chunk size, boundaries, overlap and strategy is task-specific and corpus-specific. There is no universal best setting.
The RAG chunking strategies guide covers the selection and evaluation of chunking strategies in detail, including fixed-size, recursive, structural and semantic approaches.
Metadata projection carries the attributes needed to interpret a chunk, such as its source, version and relevant permission references. Metadata does not itself enforce a permission. The runtime still has to resolve the current subject and apply the appropriate authorisation decision before a passage reaches context. Missing, stale or wrongly mapped metadata can break that chain.
Embeddings convert chunks to vector representations, while indexing writes those representations and their metadata to a searchable store. As an editorial design consideration, treat the embedding model and index implementation as workload-specific choices to compare with representative content and queries. This guide does not prescribe a universal best embedding model, vector database, search engine or hybrid design.
Query handling, retrieval and reranking
Query handling prepares the user query for retrieval. Retrieval then selects candidate passages from the index, and an optional reranking stage can reorder those candidates.
Query rewriting, expansion, decomposition, vector similarity, keyword search, hybrid retrieval, retrieval count and reranking are editorial design options, not defaults endorsed by the cited lifecycle guidance. Compare only the options relevant to the task against representative queries and declared evaluation criteria; do not assume that adding a stage improves the result.
Context assembly, generation and attribution
Context assembly combines the retrieved passages with instructions, system context and any interaction history into the final context supplied to the model. The context must be within the model's window and within the allocated budget.
Generation produces the answer from the assembled context. The model's behaviour is constrained by the context, but it is not guaranteed to follow it. An answer can be well grounded in supplied context while still being incorrect.
Attribution links the answer to its source passages. This makes the supplied evidence easier to inspect and supports evaluation. It is not a certificate of trust: a reference can be stale, misquoted, wrongly retrieved or outside the reader's authority, and attribution does not prove the answer is correct.
Identity, permissions and provenance across the flow
Permissions must survive the entire flow from source to answer. The Azure document-level access control documentation illustrates one implementation in which permission metadata is projected to indexed chunks and applied during retrieval. That implementation is Azure-specific and partly in preview; permissions can be absent, stale or misapplied.
The RAG access control guide covers the identity-to-retrieval authorisation path in detail, including revocation and derived artefacts.
Provenance records can show where content came from, which version was processed and which owner or policy applies. They give reviewers evidence for currency, permission checks and deletion; they do not make a source trustworthy by themselves. Enforcement should be tied to the requested resource and the current subject rather than inferred from network location or index membership.
Evaluate individual stages and the system end to end
Evaluation should cover both individual stages and the system end to end. The Azure RAG LLM evaluation guide treats groundedness, completeness, context utilisation, relevance and correctness as distinct, workload-dependent dimensions. It recommends repeating evaluation as the corpus, queries and system change. No single metric or threshold proves correctness, access control or safety.
Stage-level evaluation includes:
- Parsing fidelity. Does the conversion preserve the source's structure and meaning?
- Metadata and permission propagation. Are permissions and provenance correctly attached to chunks?
- Retrieval relevance. Does retrieval return the right passages for representative queries?
- Answer faithfulness. Does the generated answer follow the supplied context?
- Factual correctness. Is the answer factually accurate?
System-level evaluation includes:
- End-to-end task performance. Does the system reduce the time or effort for the defined task?
- Access-control behaviour. Does the system correctly exclude restricted content?
- Adversarial behaviour. Does the system handle instruction-like content in retrieved passages appropriately?
The RAG evaluation guide covers test sets, metrics, release gates and regression monitoring in detail.
Refresh, revocation, deletion and re-indexing
The pipeline still needs attention after launch. The following lifecycle responsibilities are editorial design considerations:
- Refresh. Detect source changes, reprocess affected documents and update the index.
- Revocation. When a user's permissions change, the system must reflect the change in retrieval.
- Deletion. When a document is withdrawn, the system must remove or suppress the affected content and its derived chunks.
- Re-indexing. When an embedding model or index implementation changes, define which corpus artefacts need to be regenerated and how the changed system will be verified.
Treat each of these as a testable lifecycle event. Record which source version caused the event, which derived records were affected, when the serving index changed and which checks passed afterwards. Caches, replicas, evaluation sets and generated artefacts need an explicit disposition; deleting one vector row does not demonstrate that every downstream copy has disappeared.
Fixed, workflow-based and agentic design choices
RAG pipelines can be designed with different levels of orchestration:
- Fixed. The retrieval and generation steps are predefined. The system retrieves, assembles context and generates. No dynamic decision-making.
- Workflow-based. The system follows a predefined workflow that may include conditional branches, multiple retrieval steps or tool calls. The path is fixed but the content is dynamic.
- Agentic. The system dynamically decides which retrieval steps to take, which tools to call and when to stop, based on feedback from the environment.
Agentic RAG is one possible orchestration pattern. It is not inherently better than a fixed or workflow-based design. As an editorial design consideration, compare any added orchestration with the simplest candidate that can perform the task, using declared evaluation criteria rather than an assumption that greater autonomy is better.
For a practical local implementation of a RAG pipeline, see apply the architecture to a local RAG build, which covers deployment choices and a vendor-neutral build plan.
Check the exact document version before ingestion
Source approval, classification, minimisation and permissions establish whether a document belongs in the pipeline. A separate document checkpoint can then decide how to handle the exact version before transformation and indexing.
.mdSiren is a document security workspace for AI. When it launches, Standard Scan will check the exact version for supported prompt-injection and document-borne risks, route it to Approved, Needs review or Quarantine, and keep the Approved exact version in your private Library. If the file changes, check the new version. This decision supports document intake; preparation, indexing, retrieval, runtime authorisation and security remain responsibilities of the wider RAG system.
Frequently asked questions
What is a RAG pipeline? It is the connected ingestion and runtime processes through which selected external knowledge is prepared, indexed, retrieved and supplied to a generative model for a defined task.
What is the difference between a RAG pipeline and RAG architecture? "Pipeline" emphasises the flow of information through stages. "Architecture" emphasises the components, their interfaces and their control points. A useful guide covers both.
What are the ingestion and runtime stages? Ingestion covers source approval, acquisition, preparation, chunking, metadata, embeddings and indexing. Runtime covers identity, query handling, retrieval, permission filtering, context assembly, generation, attribution and evaluation.
Does RAG prevent hallucinations? No. RAG supplies context to the model, but the model is not guaranteed to follow it. An answer can be well grounded in supplied context while still being incorrect.
Where should permissions be enforced? Before retrieved content reaches model context, the runtime needs an authorisation decision for the current subject and requested resource. Some architectures filter during retrieval; others combine pre-filtering with checks at additional boundaries. Permission references or attributes need to remain traceable to the source, but copying metadata into an index is not enforcement.
How do you update or delete indexed knowledge? Define a refresh mechanism that detects source changes and reprocesses affected documents. Define a deletion path that removes or suppresses content and its derived chunks. Verify that deletion is complete, including in caches and logs.
