ConceptRetrieval & RAG
RAG Pipeline
At a glance
The production path that turns documents into indexed passages and turns a question into a grounded answer with citations.
- Who this is for
- Engineers and technical readers learning the terms used in AI systems.
- Topics
- Retrieval & RAG
- Concept
In short
A RAG pipeline is the production path that turns documents into indexed passages and turns each question into a grounded answer with citations. Bigcircle builds it against a labelled set of fifty to a hundred real questions, with hybrid search, fitted chunking, reranking, and a refusal path when nothing relevant is found. The RAG systems engagement measures recall before anyone tunes the prompt.
Key facts
Definition
A RAG pipeline is the end-to-end system that prepares a corpus for search and answers each question from retrieved passages, covering ingestion, chunking, embedding, indexing, retrieval, reranking, grounded generation and evaluation.
How Bigcircle does it
- On RAG systems engagements, Bigcircle labels fifty to a hundred real questions with the passages that answer them, then tunes hybrid search, document-fitted chunking and reranking until recall at k is where it needs to be.
- On Prospex, Bigcircle runs semantic and keyword search together over a custom chunker, and a claim with no retrieved passage behind it does not appear in the output.
- On DocVerse, Bigcircle parses filing layout before any split and carries entity, period, unit and framework as hard metadata, because a correct number under the wrong metadata passes every later check.
Comparison
The RAG glossary page owns the retrieve-augment-generate pattern and agentic RAG; this page owns the production stage list from ingest through evaluation; the production guide owns failure modes in debug order.
Key number
If recall at 10 is 70%, answer accuracy cannot beat 70% whatever model writes the reply, so Bigcircle measures retrieval before changing the prompt.
A RAG pipeline is the full production path behind retrieval-augmented generation: offline work that turns documents into indexed passages, and a query-time path that finds the right passages, puts them in the prompt, and generates an answer that cites them. The model call is one stage. Everything else is the pipeline: every stage that decides whether the right evidence arrives at all, and whether the answer stays with it once it does.
RAG keeps the pattern itself: retrieve, then augment the prompt, then generate. This page is the shipping shape of that pattern, stage by stage, for a team that has to own each failure. The production RAG guide owns failure modes in debug order. The RAG vs fine-tuning guide owns when retrieval loses to weights or to an agent loop. Neighbour pages own chunking, embeddings, vector search, hybrid search, reranking, retrieval metrics and groundedness.
What is a RAG pipeline?#
For a team shipping retrieval, a RAG pipeline is the end-to-end system that prepares a corpus for search and answers each question from retrieved text rather than from the model's training memory alone. Patrick Lewis and colleagues named retrieval-augmented generation in 2020 (arXiv 2005.11401, checked 20 November 2026). Their recipe paired a pretrained seq2seq generator with a dense vector index of text, so the model conditions on retrieved passages when it writes.
That paper is the origin of the term, not a blueprint for every production stage. The pipeline teams ship today adds the write-time path (parse, chunk, embed, index) and the query-time controls (hybrid retrieval, rerank, refuse, measure) that a demo usually skips.
In plain terms this is operational rather than academic. At write time you turn messy sources into passages a search can return, and at query time you find those passages, order them, and ask the model to answer only from what arrived. For a shipping team, that split is the useful answer: an offline path that builds the index, and a live path that uses it.
RAG pipeline architecture, stage by stage#
Production architecture here is two paths that share one corpus contract. The write path prepares passages, and the query path uses them. A rag pipeline architecture diagram does not need exotic components; it needs every stage named so a team can own the failure when that stage breaks.
| Stage | What it does for a shipping team |
|---|---|
| Ingestion and parsing | Pull sources in, keep layout where meaning lives in tables or footnotes, and attach permissions and version metadata before any split |
| Chunking | Cut documents into passages search can return. A bad boundary is a retrieval bug written at index time |
| Embeddings | Map each passage to a vector so meaning becomes distance. Changing the embedder means re-embedding the corpus |
| Indexing | Store vectors and, for hybrid setups, a keyword index, with filters for access and freshness |
| Retrieval (vector search and hybrid search) | At query time, find candidates. Hybrid runs keyword and vector together so exact codes and paraphrases both survive |
| Reranking | Rescore a short candidate list so the best evidence reaches the prompt. It cannot recover a passage the first pass never fetched |
| Generation with grounding | Put the kept passages in the prompt, require citations, and refuse when nothing relevant is found (groundedness) |
| Evaluation | Label real questions, measure recall at k and faithfulness separately, and debug in pipeline order |
Measure each stage rather than hoping a stronger model hides a miss. If recall at 10 is 70%, answer accuracy cannot beat 70% whatever model writes the reply. That ceiling is already stated on the RAG page and in the production guide.
How Bigcircle does it#
We build a RAG pipeline against a labelled set first. On RAG systems that means fifty to a hundred real questions with the passages that answer them, then hybrid search, chunking fitted to the documents, and reranking until recall at k is where it needs to be. No prompt fixes a passage that was never fetched.
Week 3-4 adds grounding instructions, citation format, and a refusal path when the index holds nothing relevant to the question. Week 4-6 wires incremental reindexing, freshness checks, and monitoring for retrieval drift.
On Prospex, our prospect research engine, semantic and keyword search run together rather than as a fallback, over a custom chunker that keeps a role attached to its company. A claim with no retrieved passage behind it does not appear. That is intentional. A short profile is easy to spot, and an invented one is not.
On DocVerse, filings are parsed for layout before any split. Entity, period, unit and framework travel as hard metadata, because a correct number under the wrong metadata passes every later check. Document intelligence states the same shape for regulated corpora: hybrid search, reranking, and chunking tuned for the documents, with a citation on every extracted field.
We measure retrieval quality and answer faithfulness separately, the split already on the RAG and groundedness pages. The production RAG guide is the debug order when something fails: corpus first, then retrieval, context assembly, and generation.
When the full pipeline is the wrong build#
Skip a dedicated index while the whole knowledge base still fits in the context window and prompt caching makes repeated reads affordable. The RAG page already covers when long context plus caching beats building an index.
Stay on a single search while one query finds the answer. Move to agentic search only when a simpler pipeline has visibly fallen short, which is the boundary the RAG page and the RAG vs fine-tuning guide already draw.
Build the labelled set and the refusal path before you buy a more unusual index. If recall is low, fix parsing, chunking or hybrid retrieval first. If recall is fine and answers still drift, fix groundedness and citation checks instead. The stages fail separately, so treat them as separate owners rather than one model swap.