Glossary

ConceptRetrieval & RAG

RAG Pipeline

At a glance

The production path that turns documents into indexed passages and turns a question into a grounded answer with citations.

Who this is for
Engineers and technical readers learning the terms used in AI systems.
Topics
  • Retrieval & RAG
  • Concept

In short

A RAG pipeline is the production path that turns documents into indexed passages and turns each question into a grounded answer with citations. Bigcircle builds it against a labelled set of fifty to a hundred real questions, with hybrid search, fitted chunking, reranking, and a refusal path when nothing relevant is found. The RAG systems engagement measures recall before anyone tunes the prompt.

Key facts

Definition

A RAG pipeline is the end-to-end system that prepares a corpus for search and answers each question from retrieved passages, covering ingestion, chunking, embedding, indexing, retrieval, reranking, grounded generation and evaluation.

How Bigcircle does it

  1. On RAG systems engagements, Bigcircle labels fifty to a hundred real questions with the passages that answer them, then tunes hybrid search, document-fitted chunking and reranking until recall at k is where it needs to be.
  2. On Prospex, Bigcircle runs semantic and keyword search together over a custom chunker, and a claim with no retrieved passage behind it does not appear in the output.
  3. On DocVerse, Bigcircle parses filing layout before any split and carries entity, period, unit and framework as hard metadata, because a correct number under the wrong metadata passes every later check.

Comparison

The RAG glossary page owns the retrieve-augment-generate pattern and agentic RAG; this page owns the production stage list from ingest through evaluation; the production guide owns failure modes in debug order.

Key number

If recall at 10 is 70%, answer accuracy cannot beat 70% whatever model writes the reply, so Bigcircle measures retrieval before changing the prompt.

A RAG pipeline is the full production path behind retrieval-augmented generation: offline work that turns documents into indexed passages, and a query-time path that finds the right passages, puts them in the prompt, and generates an answer that cites them. The model call is one stage. Everything else is the pipeline: every stage that decides whether the right evidence arrives at all, and whether the answer stays with it once it does.

RAG keeps the pattern itself: retrieve, then augment the prompt, then generate. This page is the shipping shape of that pattern, stage by stage, for a team that has to own each failure. The production RAG guide owns failure modes in debug order. The RAG vs fine-tuning guide owns when retrieval loses to weights or to an agent loop. Neighbour pages own chunking, embeddings, vector search, hybrid search, reranking, retrieval metrics and groundedness.

What is a RAG pipeline?#

For a team shipping retrieval, a RAG pipeline is the end-to-end system that prepares a corpus for search and answers each question from retrieved text rather than from the model's training memory alone. Patrick Lewis and colleagues named retrieval-augmented generation in 2020 (arXiv 2005.11401, checked 20 November 2026). Their recipe paired a pretrained seq2seq generator with a dense vector index of text, so the model conditions on retrieved passages when it writes.

That paper is the origin of the term, not a blueprint for every production stage. The pipeline teams ship today adds the write-time path (parse, chunk, embed, index) and the query-time controls (hybrid retrieval, rerank, refuse, measure) that a demo usually skips.

In plain terms this is operational rather than academic. At write time you turn messy sources into passages a search can return, and at query time you find those passages, order them, and ask the model to answer only from what arrived. For a shipping team, that split is the useful answer: an offline path that builds the index, and a live path that uses it.

Write path, then query pathThe index is built once and refreshed. Every question walks the right side.ingestparsechunkpassagesembedvectorsindexstoreretrievehybridreranktop-kgeneratecite / refuseevaluaterecall + faith

RAG pipeline architecture, stage by stage#

Production architecture here is two paths that share one corpus contract. The write path prepares passages, and the query path uses them. A rag pipeline architecture diagram does not need exotic components; it needs every stage named so a team can own the failure when that stage breaks.

StageWhat it does for a shipping team
Ingestion and parsingPull sources in, keep layout where meaning lives in tables or footnotes, and attach permissions and version metadata before any split
ChunkingCut documents into passages search can return. A bad boundary is a retrieval bug written at index time
EmbeddingsMap each passage to a vector so meaning becomes distance. Changing the embedder means re-embedding the corpus
IndexingStore vectors and, for hybrid setups, a keyword index, with filters for access and freshness
Retrieval (vector search and hybrid search)At query time, find candidates. Hybrid runs keyword and vector together so exact codes and paraphrases both survive
RerankingRescore a short candidate list so the best evidence reaches the prompt. It cannot recover a passage the first pass never fetched
Generation with groundingPut the kept passages in the prompt, require citations, and refuse when nothing relevant is found (groundedness)
EvaluationLabel real questions, measure recall at k and faithfulness separately, and debug in pipeline order

Measure each stage rather than hoping a stronger model hides a miss. If recall at 10 is 70%, answer accuracy cannot beat 70% whatever model writes the reply. That ceiling is already stated on the RAG page and in the production guide.

How Bigcircle does it#

We build a RAG pipeline against a labelled set first. On RAG systems that means fifty to a hundred real questions with the passages that answer them, then hybrid search, chunking fitted to the documents, and reranking until recall at k is where it needs to be. No prompt fixes a passage that was never fetched.

Week 3-4 adds grounding instructions, citation format, and a refusal path when the index holds nothing relevant to the question. Week 4-6 wires incremental reindexing, freshness checks, and monitoring for retrieval drift.

On Prospex, our prospect research engine, semantic and keyword search run together rather than as a fallback, over a custom chunker that keeps a role attached to its company. A claim with no retrieved passage behind it does not appear. That is intentional. A short profile is easy to spot, and an invented one is not.

On DocVerse, filings are parsed for layout before any split. Entity, period, unit and framework travel as hard metadata, because a correct number under the wrong metadata passes every later check. Document intelligence states the same shape for regulated corpora: hybrid search, reranking, and chunking tuned for the documents, with a citation on every extracted field.

We measure retrieval quality and answer faithfulness separately, the split already on the RAG and groundedness pages. The production RAG guide is the debug order when something fails: corpus first, then retrieval, context assembly, and generation.

When the full pipeline is the wrong build#

Skip a dedicated index while the whole knowledge base still fits in the context window and prompt caching makes repeated reads affordable. The RAG page already covers when long context plus caching beats building an index.

Stay on a single search while one query finds the answer. Move to agentic search only when a simpler pipeline has visibly fallen short, which is the boundary the RAG page and the RAG vs fine-tuning guide already draw.

Build the labelled set and the refusal path before you buy a more unusual index. If recall is low, fix parsing, chunking or hybrid retrieval first. If recall is fine and answers still drift, fix groundedness and citation checks instead. The stages fail separately, so treat them as separate owners rather than one model swap.

Common questions about rag pipeline

  • What is a RAG pipeline?

    The production path behind retrieval-augmented generation: offline work that turns documents into searchable passages, and a query-time path that retrieves, reranks, generates with citations, and evaluates. The model call is one stage inside that path.

  • How do you build a RAG pipeline?

    Start with fifty to a hundred labelled questions, then get retrieval right before generation: parse and chunk the corpus, embed and index, run hybrid search with reranking, require citations and a refuse path, and measure recall and faithfulness separately.

  • What are the stages of a RAG pipeline architecture?

    Write path: ingest and parse, chunk, embed, index. Query path: retrieve with hybrid search, rerank, generate with grounding and citations, then evaluate. Neighbour glossary pages own the detail of each stage.

  • How is a RAG pipeline different from RAG?

    RAG names the pattern of retrieving passages and generating from them. A RAG pipeline is the shipping system that implements that pattern, including the write-time index path and the evals that catch which stage broke.

Let's build something that ships.

Tell us what you're building. We'll tell you whether it needs a fixed-scope build, a prototype taken to production, or a programme we run, what's achievable, and where the real bottlenecks are.

Reply within 2h

We store your name, email, company, and message, and email a copy to hello@bigcircle.ai. Read the privacy page and the terms.