Project: Production-Ready RAG System
Production RAG is not a demo that stuffs three chunks into a prompt. It is a retrieval product with quality, security, observability, and operations.
Architecture
text
source systems
-> ingestion jobs
-> parsing and cleaning
-> chunking
-> embeddings
-> vector plus keyword index
-> retrieval and reranking
-> grounded generation
-> evals and monitoring
Production requirements
| Area | Requirement |
|---|---|
| freshness | documents update on schedule or event |
| permissions | users only see allowed sources |
| citations | answers point to source chunks |
| no-answer behavior | model admits missing evidence |
| monitoring | traces capture retrieval and generation |
| evals | changes are tested before release |
| rollback | prompts, indexes, and models can revert |
Ingestion quality
Track document metadata:
- source ID
- title
- owner
- version
- timestamp
- permissions
- parser version
- chunking version
If you cannot trace an answer back to a source version, debugging becomes painful.
Retrieval stack
Use layers:
- metadata filtering
- dense vector search
- keyword or BM25 fallback
- reranking
- context compression
- source diversity
Each layer should be measurable.
Answer contract
Production RAG answers should include:
- direct answer
- citations
- uncertainty if evidence is weak
- no-answer response when needed
- no hidden instructions from retrieved documents
Monitoring
Log safe metadata:
- query type
- retrieved doc IDs
- retriever scores
- reranker scores
- final citations
- model
- prompt version
- latency
- token usage
- user feedback
Do not log sensitive source text unless policy allows it.
Release checklist
- Build a golden query set.
- Test retrieval recall.
- Test grounded answer quality.
- Add prompt-injection test documents.
- Run canary traffic.
- Compare cost and latency.
- Review failed traces.
- Roll out gradually.
Knowledge check
Q1: What separates production RAG from demo RAG?
Freshness, permissions, citations, evals, monitoring, and rollback.
Q2: Why version chunks and prompts?
So you can explain and reproduce why an answer happened.