RAG & Retrieval (Enterprise)
RAG: Retrieval Re-ranking
Re-rank retrieved docs to keep only the most relevant.
Re-ranking is one of the highest-ROI optimizations for RAG systems. Instead of sending all retrieved documents to the LLM, you re-rank to keep only the most relevant ones.
In simple terms
Don't send all retrieved docs — re-rank and keep only the most relevant.
How it works
- 1A user question runs an initial vector search retrieving, say, the top 20 documents.
- 2A re-ranker model (second pass) scores and reorders them by relevance.
- 3Only the top 5 final documents are kept.
- 4These are passed to the LLM to generate the answer.
- 5Result: a focused, high-quality context.
Key points
- Without re-ranking, weak/irrelevant docs reach the LLM, lowering quality and raising cost.
- With re-ranking, fewer, better docs mean higher precision and better answers.
- Re-ranking typically uses cross-encoder models (BGE, Cohere, Jina).
- Noisy context = poor answers; relevant context = better answers.
Why it matters
Re-ranking is a small addition with a large impact: higher accuracy, lower token usage and more trustworthy answers — one of the best RAG optimizations.
Frequently asked questions
- What model does re-ranking use?
- A cross-encoder (e.g. BGE-reranker, Cohere Rerank) that scores query and document together.
- Why not send all retrieved docs?
- Extra weak docs add noise and cost; re-ranking keeps only the best few.
More in RAG & Retrieval (Enterprise)
Where Hallucinations Come From
Failures across the query, retrieval and model layers.
RAG Retrieval Optimization Layer
Retrieve the right context with high recall and relevance.
RAG Optimization 1 · Hybrid Search
Keyword + semantic search for higher recall.
RAG Optimization 2 · Re-ranking
Find the most relevant docs before the LLM.
RAG Optimization 3 · Context Pruning
Remove noise; keep only what matters.
RAG Optimization 4 · Semantic Cache
Stop asking the same question twice.