RAG Retrieval Optimization Layer

This is a RAG retrieval optimization layer whose goal is to retrieve the right context with high recall and high relevance, using a sequence of techniques from query rewriting to reranking.

How it works

1. 1Query rewriting: expand, decompose or clarify the query.
2. 2Query expansion adds synonyms and related terms; hybrid retrieval combines BM25 and vector search.
3. 3Metadata filtering keeps only authorized, relevant content.
4. 4Reranking (cross-encoder) reorders by true relevance; context compression removes noise.
5. 5Diversity selection and top-K selection pick the best few chunks to send to the LLM.

Key points

• Optimise for recall (find all relevant docs) and precision (return only relevant docs).
• Always rerank before sending to the LLM; filter early with metadata.
• Compress context and continuously evaluate and iterate.
• Common pitfalls: relying only on vector search, sending too many chunks, no reranking.

Why it matters

Retrieval quality caps RAG quality. This optimization layer is how you get high-recall, high-relevance context — the biggest lever on RAG accuracy.

Frequently asked questions

Why rerank before the LLM?

It filters out weak matches so the model only sees the truly most relevant chunks, improving accuracy and cost.

What's a common pitfall?

Relying on vector search alone (low recall) or dumping too many chunks into the model (noisy context).