13 · Retrieval Pipeline

This retrieval pipeline goes from a user query to the most relevant chunks, combining vector search, keyword search, filtering, top-K selection and reranking.

How it works

1. 1User query → query embedding (convert to a vector).
2. 2Vector search finds semantically similar chunks; BM25 finds keyword matches.
3. 3Hybrid search combines both for better recall.
4. 4Metadata filter applies ACL, date, source and type constraints.
5. 5Top-K results are kept, then a cross-encoder re-ranks for true relevance.

Key points

• Combines semantic and lexical search for recall and precision.
• Metadata filtering enforces permissions and narrows results.
• Re-ranking uses a cross-encoder for final relevance.
• Components: embeddings model, vector store, keyword search, re-ranker.

Why it matters

Good retrieval means more relevant context, less hallucination and better answers. This pipeline is the retrieval half of any strong RAG system.

Frequently asked questions

Why both vector and BM25?

Vector catches meaning; BM25 catches exact terms like IDs and names. Together they maximise recall.

What is top-K?

Keeping the K most relevant candidates (e.g. 20–100) before final ranking.