Retrieval is the process of finding relevant information from a knowledge source. It's the first key step in RAG systems, before the model generates an answer.
In simple terms
Searching a knowledge base and pulling back the most relevant pieces.
How it works
- 1User question: the user asks a question.
- 2Convert to vector: the question is turned into an embedding.
- 3Search knowledge base: the vector is used to search the vector database.
- 4Retrieve top results: the top-K most relevant documents are retrieved.
- 5Pass to LLM: the retrieved context is sent to the model to generate the final answer.
Key points
- Finds information that is semantically similar, even if the exact words differ.
- Techniques include vector search, BM25 keyword search and hybrid search.
- Metadata filtering narrows results before or after search.
- Good retrieval means more relevant context and fewer hallucinations.
Why it matters
Retrieval quality decides RAG quality: if the right context isn't retrieved, even the best LLM can't answer well. It's the foundation of grounded, accurate AI systems.
Frequently asked questions
- What is top-K retrieval?
- Returning the K most relevant chunks (e.g. top 5) to pass to the model as context.
- Retrieval vs generation?
- Retrieval finds relevant information; generation writes the answer using it. RAG combines both.