RAG
Hybrid search: BM25 and vector search in one query
Hybrid search runs the same query twice: once with BM25, which finds exact words and codes, and once against vectors, which find text that is close in meaning. The two result lists are merged, usually with Reciprocal Rank Fusion, and a reranker picks what finally reaches the model.
Updated
What is BM25?
A full-text search algorithm that looks for the exact words in the query. It catches what is written character for character: a name, a product code, an error code.
Why is semantic search not enough?
Embedding search is good at finding text that means the same thing in other words. It can miss the exact match, though: ask about one specific error code and it returns pages about error codes in general. BM25 looks for that exact string.
How do you combine BM25 and vector results?
With Reciprocal Rank Fusion (RRF). Each document gets points for its position in each list, so documents ranked high in both move up. RRF works on positions, so the two searches' scores do not need to be related to each other, and the Elasticsearch docs note that it requires no tuning.
What does a reranker do?
A model rereads the candidates and reorders them by how well they answer the query. Only the most relevant reach the model that writes the answer, so it reads less text and costs less, though the reranking step itself adds a little latency.
Frequently asked questions
BM25 vs vector search: which one should I use?
BM25 for names, codes and exact terms. Vectors for questions worded differently from the document. Anthropic reports that the two together return more complete results than either one alone.
What is semantic search?
Search by meaning. The query and the documents become embeddings, and the search returns the text closest in meaning to the query.
Are there open source tools for hybrid search?
Yes. qmd, for example, is a search engine that runs on your own machine, under the MIT license. It has BM25 search, vector search and a hybrid mode that merges the lists with RRF and reorders them with an LLM reranker. It also exposes an MCP server, so an agent can use it as a tool.
What is query expansion?
A model writes variants of the query, and the search runs all of them alongside the original. In qmd's hybrid mode the original query gets extra weight in the fusion step.
Does hybrid search replace RAG?
No. It is the retrieval step inside RAG: its results go into the prompt, and the model answers from them.
Sources
Further reading
Related articles
What is RAG in AI, and how does it work step by step?
RAG (Retrieval-Augmented Generation) means the model searches your documents first and writes the answer from what it found. The documents are split into chunks, each chunk becomes an embedding in an index, and for every question the chunks closest in meaning are added to the prompt next to the question.ReadAI answers with citations: grounding an LLM in your sources
An answer with citations puts the passage that supports each claim right next to it. The model is allowed to say "I don't know", pulls the quotes first and answers second, and any claim without a quote is retracted. Hallucinations drop a lot but do not disappear, so a person checks the decisions that matter.Read
Case studies
Related services
Want search that works on your company's documents?
Tell us where the documents live and what your team looks for most often. We will work out which search fits.
Write to us