Hybrid retrieval and reranking: find broadly, choose carefully

An exact workshop code and a paraphrased request need different retrieval clues. How can you combine those clues without comparing scores that use different scales? Build the candidate pool first, then inspect the ordering and evidence it preserves.

Before you begin: You understand keyword search, embeddings, and top-k retrieval.

Hybrid retrieval brings together signals from more than one retrieval method. Reranking then reorders a smaller candidate set using a more detailed relevance assessment. They solve related but different problems.

Build a candidate pool before spending more compute

Retrieve candidates from lexical search and dense search under the same access and version rules. Deduplicate by stable document or chunk identity. If one path ignores permissions, the combined system inherits that exposure even if the final answer does not display the offending document.

Raw scores from different retrievers may use incompatible scales. A cosine similarity of 0.8 is not directly comparable to a BM25 score of 12. Rank fusion offers a way to combine positions without pretending the scores share units.

Calculate reciprocal rank fusion

For each candidate, add 1 / (constant + rank) over the lists in which it appears, with ranks starting at one. A candidate near the top of several lists gets support from several signals.

# Runnable: Python 3, standard library.
def rrf(rankings, constant=60):
    scores = {}
    for ranking in rankings:
        unique = list(dict.fromkeys(ranking))
        for rank, item in enumerate(unique, start=1):
            scores[item] = scores.get(item, 0) + 1 / (constant + rank)
    return sorted(scores, key=lambda item: (-scores[item], item))

result = rrf([
    ['exact-code', 'evening', 'general'],
    ['evening', 'beginner', 'exact-code'],
])
assert result[0] == 'evening'
print(result)

The constant controls how sharply rank differences matter. Sixty is a familiar baseline, not a universal optimum. This implementation prevents repeated IDs inside one list from receiving extra votes and makes ties deterministic.

Rerank the candidates with the question

A cross-encoder processes a query and candidate together and estimates relevance. This can use finer interactions than independently computed embeddings, but it costs more per pair. Apply it to a bounded candidate set rather than the entire corpus.

Reranking cannot recover a relevant document that never entered the candidate pool. First measure candidate recall, then measure ordering quality. If required evidence is absent, tuning the reranker alone attacks the wrong stage.

Respect the reranker's input limits. Truncating a passage can remove the exact condition that makes it relevant or irrelevant. Long candidates may need structured selection or multiple windows, with evaluation of the aggregation rule.

Remove the relevant document before reranking

Create a candidate list containing three plausible but wrong workshop pages. Give the reranker the exact question and the best possible ordering task. Can any permutation produce the missing authoritative record?

Now add the correct record but truncate away its date before reranking. The record ID is present, yet the distinguishing evidence is missing. Evaluate the actual text presented to the reranker and generator, including truncation and expansion. Coverage of IDs and coverage of complete support are different tests.

Inspect the final context, not only the ranking

Deduplicate overlapping passages and preserve enough evidence diversity for multi-part questions. A top-five list of nearly identical chunks can have excellent individual relevance while omitting a necessary second source.

Measure ranking quality, final-answer correctness, latency, and cost together. A higher reranker score is not calibrated proof that a passage supports the answer.

Exercise: reranking improves the position of every relevant candidate, but half the questions still have no relevant candidate. What is the next priority?

Next, investigate query transformations that can widen candidate coverage without changing what the user asked.

Sources

See Elasticsearch's reciprocal rank fusion reference and Sentence Transformers' retrieve-and-rerank guide. Compare implementations and score conventions before combining systems.

Continue: Query Expansion Techniques.