Re-ranking

Definition

An advanced, secondary step in RAG architectures where an initial large list of retrieved documents is passed through a highly precise 'Cross-Encoder' model to re-score and re-sort them based on deep contextual relevance.

Explain Like I'm New

The Vector Database acts as a fast but somewhat sloppy net, catching 50 possible documents. The Re-ranker acts as a slow, meticulous human, reading those 50 documents thoroughly and picking the absolute best 3.

Real World Example

Cohere's Rerank API. You search the database and get 20 results. You send the user's query and those 20 results to Cohere. Cohere calculates a deep contextual score for each one and outputs the top 3, drastically improving the final LLM answer.

Common Use Cases

•Advanced RAG
•Improving search accuracy

Interview Questions

basic

Why don't we just use the Re-ranker on the entire database from the start?

intermediate

What is the difference between a Bi-Encoder (Vector Database) and a Cross-Encoder (Re-ranker)?

Flash Cards

Question

Why not use it initially?

Click to reveal answer

Answer

Because Re-ranking models are extremely computationally expensive and slow. Running a Re-ranker on a database of 1 million documents would take hours. Vector math takes milliseconds.

Question

Bi-Encoder vs Cross-Encoder?

Click to reveal answer

Answer

Bi-Encoders (Vectors) evaluate the Query and the Document completely separately, reducing them to math before comparing them (Fast but shallow). Cross-Encoders evaluate the Query and Document *together simultaneously*, understanding how the words interact with each other (Slow but deeply accurate).