Home / Reranking / Best Reranking Models Compared

Best Reranking Models Compared

The best reranking model is the one that ranks your own queries best within your latency, cost and data-handling limits; no single model wins everywhere. The realistic shortlist splits into hosted rerankers and open models. Hosted rerankers come from Cohere, Voyage AI, Jina AI, NVIDIA and Contextual AI and are called over an API. Open cross-encoders, such as the MS MARCO MiniLM models and the BAAI bge-reranker family, run on your own hardware. Language support, input length, instruction following and price separate them more than headline accuracy. Weaviate integrates all of these: hosted models through reranker modules, and open models through a locally hosted Transformers container. Choose two or three candidates that fit your constraints, run them over a labelled set of your own queries, and keep the one with the best ranking for the latency you can afford.

What are the different types of reranking models?

The different types of reranking models are cross-encoders, late interaction models, instruction-following rerankers and general LLM rerankers. They trade cost per candidate against flexibility.

Cross-encoders read the query and each candidate together and output a relevance score; most hosted reranking APIs and open rerankers are of this kind. Late interaction models, such as Jina’s ColBERT model, keep a vector per token and match query tokens to passage tokens, which is faster per candidate at the cost of larger stored representations. Instruction-following rerankers, such as Contextual AI’s instruct models, accept a natural language instruction that changes how relevance is judged.

LLM rerankers prompt a general language model to score or order passages. They handle complex criteria but cost far more per query, as covered in LLM reranking. A reranking algorithm, in the general sense, is any second-stage method that reorders first-stage results; what is reranking covers the idea and cross-encoder rerankers covers the dominant type. The models themselves fall into hosted and open groups.

Which hosted reranking models are available?

Hosted reranking models available through Weaviate’s integrations come from Cohere, Voyage AI, Jina AI, NVIDIA and Contextual AI, each called with an API key. Weaviate’s documentation lists their models and defaults.

The table shows a trap worth knowing. Weaviate’s documentation warns that the Voyage AI module’s built-in default, rerank-lite-1, is a retired first-generation model, so a collection created without a model setting points at it. The fix is to set the model on every collection. Weaviate also stopped validating Cohere model names in version 1.33, so any model the account can reach, including fine-tuned ones, can be named.

Hosted rerankers trade control for convenience. They need no GPU and stay current, but every query sends text to the provider and incurs a fee. Data that must stay in-house points to open models.

Which open reranking models can you run yourself?

Open reranking models you can run yourself include the MS MARCO cross-encoders from the Sentence Transformers project and the BAAI bge-reranker family, both available on Hugging Face. They run on your own hardware with no per-query fee.

The MS MARCO cross-encoders, such as cross-encoder/ms-marco-MiniLM-L6-v2, are small English models trained on web search questions. They are fast and a common default; Weaviate’s reranker-transformers container offers a ready image for it. The bge-reranker models from the Beijing Academy of Artificial Intelligence are larger, and the v2-m3 version is multilingual. Weaviate’s container can host other Hugging Face cross-encoders too, selected by image tag.

Running open models means owning the hardware and the updates. On a CPU, larger rerankers can be slow over many candidates; a GPU changes that, and Weaviate’s container supports CUDA. Weaviate’s documentation notes that the Transformers container is not available on Weaviate Cloud, since it needs its own inference service. Reranking latency and cost covers sizing. Language support is often the deciding factor.

Which multilingual reranker should you use?

You should use a reranker trained on multilingual data, such as bge-reranker-v2-m3, jina-reranker-v2-base-multilingual, Cohere’s multilingual-capable rerankers or Contextual AI’s multilingual models, and test it on your own languages. English-only models degrade on other languages.

Many early rerankers, including the MS MARCO MiniLM models, were trained on English questions. On other languages, or on queries in one language over documents in another, they rank poorly because they never learned the vocabulary. Models whose names include “multilingual” or “m3” were trained across many languages. Weaviate’s Jina AI integration defaults to the multilingual v2 model for this reason.

Coverage varies by language even among multilingual models; a model strong in Spanish may be weaker in Hungarian or Thai. Weaviate’s academy course on evaluating embedding models notes the same pattern of language-specific performance. Build a small test set in each language you serve and compare candidates directly. Speed is the next filter.

How do reranking models differ in latency and cost?

Reranking models differ in latency and cost through model size, input length and hosting. Small open models are cheapest per query on your own hardware, large models are slowest, and hosted APIs add network time and a fee. Every model runs once per candidate.

Latency grows with the number of candidates and the size of the model. A six-layer MiniLM cross-encoder runs much faster per pair than a multi-billion-parameter reranker such as NVIDIA’s Mistral-based default. Hosted models move compute off your servers but add a round trip and per-query pricing that changes over time, so check each provider’s current price list at decision time.

The cheapest improvement is often fewer candidates rather than a smaller model. Reranking 20 candidates with a strong model can beat reranking 100 with a weak one. Reranking in RAG explains how to choose the candidate count. Domain fit can matter more than any of these.

How does input length limit a reranking model?

Input length limits a reranking model because each model reads only a fixed number of tokens of query plus passage and silently truncates the rest. A passage longer than the window is judged on its opening alone.

Small cross-encoders built on BERT-style encoders typically read a few hundred tokens per pair, while newer rerankers accept far longer inputs. If chunks run longer than the window, the sentence that answers the question may never be read, and the reranker can demote the right passage. The problem is invisible in the output, since the model still returns a confident score. Check the maximum input length in each candidate model’s documentation and compare it with your chunk sizes.

Two fixes work. Keep chunks within the reranker’s window, which also tends to help retrieval, as covered in chunking for semantic search. Or choose a model with a longer window, accepting slower scoring, since longer inputs cost more per pass. Weaviate’s reranker query names the property to score, so a collection can store a shorter text field purely for reranking. With length, language and speed settled, domain fit remains.

How do you choose a reranker for a specialised domain?

You choose a reranker for a specialised domain by testing several candidates on labelled queries from that domain, and fine-tuning one if none ranks well enough. General rerankers can rank specialised text worse than the first stage did.

Rerankers trained on web search questions learn what web relevance looks like. Scientific claims, legal clauses, medical notes and internal product language look different, and a general model may misjudge them. Weaviate’s documentation names legal, medical and scientific literature as cases needing nuanced reranking, which is also where a mismatched model is most likely to hurt.

Fine-tuning closes the gap. Cohere supports fine-tuned rerankers, which Weaviate can call by model ID, and open cross-encoders can be fine-tuned with Sentence Transformers on in-domain query and passage pairs. Instruction-following rerankers offer a lighter option, steering a general model with written criteria. Testing is what turns these options into a choice.

How do you choose the best reranking model for your search?

You choose the best reranking model by filtering candidates on hard constraints, such as data location, language and latency, then comparing the survivors on your own labelled queries. The test decides; leaderboards only shortlist.

Build a test set of real queries with known relevant results, retrieve candidates with your first stage, and rerank them with each model. Compare nDCG at 10 or recall in the top few, and record latency. In Weaviate, switching models is a configuration change, since reranker settings are mutable on collections from versions 1.25.23, 1.26.8 and 1.27.1 onward.

from weaviate.classes.config import Configure, Reconfigure

client.collections.create("HelpArticle",
    reranker_config=Configure.Reranker.voyageai(model="rerank-2.5"))   # set a current model

articles = client.collections.use("HelpArticle")
articles.config.update(reranker_config=Reconfigure.Reranker.cohere(model="rerank-v3.5"))

Diversity is a separate concern from relevance. If the top results repeat each other, add maximal marginal relevance after reranking. Revisit the choice when providers release new models, since the field moves quickly.

The best reranking model is the one that ranks your own queries best within your limits on latency, cost, language and data handling. Hosted options in Weaviate come from Cohere, Voyage AI, Jina AI, NVIDIA and Contextual AI, with defaults worth checking, such as Voyage AI’s retired rerank-lite-1. Open cross-encoders such as the MS MARCO MiniLM and bge-reranker models run in Weaviate’s Transformers container on your own hardware. Filter by constraints, test two or three candidates on labelled queries, fine-tune for specialised domains, and keep the model that ranks best for the latency you can afford.