Home / Reranking / LLM Reranking: Pointwise, Pairwise and Listwise

LLM Reranking: Pointwise, Pairwise and Listwise

LLM reranking is a second-stage search step in which a large language model reads a query and a short list of retrieved candidates and reorders them by relevance. A first-stage search, such as vector, keyword or hybrid search, finds a few dozen candidates quickly. The language model then judges them more carefully than an embedding can. There are three ways to prompt it. Pointwise prompts score one document at a time, pairwise prompts compare two documents, and listwise prompts rank many documents in a single call. LLM rerankers need no task-specific training and can follow instructions, such as boosting recent documents. They are also slower and more expensive than cross-encoder rerankers, and their output must be parsed and checked. Weaviate’s built-in reranker modules use dedicated reranking models, so an LLM reranker usually runs in application code on the results of a Weaviate query.

What is LLM reranking?

LLM reranking is the use of a general large language model as the reranker in a two-stage search pipeline. The model receives the query and the candidate texts and returns scores or an order.

Reranking itself is an old idea, explained in what is reranking. A fast first stage retrieves candidates from the whole collection, and a slower, more accurate model reorders a small number of them. Weaviate’s concepts documentation describes reranking as computing a relevance score between the query and each data object, as a second stage, because scoring every pair in the collection would be too slow.

The usual reranker is a cross-encoder, a model trained to output one relevance score for a query and document pair, covered in cross-encoder rerankers. An LLM reranker replaces that trained model with a language model and a prompt. Weaviate’s blog on large language models and search notes that LLMs can now do this off the shelf without any extra training.

The appeal is flexibility. The same model can rank support articles, legal passages or product listings, and the prompt can carry extra rules. The cost is speed, money and the need to turn free text into a usable ranking. The ranking itself can be requested in three different shapes.

What are pointwise, pairwise and listwise LLM reranking?

Pointwise, pairwise and listwise reranking are the three prompt designs for LLM rerankers. They differ in how many documents the model sees per call and what it outputs.

Pointwise reranking works like a cross-encoder. The model sees one query and one document and outputs a relevance score. Weaviate’s blog on ranking models gives the example prompt “please output a relevance score on a scale of 1 to 100” and notes that bounding the score is the tricky part. A simpler variant asks for yes or no and uses the model’s confidence as the score.

Pairwise reranking asks which of two documents better answers the query. Pairwise ranking prompting, from Zhen Qin and colleagues in 2023, built on this idea. Comparing every pair of n candidates takes n(n-1)/2 calls, so 20 candidates need 190 comparisons. Sorting algorithms reduce the count but still need many calls.

Listwise reranking puts many documents in one prompt and asks for an ordered list of their IDs. Weaviate’s blog on fine-tuning Cohere’s reranker contrasts the two ends: pointwise rerankers take each candidate separately, while listwise rerankers take all the candidates at once and output a reranked list. Listwise is the design most associated with LLM reranking.

How does listwise LLM reranking work?

Listwise LLM reranking works by numbering the candidates in a prompt and asking the model to output their identifiers from most to least relevant. A window moves over the list when it is too long for one prompt. The model can compare candidates directly.

A typical prompt lists passages as [1], [2], [3] and so on, states the query, and asks for output such as [3] > [1] > [2]. RankGPT, from Weiwei Sun and colleagues in 2023, popularised this format with general chat models. Seeing the candidates side by side lets the model prefer the passage that answers the question over one that only repeats its words.

Context windows limit how many passages fit. Weaviate’s blog on ranking models raises the same problem: reranking 100 documents five at a time needs a tournament-style decomposition. The RankGPT paper used a sliding window of 20 passages moving by 10, starting from the bottom of the list. With 100 candidates, start positions 81, 71 and so on down to 1 give 9 windows.

Because each window overlaps the next by 10, a relevant passage near the bottom can move up one window at a time. After the final window, the top 10 positions have been compared directly. Positions lower in the list get less attention, which is acceptable when only the top few results are shown or passed to a generator. Open listwise models were trained on this pattern.

Which models are used for LLM reranking?

LLM reranking uses either general chat models prompted to rank, or open models trained specifically to produce listwise rankings. Trained models are smaller and more predictable in their output format.

General chat models are the quickest way to start, since a prompt is all that is needed. Their quality depends on the model, and their output sometimes breaks the requested format, repeats an identifier or omits one. Every response therefore needs parsing and repair before use.

Trained listwise rerankers address this. Weaviate’s blog on fine-tuning Cohere’s reranker summarises RankZephyr, in which Ronak Pradeep and colleagues trained a 7 billion parameter model to copy outputs from a two-stage training process with GPT-3.5 and GPT-4. The resulting model runs locally and emits rankings in a fixed format.

Dedicated rerankers remain the main alternative. Weaviate’s reranker integrations cover providers such as Cohere, Voyage AI, Jina AI, NVIDIA, Contextual AI and locally hosted transformers models. A comparison of options is in the best reranking models. Instruction following is where LLMs add something those models lack.

Can an LLM reranker follow ranking instructions?

An LLM reranker can follow ranking instructions written in plain language, such as preferring recent documents or a particular author, which a fixed relevance model cannot. The rules live in the prompt rather than in code.

Weaviate’s blog on large language models and search gives an example prompt that asks the model to rank results by relevance and to boost relevance based on recency and a named author. Each result is passed with its metadata as key-value pairs. The blog notes that this lets practitioners swap ranking logic easily, and that the model can explain its ranking.

The same blog links this to choosing how many results to keep. Given k search results, the model can be asked how many are relevant enough to show a user or to pass to the next step. That is an LLM version of a cutoff, related to Weaviate’s autocut feature for vector distances.

Instructions also bring risks. A model may follow a boost too strongly and demote the best answer, and wording changes can change rankings. Rules that can be expressed as filters or sort orders are cheaper and more predictable in the database. Diversity is a separate goal, handled by methods such as maximal marginal relevance. The cost of all this flexibility is easy to underestimate.

What does LLM reranking cost in latency and money?

LLM reranking costs one or more language model calls per query, each processing every candidate’s text, which adds noticeable delay and per-token charges compared with a cross-encoder. The number of candidates drives both.

A listwise call over 20 passages sends all 20 passages as input tokens and waits for the model to write the ranking. Longer passages mean more tokens. A sliding window over 100 candidates multiplies the calls by 9, and the windows run in sequence, since each depends on the previous order.

Pointwise calls are independent and can run in parallel, so latency stays close to one call, but the total token count is similar or higher. Pairwise all-pairs comparison is the most expensive, with 190 calls for 20 candidates. Weaviate’s academy course sums up rerankers generally as better quality for top results at the price of added latency and cost.

Practical systems limit the damage. They rerank only the top 20 or so candidates, truncate passages, cache results for repeated queries and reserve LLM reranking for queries where quality matters most. The trade-offs for every reranker type are covered in reranking latency and cost. The implementation pattern is short.

How do you add an LLM reranker to Weaviate search results?

You add an LLM reranker to Weaviate search results by retrieving candidates with a hybrid query and sending the numbered passages to a language model. The objects are then reordered by the identifiers it returns. Parsing must tolerate bad output.

import re

def llm_rerank(collection, query, ask_llm, candidates=20, top=5):
    res = collection.query.hybrid(query=query, alpha=0.5, limit=candidates)
    objs = res.objects
    passages = "\n".join(f"[{i+1}] {o.properties['text'][:500]}" for i, o in enumerate(objs))
    prompt = (f"Rank the passages by relevance to the query: {query}\n{passages}\n"
              "Answer only with identifiers, most relevant first, like [2] > [1] > [3].")
    order = []
    for n in re.findall(r"\[(\d+)\]", ask_llm(prompt)):
        i = int(n) - 1
        if 0 <= i < len(objs) and i not in order:
            order.append(i)                       # drop repeats and invalid IDs
    order += [i for i in range(len(objs)) if i not in order]   # keep anything omitted
    return [objs[i] for i in order[:top]]

ask_llm is any function that sends a prompt to a language model and returns its text. The parser keeps the first mention of each valid identifier, ignores numbers outside the list, and appends omitted candidates in their original order, so a malformed answer degrades to the first-stage ranking. The parsing logic was run against sample malformed answers as a check.

Weaviate’s built-in rerank parameter suits dedicated reranking models. Its academy course points out that the query limit is both the reranker’s input set and the number of results returned, so a wider pool means raising the limit and keeping the top n yourself. The same applies to the LLM pattern. RAG pipelines use the reranked list as context, as described in reranking in RAG. Knowing when to use this approach decides whether it earns its cost.

When should you use an LLM reranker instead of a cross-encoder?

You should use an LLM reranker when queries need reasoning or custom ranking rules, when there is no training data for a domain, or when only a few queries need the highest quality. A cross-encoder remains the default for high traffic.

LLM rerankers suit complex questions where the right passage answers the question rather than echoing its words. They suit rules that change often, such as editorial boosts. They also suit prototypes, since no model needs to be trained or hosted, and offline work such as building judged test sets.

Cross-encoders suit search at scale. They are faster, cheaper per query, produce a numeric score directly and never break an output format. Weaviate’s reranker modules run them inside the query, with no parsing step. Many systems combine both, with a cross-encoder for all traffic and an LLM pass for hard or high-value queries.

Measure before deciding. Compare the first-stage ranking, a cross-encoder and an LLM reranker on judged queries with metrics such as nDCG, alongside latency and cost per query. If the LLM’s gain is small on your data, the cheaper reranker wins.

LLM reranking uses a large language model to reorder a retrieved shortlist. Pointwise prompts score one passage at a time, pairwise prompts compare two, and listwise prompts rank many together. For long lists a sliding window helps: 9 windows of 20 cover 100 candidates at a step of 10. LLMs need no training and can follow plain-language ranking rules. They also add latency, per-token cost and output that must be parsed. In Weaviate, run the first-stage hybrid query, rerank in application code, and fall back to the original order when the model’s answer is malformed.