Home / Hybrid search / Score Normalization in Hybrid Search: Min-Max, Z-Score and More

Score Normalization in Hybrid Search: Min-Max, Z-Score and More

Score normalization in hybrid search is the step that rescales keyword and vector scores onto a common scale so they can be weighted and added into one ranking. It is needed because BM25 scores are unbounded and vary by query, while vector similarities sit in a narrow fixed range, so adding them raw lets one side swamp the other. The common methods are min-max scaling, which maps each list’s best score to 1 and worst to 0, and z-score standardization, which centres each list on its mean and divides by its spread. L2 scaling divides by the vector length, and distribution-based scaling clips at a few standard deviations. Rank-based fusion avoids normalization by discarding scores and using positions. Weaviate’s default, relative score fusion, uses min-max scaling per result list before applying the alpha weight.

What is score normalization in hybrid search?

Score normalization in hybrid search is the transformation that puts the keyword score and the vector score of each document on the same scale before they are combined. Without it, the weights in a hybrid query do not mean what they say.

Hybrid search runs a keyword search, usually BM25, and a vector search, then fuses their results. A weighted sum such as alpha times the vector score plus one minus alpha times the keyword score only works if a point of vector score means as much as a point of keyword score. Normalization makes that true. The surrounding mechanics are in how hybrid search works, and the concept itself in what is hybrid search.

Normalization is applied per query and per result list. Each search’s top candidates are rescaled using only that list’s own scores, which is why the candidate depth matters: the minimum, maximum or mean depend on how many results each side returned. Weaviate’s documentation notes that relative score fusion is sensitive to a small limit for this reason, and searches with a higher internal limit before trimming. The scale problem is easiest to see on real numbers.

Why do keyword and vector scores need normalization?

Keyword and vector scores need normalization because BM25 scores are unbounded and change with the query, while vector similarities fall in a fixed narrow range. Raw sums are then dominated by whichever side happens to be larger. The imbalance changes from query to query.

Weaviate’s documentation uses the example in the figure. The keyword scores for five documents run from 5 down to 0.09, and the vector scores from 0.6 down to 0.009. Added raw, the keyword side would decide almost everything, since its spread is several times larger. A query with rarer terms could produce BM25 scores in the tens, and the imbalance would grow further.

Weaviate’s documentation also explains why it provides no threshold for the keyword component of hybrid search: BM25 scores are not normalized or bounded like vector distances, so a universal cut-off makes little sense. The same property is what makes normalization necessary for fusion. The simplest fix is min-max scaling.

How does min-max normalization work?

Min-max normalization rescales each list so its highest score becomes 1, its lowest becomes 0, and every other score falls in between in proportion to its distance from those ends. The formula is the score minus the minimum, divided by the maximum minus the minimum.

In Weaviate’s example, the keyword scores 5, 2.6, 2.3, 0.2 and 0.09 become 1.0, 0.511, 0.450, 0.022 and 0.0. The vector scores keep their shape: the top four, all near 0.6, map close to 1, and the outlier at 0.009 maps to 0. Weaviate’s relative score fusion, the default since version 1.24, uses this scaling, then multiplies the vector side by alpha and the keyword side by one minus alpha. Weaviate reports that relative score fusion gave about 6 percent better recall than ranked fusion on the FIQA dataset in its internal benchmarks.

Min-max keeps the gaps between scores, so a document far ahead in one list stays far ahead. Its weakness is sensitivity to the extremes. A single very low score stretches the range and squeezes everyone else together near 1, as the vector list shows. A single very high score does the opposite. That is the problem z-score normalization addresses. Relative score fusion covers Weaviate’s implementation in depth.

How does z-score normalization differ from min-max normalization?

Z-score normalization differs from min-max normalization by centring each list on its mean and scaling by its standard deviation. Scores then show how unusual each result is rather than where it sits between the best and worst. Outliers stand out instead of setting the scale.

The formula is the score minus the list’s mean, divided by its standard deviation. Applied to Weaviate’s example keyword scores, document 1’s 5 becomes about 1.64 and document 3’s 0.09 about -1.08. For the vector list, the four near-tied documents all land near 0.5, while the outlier, document 3, drops to -2.0. Unlike min-max, the results are not confined to 0 to 1, and negative values are normal.

The choice matters when lists have different shapes. If one leg returns a single standout and a long tail, min-max compresses the tail; z-score keeps more spread. If a list has few results, its mean and standard deviation are noisy, and z-score can swing. Weaviate applies min-max scaling inside relative score fusion. OpenSearch’s search pipelines document min-max, L2 and z-score techniques, and Qdrant documents a distribution-based score fusion that normalizes using the mean and three standard deviations. Several other options exist.

Which other score normalization methods are used in hybrid search?

Other score normalization methods used in hybrid search are L2 scaling, distribution-based scaling, theoretical-bound scaling, and abandoning scores for ranks. Each makes a different assumption about what a score means.

• L2 scaling divides each score by the square root of the sum of squared scores in the list, keeping ratios intact.
• Distribution-based scaling maps the range from the mean minus three standard deviations to the mean plus three onto 0 to 1, clipping beyond.
• Theoretical-bound scaling uses fixed limits, such as cosine’s known range, instead of the observed minimum and maximum.
• Rank-based fusion replaces each score with a function of its position, as reciprocal rank fusion does.

Theoretical bounds suit vector scores, whose range is known, but not BM25, which has no fixed maximum. Some systems combine approaches, scaling the vector side by its theoretical range and the keyword side by observed min-max. Bruch and colleagues’ 2023 analysis of fusion functions for hybrid retrieval compared convex combinations of normalized scores with reciprocal rank fusion and found the convex combination competitive or better, with alpha tunable from few examples. Score fusion methods compares the combinations. Normalization choices change real rankings.

How do normalization methods change the final ranking?

Normalization methods change the final ranking when the two result lists disagree and their score distributions differ, as in Weaviate’s five-document example, where score-based and rank-based fusion pick different winners. Small changes in scaling can reorder close documents.

With equal weights, min-max, z-score and L2 all rank document 1 first. It is the clear keyword winner by a wide margin and sits inside the near-tie at the top of the vector list. Rank fusion instead puts document 2 first, because it ranks first in the vector list and third in the keyword list, and ranks alone cannot see that document 1’s keyword lead was large. Weaviate’s documentation draws the same conclusion from this example.

The score-based methods agree here because both lists have one clear outlier at the bottom. With other distributions they can diverge, especially for documents found by only one leg, whose missing score is usually treated as 0. Testing on labelled queries is the only reliable way to pick. The practical question is how to choose.

Which score normalization should you use?

You should start with your search engine’s default, such as Weaviate’s relative score fusion with min-max scaling. Switch only if a labelled evaluation shows another method or rank fusion ranks better on your queries. Alpha usually matters more than the normalization formula.

Min-max is simple, keeps score gaps and works well when both lists have reasonable spread. Z-score suits lists with outliers and enough results for stable statistics. Rank fusion is the safest when score distributions are erratic or unknown, at the cost of ignoring how decisive a lead is. Weaviate’s documentation recommends relative score fusion as the default and ranked fusion when score magnitudes should be ignored entirely.

from weaviate.classes.query import HybridFusion

articles = client.collections.use("HelpArticle")
by_score = articles.query.hybrid(query="refund ERR-4012", alpha=0.5,
                                 fusion_type=HybridFusion.RELATIVE_SCORE, limit=10)
by_rank = articles.query.hybrid(query="refund ERR-4012", alpha=0.5,
                                fusion_type=HybridFusion.RANKED, limit=10)

Run both on a set of real queries with known relevant results and compare nDCG at 10. Then tune alpha, covered in balancing keyword and vector scores. Rank fusion itself is explained in reciprocal rank fusion.

Normalization interacts with the rest of the hybrid design. Whether hybrid search is worth it at all is covered in when to use hybrid search and hybrid search vs vector search. Engines differ in which normalizations they offer, as compared in databases that support hybrid search and hybrid search in SQL databases. Sharded clusters raise the question of whether to normalize per shard or over the whole result, covered in hybrid search on distributed databases. Applied settings are walked through in the hybrid search tutorial, hybrid search for RAG and hybrid search for enterprise search.

Score normalization puts keyword and vector scores on one scale so a weighted sum means what its weights say. BM25 scores are unbounded and vector similarities sit in a narrow band, so raw sums are lopsided. Min-max maps each list to 0 to 1 and keeps gaps, z-score centres on the mean so outliers stand out, L2 and distribution-based scaling offer other trade-offs, and rank fusion skips scores entirely. In Weaviate’s example, all score-based methods ranked the strong keyword match first while rank fusion favoured the vector leader. Start with Weaviate’s relative score fusion, compare it with ranked fusion on labelled queries, and tune alpha before changing formulas.