Home / How it works / How Relevance Scores Are Calculated in Semantic Search

How Relevance Scores Are Calculated in Semantic Search

A relevance score is the number a search system assigns to each result to say how well it matches the query, and the results are sorted by it. In semantic search the score usually comes from comparing the query’s vector with each document’s vector. The comparison uses cosine similarity, dot product or Euclidean distance. Keyword search scores with BM25, and hybrid search fuses the two. These scores are good for ranking within one query but poor as absolute measures of relevance. Their scale depends on the embedding model, the distance metric, the query and the collection. A cosine similarity of 0.8 can be an excellent match for one model and a weak one for another. That is why fixed thresholds often fail. Weaviate, for example, returns a raw distance for vector search, where lower is better. BM25 and hybrid scores are relative to the dataset and the query.

What is a relevance score?

A relevance score is a number that estimates how well a document matches a query, used to order search results from most to least relevant. Each retrieval method computes it differently.

The score is the system’s estimate, not a judgment of truth. A high score means the document is close to the query in the way the system measures closeness. For a vector search that is closeness in embedding space; for keyword search it is weighted term overlap. Whether the result actually helps the user is a separate question, answered by evaluation. Semantic matching covers what closeness captures.

“Semantic relevance” means relevance judged by meaning rather than exact words. A query for “laptop battery dies fast” is semantically relevant to a page titled “Improving notebook battery life” although they share almost no words. Vector scores are designed to reflect that kind of relevance, which is why they power vector search.

How are relevance scores calculated in semantic search?

Relevance scores in semantic search are calculated by embedding the query, then measuring the distance or similarity between the query vector and each candidate document’s vector. The measure is set when the index is created.

The common measures are cosine similarity, which compares direction; dot product, which also reflects length; and Euclidean distance, which measures straight-line separation. Cosine vs dot product vs Euclidean distance explains when each fits. Use the measure the embedding model was trained with. Weaviate’s default is cosine, and its documentation advises matching the model’s training metric.

Databases often report a distance rather than a similarity, so that lower always means closer. Weaviate’s cosine distance is 1 minus cosine similarity, ranging from 0 to 2. Its dot metric is the negative dot product, and its Euclidean option is squared distance. All three keep the rule that a smaller distance means a closer match. The search returns the k closest documents, as covered in top-k retrieval.

Keyword search scores in a different way. BM25 adds up a contribution for each query term found in a document. Each contribution grows with how often the term appears, with diminishing returns, and with how rare the term is across the collection. Longer documents are adjusted so they do not win just by containing more words. Weaviate uses BM25F, a variant that scores each field separately and lets some fields count more. Because rarity is measured over the collection, the same document can score differently after other documents are added.

Why do relevance scores come on different scales?

Relevance scores come on different scales because each metric and retrieval method has its own range and direction, from bounded cosine values to unbounded BM25 scores. Reading a score needs its unit.

Weaviate’s documentation describes certainty as an opinionated measure between 0 and 1, available only for cosine distance. For other metrics, distances may be unbounded, so Weaviate recommends distance over certainty. Its BM25 score is relative to the dataset and the query, and the hybrid score depends on the chosen fusion method.

This variety is why combining scores needs care. Adding a BM25 score to a cosine similarity mixes units, so hybrid search normalises or ranks first, as in relative score fusion. The same caution applies when comparing scores from two different systems during a migration.

How do cosine-based scores convert into each other?

For normalised vectors, cosine similarity, cosine distance, certainty and squared Euclidean distance are exact transformations of each other, so they produce the same ranking. Only the numbers differ.

Cosine distance is 1 minus similarity. Certainty is 1 minus half the distance, which maps the range onto 0 to 1. For unit-length vectors, squared Euclidean distance equals 2 minus 2 times the similarity, which is twice the cosine distance. So a cosine similarity of 0.8 is a distance of 0.2, a certainty of 0.9 and a squared Euclidean distance of 0.4.

Confusing these units is a common bug. A threshold written as “similarity greater than 0.8” becomes “distance less than 0.2” in a database that returns distance. Check which number your client returns before setting any cut-off. Dot product only matches this pattern when vectors are normalised.

Why do fixed relevance thresholds fail?

Fixed relevance thresholds fail because the score of a good match varies with the model, the query and the collection, so one cut-off is too strict for some queries and too loose for others. Scores rank well but calibrate poorly.

Different embedding models spread their similarities differently. Some place most unrelated texts at moderate similarity, others near zero, a property discussed in high-dimensional vector space. Short queries, long documents and specialist vocabulary all shift scores. A threshold tuned on one set of queries often removes good results for another.

Better options exist. Set thresholds per model from judged examples, and recheck them after any model change. Use autocut, which cuts results where scores jump, instead of a fixed number. Add a reranker, whose scores are trained to judge relevance directly. When no result clears the bar, show a “no good match” state instead of weak results, which matters most in RAG.

What is context relevance in RAG?

Context relevance in RAG is a measure of whether the passages retrieved for a question actually contain information needed to answer it. It is usually scored after retrieval, often by a language model.

Evaluation frameworks such as Ragas score context relevance, or context precision and recall, by asking a language model to judge each retrieved passage against the question. Weaviate’s overview of RAG evaluation describes these prompt-based checks alongside classic metrics. Unlike vector distances, these scores try to judge usefulness directly, at the cost of an LLM call per check.

Context relevance is a quality check, not a ranking signal for every query. Use it in offline evaluation and spot checks in production. Low context relevance points back to retrieval: chunking, the embedding model, hybrid weighting or filters. Chunking for semantic search is often the first place to look.

How do you tune relevance scoring in practice?

You tune relevance scoring by measuring ranking quality on judged queries, then adjusting the model, metric, hybrid weights and reranking, rather than adjusting raw score thresholds. Ranking metrics show whether changes help.

Start with a test set of real queries and graded labels, and compute nDCG and recall for each change. For hybrid search, tune alpha and the fusion method, and inspect the explanation of each score, which Weaviate returns through explainScore. For approximate search, confirm the index is not the bottleneck, as covered in exact vs approximate search.

Relevance also depends on what the vectors capture. Synonyms and related terms and multilingual search depend on the model. Dense vectors and sparse vectors score differently, as compared in dense vs sparse vectors.

Where do relevance scores fit in a search system?

Relevance scores appear at each stage of a search system: first-stage retrieval scores from the index, fused scores in hybrid search, and reranker scores at the end. Each stage refines the order.

The first stage, from the vector or keyword index, is fast and approximate. Fusion combines stages, and a cross-encoder reranker scores the top candidates by reading query and document together. Only the final order is shown to users. How semantic search works and semantic search architecture show the full pipeline.

Keep score handling explicit in code. Log the score type and value with each result, so analysis can tell a distance from a similarity. Store which model and metric produced it, as described in semantic indexing and vector indexes. Similarity search covers the general problem these scores serve.

A relevance score estimates how well a result matches a query and sets the order of results. Semantic search computes it from vector similarity, keyword search from BM25, and hybrid search by fusing the two. Scores differ in range and direction: Weaviate’s cosine distance runs from 0 to 2 with lower better, while BM25 and hybrid scores are relative to each query. For normalised vectors, cosine similarity, distance, certainty and squared Euclidean distance convert exactly. Avoid fixed global thresholds; tune on judged queries and use reranking for the final order.