Home / Search evaluation / Recall@k Explained: Formula, Meaning and How to Measure It

Recall@k Explained: Formula, Meaning and How to Measure It

Recall@k is an offline retrieval metric that measures the fraction of all known relevant items found within the first k ranked results. Read “@k” as “at k,” where k is a fixed cutoff such as 5 or 10. The metric divides relevant results inside that cutoff by all known relevant items for the query. Higher recall means fewer known relevant items were missed, but it does not reward better ordering inside the cutoff. Reliable use therefore requires explicit relevance judgments, a product-based choice of k, and a controlled comparison across the same queries and corpus.

What does recall@k measure?

Recall@k measures retrieval coverage within a fixed top-k result set. For one query, the numerator counts items that are both relevant and present among the first k results. The denominator counts every item marked relevant for that query. Weaviate’s “Evaluation Metrics for Search and Recommendation Systems,” read October 8, 2026, defines recall@k with this relevant-items-found fraction and classifies it as non-rank-aware.

The score runs from 0 to 1 when the relevance set is not empty. A score of 0 means the top k contains none of the known relevant items. A score of 1 means the top k contains all of them. Higher recall is better for coverage at the chosen cutoff, but higher recall alone does not prove a better user experience. Returning the whole corpus eventually reaches complete recall while also returning every irrelevant item.

Recall in classification uses the same true-positive denominator idea, but search evaluation applies it to retrieved items for an information need. Christopher Manning, Prabhakar Raghavan, and Hinrich Schütze describe ranked retrieval evaluation through top-k document sets in Introduction to Information Retrieval, Chapter 8. The broader family of information retrieval metrics separates this coverage question from ordering, utility, and online behavior.

How do you calculate recall@k?

Recall@k equals the number of relevant items in the top-k results divided by the total number of known relevant items. For query q, write the formula as Recall@k(q) = |TopK(q) ∩ Rel(q)| / |Rel(q)|. TopK(q) is the first k retrieved items. Rel(q) is the judged relevant set. The intersection supplies the relevant hits.

The denominator creates two important boundaries. First, Recall@k is undefined when a query has zero known relevant items. An evaluation must decide before calculation whether to exclude those queries or report them separately. Second, if a query has R relevant items, its maximum score is min(k, R) / R. A query with 20 relevant items cannot exceed 0.5 at k=10.

Recall@k cannot decrease as k grows when the ranking and judgments stay fixed. A larger cutoff keeps every earlier result and adds more. The numerator may rise or stay constant, while the denominator does not change. This monotonic property makes a recall curve useful for locating the depth where additional candidates stop finding relevant material.

What does a worked recall@k example show?

A ranking with eight known relevant documents and five relevant documents in its top 10 has Recall@10 of 5/8, or 0.625. The illustrative ranking marks documents A, C, D, F, and H as relevant within the first 10 positions. Three of those documents occur in the first five positions. The same relevance set therefore produces Recall@5 of 3/8, or 0.375.

The example also separates recall from precision. Precision@5 is three divided by five, or 0.6, because three of five returned items are relevant. Precision@10 is 5/10, or 0.5. Both metrics use the same hit count. Recall divides by eight known relevant documents, while Precision@k divides by the result cutoff.

Moving document A from rank 1 to rank 5 would leave Recall@5 unchanged because the top-five set still contains three relevant documents. Swapping D (rank 4) with non-relevant G (rank 7) would lower Recall@5 to 2/8, or 0.25, while Recall@10 stays 5/8. Recall@k reacts when a relevant item crosses the cutoff, not when items change positions on the same side of it.

How should you choose k for recall@k?

Choose k from the number of results the product displays or the downstream stage can process, then report nearby cutoffs. Tie k to the product surface or processing limit. Search results, recommendation shelves, reranker pools, and RAG context windows impose different useful depths.

Recall@5 means coverage within five results. Recall@10 means coverage within 10. Neither cutoff is universally correct. A voice interface may depend on the first result, while a legal search workflow may inspect a deeper set. A RAG retriever may fetch more chunks than its generator can accept, then use reranking to reduce that pool.

Report a small curve at operationally meaningful cutoffs instead of choosing k after seeing the best score. Recall@1, Recall@5, and Recall@10 may reveal whether relevant items exist but sit too deep. Larger k can add relevant material, irrelevant material, latency, and context noise. The complete semantic search evaluation workflow connects this cutoff choice to latency and product outcomes.

A meaningful cutoff still depends on a trustworthy denominator. That denominator comes from relevance judgments, not from the retriever’s own scores.

How do relevance judgments affect recall@k?

Recall@k is only as complete and consistent as the relevance set used as its denominator. A test collection needs queries, a stable corpus, and judgments that identify relevant items. Manning, Raghavan, and Schütze note that judged relevant sets may be incomplete when assessors judge pooled results from selected systems. An unjudged document is therefore unknown, not automatically proven irrelevant.

The retrievable unit must remain consistent. Document-level judgments do not directly score a passage-level system when one document becomes many chunks. The evaluator needs chunk judgments, a documented document-to-chunk rule, or another explicit mapping. Filters also alter the eligible corpus. Comparing an unfiltered baseline with a filtered candidate confounds retrieval quality with corpus eligibility.

Binary Recall@k needs relevant and non-relevant labels. Graded judgments may be converted through a declared threshold, but the threshold must stay fixed across runs. Queries with no known relevant items need a predeclared policy because their denominator is zero. A sound search test set and relevance-judgment process prevents that policy from changing after results appear.

How do you average recall@k across queries?

Average Recall@k by calculating one score per eligible query and taking their arithmetic mean. This macro-average gives each information need one vote. Manning, Raghavan, and Schütze use arithmetic means across information needs for ranked retrieval measures. The report must also state how many queries entered the mean and how zero-relevant queries were handled.

Pooling every relevant hit and every relevant judgment before division produces a different measure. That pooled ratio weights queries with larger relevance sets more heavily. It may be intentional, but it is not interchangeable with the usual per-query macro-average. Naming the aggregation rule prevents two teams from reporting different values under the same Recall@k label.

An overall mean can hide failure clusters. Break it out by query intent, language, difficulty, tenant, content type, or other predeclared segment. Report query counts with each slice. A retrieval benchmark becomes comparable only when the corpus snapshot, queries, judgments, cutoff, and aggregation remain fixed.

How does recall@k work for RAG retrieval?

RAG retrieval Recall@k asks whether the top-k chunks cover the reference contexts or facts required by each evaluation query. The denominator may contain reference chunk IDs, relevant documents, or answer-supporting claims. Ragas documentation, read October 8, 2026, distinguishes ID-based context recall from claim-based context recall. These definitions require different annotations and cannot share a score without conversion.

A dataset with exactly one gold chunk per query reduces the per-query calculation to a binary hit: 1 when that chunk appears in the top k, and 0 otherwise. Averaging those hits is often called hit rate or single-target Recall@k. The label should state this special case because ordinary set recall permits several relevant items per query.

Retrieval recall does not measure whether the generator used the evidence faithfully or wrote a correct answer. It measures whether required context survived retrieval. The generator, prompt, ordering, and context capacity create later failure points. RAG evaluation should therefore pair retrieval coverage with answer-level and faithfulness measures inside the broader semantic search evaluation plan.

How does recall@k differ from precision@k?

Recall@k divides relevant hits by all known relevant items, while Precision@k divides the same hits by k returned items. Recall asks how much relevant material the system captured. Precision asks how concentrated the top-k list is with relevant material. Weaviate’s evaluation guide defines both metrics with this shared numerator and different denominators.

The worked example has five relevant hits in the top 10 and eight known relevant documents. Recall@10 is 5/8, or 0.625. Precision@10 is 5/10, or 0.5. At k=5, recall falls to 3/8, or 0.375, while precision rises to three divided by five, or 0.6. The change reflects the cutoff, not a contradiction.

Returning more items often gives recall more opportunities to rise, but those additions may lower precision. Recall alone also treats every position inside k equally. Pairing both metrics exposes coverage and concentration without forcing them into one score. The dedicated Precision@k guide covers its ceiling and denominator in depth.

When should recall@k be paired with MRR or nDCG?

Pair Recall@k with MRR when the first relevant result matters, or with nDCG when positions and graded relevance matter. Recall@k is set-based inside its cutoff. Swapping a relevant document from rank 1 to rank 10 leaves Recall@10 unchanged if the top-10 set contains the same relevant documents.

Mean reciprocal rank uses the reciprocal position of the first relevant result, then averages across queries. It distinguishes a first relevant result at rank 1 from one at rank 5. It does not measure how many later relevant items were recovered. MRR and Recall@k therefore answer complementary questions for RAG and search.

Normalized discounted cumulative gain discounts lower positions and accepts graded relevance. Weaviate’s evaluation guide classifies MRR and nDCG as rank-aware, unlike recall. Use Recall@k for candidate coverage, MRR for first-hit placement, and nDCG for ordered graded utility. No single choice replaces a metric set tied to product behavior.

How do you measure int8 quantization impact on recall@10?

Measure int8 quantization impact by comparing identical Recall@10 runs before and after quantization against the same queries, corpus, filters, judgments, and full-precision baseline. Weaviate’s vector-index documentation, read October 8, 2026, explains that flat search checks all vectors while HNSW builds an approximate candidate list. It also identifies query-time search breadth as a speed-versus-recall control.

Elastic’s “Optimize approximate kNN search performance and accuracy” documentation, read October 8, 2026, describes int8_hnsw as 8-bit quantized HNSW search. The same documentation reports about fourfold lower search RAM than unquantized float vectors, warns that quantization may reduce recall, and states that original vectors remain available for rescoring. This vendor behavior is dated because defaults and controls change.

Two measurements share the Recall@10 name. Relevance Recall@10 compares the returned set with human relevance judgments. ANN recall compares approximate neighbors with the exact full-precision top 10. Quantization can affect ANN overlap without producing the same change in human relevance recall. Report both when engineering and user relevance are separate decision criteria.

1. Freeze evaluation inputs by holding queries, embeddings, corpus snapshot, filters, distance function, retrievable unit, judgments, and k constant.
2. Record a full-precision reference with exact search for ANN Recall@10 and an unquantized production configuration for operational comparison.
3. Run the int8 candidate with candidate breadth and other search settings documented rather than silently retuned.
4. Compare per-query deltas for relevance Recall@10, ANN Recall@10, latency, and memory under the same workload.
5. Test recovery controls separately by changing candidate breadth or full-vector rescoring one factor at a time.

No universal recall loss applies to every embedding model, corpus, or query mix. The controlled experiment supplies the value. Production changes then belong in search-quality monitoring with the exact configuration recorded.

How should recall@k results be reported?

Report Recall@k with its cutoff, query count, relevance policy, corpus snapshot, aggregation rule, comparison baseline, and query slices. These seven fields make a score reproducible. A bare statement such as “recall is 0.8” omits the depth, denominator, test size, and evaluation conditions that give the number meaning.

Include multiple cutoffs only when they match real product stages. Record both the ratio and percentage when different audiences need them, such as 0.625 and 62.5%. Do not compare scores from different datasets as if they measured the same difficulty. Inspect failed queries and segment means because an aggregate improvement may conceal a damaged language, tenant, or query type.

Offline Recall@k also needs latency, resource cost, and an ordering metric when those attributes matter. Online behavior belongs to search metrics and analytics, not inside the offline denominator. Production monitoring cannot recalculate trustworthy recall without refreshed judgments or a stable labelled sample.

Recall@k remains a precise coverage measure when its denominator, cutoff, and test conditions stay explicit. Calculate it per query, average it by a declared rule, and compare systems on frozen inputs. Then use Precision@k, MRR, nDCG, latency, and online evidence to decide whether extra retrieved material improves the search or RAG system.