Home / Search evaluation / Precision@k vs Recall@k

Precision@k vs Recall@k

Precision@k is the share of the top k search results that are relevant, while recall@k is the share of all relevant items that appear in the top k. Precision asks how clean the result list is; recall asks how complete it is. The two pull in opposite directions as k grows: recall can only rise or stay the same, while precision usually falls once the relevant items run out. Neither metric looks at order inside the top k, so a relevant result at rank 1 counts the same as one at rank k. A person scanning a short result page notices irrelevant results, so precision at a small k matters. A RAG pipeline that passes passages to a language model needs the right passage to be present, so recall at the context size matters most, with precision keeping noise down. Reporting both, at a k that matches real use, gives the clearest picture.

What does precision@k measure in search evaluation?

Precision@k measures the fraction of the first k results that are relevant to the query. It answers how much of what the user sees in the top k is worth seeing.

Weaviate’s article on evaluation metrics defines precision@k as the number of relevant items within the top k results divided by k. It describes the metric as not rank-aware: it counts relevant results but ignores their order. A list with relevant items at ranks 1 and 2 and one with relevant items at ranks 3 and 4 have the same precision at 4.

Precision needs relevance judgments, a record of which documents are relevant to each test query. Without them there is nothing to count against. How judgments are collected is covered in search test sets and relevance judgments.

The k matters as much as the metric. Precision at 1 asks whether the single top result is right. Precision at 10 asks how much of the first page is useful. Each answers a different product question. Precision on its own also misses half the picture, which is why it is almost always paired with recall. The wider metric family is surveyed in information retrieval metrics.

How does recall@k differ from precision@k?

Recall@k differs from precision@k by dividing the relevant results in the top k by the total number of relevant items, not by k. It measures coverage rather than cleanliness.

Weaviate’s evaluation article defines recall@k as the number of relevant items within the top k divided by the total number of relevant items in the dataset. Like precision, it is not rank-aware. The two metrics share a numerator, the relevant results found, and differ only in the denominator.

That small difference changes what each rewards. A system that returns one perfect result and stops has high precision but low recall when several relevant items exist. A system that returns many results, including every relevant one, has high recall but low precision. Neither number alone tells you whether the system is good.

Recall depends on knowing how many relevant items exist, which is often uncertain. Precision only needs judgments for the returned results. That makes precision easier to measure on large collections and recall more dependent on thorough judging. Recall is covered in depth in recall at k. A worked example shows both on the same list.

How do you calculate precision@k and recall@k step by step?

You calculate precision@k and recall@k by counting the relevant results in the top k, then dividing by k for precision and by the total relevant count for recall. The example comes from Weaviate’s evaluation article.

The query is “sweet pastry”. The relevant items are donut, muffin and scone. The system returns donut, bagel, muffin and croissant, in that order. Two of the four results are relevant, so precision at 4 is 2 divided by 4, which is 0.5. Two of the three relevant items were found, so recall at 4 is 2 divided by 3, about 0.667. Weaviate’s article reports the same values from pytrec_eval.

Stepping through the cut-offs shows how each behaves. At k = 1, donut is relevant: precision 1, recall 0.333. At k = 2, bagel is not: precision 0.5, recall unchanged. At k = 3, muffin is relevant: precision 0.667, recall 0.667. At k = 4, croissant is not: precision 0.5, recall unchanged.

Scone was never returned, so recall never reaches 1. More results could raise it, but each extra irrelevant result would lower precision. That tension is the central trade-off between the two metrics.

Why do precision and recall trade off as k grows?

Precision and recall trade off as k grows because each extra result can only add to recall, while it lowers precision whenever it is not relevant. Once the relevant items are used up, every new result is a miss.

Recall at k never decreases as k grows, because a larger cut-off includes everything a smaller one did. Precision has no such guarantee. It goes up when the next result is relevant and down when it is not. In most systems relevant results cluster near the top, so precision tends to fall as k increases.

The number of relevant items caps precision. With three relevant items, precision at 10 can be at most 3 divided by 10, which is 0.3, even for a perfect system. In the example, pytrec_eval reports precision at 10 of 0.2, because the run had only four results and two relevant ones, and the standard definition still divides by 10. Short result lists therefore score low precision at large k.

The cap means precision@k values are only comparable across queries with similar numbers of relevant items. A query with one relevant document and a query with fifty play by different rules. This is one reason rank-aware metrics normalize by the best possible score, as nDCG does in nDCG explained. Some single-number summaries try to balance precision and recall directly.

How do F1@k and R-precision combine precision and recall?

F1@k combines precision and recall as their harmonic mean, and R-precision sets k to the number of relevant items so that precision and recall become equal. Both give one number instead of two.

F1@k is 2 times precision times recall, divided by precision plus recall. In the example, precision at 4 is 0.5 and recall at 4 is about 0.667, so F1 at 4 is 2 × 0.5 × 0.667 / 1.167, about 0.571. The harmonic mean punishes imbalance: a system cannot score well by maximizing one metric and ignoring the other.

R-precision avoids choosing k at all. For a query with R relevant items, it measures precision at rank R. Because precision and recall share a numerator and both denominators equal R at that point, the two metrics are equal there. In the example, R is 3, and precision at 3 is about 0.667, which pytrec_eval reports as the R-precision. The measure is part of the trec_eval tool used in TREC evaluations.

Single numbers hide detail. F1 does not say whether the problem is noise or gaps, and R-precision uses a different k for every query. Report the components alongside any combined score. The right k for the components comes from how results are consumed.

Which one should you optimise for search versus RAG?

You should optimise precision at a small k for result pages people read, and recall at the context size for RAG, while keeping precision high enough that noise does not mislead the model. The consumer decides the priority.

A person reading a result page sees only the first few results and loses trust with each irrelevant one. Precision at the number of visible results, often 5 or 10, tracks that experience. Recall beyond the visible page matters less, since few users scroll far.

A RAG pipeline is different. If the passage that holds the answer is not in the context, the model cannot use it, and no amount of precision helps. Recall at the number of passages sent, such as 5, is the first test, as covered in reranking in RAG. Precision still matters, because irrelevant passages can distract the model or crowd out useful ones.

A single shown answer, such as a featured result or a voice reply, fits precision at 1, or MRR, which also rewards how early the first good result appears, as described in mean reciprocal rank. Whatever the choice, the metrics must be computed the same way every time.

How do you compute precision@k and recall@k in Python?

You compute precision@k and recall@k in Python with pytrec_eval, passing the relevance judgments and the system’s scored results and asking for the P and recall measures at each cut-off. It reproduces trec_eval’s definitions.

import pytrec_eval

qrels = {"sweet pastry": {"donut": 1, "muffin": 1, "scone": 1}}
run = {"sweet pastry": {"donut": 0.95, "bagel": 0.9, "muffin": 0.8, "croissant": 0.7}}

evaluator = pytrec_eval.RelevanceEvaluator(qrels, {"P.1,3,4,10", "recall.4", "Rprec"})
scores = evaluator.evaluate(run)["sweet pastry"]
print(scores["P_4"], scores["recall_4"], scores["Rprec"])   # 0.5 0.666... 0.666...

The qrels hold the judgments and the run holds the system’s scores, which pytrec_eval sorts into a ranking. The measure names take comma-separated cut-offs. The code was run as a check and returns the values in the comment, matching Weaviate’s article for precision and recall at 4.

Averages across queries are usually plain means of the per-query values, sometimes called macro-averages. Report the k, the number of queries and how queries with no relevant items were handled. Weaviate’s evaluation article walks through the same library for each metric. The full evaluation workflow is in how to evaluate semantic search. Precision also behaves differently from what users call relevance, a gap worth understanding.

What do precision@k and recall@k miss?

Precision@k and recall@k miss the order of results within the top k, the degree of relevance, and anything users never judged. They count hits; they do not weigh them.

Order is the first gap. Two systems can tie on precision at 10 while one puts its relevant results at ranks 1 to 3 and the other at ranks 8 to 10. Users notice the difference immediately. Rank-aware metrics such as MRR and nDCG capture it, which is why they are usually reported alongside.

Graded relevance is the second. Precision treats a perfect answer and a marginal one the same, once both pass the relevance cut-off. Unjudged documents are the third: most tools count them as irrelevant, which penalizes systems that find good documents the judges never saw. The broader distinction is discussed in relevance vs recall.

Engineering metrics are separate again. ANN recall measures whether an approximate index finds the true nearest neighbours, not whether results are relevant, as explained in ANN recall. Public benchmarks and production signals fill other gaps, as covered in retrieval benchmarks, online search metrics and analytics, monitoring search quality in production and why vector search returns irrelevant results.

Precision@k is the relevant share of the top k results, and recall@k is the share of all relevant items found in the top k. In Weaviate’s pastry example they are 0.5 and about 0.667 at k = 4. Recall can only rise with k while precision usually falls, and precision is capped when relevant items are fewer than k. Use precision at a small k for result pages, recall at the context size for RAG, and pair both with a rank-aware metric. Report the k, the definitions and the averaging so results can be compared.