Home / How it works / Top-k Retrieval: How Many Results Should Semantic Search Return?

Top-k Retrieval: How Many Results Should Semantic Search Return?

Top-k retrieval returns the k items most similar to a query, ranked by score, where k is a number you choose, such as 5, 10 or 100. In semantic search, k decides how many candidates leave each stage of the pipeline. A large first-stage k gives a reranker or RAG system more chances to include the right passage. A small final k keeps results and prompts focused. k interacts with filters, which can leave fewer than k matches, and with approximate indexes, whose search width must be at least as large as k. It also interacts with hybrid fusion, which needs enough candidates from each side. Libraries handle short result sets differently. FAISS, for example, pads missing slots with an ID of -1 and the largest float value. Choose k from measured recall and latency on your own queries, not from habit. In Weaviate, k is the limit parameter on a query.

What is top-k retrieval?

Top-k retrieval is the step that selects the k items with the highest relevance scores for a query and returns them in order. It is how almost every search system decides what to return.

A semantic search compares the query’s vector with stored vectors and ranks items by similarity. Returning everything would be useless, so the system keeps only the top k. In Weaviate, k is the limit on a query; in FAISS it is the k argument to search; in SQL with pgvector it is a LIMIT clause after ORDER BY distance. The concept is the same everywhere. Vector search covers how the scores are produced.

Choosing k looks trivial, but it shapes the whole system in practice. Too small, and the right result can be cut off. Too large, and later stages slow down, users wade through noise, or a language model’s prompt fills with irrelevant text. The right k differs by stage and by use case.

How does k differ between pipeline stages?

k differs between pipeline stages because each stage has a different job: the first stage casts a wide net, reranking sharpens the order, and the final stage shows or sends only the best few. Each needs its own value.

A first stage of vector or hybrid search might return 50 to 200 candidates. A cross-encoder reranker then scores each and reorders them, which costs one model call per candidate, as covered in what is reranking. The final page or prompt keeps a handful. The first-stage k sets a ceiling: if the right passage is not among those candidates, no later stage can recover it.

The ranges above are examples, not rules. Measure recall at several first-stage k values and watch where it levels off; that point, balanced against latency, is a good choice. Semantic search architecture shows the full pipeline.

How do you choose k?

You choose k by measuring recall at k on judged queries for several values, finding where extra results stop adding relevant items, and weighing that against latency and cost. Different stages get different answers.

For the first stage, compute recall at k for k values such as 10, 20, 50 and 100. Recall rises with k and then flattens; past that point, more candidates mostly add cost. For the final stage, think about the consumer. People rarely scan past the first page, and a language model’s answer quality depends on getting the right passages near the top of a short context. Recall at k and precision at k explain the metrics.

For RAG, also count tokens. Each passage adds to the prompt, raising cost and latency and sometimes diluting the answer with marginal context. Start small, measure answer quality, and raise k only if answers miss information that a larger k would have supplied.

Chunk size and k work together. Small chunks need a larger k to cover the same amount of text, while large chunks need fewer but bring more irrelevant sentences with them. When you change chunking, re-measure k, because the old value may no longer fit. Grouping chunks by parent document before counting k, so one long document cannot fill every slot, often gives a fairer result list. Record the k used at each stage in your query logs, so later evaluations compare like with like.

What happens when fewer than k results exist?

When fewer than k items exist or pass a filter, most databases simply return a shorter list, while some libraries such as FAISS fill the missing slots with placeholder values. Code must handle both.

FAISS returns fixed-size arrays, so when it cannot find k neighbours it uses -1 as the ID and the largest 32-bit float as the distance. This happens with tiny indexes, with IVF indexes that probe too few lists, and with filtered searches. Code that uses the IDs directly can then index the wrong row, since -1 often means the last element in Python. Drop entries with ID -1 before using results, and treat a short list as a signal that the index settings or filter may need attention.

Vector databases usually return only real matches, so a filtered query may return three results when ten were requested. That is correct behaviour, but an interface or a RAG prompt should handle short lists gracefully, for example by saying how many matches were found. Very selective filters are covered in filter selectivity.

How does k interact with approximate search?

k interacts with approximate search because the index explores a limited candidate list, and that list must be at least as long as k, and usually longer, to return accurate top-k results. In HNSW this list size is ef.

HNSW search keeps a dynamic list of the best candidates found so far, of size ef. If ef is smaller than k, the search cannot return k good results. Weaviate’s default dynamic ef scales with the requested limit, between a configured minimum and maximum, so larger k automatically widens the search. IVF indexes behave similarly through the number of lists probed. Approximate nearest neighbor search and exact vs approximate search cover the details.

Recall of the index itself also varies with k. An index tuned to find the true top 10 reliably may miss more of the true top 100. Measure ANN recall at the k you actually use, not only at 10.

How does k work with hybrid search and filters?

In hybrid search each leg retrieves its own candidate pool before fusion, so the pool sizes, not just the final k, decide which documents can appear; filters shrink the pool further. Pools should exceed the final k.

A document ranked 30th by keywords and 30th by vectors might deserve a high fused score, but if each leg only returns its top 10, it never reaches fusion. Weaviate retrieves at least 100 candidates per hybrid leg by default, configurable with the QUERY_HYBRID_MAXIMUM_RESULTS setting, so deeper pages stay consistent. How hybrid search works explains fusion.

Filters are applied during the vector search in most databases, so the top k come from items that pass the filter. Post-filtering after retrieval is different: it can return fewer than k items even when many matches exist, as covered in pre-filtering vs post-filtering. Paging with an offset adds further work, since each page needs offset plus k candidates.

Is there an alternative to a fixed k?

The alternatives to a fixed k are a similarity threshold, which keeps items closer than a set distance, and autocut, which cuts the list where scores drop sharply. Both adapt the count to the query.

A threshold returns many results for broad queries and few or none when nothing is close, which suits RAG systems that should say “I don’t know” rather than pass weak context. Thresholds depend on the model and need calibrating, as explained in relevance scores. Weaviate’s autocut, set with auto_limit, groups results by jumps in score and returns that many groups, so a query with three clear matches returns three.

Many systems combine both with a fixed maximum. They return up to k results, stop early at a score jump, and drop anything under a minimum similarity. The similarity search and semantic matching pages describe what the scores mean. The same principles apply whether vectors are dense, sparse or both, as compared in dense vs sparse vectors.

Top-k retrieval returns the k highest-scoring items for a query. Use a larger k at the first stage, so a reranker or RAG system has the right candidate available, and a small k for what users or the model see. Choose each k from measured recall and latency. Watch for short result sets: FAISS pads missing slots with -1 IDs, while databases return shorter lists. Keep ef and hybrid candidate pools larger than k, and consider autocut or a threshold when a fixed count does not suit every query. Related ideas are covered in how semantic search works, distance measures, vector indexes, synonyms, multilingual search and chunking.