Home / Query understanding / HyDE Explained: Hypothetical Document Embeddings

HyDE Explained: Hypothetical Document Embeddings

HyDE, or Hypothetical Document Embeddings, is a zero-shot retrieval method that searches with an embedding of a generated answer-like passage instead of only embedding the original query. A language model writes one or more hypothetical documents, an encoder turns them into vectors, and the system searches for nearby real documents. The invented passage is an intermediate retrieval representation, not evidence and not the final answer. HyDE can bridge wording differences between short questions and detailed documents without relevance labels. Its tradeoff is extra generation time, cost, and exposure to generator bias. Gao, Ma, Lin, and Callan introduced the method in the paper “Precise Zero-Shot Dense Retrieval without Relevance Labels,” published at ACL in 2023.

What does HyDE stand for, and where did the method come from?

HyDE stands for Hypothetical Document Embeddings, a zero-shot dense retrieval method introduced by Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. The authors submitted the paper to arXiv on December 20, 2022. ACL published it in July 2023 under the title “Precise Zero-Shot Dense Retrieval without Relevance Labels.”

“Zero-shot” describes the intended training setting. HyDE does not require judged query-document pairs to fine-tune its retriever for the target collection. Relevance labels would otherwise tell a model which corpus documents should rank for each training query. The method instead combines an instruction-following generator with an existing contrastively trained encoder.

The generated text has a narrow job. HyDE uses its embedding as a search representation, then returns real corpus documents near that vector. It does not treat invented statements as verified facts. In a retrieval-augmented system, only retrieved source material should enter the evidence context for a later answer generator.

This mechanism changes query understanding before retrieval. It converts a question into a representation shaped more like a relevant document. That distinction explains why direct query embeddings sometimes leave useful evidence outside the nearest neighborhood.

Why can direct query embeddings miss relevant documents?

Direct query embeddings can miss relevant documents when a short question and a detailed answer occupy different neighborhoods in the encoder’s embedding space. A user may supply a few conversational words, while a useful passage contains technical terms, explanations, causes, and remedies. Gao and colleagues frame HyDE as a way to bridge that query-document gap.

The problem is asymmetry, not a claim that every embedding model fails. A query asks for information, while a document expresses that information. Their vocabulary and structure can differ even when their meanings align. The guide to embeddings explains how an encoder maps both forms into numerical representations, while dense retrieval owns the deeper encoder mechanics.

HyDE asks a generator to produce document-shaped language first. That text may add the terminology and explanatory pattern likely to appear in a relevant passage. The encoder then compresses this expanded representation before similarity search. Haystack’s HyDE documentation recommends testing this approach when retrieval recall is weak or the target domain differs from the retriever’s training data.

Exact strings create the opposite case. Product codes, error identifiers, and quoted names may already carry the best literal signal, while generation can alter them. Understanding this divide makes the four-stage HyDE workflow easier to apply safely.

How does HyDE work step by step?

HyDE works in four retrieval steps: generate one or more hypothetical documents, encode them, combine their vectors, and search for nearby real documents. Gao and colleagues use separate generator and encoder models. The generator supplies answer-like language, while the encoder creates the representation used by the retrieval index.

The workflow keeps each stage’s responsibility explicit:

1. HyDE generates one or more hypothetical documents from the query and an instruction suited to the task’s register.
2. HyDE encodes each generated document with the same encoder space used for searchable corpus documents.
3. HyDE combines the generated vectors, usually by averaging them, with optional inclusion of the original query vector.
4. HyDE searches that combined vector against real indexed documents and returns the nearest candidates.

Gao and colleagues describe the output as a retrieval representation. A later RAG retrieval stage may send retrieved real documents to an answer model, but the hypothetical document itself is neither retrieved evidence nor a final response. This boundary prevents fluent invention from acquiring source status.

Implementations vary in how many passages they generate and whether they retain the query vector. The vector formula makes those choices measurable rather than implicit.

How does HyDE build one search vector?

HyDE builds one search vector by averaging N hypothetical-document embeddings, with Gao and colleagues’ Equation 8 optionally adding the original query embedding as one more vector. In plain notation, generated text is d_k = Generate(q), each document vector is v_k = Encode(d_k), and their mean is v_HyDE = (1/N) * sum(v_k).

When the query is included, the formula becomes v_HyDE+q = (Encode(q) + sum(v_k)) / (N+1). Therefore, an unweighted mean gives the query a share of 1/(N+1). The system encodes N generated texts, or N generated texts plus the query. A model may batch those encodings, so vector count does not necessarily equal API-call count.

Defaults are implementation choices, not a universal rule. The authors’ repository defaults to n=8, max_tokens=512, and temperature=0.7, then includes the query in the average. Haystack 3.3 documents five passages without the query. LangChain JavaScript documents one hypothetical answer.

Weaviate’s official hybrid-search documentation supports an original query string for the keyword branch and a supplied vector for the vector branch. That permits hybrid search to preserve literal query terms while using a HyDE vector for semantic retrieval. The next question is why invented details can still yield useful neighbors.

Why can HyDE retrieve well from a false hypothetical document?

HyDE can retrieve well from a false hypothetical document because the generated text supplies relevance patterns while the encoder compresses it into a vector used only to locate real corpus documents. Gao and colleagues call the encoder a dense bottleneck. Its lossy compression can retain broad relevance cues while discarding some unsupported detail.

False details do not become safe or true through embedding. They may still move the search representation toward a useful topical neighborhood, but they can also move a near miss closer. Retrieved documents must provide the evidence, and a later reranker may reorder candidates using a stronger relevance model.

Illustrative example: a small test used the query why does my espresso taste sour. Three hypothetical passages described sour espresso as under-extraction and recommended slower flow or a finer grind. The relevant indexed passage explained that fast water flow causes under-extraction and recommended a finer grind or longer shot. A near miss discussed bitter over-extraction from long shots or overly fine grinds. Other distractors covered descaling, steam-wand purging, bean storage, and basket size.

The following cosine similarities came from normalized all-MiniLM-L6-v2 embeddings in sentence-transformers 5.1.2. They describe one query, not a production benchmark.

Both passages moved closer, so the test demonstrates a neighborhood shift rather than proof of correctness. Controlled benchmark results show whether that shift improves ranked retrieval across judged queries.

How much did HyDE improve retrieval in the original paper?

HyDE raised TREC DL19 nDCG@10 from Contriever’s 44.5 to 61.3, while the supervised Contriever-ft scored 62.1 in Gao and colleagues’ ACL 2023 paper. BM25 scored 50.6 on the same dataset and metric. These are author-reported benchmark results, not guarantees for another corpus.

The central DL19 comparison is compact:

The nDCG@10 metric rewards placing graded-relevance documents near the top ten results. HyDE did not win every comparison. On TREC-COVID, BM25 scored 59.5 and HyDE scored 59.3. The paper’s multilingual tests improved unsupervised mContriever for Swahili, Korean, Japanese, and Bengali, but did not beat fine-tuned mContriever on those four rows.

Generator choice also mattered. Flan-T5, Cohere, and InstructGPT variants scored 48.9, 53.8, and 61.3 on DL19. Adding HyDE moved Contriever-ft from 62.1 to 67.4 and GTR-XL from 69.6 to 71.9. Mixed results make deployment fit more important than one headline score.

When does HyDE help or hurt retrieval?

HyDE helps most when relevance labels are unavailable and query wording differs from corpus language; it hurts when generation latency, domain ignorance, ambiguity, or strong literal identifiers outweigh that benefit. Haystack recommends trying it for weak recall and domains unlike the retriever’s training data. Gao and colleagues also identify novel, unexpected, or emerging queries as a continuing use case.

Selection should follow evidence and system constraints:

The paper does not establish that HyDE solves ambiguity. Its averaging model assumes a uni-modal generated-vector distribution. Samples for two meanings could produce a midpoint that represents neither intent well. A query intent stage can route distinct cases before generation.

Real-time generation adds latency and compute, and generator knowledge or bias can distort the search direction. Sending sensitive queries to an external generator is also an architecture concern. Exact strings may favor keyword search, while a strong supervised retriever may make generation unnecessary. These alternatives differ in what they change and when they operate.

How does HyDE compare with query expansion, query rewriting, and relevance feedback?

HyDE changes the dense search vector with a generated document, while query expansion adds terms or pseudo-documents to the query and ReDE-RF selects real feedback documents instead of generating one. These methods can all alter retrieval, but their outputs, branches, and failure modes are not interchangeable.

The comparison separates related names that are often collapsed:

Wang, Yang, and Wei’s EMNLP 2023 Query2doc work concatenates a pseudo-document with the original query and reports BM25 gains from 3% to 15% on MS MARCO and TREC DL. Jedidi and colleagues’ 2024 ReDE-RF method first retrieves with hybrid sparse-dense search, requests one-token relevance judgments from an LLM, then uses embeddings of selected real documents.

The other names describe different transformations. Query expansion adds cues, query rewriting changes wording, and multi-query retrieval issues several searches. Testing decides which transformation earns its operational cost.

How should a team test HyDE in a search system?

A team should test HyDE against direct dense, keyword, and hybrid baselines on the same judged queries, then compare recall, nDCG, latency, and cost by query segment. The corpus, filters, candidate count, and relevance judgments must stay fixed across variants. Otherwise, a system change can masquerade as a HyDE gain.

A practical evaluation sequence keeps the decision reproducible:

1. Freeze inputs: keep the indexed corpus, access filters, query set, and judgments constant.
2. Run baselines: compare direct dense retrieval, keyword retrieval, hybrid retrieval, and HyDE under matching candidate limits.
3. Measure outcomes: calculate recall and nDCG, then record generation time, retrieval time, and per-query cost.
4. Segment queries: separate conversational language, identifiers, domains, emerging topics, and ambiguous intents.
5. Add guardrails: fall back to direct retrieval on generator timeout and keep hypothetical text outside answer evidence.

Logs should preserve the original query, prompt version, hypothetical text, generator and encoder identifiers, retrieved document identifiers, scores, and stage latency. That record supports offline diagnosis and later production search quality checks. The espresso similarities cannot replace judged evaluation because they describe one illustrative query.

A sound decision comes from evaluating semantic search on representative traffic. Adopt HyDE only for segments where retrieval gains exceed generation latency, cost, privacy exposure, and failure risk. As relevance labels accumulate, compare it again with a supervised retriever.

HyDE is a zero-shot retrieval method that turns generated answer-like text into a search vector for finding real documents. Its value depends on whether that representation closes a query-document gap on judged queries. The practical decision is not whether generated text sounds plausible, but whether HyDE improves retrieval enough to justify generation and its safeguards.