Home / Retrieval models / What Is a Bi-Encoder?

What Is a Bi-Encoder?

A bi-encoder is a model that turns a query and a document into vectors separately, so their similarity can be measured by comparing the two vectors. Because each text is encoded on its own, every document can be embedded once in advance and stored in a vector index. At search time only the query needs encoding, followed by a fast nearest-neighbour lookup. That independence is what makes semantic search fast enough for millions of documents. Most text embedding models used for search, such as Sentence-BERT models, are bi-encoders. The trade-off is accuracy: the query and document never see each other inside the model. Fine interactions such as negation or word order are captured less well than by a cross-encoder, which reads both together. Production search therefore often retrieves with a bi-encoder and reranks the top results with a cross-encoder.

What is a bi-encoder?

A bi-encoder is a neural model architecture that encodes two texts separately into vectors and scores their relationship by comparing those vectors, usually with cosine similarity or dot product. “Bi” refers to the two encoding passes.

The name contrasts it with a cross-encoder, which feeds both texts through one model together. In a bi-encoder, the query never influences how the document is encoded, and vice versa. Each text becomes a point in a shared vector space, and closeness in that space stands for relevance. What are embeddings covers the vectors themselves.

The two passes can use one shared encoder or two separate ones. Sentence-BERT, from Reimers and Gurevych in 2019, uses a single shared network, a siamese design. Dense passage retrieval, from Karpukhin and colleagues in 2020, uses one encoder for questions and another for passages. Both are bi-encoders, because the texts are still encoded independently.

How does a bi-encoder work?

A bi-encoder works by passing each text through a transformer, pooling the token outputs into one vector, and comparing query and document vectors with a similarity function. Documents are encoded ahead of time.

Inside the encoder, a transformer produces a vector for each token, and a pooling step, often the mean of the token vectors or the first token’s vector, reduces them to one vector per text. The model all-MiniLM-L6-v2, for example, outputs 384 numbers per text using mean pooling. Tokenization vs embedding walks through these steps.

At index time, every document is encoded and its vector stored. At query time, the query is encoded once, and a vector index such as HNSW finds the stored vectors closest to it. The model is not involved in the comparison, which is plain arithmetic: a dot product over a few hundred numbers per candidate. That is the essence of dense retrieval.

Why are bi-encoders fast enough for search?

Bi-encoders are fast enough for search because the expensive model runs once per document and once per query, never once per query-document pair. Comparison is then a vector lookup.

The arithmetic is simple. With N documents and Q queries, a bi-encoder needs N plus Q model calls: 1,001,000 for a million documents and a thousand queries, and the document part is paid only once. Scoring every pair with a cross-encoder needs N times Q, a billion calls, and the cost repeats for every new query. Approximate nearest neighbour indexes then make the vector comparison itself fast, as covered in approximate nearest neighbor search.

Updates are cheap too: adding or changing a document needs one new embedding, not a rescoring of every past query. This is why bi-encoders sit at the first stage of most semantic search systems. They narrow millions of documents to a shortlist in milliseconds, and slower, more accurate models can then work on that shortlist.

How are bi-encoders trained?

Bi-encoders are trained with contrastive learning: the model is shown pairs that should match, such as a question and its answer, and learns to place them closer than non-matching pairs. Other examples in a training batch often serve as the non-matching pairs.

Training data usually consists of query-passage pairs from search logs, question-answer collections or paraphrase sets. A common loss treats each pair’s passage as the positive and the other passages in the batch as negatives, pushing the positive closer and the negatives away. Harder negatives, passages that look relevant but are not, make the model sharper. Fine-tuning embedding models covers adapting a bi-encoder to your own data.

Some bi-encoders treat queries and documents differently, for example by adding an instruction or a “query:” prefix to queries only. That helps when short questions must match long passages, the asymmetric case described in asymmetric vs symmetric semantic search. Always follow the model’s own instructions for prefixes. Training data shapes what the model treats as relevant, so a bi-encoder trained on web questions may rank differently from one trained on scientific claims or product titles.

What are the limitations of bi-encoders?

The limitations of bi-encoders come from compressing each text into one vector without seeing the other text: fine details such as negation, numbers and word order can be lost, and rare terms are matched loosely. These are known, fixable weaknesses.

A single vector summarises everything a passage says, so a long passage with several topics gets an averaged representation. Two sentences that differ only in “with” and “without” may land close together. Product codes and rare names may be blurred into nearby but different terms. The model also cannot use the query to decide which part of a document matters, because the document was encoded before the query existed.

Three remedies are common. Hybrid search adds keyword matching for exact terms, as compared in dense vs sparse retrieval. Late-interaction models such as ColBERT keep a vector per token for finer matching, covered in ColBERT and late interaction. A cross-encoder reranks the shortlist, as described next.

How do bi-encoders and cross-encoders work together?

Bi-encoders and cross-encoders work together in a two-stage pipeline: the bi-encoder retrieves a shortlist quickly, and the cross-encoder reorders that shortlist accurately. Each covers the other’s weakness.

A cross-encoder reads the query and one document together, so attention can link every query word to every document word. That catches negation and word order, but it costs one model call per pair. Running it on the top 20 to 100 candidates from a bi-encoder keeps total cost manageable while lifting precision at the top. Bi-encoder vs cross-encoder compares the two in detail.

Weaviate supports this pattern directly. A collection’s vectorizer, or vectors you supply, provides the bi-encoder stage, and a reranker integration applies a cross-encoder to the results of a near_text or hybrid query. The rest of the retrieval landscape, including sparse retrieval and SPLADE, is surveyed in retrieval models explained.

How do you use a bi-encoder in practice?

You use a bi-encoder by embedding your documents once, storing the vectors in a vector database, and embedding each incoming query with the same model before searching. Libraries and database integrations handle most of the work.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")          # a bi-encoder
doc_vectors = model.encode(documents, normalize_embeddings=True)   # once
query_vector = model.encode("get my money back", normalize_embeddings=True)
scores = doc_vectors @ query_vector                        # cosine similarity
top = scores.argsort()[::-1][:5]

This brute-force version suits a few thousand documents. For larger collections, store the document vectors in a vector database, which builds an approximate index and returns the nearest vectors quickly. In Weaviate you can supply the vectors yourself or configure a vectorizer so Weaviate calls the bi-encoder for you at import and query time.

Two rules prevent most mistakes. Use exactly the same model, version and preprocessing for documents and queries, or the vectors will not be comparable. Re-embed every document whenever the model changes, as described in re-embedding when the model changes.

How do you choose a bi-encoder model?

You choose a bi-encoder model by checking its retrieval scores on public benchmarks, its languages, vector size and input length, then testing the best few on your own queries. The best general model is not always best for your data.

Benchmarks such as the retrieval tasks in MTEB give a first ranking, as discussed in the MTEB leaderboard. Check the maximum input length against your chunk sizes, and the vector size against your memory budget. Check language coverage if content is multilingual, and the licence if you plan to host the model yourself. Then run a small evaluation on judged queries from your own users, as covered in choosing an embedding model.

A bi-encoder can also store several vectors per object, for example one for a title and one for a body, as covered in multi-vector search. Agents that call search repeatedly rely on bi-encoders for speed, as described in agentic retrieval and agentic search.

A bi-encoder encodes a query and a document separately into vectors and compares them with cosine similarity or dot product. Because documents are encoded once and reused, search needs only one model call per query plus a fast vector lookup, which scales to millions of documents. Bi-encoders are trained contrastively on matching pairs, and they power most dense retrieval. Their weakness is fine detail, such as negation and rare terms, so production systems pair them with keyword search and a cross-encoder reranker on the shortlist.