Home / Foundations / Sentence-BERT: How Sentence Embeddings Became Fast

Sentence-BERT: How Sentence Embeddings Became Fast

Sentence-BERT (SBERT) is a 2019 method by Nils Reimers and Iryna Gurevych that fine-tunes BERT in a siamese network so each sentence becomes one vector that can be compared with others by cosine similarity. Before it, getting good similarity scores from BERT meant feeding every pair of sentences through the model together, which is far too slow for search. The paper’s example: finding the most similar pair among 10,000 sentences takes about 65 hours with BERT used that way, and about 5 seconds with SBERT. The trick is to encode each sentence once, pool its token vectors into a fixed-size embedding, and train the encoder on sentence pairs so that the embeddings themselves carry the similarity. That design, a bi-encoder, is the basis of modern semantic search. Weaviate’s text2vec-transformers module pools token vectors with masked mean or CLS pooling, and its documentation points to the SBERT paper.

What is Sentence-BERT used for in search?

Sentence-BERT is a modification of BERT that produces semantically meaningful sentence embeddings, fixed-size vectors whose cosine similarity reflects how similar two sentences are. It was published at EMNLP 2019.

The paper, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks”, addressed a gap. BERT, released by Google in 2018, set new records on many language tasks, including judging whether two sentences mean the same thing. But it did so by reading both sentences together, which made it a poor tool for search, where one query must be compared with millions of documents.

SBERT kept BERT’s encoder but changed how it was used and trained. Each sentence goes through the encoder on its own, the token outputs are pooled into one vector, and the network is fine-tuned so those vectors are close for similar sentences. The result is a model whose embeddings can be precomputed, stored and searched.

The work also produced the sentence-transformers library, which became one of the most widely used tools for embeddings. The architecture behind BERT is explained in transformers and BERT in search.

What problem does Sentence-BERT solve?

Sentence-BERT solves the problem that BERT gave accurate similarity scores only for sentence pairs read together, which is too slow for search, while its single-sentence outputs made poor embeddings. It needed to be both fast and accurate.

Reading pairs together is a cross-encoder setup. It is accurate because every word of one sentence can attend to every word of the other. But each comparison needs a full pass through the model. For a collection of 10,000 sentences, comparing all pairs means 10,000 × 9,999 / 2 = 49,995,000 passes, and every new query needs a pass per document. The trade-off is explored in bi-encoder vs cross-encoder.

The obvious shortcut did not work. Taking BERT’s output for a single sentence, either the special CLS token or the average of all token vectors, gave poor sentence embeddings. The paper reports that averaged BERT embeddings performed worse on semantic similarity benchmarks than averaged GloVe word vectors, an older and much simpler method described in word2vec and GloVe. BERT had never been trained to make single-sentence vectors comparable.

How does Sentence-BERT work?

Sentence-BERT works by passing each sentence through the same BERT encoder, pooling the token vectors into one sentence vector, and comparing two sentence vectors with cosine similarity. The two encoder copies share weights.

A siamese network is two identical branches with shared weights. In SBERT, both branches are the same BERT model. Sentence A produces token vectors, which are pooled into vector u; sentence B produces vector v. Because the branches are identical, any sentence can be encoded alone, and its vector can be compared with any other.

Pooling turns a variable number of token vectors into one fixed-size vector. The paper tested three strategies: the CLS token output, the mean of all token outputs and the maximum over each dimension. Mean pooling was the default and performed well. At inference time, the cosine similarity of u and v is the similarity score.

How is Sentence-BERT trained?

Sentence-BERT is trained by fine-tuning the siamese network on labelled sentence pairs, so that the pooled vectors of similar sentences end up close and those of unrelated sentences end up apart. The paper used natural language inference data.

The main training data came from SNLI and MultiNLI, collections of sentence pairs labelled as entailment, contradiction or neutral. For these, SBERT concatenated u, v and the element-wise difference |u − v|, and trained a small classifier on top to predict the label. The gradient flowed back into the shared encoder, shaping its embeddings. For datasets with graded similarity scores, a regression objective on cosine similarity was used, and a triplet objective handled anchor, positive and negative examples.

Later models changed the data more than the design. Training on large collections of question and answer pairs, with contrastive losses that use other examples in the batch as negatives, produced embeddings tuned for retrieval rather than general similarity. That shift is discussed in asymmetric vs symmetric semantic search, and the general idea of a two-tower encoder in bi-encoders.

Why does Sentence-BERT make semantic search fast?

Sentence-BERT makes semantic search fast because documents are encoded once, ahead of time, and each query needs only one encoding plus vector comparisons, instead of one model pass per document. Vector indexes then make the comparisons cheap.

The paper’s own example frames the gain. Finding the most similar pair in 10,000 sentences took an estimated 65 hours with BERT as a cross-encoder and about 5 seconds with SBERT. SBERT needs 10,000 encodings and cheap cosine computations instead of nearly 50 million model passes.

In search the effect compounds. Document embeddings are computed at index time and stored. A query is encoded once, and an approximate nearest neighbour index, such as the HNSW index Weaviate uses by default, finds the closest documents without scanning everything. The cross-encoder still has a role: it can rerank a short list of candidates, as described in cross-encoder rerankers.

Which pooling strategies does Sentence-BERT use?

Sentence-BERT uses mean pooling by default, averaging the output vectors of all tokens, and also supports CLS pooling and max pooling. The choice is part of the model and must match how it was trained.

Mean pooling averages over real tokens only, ignoring padding, which is why it is often called masked mean. CLS pooling takes the output of the special classification token that BERT places at the start of every input. Max pooling takes the largest value in each dimension across tokens. A model fine-tuned with one strategy should be used with the same strategy, or its vectors lose their meaning.

Weaviate exposes this choice in its text2vec-transformers module, where pooling_strategy can be masked_mean, the default, or cls, and the documentation links to the SBERT paper for the background. How tokens relate to the final embedding is covered in tokenization vs embedding.

How has Sentence-BERT shaped later embedding models?

Sentence-BERT has shaped later embedding models by establishing the bi-encoder recipe, a transformer encoder, pooling and contrastive fine-tuning on pairs, that most text embedding models still follow. Its library spread the recipe widely.

The sentence-transformers library made it simple to train and use such models, and thousands of models built with it are shared publicly. Weaviate’s article on choosing a sentence transformer from Hugging Face and its text2vec-transformers module both build on that ecosystem. Weaviate added transformer vectorizers, including Sentence-BERT models, in its 1.2 release.

Successors extended the idea in several directions: multilingual models trained by distillation, smaller and faster encoders, and retrieval models trained on far larger pair collections. Comparing such models is covered in choosing an embedding model, and the step from word vectors to sentence vectors in word vs sentence embeddings.

How do you use a Sentence-BERT model?

You use a Sentence-BERT model by loading it with the sentence-transformers library, encoding texts into vectors and comparing them with cosine similarity, or by letting a vector database run the model for you. Both paths use the same pooling.

from sentence_transformers import SentenceTransformer, util
from weaviate.classes.config import Configure

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")   # mean pooling, 384 dims
emb = model.encode(["A man is playing a guitar.",
                    "Someone plays an instrument.",
                    "The stock market fell today."], normalize_embeddings=True)
scores = util.cos_sim(emb, emb)          # 3 x 3 cosine similarity matrix

# Or let Weaviate run a transformers model in its inference container
vector_config = Configure.Vectors.text2vec_transformers(pooling_strategy="masked_mean")

The snippet was run as an API check with sentence-transformers 5.1.2: the model loaded with mean pooling and produced 384-dimensional vectors and a 3 × 3 similarity matrix. No scores are reported, because the point is the interface, not a result. The Weaviate configuration was built with the Python client 4.18.3.

Fine-tuning on your own pairs follows the SBERT recipe, as described in fine-tuning embedding models.

What are the limits of Sentence-BERT?

Sentence-BERT’s limits are those of any bi-encoder: one vector per text loses some detail, long texts must be truncated or chunked, and similarity is not always relevance. Later work addresses each.

A single vector compresses a whole passage, so fine distinctions can be lost; late-interaction models keep one vector per token instead, as covered in ColBERT and late interaction. Input length is bounded by the encoder, so documents are chunked. And models trained for general similarity may favour related rather than answering passages, a distinction covered in semantic similarity vs relatedness.

The history around SBERT is covered in the vector space model, latent semantic indexing and semantic search at Google. Background topics include what is information retrieval, high-dimensional vector space, the curse of dimensionality, knowledge graphs in search and cache hits and misses.

Sentence-BERT turned BERT into a practical embedding model by encoding each sentence separately with shared weights, pooling the token vectors and fine-tuning on sentence pairs. That made similarity a matter of cosine between precomputed vectors, cutting the paper’s 10,000-sentence example from about 65 hours to about 5 seconds and avoiding 49,995,000 cross-encoder passes. Its bi-encoder recipe, with mean pooling and contrastive training, still underlies most text embedding models and the semantic search they power in vector databases such as Weaviate.