Home / Retrieval models / SPLADE Explained: Learned Sparse Retrieval

SPLADE Explained: Learned Sparse Retrieval

SPLADE is a learned sparse retrieval model that turns a query or document into a weighted list of vocabulary terms, including related terms absent from the text. Matching then works through an inverted index, like keyword search. The name stands for Sparse Lexical and Expansion model. It uses the masked language model head of BERT to score every term in a vocabulary of about 30,000 word pieces, then keeps only a small set of non-zero weights. A document about a “refund for a broken item” can gain weight on “return” and “money”, so a query that shares none of its words still matches. Learned sparse retrieval sits between BM25, which only counts the words present, and dense embeddings, which match meaning in a vector space. Weaviate’s hybrid search uses BM25F for its sparse side, and SPLADE output can be added through document expansion or fused with Weaviate results.

What is learned sparse retrieval?

Learned sparse retrieval is a family of retrieval methods in which a neural model assigns weights to vocabulary terms for each query and document, producing sparse vectors that run on an inverted index. The model learns which terms matter and which related terms to add.

Classic sparse retrieval, explained in sparse retrieval, weights terms with fixed formulas. BM25 counts how often a word appears and how rare it is. Its vector has non-zero values only for words present in the text, so a missing synonym contributes nothing. Weaviate’s article on hybrid search explains that sparse vectors have mostly zero values with only a few non-zero values, while dense vectors mostly contain non-zero values.

Learned sparse models replace the formula with a trained model. Some only re-weight the words present. DeepCT, from Zhuyun Dai and Jamie Callan in 2019, learned term importance from context. Others also expand the text. Doc2query generated likely questions and appended them to documents before indexing.

SPLADE does both in one step: it re-weights present terms and adds expansion terms, all as one sparse vector. The approach keeps every advantage of an inverted index, including fast lookup and readable matches. The overview of retrieval families is in retrieval models explained. The encoding step is where SPLADE differs from BM25.

How does SPLADE encode text into a sparse vector?

SPLADE encodes text by passing it through BERT’s masked language model head, which scores every vocabulary term at every token position, then applying log(1 + ReLU) and taking the maximum over positions. The result is one weight per vocabulary term.

BERT was pretrained to predict masked words, so its output head already knows which words fit a context. For each token, it gives a score for every term in its vocabulary, about 30,000 word pieces. SPLADE reuses this head. A token such as “refund” produces high scores for “refund” itself and for related words such as “return” or “money”.

Two operations make the output sparse and stable. The ReLU sets negative scores to zero, and the logarithm of one plus the score dampens very large values. SPLADE v2, from Thibault Formal and colleagues in 2021, then takes the maximum over all token positions for each term. The original SPLADE paper summed over positions instead.

Training adds a sparsity penalty. SPLADE uses the FLOPS regulariser, which penalises terms that are active in many documents, because those make postings lists long and queries slow. The model is trained on query and passage pairs so that relevant pairs score higher than irrelevant ones. Scoring two such vectors is simple.

How does SPLADE score a query against a document?

SPLADE scores a query against a document with the dot product of their sparse vectors, summing the product of the two weights for every term that is non-zero in both. Terms present on only one side add nothing.

In the example, the query “get my money back” and the document “refund for a broken item” share no words. BM25 gives them a score of 0. The illustrative SPLADE vectors share two terms: the document has “money” as an expansion term, and the query has “refund” as one. The dot product adds 1.0986 × 0.5878 for refund, about 0.6457, and 0.4055 × 0.7885 for money, about 0.3197, giving 0.9654.

The weights come from log(1 + x) applied to illustrative model scores, for example log(1 + 2.0) ≈ 1.0986 for “refund” in the document. A real model would produce different numbers and many more non-zero terms. The mechanism is the same: matching depends on shared terms, but the model decides which terms each side contains.

Because the score is a sum over shared terms, an inverted index can compute it efficiently. The engine fetches the postings for each non-zero query term and accumulates weights. The problem this solves is described in the vocabulary mismatch problem. The comparison with dense models follows naturally.

How does SPLADE compare with BM25 and dense retrieval?

SPLADE compares with BM25 and dense retrieval as a middle path: it eases vocabulary mismatch like dense models, while keeping exact-term matching and an inverted index like BM25. Each method has its own costs.

BM25 needs no model, indexes quickly and handles rare exact terms such as product codes well. It fails when the query and document use different words. Dense retrieval, explained in dense retrieval, embeds whole passages as vectors and matches meaning, but can blur rare terms and gives no readable reason for a match. The trade-offs are compared in dense vs sparse retrieval.

SPLADE expands both sides with related terms, so it handles many synonym cases. Its vectors stay interpretable: every match can be explained by the terms and weights that produced it. It uses the same transformer encoders as bi-encoders, encoding queries and documents separately, so documents can be indexed ahead of time.

Costs differ too. SPLADE needs a model at index time and usually at query time. Its expanded vectors contain more terms than BM25’s, which makes postings lists longer and queries slower than plain BM25. Some variants drop the query encoder and use the query’s own terms, which removes model latency at query time at some cost in quality. Other learned sparse models make different choices.

Which other learned sparse models are there?

Other learned sparse models include DeepCT for term re-weighting, doc2query and docTTTTTquery for document expansion, and uniCOIL, which combines expansion with learned weights. They differ in whether they re-weight, expand or both.

DeepCT learns how important each present term is in context, then writes those weights into a standard index. It cannot add new terms. Doc2query and its successor docTTTTTquery, from Rodrigo Nogueira and colleagues in 2019, generate questions a document might answer and append them to the text, so the index gains new terms. They leave weighting to BM25.

uniCOIL, from Jimmy Lin and Xueguang Ma in 2021, learns a single weight per term on top of an expansion step. SPLADE learns expansion and weighting jointly through the masked language model head, which is why it became a common reference point for the family.

All of these sit apart from late-interaction models, which keep one vector per token, covered in ColBERT and late interaction and multi-vector search. Learned sparse models keep one sparse vector per text. The encoding takes only a few lines.

How do you compute SPLADE vectors in Python?

You compute SPLADE vectors in Python by loading a SPLADE checkpoint as a masked language model, applying log(1 + ReLU) to its output scores and taking the maximum over tokens. The non-zero entries are the sparse vector.

import torch
from transformers import AutoTokenizer, AutoModelForMaskedLM

name = "naver/splade-cocondenser-ensembledistil"   # a public SPLADE checkpoint
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForMaskedLM.from_pretrained(name)

def splade(text):
    batch = tok(text, return_tensors="pt", truncation=True)
    with torch.no_grad():
        logits = model(**batch).logits                       # [1, tokens, vocab]
    weights = torch.log1p(torch.relu(logits))
    weights = weights * batch["attention_mask"].unsqueeze(-1)   # ignore padding
    vec = weights.max(dim=1).values.squeeze(0)               # max over tokens
    idx = vec.nonzero().squeeze(1)
    return {tok.convert_ids_to_tokens(int(i)): float(vec[i]) for i in idx}

def score(q, d):
    return sum(w * d.get(t, 0.0) for t, w in q.items())

The function returns a dictionary from vocabulary term to weight, which makes expansions visible: printing the top entries of a document’s vector shows which related terms the model added. The score function is the sparse dot product. The code follows the SPLADE v2 formula and the standard transformers API. It was not run, since no SPLADE checkpoint was available locally.

Production systems store these vectors in an engine that supports weighted sparse postings, rather than scoring in Python. The vocabulary is fixed by the tokenizer, so rare product codes split into word pieces, which can weaken exact matching for them. Fitting SPLADE into a Weaviate pipeline needs a choice of pattern.

How can you use SPLADE with Weaviate?

You can use SPLADE with Weaviate by writing its top expansion terms into a text property for BM25F, or by fusing a separate sparse index with Weaviate results. Weaviate’s own hybrid search pairs BM25F with vector search.

Weaviate’s article on hybrid search names BM25 and SPLADE as ways to produce sparse embeddings, and notes that its hybrid search implementation uses BM25 or BM25F with vector search. The 1.17 release article gives the motivating example: in “How to catch an Alaskan Pollock?”, dense vectors read “catch” as fishing, while sparse methods capture the entity “Alaskan Pollock”.

The document-expansion pattern is simple. For each document, compute its SPLADE vector, take the strongest terms not already in the text, and store them in an extra property such as expansion_terms. BM25F then matches queries against those terms too, and property weights control their influence. The weights themselves are lost, so this captures expansion but not learned weighting.

The fusion pattern keeps the full model. A separate sparse index returns its ranking, Weaviate returns a hybrid or vector ranking, and the application merges the two with reciprocal rank fusion. This adds an index to operate, so it suits teams that have measured a clear gain. Agent-driven pipelines that pick a retriever per query are covered in agentic retrieval and agentic search. Choosing whether to adopt SPLADE at all depends on the data.

When should you use SPLADE?

You should use SPLADE when keyword search misses relevant documents because of vocabulary mismatch, but you still need exact-term matching, an inverted index and explainable matches. Hybrid search is the main alternative.

Good fits include collections where users and authors use different words, such as support content, and settings where explaining a match matters, such as legal or regulated search. SPLADE is also attractive where an inverted index is already in place and adding a vector index is not.

Hybrid search with BM25F and a dense model often reaches a similar goal with standard tooling, as explained in hybrid search. A reranker on top improves the final order regardless of the first stage, as covered in cross-encoders and bi-encoder vs cross-encoder.

Measure on your own queries before committing. Compare BM25, BM25 with SPLADE expansion terms, and hybrid search on judged relevance, along with index size and query latency. If hybrid search already closes the gap, an extra model may not be worth running.

SPLADE is a learned sparse retrieval model that turns text into weighted vocabulary terms, adding related terms that never appeared, and scores matches with a sparse dot product on an inverted index. It applies log(1 + ReLU) to BERT’s masked language model scores and takes the maximum over tokens, with a FLOPS penalty to keep vectors sparse. In the illustrative example, a query and document with no shared words still score 0.9654 through expansion terms, where BM25 scores 0. SPLADE eases vocabulary mismatch while keeping exact matching and explainability. With Weaviate, add its expansion terms to a BM25F property or fuse a separate sparse index with Weaviate’s hybrid results.