Home / Embeddings / Matryoshka Embeddings: Truncatable Vectors Explained

Matryoshka Embeddings: Truncatable Vectors Explained

Matryoshka embeddings are vectors trained so that their first dimensions carry the most information, which lets you cut one embedding to a shorter length and still use it for search. The name comes from Russian nesting dolls: a 1,024-dimensional vector contains a usable 512-dimensional vector, which contains a usable 256-dimensional one, and so on. The technique, Matryoshka Representation Learning, was introduced by Kusupati and colleagues in 2022. It changes only the training loss, so the model costs nothing extra to run. Shorter vectors use less memory and compare faster, at the price of some accuracy. To truncate correctly, keep the first dimensions and normalise the result again. Weaviate’s integrations expose a dimensions setting for Matryoshka-trained models such as OpenAI’s text-embedding-3 family and Snowflake Arctic Embed, served by Weaviate Embeddings.

What are Matryoshka embeddings?

Matryoshka embeddings are embeddings whose leading dimensions form a valid, lower-resolution version of the full vector, so any prefix of the vector can be used as an embedding in its own right. Ordinary embeddings do not have this property.

In a conventional embedding model, information is spread across all dimensions with no particular order. Dropping the last half of the vector throws away an arbitrary half of the information, and search quality falls sharply. In a Matryoshka model, earlier dimensions are trained to hold the broad meaning and later dimensions add detail. Weaviate’s article on the technique compares it to viewing an image at several resolutions.

The practical result is flexibility. One model produces one embedding, and each application chooses how many dimensions to keep: fewer for cheap, fast search over huge collections, more for the highest accuracy. The choice can even differ within one system, with short vectors for a first pass and full vectors for a final check.

Matryoshka embeddings are a property of the model, not of the database. The general background on vectors and their size is covered in what are embeddings and embedding dimensions.

How does Matryoshka representation learning work?

Matryoshka representation learning trains a model with a loss that adds up the training loss computed on several prefixes of the embedding, so every prefix is pushed to be useful on its own. Nothing changes at inference time.

Standard training computes one loss on the full vector. Matryoshka training computes the same loss several times, once on the first 8 dimensions, once on the first 16, and so on up to the full length, then adds them together. Weaviate’s article writes the total as the loss up to 8 dimensions plus the loss up to 16 dimensions and onward to 2,048. To reduce the total, the model must make each prefix work well, so it learns to put the most useful information first.

The technique is cheap to adopt. It works with most loss functions, including the contrastive losses used for retrieval models, and existing models can be fine-tuned to produce Matryoshka vectors. The model size and speed stay the same, because only the training objective changes. Training approaches in general are covered in fine-tuning embedding models.

Sizes between the trained prefixes also work reasonably well. Weaviate’s analysis of OpenAI’s text-embedding-3-large embeddings found that the spread of values per dimension falls in steps at 512, 1,024, 1,536 and 3,072, suggesting those were the trained prefixes. Within each step the information is spread fairly evenly, which explains why intermediate sizes still give gradual gains.

How do you truncate a Matryoshka embedding correctly?

You truncate a Matryoshka embedding by keeping its first d dimensions and dividing the result by its new length, so the shortened vector has unit length again. Skipping the normalisation breaks dot-product search.

Most embedding APIs return unit-length vectors. Cutting off the tail removes part of the length, so the prefix is shorter than 1. Cosine similarity ignores length, but dot product does not, and many systems use dot product on the assumption that vectors are normalised. Normalising after truncation restores that assumption. Weaviate’s article notes this requirement when the dimension is changed after generation.

Providers that support Matryoshka models often do this for you. OpenAI’s text-embedding-3 models accept a dimensions parameter. Weaviate’s article shows that an 8-dimensional result from the API equals the first 8 dimensions of the full 1,536-dimensional vector multiplied by a constant scale factor, so their cosine similarity is 1. The API returns the truncated, renormalised vector.

Never mix models or sizes. Weaviate’s article also shows that the first 1,536 dimensions of text-embedding-3-large are unrelated to text-embedding-3-small embeddings, with a cosine similarity close to 0. Queries and documents must use the same model and the same truncation. Changing either means re-embedding, as covered in re-embedding when the model changes.

Which embedding models support Matryoshka truncation?

Many current embedding models support Matryoshka truncation, including OpenAI’s text-embedding-3 models, Snowflake Arctic Embed and Jina AI’s v3 embeddings, all available through Weaviate integrations. Each model documents which sizes it supports.

Weaviate’s own embedding service offers two Matryoshka-trained models. Snowflake Arctic Embed L v2.0 is a 1,024-dimensional multilingual model that also accepts 256 dimensions. Arctic Embed M v1.5 is a 768-dimensional English model that also accepts 256. Weaviate’s documentation states that using scalar quantization and 256 dimensions provides 99% of unquantized, full-precision performance for these models.

Other integrations expose the same idea. OpenAI’s text-embedding-3-small and text-embedding-3-large take a dimensions setting in Weaviate’s text2vec-openai vectorizer. Weaviate’s 1.27 release added Jina AI’s v3 embeddings with an optional dimensions parameter, noting that the model produces Matryoshka-style embeddings. Jina’s multimodal jina-clip-v2 accepts any size from 64 to 1,024.

Support also depends on the provider’s API. Weaviate’s DigitalOcean integration does not forward a dimensions value, because that endpoint does not accept one, so vectors always arrive at full size. In that case you can truncate and normalise yourself and import the vectors. Choosing among models is covered in choosing an embedding model and the MTEB leaderboard.

What do you lose by using fewer dimensions?

Using fewer dimensions loses some retrieval accuracy, but for Matryoshka models the loss is usually small at moderate truncation and grows as the vector gets very short. The exact cost depends on the model and your data.

Published figures show the scale. OpenAI reported that text-embedding-3-large shortened to 256 dimensions still outperformed its older, unshortened text-embedding-ada-002 at 1,536 dimensions on the MTEB benchmark, as Weaviate’s article relays. Weaviate’s documentation for Arctic Embed states 99% of full-precision performance at 256 dimensions with scalar quantization. Both are vendor-reported benchmark results, not guarantees for your data.

The loss is not uniform. Fine distinctions, such as between near-duplicate documents or closely related technical topics, depend on detail held in later dimensions. Broad topical matches survive heavy truncation better. Domain-specific collections may lose more than general benchmarks suggest, as discussed in domain-specific embeddings.

Test before committing. Build a set of real queries with known relevant results, run them at full size and at the candidate sizes, and compare recall at the depth your application uses. Pick the smallest size whose quality you accept.

How much memory do shorter embeddings save?

Shorter embeddings save memory in direct proportion to the dimensions removed, so cutting 1,024 dimensions to 256 reduces raw vector storage by a factor of four. Combining truncation with quantization multiplies the saving.

The arithmetic is simple. A float32 value takes 4 bytes, so one million 1,024-dimensional vectors take 1,000,000 × 1,024 × 4 bytes, which is 4.096 GB. At 256 dimensions the same collection takes 1.024 GB. Scalar quantization to 8 bits stores 1 byte per value, bringing 256 dimensions to 0.256 GB, a sixteenth of the starting size.

Speed improves too. Distance calculations scale with the number of dimensions, so shorter vectors compare faster and more of them fit in processor caches. The graph index itself also takes memory, which truncation does not reduce. Full sizing is covered in the vector storage and memory calculator.

How does adaptive retrieval use Matryoshka embeddings?

Adaptive retrieval uses short Matryoshka prefixes to find a candidate shortlist quickly, then rescores only those candidates with the full-length vectors to restore accuracy. The original Matryoshka paper proposed this two-stage approach.

The idea mirrors how vector databases handle compression. A compressed or shortened index finds the neighbourhood fast, and a more precise comparison orders the final few. Because the short vector is a prefix of the full one, no second model is needed: both stages use the same embedding.

In Weaviate, this can be built with two named vectors on each object, one short and one full. The search runs on the short vector, returns a shortlist with the full vectors attached, and the application reorders the shortlist by full-length similarity. Weaviate’s own compression features, covered in embedding quantization, apply the same shortlist-then-rescore idea inside the database.

How do you use Matryoshka embeddings in Weaviate?

You use Matryoshka embeddings in Weaviate either by setting dimensions on a supported vectorizer, or by truncating vectors yourself and storing a short and a full named vector. The second option enables adaptive retrieval.

import numpy as np
from weaviate.classes.config import Configure, Property, DataType

# Option 1: let the vectorizer return 256-dimensional Matryoshka vectors
client.collections.create(
    "DocShort",
    vector_config=Configure.Vectors.text2vec_weaviate(
        model="Snowflake/snowflake-arctic-embed-l-v2.0", dimensions=256),
    properties=[Property(name="text", data_type=DataType.TEXT)],
)

# Option 2: store a short and a full vector, shortlist on short, rescore on full
def truncate(v, d):
    v = np.asarray(v[:d], dtype=np.float32)
    return v / np.linalg.norm(v)              # normalise again after cutting

docs = client.collections.use("Doc")          # named vectors: "short" (256), "full" (1024)
res = docs.query.near_vector(truncate(q_full, 256).tolist(), target_vector="short",
                             limit=100, include_vector=["full"])
ranked = sorted(res.objects, key=lambda o: -np.dot(q_full, o.vector["full"]))[:10]

Option 1 keeps everything inside Weaviate. Option 2 stores both lengths and does the final ordering in the application. Option 2 was run on Embedded Weaviate 1.30.5 with random unit vectors as an API check only; the target document came first after rescoring, which shows the code path works, not that it improves quality.

Embedding large collections at either size benefits from batching, as described in batch embedding. Query and document prompts still matter for models that use them, as covered in asymmetric vs symmetric semantic search.

How do Matryoshka embeddings compare with quantization?

Matryoshka truncation reduces the number of dimensions, while quantization reduces the bits stored per dimension, so the two are independent and can be combined. Each has a different accuracy profile.

Truncation removes information outright, but in the order the model chose, least important last. Quantization keeps every dimension at lower precision, which adds a little noise everywhere. Weaviate’s 1.27 release notes suggest combining the Jina v3 dimensions setting with quantization to manage resources, and its Arctic Embed figure of 99% already combines 256 dimensions with scalar quantization.

Matryoshka training is a model feature and must be present from the start. Quantization works with any model. Related topics include multimodal embeddings, where vectors are often large, tokenization vs embedding and word vs sentence embeddings.

Matryoshka embeddings let one vector serve at several lengths, because training adds the loss on each prefix and pushes the most information into the first dimensions. Truncate by keeping the first dimensions and normalising again, and never mix sizes or models in one index. Supported models include OpenAI’s text-embedding-3 family, Snowflake Arctic Embed and Jina v3, all usable through Weaviate’s dimensions setting. Cutting 1,024 dimensions to 256 cuts raw storage four times, and 8-bit quantization cuts it four times more. Test the accuracy on your own queries, and use a short shortlist with full-length rescoring when you need both speed and precision.