Home / Embeddings / Embedding Dimensions: What Determines Them and Does Size Matter?

Embedding Dimensions: What Determines Them and Does Size Matter?

An embedding dimension is one of the numbers in an embedding vector, and the dimension count, such as 384, 768, or 1536, is how many numbers the model outputs for each piece of text. The count is fixed by the model’s design, usually the width of its final layer. More dimensions give the model more room to encode meaning, but they also raise memory and slow comparisons in proportion. A million 1536-dimension vectors take about 6 GB in float32, while a million 384-dimension vectors take about 1.5 GB. More dimensions do not guarantee better search. OpenAI reports that its text-embedding-3-large model, shortened to 256 dimensions, still outperforms its older 1536-dimension model on the MTEB benchmark. The right size is the smallest one that holds quality on your own queries.

What is an embedding dimension?

An embedding dimension is a single position in an embedding vector, and the embedding’s dimensionality is the total number of positions. A 384-dimensional embedding is a list of 384 numbers that together represent the meaning of a piece of text.

Each dimension is one axis of the space the model learned. A point in that space is one embedding. Texts with similar meaning land near each other, so search becomes a question of distance between points. The embeddings guide explains how that space is learned.

Individual dimensions rarely mean anything a person could name. Meaning is spread across many of them at once, so no single number stands for “sports” or “price.” What matters is the vector as a whole and its distance to other vectors.

The same word, dimension, also describes vectors in general. A vector dimension is just the length of the list. In vector databases, every vector in a collection must have the same dimension count, because distances are only defined between vectors of equal length. That rule shapes what happens when a model, and therefore its dimension count, changes. The count itself comes from the model.

What determines embedding dimensions, and can you change them?

The number of dimensions in a vector embedding is determined by the embedding model’s architecture, usually the width of its final layer or a projection layer added on top. The user does not choose it per document, though some models offer a setting to shorten the output.

Transformer models carry a hidden size through every layer. BERT, from Devlin and colleagues in 2018, uses a hidden size of 768 in its Base version and 1,024 in its Large version. Embedding models built on such encoders often output vectors of the same width, which is why 768 and 1,024 are so common. Other models add a projection layer that maps the final state to a chosen size.

The choice reflects a trade-off made during model design. Wider layers can represent more distinctions but cost more to train and run. Smaller models, such as compact sentence encoders, often output 384 dimensions to stay fast.

Some newer models let the caller pick the output size. OpenAI’s text-embedding-3 models accept a dimensions parameter, and Weaviate’s Jina AI integration exposes a similar setting for Jina’s v3 model. These models were trained so that shorter outputs remain useful, a method covered in the section on shortening embeddings. The resulting sizes cluster around a few familiar numbers.

What do common sizes like 384, 768, and 1536 dimensions mean?

Common sizes such as 384, 768, 1024, 1536, and 3072 dimensions each come from a family of models, and each size sets how much memory every vector needs. A float32 dimension takes 4 bytes, so the arithmetic is direct.

The table counts vector storage only. An index such as HNSW adds graph links on top. Weaviate’s compression benchmark gives a matching figure: one million 1536-dimension vectors take 5,859 MB uncompressed, which is the same 6.14 GB measured in binary megabytes.

A 384-dimensional vector therefore costs a quarter of a 1536-dimensional one. Weaviate’s training material makes the general point with two real models. A 4096-dimension model needs about 16 KB per vector and a 384-dimension model about 1.5 KB. A roughly tenfold difference in dimensions becomes a tenfold difference in memory. Choosing an embedding model compares model families. Memory is only one of the costs.

How do embedding dimensions affect memory and speed?

Embedding dimensions affect memory and speed linearly: doubling the dimensions doubles the bytes per vector and roughly doubles the work of every distance calculation. At scale, that difference decides hardware size and cost.

Weaviate’s resource planning guide calls reducing dimensionality the most effective single way to cut memory, and gives the example that a 384-dimension model uses far less memory than a 1536-dimension one. Memory matters because graph indexes such as HNSW keep vectors in RAM for fast access. A collection that fits in memory at 384 dimensions may need several machines at 3072.

Speed follows the same arithmetic. Each comparison between a query and a stored vector multiplies and adds every dimension. An approximate index reduces how many comparisons run, but not the cost of each one. Embedding time also grows with model size, which affects how quickly new content can be indexed.

Compression changes the picture without changing the model. Weaviate’s benchmark on one million 1536-dimension vectors shows 8-bit quantization cutting 5,859 MB to 1,465 MB while keeping recall between 97 and 99.9 percent. Embedding quantization covers the options. The harder question is whether the extra dimensions buy better results.

Do more dimensions mean better search quality?

More dimensions do not by themselves mean better search quality; the model’s training matters far more than its width. A well-trained small model often beats a larger, older one.

OpenAI’s own results show this clearly. As reported in OpenAI’s documentation and summarized in Weaviate’s analysis, a text-embedding-3-large embedding shortened to 256 dimensions still outperforms an unshortened text-embedding-ada-002 embedding with 1,536 dimensions on the MTEB benchmark. One-sixth of the dimensions, from a newer model, beat the full size of an older one.

Benchmarks across many models point the same way. Muennighoff and colleagues’ MTEB study, published at EACL 2023, found that no single embedding method dominates across all tasks. A model’s size, dimension count, and training data all interact, and the best choice depends on the task.

Within one model family, larger outputs usually do help a little, which is why providers offer both small and large versions. The gain shrinks as dimensions grow, while the memory cost keeps rising in a straight line. That trade is why shortening embeddings has become common.

How can you shorten embeddings without retraining?

You can shorten embeddings without retraining when the model was trained with Matryoshka representation learning, which packs the most important information into the first dimensions. Cutting the end off such a vector keeps most of its meaning.

Kusupati and colleagues introduced the method in their 2022 paper, “Matryoshka Representation Learning.” It trains a model so that its embeddings work at several nested lengths, coarse to fine. The paper reports up to 14 times smaller embeddings at the same accuracy on ImageNet-1K classification, and up to 14 times faster large-scale retrieval.

OpenAI’s text-embedding-3 models use this idea. Their documentation says developers can remove numbers from the end of an embedding through the dimensions parameter without losing its concept-representing properties. Weaviate’s analysis of these models adds two cautions. A shortened vector should be normalized again before cosine comparison. And embeddings from different models, such as 3-small and 3-large, are not compatible even when cut to the same length.

Shortening only works for models trained for it. Truncating an ordinary embedding throws away information spread across all dimensions and usually hurts quality badly. For other models, quantization reduces memory instead, by storing each dimension in fewer bits. Matryoshka embeddings covers the technique in depth. With these options, picking a size becomes a measurable choice.

What is a good embedding size?

A good embedding size is the smallest one that keeps search quality acceptable on your own queries, which for many applications falls between 384 and 1,536 dimensions. The answer comes from a test, not a rule.

A practical way to choose:

1. Estimate the collection size and multiply by bytes per vector to see the memory each option needs.
2. Shortlist two or three models, including a compact one and a larger one.
3. Test each on judged queries, and try shortened sizes for Matryoshka models.
4. Pick the smallest size whose quality is within an acceptable margin of the best.
5. Add quantization if memory is still the limit, and check recall again.

The sequence often lands on a smaller size than expected. A compact model may be enough for short product descriptions, while long technical documents may benefit from a larger one. The MTEB leaderboard helps build the shortlist, and the evaluation guide covers the test.

Two constraints are easy to miss. Every vector in a collection must share one dimension count, so changing size means re-embedding everything, as re-embedding when the model changes explains. And some compression settings depend on dimensions: Weaviate’s product quantization requires the segment count to divide the dimension count evenly. A final pair of common questions concerns large language models.

What are the embedding sizes of GPT-3 and GPT-4?

GPT-3’s largest model uses a hidden size of 12,288, while OpenAI has not published GPT-4’s internal dimensions; for search, OpenAI provides separate embedding models of 1,536 and 3,072 dimensions. The language model and the embedding model are different things.

Brown and colleagues’ 2020 paper on GPT-3, “Language Models are Few-Shot Learners,” lists the model sizes in a table. The 175-billion-parameter model has a model dimension of 12,288. That is the width of its internal representations, not a vector offered for search.

OpenAI has not published GPT-4’s architecture details, including its hidden size, so any figure quoted for GPT-4 embeddings is a guess. For retrieval, OpenAI offers dedicated embedding models. Its documentation lists text-embedding-3-small and text-embedding-ada-002 at 1,536 dimensions and text-embedding-3-large at 3,072.

The distinction matters in practice. A search system embeds documents and queries with an embedding model, then may pass the retrieved text to a language model to write an answer. The two models are chosen separately. Related topics include word versus sentence embeddings, tokenization versus embedding, multimodal embeddings, domain-specific embeddings, fine-tuning embedding models, batch embedding, and asymmetric versus symmetric search.

An embedding dimension is one number in an embedding vector, and the dimension count is fixed by the model, usually its layer width. Each float32 dimension costs 4 bytes, so memory and comparison time grow in direct proportion. More dimensions do not guarantee better search: OpenAI’s newer model at 256 dimensions outperforms its older one at 1,536. Matryoshka models can be shortened, and quantization can compress any model. The decision is to test models and sizes on your own queries and keep the smallest one that holds quality.