Home / Embeddings / Embedding Quantization: Binary, Scalar and Product Quantization

Embedding Quantization: Binary, Scalar and Product Quantization

Embedding quantization stores each vector with fewer bits, so a vector index needs less memory and searches faster, at some cost in recall. Uncompressed embeddings use a 32-bit float per dimension, so a 1,536-dimension vector takes 6,144 bytes. Scalar quantization keeps one 8-bit integer per dimension, about 4 times smaller. Binary quantization keeps one bit per dimension, 32 times smaller. Product quantization splits the vector into segments and stores one centroid ID per segment, often 20 times smaller or more. Rotational quantization, Weaviate’s recommended method, rotates the vector before quantizing and needs no training. Its 8-bit version reaches 98 to 99 percent recall in Weaviate’s internal testing. Most systems recover lost accuracy by rescoring: fetching extra candidates with compressed vectors and re-ranking them with the originals.

What is embedding quantization?

Embedding quantization is the compression of embedding vectors by representing each number, or each group of numbers, with fewer bits than the original 32-bit float. It is also called vector quantization or vector compression.

An embedding model outputs floating-point numbers, usually 32 bits each. A vector index such as HNSW keeps those vectors in memory for fast search, and memory is the main cost of running vector search at scale. A million 1,536-dimension vectors take about 6.1 GB as float32 before the graph is counted. Quantization shrinks that footprint, and because smaller numbers are faster to compare, it can speed up search too.

Embedding quantization is different from model quantization. Quantizing an AI model, for example to int8 weights, makes the model itself smaller and faster to run. Quantizing embeddings compresses the vectors the model produces. The two can be combined, but they solve different problems. What are embeddings covers the vectors being compressed.

How much memory does each quantization method save?

Each quantization method saves a fixed share of vector memory set by its bits per dimension: 8-bit methods save 75 percent, 4-bit methods 87.5 percent, and 1-bit methods almost 97 percent. Product quantization depends on its segment count.

The arithmetic is simple. Bytes per vector equal the dimension count times the bits per dimension, divided by 8. A 1,536-dimension vector takes 6,144 bytes as float32, 1,536 bytes at 8 bits, 768 at 4 bits and 192 at 1 bit. With 256 centroids, product quantization stores one byte per segment, so 256 segments take 256 bytes.

Total memory falls by less than the vector size, because an HNSW index also stores its graph of connections, which quantization does not shrink. Weaviate also keeps the uncompressed vectors on disk for rescoring, so disk use rises slightly while memory falls. The vector storage and memory calculator estimates totals for a given collection, and embedding dimensions covers the other lever on size.

How does scalar quantization work?

Scalar quantization works by mapping each 32-bit float dimension to an 8-bit integer, one of 256 buckets spanning the range of values that dimension takes. It is often called int8 quantization.

The quantizer first learns the range of values, either per dimension from a training sample or per vector. It then divides the range into 256 even steps and replaces each value with the number of its step. Distances are computed on the integers, which modern processors handle quickly. Weaviate’s SQ trains its buckets on a sample, with a default training limit of 100,000 objects per shard.

Scalar quantization is predictable and robust. Weaviate’s documentation cites typical recall of 95 to 97 percent before rescoring. It suggests SQ as a good starting point when you have a representative sample of the final data. The weak spot is training: if later data differs from the training sample, values can fall outside the learned ranges. Some embedding APIs can also return int8 or binary embeddings directly, which applies the same idea at the source.

How does binary quantization work?

Binary quantization works by keeping only the sign of each dimension, one bit, so a vector becomes a bit string compared with fast bitwise operations. It gives the largest compression of the common methods.

A positive value becomes 1 and a negative value becomes 0. Distance between two bit strings is the Hamming distance, the count of positions that differ, which processors compute in a handful of instructions. The result is a 32-fold reduction in vector size and very fast candidate scoring.

The cost is accuracy. Weaviate’s documentation says BQ works best for high-dimensional vectors and that its recall varies widely with the data and model. Some embedding models are trained to keep quality under binary quantization, and pairing it with Matryoshka embeddings shrinks vectors further. In practice, binary quantization is nearly always paired with rescoring on a larger candidate set.

How does product quantization work?

Product quantization works by splitting each vector into segments, replacing every segment with the ID of its nearest centroid from a trained codebook, and storing only those IDs. Jégou, Douze and Schmid introduced it in 2011.

With m segments and k centroids per segment, the codebook holds m times k centroids, each of d divided by m dimensions. That is k times d floats in total, shared by every vector. For 768 dimensions, 128 segments and 256 centroids, it holds 196,608 floats. Each vector then needs m codes, one byte each when k is 256. Weaviate’s example turns 3,072 bytes into 128 bytes, almost 24 times smaller.

Search uses asymmetric distance computation, known as ADC. For each query, the system builds a lookup table of m times k distances, from each query segment to each centroid. A vector’s approximate distance is then the sum of m table lookups, with no floating-point maths per vector. Optimized product quantization, from Ge and colleagues in 2013, adds a learned rotation before splitting so the segments carry information more evenly.

What is rotational quantization in Weaviate?

Rotational quantization in Weaviate rotates each vector with a fast pseudorandom transform and then quantizes every dimension, so it needs no training and keeps high recall. Weaviate’s documentation recommends it as the default choice.

The rotation, based on the Walsh-Hadamard transform, spreads information evenly across dimensions, so no single dimension carries outsized values. The rotated vector is then quantized to 8, 4 or 1 bits per dimension, using that vector’s own minimum and maximum. Weaviate reports that 8-bit RQ keeps 98 to 99 percent recall in internal testing. The 4-bit and 1-bit versions compress further and rely more on rescoring.

from weaviate.classes.config import Configure

client.collections.create(
    name="Docs",
    vector_config=Configure.Vectors.self_provided(
        vector_index_config=Configure.VectorIndex.hnsw(
            quantizer=Configure.VectorIndex.Quantizer.rq(bits=8)
        )
    ),
)

Because RQ has no training step, it can be enabled when a collection is created, before any data arrives. From version 1.33, Weaviate can apply a default quantization to new collections through the DEFAULT_QUANTIZATION setting. Weaviate’s academy notes that Weaviate Cloud sets that default to 8-bit RQ. Once set on a collection, quantization cannot be disabled.

How does rescoring recover recall after quantization?

Rescoring recovers recall by fetching more candidates than requested using the compressed vectors, then re-ranking those candidates with their full-precision vectors. Compression decides the shortlist; the originals decide the order.

Compressed distances are approximate, so a true neighbour can fall just outside the top results. Over-fetching widens the net, and recomputing exact distances for that small set fixes the ranking. Weaviate’s default rescore limit is 20 candidates for 8-bit and 4-bit RQ on HNSW, and 512 for 1-bit RQ. In Weaviate’s internal tests on 1,536-dimension data, rescoring the top 50 brought 8-bit and 4-bit RQ to 99.8 percent recall or better.

The trade is a little extra work per query, reading a few dozen or hundred uncompressed vectors from disk or cache. Measure the result with ANN recall: compare compressed search against exact search on the same queries. Exact vs approximate search explains the baseline those numbers rest on.

Which quantization method should you choose?

You should choose 8-bit rotational or scalar quantization by default, product quantization when memory is tight and you can train on representative data, and binary quantization for very high-dimensional vectors with generous rescoring. Test each on your own queries.

Weaviate’s documentation stresses that compressed recall depends heavily on the embedding model. A model change can shift the best method, so retest after re-embedding when the model changes. Weaviate’s PQ guidance recommends 10,000 to 100,000 objects per shard to train the codebook. Choosing an embedding model and the MTEB leaderboard help shortlist models before compression testing.

A simple test plan covers four methods: none, 8-bit, PQ and binary with rescoring. Record recall at 10 against exact search, memory per million vectors and query latency for each. Pick the smallest setting whose recall your application tolerates. Quantization also interacts with domain-specific embeddings and fine-tuned models, whose value distributions may differ from general models.

Which related quantization terms are easy to confuse?

Related quantization terms are easy to confuse because the same words describe embedding compression, model compression and signal coding in different fields. Context decides the meaning.

“Vector quantization” in classic signal processing means mapping vectors to the nearest entry in a codebook, of which product quantization is a structured form. “Finite scalar quantization” is a 2023 technique by Mentzer and colleagues for the latent codes of generative models, not a vector search index method. “Adaptive quantization” usually means fitting quantization ranges to the data, as trained SQ buckets do. Each idea shares the goal of fewer bits with acceptable error.

Quantization is also separate from tokenization, batching and embedding type. A text becomes tokens and then a vector, as in tokenization vs embedding; the vector is then compressed. Batch embedding produces the vectors, and word vs sentence embeddings, multimodal embeddings and asymmetric vs symmetric search describe what they represent.

Embedding quantization stores vectors with fewer bits to cut memory and speed up search. Scalar and 8-bit rotational quantization shrink vectors 4 times, binary quantization 32 times, and product quantization by a ratio set by its segments. Weaviate recommends 8-bit RQ, which needs no training and keeps 98 to 99 percent recall in its testing. Rescoring with the original vectors recovers most lost recall for a small cost per query. Choose by testing recall, memory and latency on your own data and model.