Home / How it works / Semantic Indexing: How Vector Indexes Are Built

Semantic Indexing: How Vector Indexes Are Built

Semantic indexing stores representations of meaning for fast search, and a vector index is the structure that makes it fast by skipping most comparisons. An embedding model turns each item into a vector. The index organizes those vectors so a query can jump to the right neighborhood. Different index types organize them differently. A flat index keeps a plain list and compares everything. A graph index such as HNSW links each vector to its neighbors. A cluster index such as IVF groups vectors around centroids and searches only the nearest groups. IVF-PQ adds compression on top. The index you choose decides the balance between speed, recall, memory, and how easily data can be added or removed.

What is semantic indexing?

Semantic indexing is the step that converts content into meaning-based representations and stores them in a structure built for similarity search. It is the indexing half of semantic search: everything that happens to content before the first query arrives.

The term has an older meaning too. Latent semantic indexing, described by Deerwester and colleagues in 1990, mapped terms and documents into a reduced space of concepts. Modern semantic indexing follows the same idea with neural embeddings instead of matrix factorization. A model embeds each chunk of content, and the vectors go into an index.

Semantic indexing and vector indexes are related but not the same. Semantic indexing is the whole process: choosing chunks, embedding them, storing metadata, and building the index. A vector index is the data structure at the end of that process. A team can change its index type without changing its embeddings, but changing the embedding model means re-indexing everything.

The semantic search pipeline shows where indexing sits beside querying, and chunking for semantic search covers the step before embedding. The index itself is where most performance decisions live.

What is a vector index?

A vector index is a data structure that organizes stored vectors so a nearest-neighbor query can find close matches without checking every vector. Without one, each query compares itself with the whole collection, which grows linearly with its size.

Weaviate’s indexing documentation describes the job as balancing search speed against accuracy and resource use. Most vector indexes are approximate: they skip most of the collection and accept that they may miss a few true neighbors. Microsoft’s SQL documentation makes the same point plainly. A vector index returns approximate results, which is different from a traditional relational index.

A vector index is also different from a vector database. The index is one component. A vector database stores the original objects, their metadata, and one or more indexes. It adds filtering, updates, replication, and an API. Libraries such as FAISS provide indexes without the database around them. Hybrid indexing usually means a system keeps both a vector index and a keyword inverted index for the same objects, so hybrid search can query both.

The index serves vector search, the database operation, which in turn serves semantic search, the user-facing goal. The semantic search versus vector search comparison and similarity search separate those terms in detail. How an index gets built depends on its type.

How is a vector index built?

A vector index is built by inserting embeddings one at a time into a structure such as a graph, or by first learning clusters from a sample and then assigning every vector to one. The two approaches have very different build and update behavior.

Graph indexes grow incrementally. In HNSW, each new vector is placed by searching the existing graph for its nearest neighbors and linking to them, layer by layer. Malkov and Yashunin’s 2016 paper describes the multilayer structure. Weaviate’s documentation notes that HNSW needs no training step, so a collection can start empty and grow.

Cluster indexes need data first. An IVF index runs a clustering step, usually k-means, on a sample to find centroids, then assigns each vector to its nearest centroid. pgvector’s documentation advises creating an IVFFlat index only after the table already contains data, because the centroids are learned from it. If the data changes a lot later, the centroids may stop representing it well.

Either way, build cost matters at scale. Weaviate’s FAQ notes that imports are almost always CPU-bound because building the HNSW index is costly. The index settings chosen at build time, such as graph connectivity or cluster count, set the ceiling on later recall. With the build process in view, the main index types can be compared.

What types of vector indexes are there?

The main types of vector indexes are flat, graph-based, cluster-based, tree-based, and disk-based indexes, each trading speed, memory, and recall differently. Weaviate’s documentation groups them into graph, tree, cluster, and flat families.

The table shows that no family wins outright. Flat indexes suit small collections. Weaviate suggests them for collections expected to stay under about 100,000 vectors, and its dynamic index starts flat and switches to HNSW once a collection passes 10,000 objects by default. Microsoft’s SQL documentation suggests exact search under about 50,000 vectors.

Graph indexes dominate when speed and recall both matter. Cluster indexes save memory and build quickly. Weaviate’s HFresh index is a cluster-based design that keeps an HNSW index of centroids in memory and stores the lists of vectors on disk. Approximate nearest neighbor search and exact versus approximate search cover when approximation is acceptable. IVF deserves a closer look, because it is the most common alternative to HNSW.

How does an IVF index work?

An IVF index, short for inverted file index, groups vectors into clusters around learned centroids and searches only the clusters closest to the query. The name comes from the inverted lists used in keyword search: each centroid keeps a list of the vectors assigned to it.

FAISS documents the mechanism. A coarse quantizer, usually trained with k-means, splits the space into nlist cells. At query time, the index finds the nearest centroids and searches only the vectors in the nprobe closest cells. That cuts the work to roughly nprobe divided by nlist of the collection.

Two settings control the trade-off. More clusters make each cluster smaller and faster to scan, but raise the chance that a true neighbor sits in a cluster the query does not probe. Probing more clusters raises recall and costs time. pgvector gives starting points for its IVFFlat index. Set lists to the row count divided by 1,000 for up to one million rows, or the square root of the row count above that. Start probes at the square root of lists.

Worked through, a table of 500,000 rows would start with 500 lists and about 22 probes. Those are starting values to tune against measured recall, not final settings. IVFFlat stores full vectors in each list. The next step stores compressed ones.

How does IVF-PQ compress a vector index?

IVF-PQ compresses a vector index by replacing each stored vector with a short product-quantization code, while keeping the IVF clusters to narrow the search. It combines two ideas: partitioning to search less, and compression to store less.

Product quantization splits each vector into several sub-vectors and replaces each with the ID of its nearest entry in a small learned codebook. Jégou, Douze, and Schmid described the method, including its combination with an inverted file, in IEEE Transactions on Pattern Analysis and Machine Intelligence in 2011. FAISS’s IVF-PQ index stores codes of a few bits per sub-vector. Its documentation gives the size as the number of sub-quantizers times bits per code, divided by 8, plus 8 bytes for the vector’s ID.

The savings are large. Weaviate’s quantization documentation gives an example: a 768-dimensional float vector takes 3,072 bytes, and its product-quantized form takes 128 bytes. Weaviate applies product quantization to HNSW rather than IVF, which shows the technique works with either index family.

Compression costs recall, because distances are computed on approximations. Systems recover some of it by rescoring the top candidates with full vectors. With IVF and IVF-PQ explained, the most searched comparison follows.

How do IVF and HNSW indexes compare?

IVF and HNSW compare as a lighter, training-based cluster index against a heavier, incrementally built graph index with a better speed-recall trade-off. pgvector’s documentation states the core difference: HNSW has better query performance than IVFFlat in terms of the speed-recall trade-off, but builds more slowly and uses more memory.

The practical differences follow from structure:

• Training: IVF learns centroids from existing data, while HNSW needs no training step.
• Updates: HNSW absorbs new vectors incrementally, while IVF assigns them to existing clusters that may drift from the data.
• Memory: HNSW stores graph links for every vector, while IVF stores only list membership.
• Compression: IVF-PQ shrinks vectors sharply, while HNSW can use quantization too.
• Tuning: IVF tunes lists and probes, while HNSW tunes connections and candidate list sizes.

The list explains why HNSW is the default in many vector databases and IVF remains common in libraries and relational extensions. Weaviate uses HNSW by default and offers HFresh, a cluster-based design, when memory is the main constraint. For a growing collection with frequent updates, HNSW is usually the simpler choice. For very large, mostly static collections on a tight memory budget, IVF-PQ or a disk-based index can win. The choice also determines how the index ages.

How are vector indexes updated and maintained?

Vector indexes are maintained by handling inserts, deletes, and data drift in ways that keep recall stable over time. An index that was accurate at launch can degrade quietly as content changes.

Inserts are straightforward for HNSW, which links each new vector into the graph. Deletes are harder. Weaviate’s documentation explains that deleted objects are cleaned up by an asynchronous process that rebuilds the graph connections around them. Heavy delete traffic therefore costs background work.

IVF indexes face drift. New vectors are assigned to clusters learned from older data. If content shifts, clusters become uneven and recall falls. The usual fix is to rebuild the index periodically from current data.

Settings need revisiting too. Weaviate’s dynamic ef setting adjusts the query-time candidate list with the requested result count. Recall should be measured against exact search on a sample after major data changes. Top-k retrieval and relevance scores cover the query side, and distance metrics covers the metric fixed at build time. Relational databases now offer these indexes as well.

Is SQL a vector database?

SQL is a query language rather than a vector database, but several SQL databases now store vectors and build vector indexes, so they can serve vector search alongside relational data. The line between the categories has blurred.

Microsoft’s SQL Database Engine documentation describes a vector data type that stores 4-byte floats, a VECTOR_DISTANCE function for exact nearest-neighbor search, and vector indexes based on the DiskANN algorithm, queried with VECTOR_SEARCH. As of its September 2026 documentation, vector indexes are generally available in Azure SQL Database and in preview in SQL Server 2025. In PostgreSQL, the pgvector extension adds IVFFlat and HNSW indexes.

A relational database with vector support suits teams whose data already lives there and whose search needs are moderate. A dedicated vector database adds features built for the workload, such as hybrid search, built-in vectorization, quantization options, and multi-tenancy. Hybrid search in SQL databases and the semantic search architecture guide cover the trade-off.

The vectors themselves matter as much as the index. Dense vectors, sparse vectors, and dense versus sparse vectors need different structures. Related topics include semantic matching, synonyms and related terms, and multilingual and cross-lingual search.

Semantic indexing stores meaning as vectors, and a vector index makes those vectors fast to search by skipping most comparisons. Flat indexes compare everything, HNSW walks a layered graph, IVF searches the nearest clusters, and IVF-PQ adds compression. HNSW offers the better speed-recall trade-off at a memory cost, while IVF builds lighter but needs training data and periodic rebuilds. The decision is to start flat for small collections, use HNSW for growing ones, and consider IVF-PQ or a disk-based index when memory is the binding limit, measuring recall against exact search after every change.