Home / Scale and cost / Keeping a Semantic Search Index Fresh: Real-Time Indexing

Keeping a Semantic Search Index Fresh: Real-Time Indexing

Real-time indexing keeps a semantic search index in step with changing data, so new, edited, and deleted content shows up correctly in search results within seconds rather than after a scheduled rebuild. Each change must be embedded, written to storage, and inserted into the vector index, while deletions must disappear from results at once. Weaviate does this by making objects queryable as soon as they are written, by hiding deleted objects immediately with tombstones, and by cleaning up the index in the background. Freshness has costs. Inserting into a graph index takes CPU time, and cleaning up deletions takes more. The right level of freshness depends on how fast content changes and how much a stale result costs, which ranges from minutes for a product catalog to never for an archive.

What is real-time indexing in semantic search?

Real-time indexing in semantic search is the practice of applying every content change to the search index as it happens, rather than in periodic batches. A new product, an edited policy, or a withdrawn article should be reflected in results almost immediately.

An index is a data structure that makes lookups fast without scanning every record. A database index on a customer table speeds up lookups by name. A semantic search index does the same for meaning: it stores embeddings in a vector index so similar items can be found quickly. Keeping it fresh means the vectors, the stored objects, and any keyword index all reflect the current data.

Freshness is harder for vector indexes than for simple lookups. Each new item must first be embedded by a model, which takes time or an API call. It then has to be placed into a graph or cluster structure, which takes more time as the index grows. Deletions must remove the item from results without breaking the structure around it.

Weaviate’s engineering blog describes its design goal plainly: any imported object can be queried immediately by ID, keyword search, or vector search, and objects can be updated or deleted while queries run. That makes it a real-time vector database. Whether that speed is needed depends on the workload, as scaling semantic search also notes.

Does real-time indexing matter for your use case?

Real-time indexing matters when content changes often and stale results cost money, trust, or safety; it matters little when content is stable. The answer comes from two questions: how fast does the data change, and what does a stale result cost?

The table shows that freshness is a business decision, not a technical default. Deletions often matter more than additions. A withdrawn document or a revoked permission that still appears in results can do more harm than a new document that appears an hour late.

Real-time knowledge indexing for AI assistants follows the same logic. A retrieval-augmented system answers from whatever the index holds. If the index lags, the assistant answers from old facts. Semantic search in RAG covers that pipeline. Once the need is clear, the mechanics of a single change explain the costs.

What happens when a document is added to a vector index?

When a document is added to a vector index, it is embedded, stored as an object, and inserted into the graph or cluster structure so nearest-neighbor search can find it. Each step takes time, and the slowest one sets how quickly the document becomes searchable.

Embedding comes first. The document’s text goes through an embedding model, either inside the database through a vectorizer module or in the application before the write. Then the object and its metadata are stored. Weaviate’s storage documentation explains that its object store and inverted index use an LSM tree: writes land in memory, are flushed to sorted disk segments, and are merged in the background. The vector index sits beside them, protected by a write-ahead log.

The vector insert is the expensive step. In an HNSW graph, each new vector is linked to its nearest neighbors, which requires a small search of its own. Weaviate offers asynchronous indexing to keep writes fast. With it enabled, vector indexing goes through a queue, and since version 1.28 that queue is stored on disk and covers single imports, updates, and deletes as well as batches. Weaviate’s documentation notes the trade-off: there is a short delay between creating an object and finding it through the vector index.

Weaviate’s 1.22 release notes describe how searches behave meanwhile. Vector queries search the index built so far plus up to 100,000 not-yet-indexed objects by brute force. Changes to existing objects need more care.

How do vector indexes handle updates and deletes?

Vector indexes typically handle an update as a delete plus an insert, and handle a delete by marking the entry as gone first and removing it from the structure later. Graph indexes cannot simply erase a node without repairing the links around it.

Weaviate’s engineering blog describes the mechanism. Each object has an internal document ID. When an object changes, Weaviate deletes the old document ID, assigns a new one, and inserts the object again. A delete attaches a tombstone to the document ID, an approach the blog credits to Cassandra. The object is hidden from query results immediately.

The cleanup happens later. Weaviate’s index documentation explains that tombstoned objects stay connected to the HNSW graph until an asynchronous cleanup reassigns their edges and removes them for good. Cleanup runs on a schedule set by cleanupIntervalSeconds, which defaults to 300 seconds. Doing the work in bulk is cheaper than repairing the graph after every single delete.

Heavy delete traffic therefore creates background work. For very large shards, Weaviate exposes limits on how many tombstones each cleanup cycle processes. Its reference suggests starting points of 1 million minimum and 10 million maximum per cycle for a shard of 300 million objects. Cleanup threads default to half the available CPU cores. The tombstone approach is one form of a broader pattern.

What is a soft delete in a vector database?

A soft delete in a vector database marks an item as deleted so it no longer appears in results, while the physical removal from the index happens later. Tombstones are the database’s internal form of soft delete.

Soft deletes exist because physical removal is expensive in a graph index. Removing a node means relinking its neighbors so searches can still pass through that part of the graph. Doing that immediately for every delete would slow writes. Marking first and cleaning in batches spreads the cost.

Applications sometimes add their own soft delete on top. A property such as archived or deleted_at lets the application hide items with a filter while keeping them for audit or undo. That is useful, but every query must then include the filter, and the hidden items still occupy memory and index space. A true delete removes them for good once cleanup runs.

Replication adds one more case. When an object is deleted on some replicas but not yet on others, the system must decide which wins. Weaviate offers three deletion resolution strategies. Its default from version 1.36 resolves conflicts by timestamp, so the most recent operation wins. Keeping deletes correct is one part of keeping the whole index in sync.

How do you keep a search index in sync with source data?

You keep a search index in sync with source data by sending every change from the system of record to the index through a pipeline, and by monitoring how far the index lags behind. The pipeline pattern depends on how changes are produced.

Three patterns cover most systems:

• Event-driven updates send each create, update, and delete from the source to the index as it happens.
• Scheduled batches collect changes for a period and apply them together, trading freshness for simplicity.
• Dual writes make the application write to the database and the index in the same request.

The list runs from freshest to simplest. Event-driven pipelines often read a database’s change log, so no change is missed. Scheduled batches suit content that changes slowly. Dual writes are easy to build but can leave the two systems out of step if one write fails, so they need retries and periodic checks.

Whatever the pattern, a few habits prevent drift. Store a content hash or update time with each object, so unchanged items are skipped and edited ones are re-embedded. Make every write idempotent, using stable IDs so a retried write replaces rather than duplicates. Treat deletes as first-class events, not afterthoughts. Measure freshness lag directly: the time between a change in the source and its appearance in search results. With asynchronous indexing, also watch the queue length, which Weaviate reports per node. Sometimes incremental updates are not enough.

Should you run a full reindex or incremental updates?

Run incremental updates for day-to-day content changes, and run a full reindex only when something changes every vector, such as the embedding model or the chunking rules. A corpus that changes daily almost always suits incremental updates.

Incremental updates touch only what changed. A new article is embedded and inserted; an edited one is re-embedded and replaced; a removed one is deleted. Cost scales with the volume of change, not the size of the collection.

A full reindex rebuilds everything. It is necessary when the embedding model changes, because vectors from two models cannot be compared. It is also needed when chunking, preprocessing, or the set of embedded fields changes, since every vector would differ. Some teams also rebuild periodically to compact an index that has absorbed heavy churn.

A full reindex should not interrupt search. The safe pattern builds a new collection beside the live one, verifies it on judged queries, and then switches traffic. Weaviate’s collection aliases make that switch a single atomic operation. Re-embedding when the model changes walks through it, and managed versus self-hosted search covers who runs the extra capacity. The final question is which systems handle real-time data well.

Which databases are good for real-time data?

Databases that are good for real-time data accept writes continuously and make them queryable within seconds, and no single one is best for every real-time workload. The right choice depends on whether the data is transactional, analytical, or search-oriented.

A real-time database is one that serves reads on data that is still arriving, such as a live order book, a chat history, or a product catalog with changing stock. Indexing databases are those that build structures for fast lookup, from the B-tree indexes in relational databases to inverted indexes in search engines and vector indexes in vector databases.

For semantic search on changing data, the requirements are specific: incremental inserts into the vector index, immediate hiding of deletes, filters that apply to fresh data, and a way to rebuild without downtime. Weaviate meets these through its incremental HNSW index, tombstones, asynchronous indexing, and collection aliases. Other systems in databases that support hybrid search handle updates in their own ways, and some cluster-based indexes need periodic retraining as data drifts.

Throughput and latency under write load deserve a test before choosing. Search latency covers how writes affect query times, the cost of semantic search covers the compute that freshness consumes, and the recall versus latency trade-off and GPU acceleration cover tuning under load.

Real-time indexing keeps a semantic search index in step with changing data by embedding, storing, and indexing each change as it happens. Weaviate makes new objects queryable at once, hides deletes with tombstones, treats updates as delete plus insert, and cleans up in the background. Asynchronous indexing speeds writes at the cost of a short lag. The decision is to match freshness to the cost of staleness. Use incremental updates for daily change and a full rebuild behind an alias for model changes, and measure freshness lag so the index never quietly falls behind.