Home / Hybrid search / Hybrid Search on Distributed Databases at Scale

Hybrid Search on Distributed Databases at Scale

Hybrid search on a distributed database runs a keyword query and a vector query across data split into shards on many nodes, then merges and fuses the results into one ranked list. A coordinating node sends the query to every shard that holds part of the collection. Each shard searches its own inverted index and vector index and returns its best candidates. The coordinator combines the shard lists and fuses keyword and vector scores into the final top results. Sharding lets a collection grow past one machine and speeds up imports. Replication copies shards to other nodes for availability and more query throughput. In Weaviate, every shard has its own vector index, inverted index and object store, and objects are assigned to shards by a hash of their UUID. What tends to break at scale is candidate depth, score consistency between shards, filtered queries and memory per node.

What is hybrid search on a distributed database?

Hybrid search on a distributed database is the combination of keyword and vector retrieval in a system that spreads its data and work across multiple machines. It is how hybrid search scales past one server.

Hybrid search itself fuses two rankings: keyword relevance, usually BM25, and vector similarity. What is hybrid search and how hybrid search works cover the method on one machine. A distributed database runs the same logic over many nodes, so the collection can hold more data than one node’s memory and serve more queries than one node’s CPUs.

“Distributed database vs vector database” is a false choice. Most production vector databases, Weaviate included, are distributed systems that can run on one node or a cluster. Some general distributed databases have added vector search, and some search engines have added vectors to their keyword core. Databases that support hybrid search compares the options.

How does a hybrid query run across shards?

A hybrid query runs across shards in three steps: the coordinating node scatters the query to every relevant shard, each shard returns keyword and vector candidates, and the coordinator gathers, merges and fuses them. This pattern is called scatter-gather.

In Weaviate, a collection is made of one or more shards, and each shard contains its own vector index, inverted indexes and object store. Shards can sit on different nodes, and Weaviate handles the orchestration at import and query time automatically. Objects are placed by a 64-bit Murmur-3 hash of their UUID, which spreads data evenly without a user-chosen key.

Because every shard searches only its own data, a query’s latency is set by the slowest shard plus the merge. More shards spread memory and import work, but every query still touches all of them. That is why sharding helps capacity more than throughput, and why replication is the tool for query volume.

How do sharding and replication scale hybrid search?

Sharding scales hybrid search in data size and import speed, while replication scales it in availability and query throughput. Production clusters usually combine both.

Weaviate’s documentation spells out the trade-off. More shards let a collection exceed one node and speed up imports, even on a single node, but do not improve query throughput. More replicas make the system highly available and give near-linear gains in query throughput, but do not speed up imports. Its example combines three shards with a replication factor of 3, for nine shard copies across the cluster.

Plan shard count early. Weaviate’s academy notes that existing collections cannot be re-sharded, and recommends more shards than initial nodes so the cluster can grow. Its example is 8 shards on 2 nodes, which can later spread across 8 nodes. Replica movement, added in version 1.32, can relocate individual shard replicas to rebalance after adding nodes.

What fails in hybrid search when data grows?

When data grows, hybrid search tends to fail on candidate depth, keyword score consistency across shards, filtered query speed, and memory per node. Each has a known mitigation.

Candidate depth is the first. Each leg of a hybrid query fetches a pool of candidates before fusion. If the pool is too small, a document that ranks well on the fused score but modestly on each leg can be lost. Weaviate’s QUERY_HYBRID_MAXIMUM_RESULTS setting sets the minimum pool per leg, 100 by default, and its documentation advises raising it if hybrid results shift between pages. Deep paging multiplies the work on every shard, because each one must return enough candidates to cover the requested offset.

Keyword statistics are the second. BM25 depends on how common a term is across documents. In any sharded engine, statistics gathered per shard can differ from global ones unless the engine shares them, which can nudge keyword scores between shards. Uniform hash-based placement keeps shards statistically similar, which limits the effect. Fusion by rank, as in reciprocal rank fusion, is less sensitive to raw score scale than relative score fusion.

How do filters and multi-tenancy affect distributed hybrid search?

Filters and multi-tenancy affect distributed hybrid search by changing how many shards a query must touch and how much of each shard the search must scan. Both can make queries much cheaper or much slower.

A restrictive filter makes vector graph traversal inefficient, because most neighbours fail the filter. Weaviate switches to a flat scan of the matching objects when a filter matches fewer than its flatSearchCutoff, 40,000 objects by default. On a sharded collection, each shard applies the filter to its own data, so a filter that is selective globally stays selective on every shard.

Multi-tenancy changes the shape entirely. In Weaviate, each tenant is its own shard, so a tenant’s query touches one shard instead of all of them. That turns a scatter-gather over the whole collection into a single-shard search, which is why per-tenant hybrid search scales to very large tenant counts. Multi-tenant semantic search covers the pattern.

How do you tune hybrid search on a distributed cluster?

You tune hybrid search on a distributed cluster by measuring per-shard timing, sizing candidate pools, choosing the fusion method and weight, and adding replicas for throughput or shards for capacity. Measure before and after each change.

Weaviate’s query profiling returns per-shard timing for vector search, keyword scoring and filters, so you can see whether one shard or one leg is slow. Prometheus metrics give query latency and request rates across the cluster. Use them to separate a slow keyword leg, a slow vector leg, and a slow merge.

Relevance tuning is the same as on one node. Set the balance between keyword and vector scores, as in balancing keyword and vector scores, and pick a fusion method from score fusion methods. Normalisation details are in score normalization. Recheck relevance after scaling changes, since candidate pools and shard counts can shift results slightly.

How do you deploy a distributed hybrid search database?

You deploy a distributed hybrid search database on a managed service or on your own infrastructure, often Kubernetes, with a replication factor for availability and a planned shard count. The choice depends on team capacity and data rules.

Weaviate offers a Helm chart for Kubernetes and a managed cloud service. Its documentation recommends replication for zero-downtime rolling upgrades and for tolerating node failures; without replication, losing a shard means losing its data. Replicas can sit in different regions to reduce latency for distant users. Managed vs self-hosted search compares the two routes.

Data residency adds constraints. If regulated data must stay within a country, replicas cannot be placed freely across regions. Common answers are a separate cluster per region, or a tenant or collection per jurisdiction on regional infrastructure. Agents and RAG pipelines that query the cluster inherit these boundaries, as discussed in hybrid search for RAG.

When does hybrid search need a distributed database?

Hybrid search needs a distributed database when the data no longer fits on one node, when query volume exceeds one node, or when downtime is unacceptable. Many applications never reach that point.

A single node handles a large share of real workloads, and scaling up a node is simpler than running a cluster. Weaviate’s academy suggests scaling up when cost and convenience come first and some downtime is tolerable. It suggests scaling out for very large collections, high-throughput consumer apps, or strict uptime. Compression and disk-based indexes can also delay the need to shard for memory.

Start from your requirements, not the architecture. When to use hybrid search and hybrid search vs vector search settle whether hybrid is needed at all. Hybrid search in SQL databases, hybrid search for enterprise search and the hybrid search tutorial cover the main implementation paths.

Hybrid search on a distributed database scatters each query to every shard, gathers keyword and vector candidates, and fuses them into one ranking. Sharding raises capacity and import speed; replication raises availability and query throughput. Plan shard counts early, size candidate pools for deep or filtered queries, and prefer rank-based fusion where score scales vary. Multi-tenancy turns many queries into single-shard searches. In Weaviate, each shard has its own indexes, query profiling shows per-shard timing, and replication supports rolling upgrades and regional placement.