Home / Build it / Semantic and Neural Search in OpenSearch

Semantic and Neural Search in OpenSearch

Semantic search in OpenSearch, which the project calls neural search, uses four pieces. They are an embedding model registered through ML Commons, an ingest pipeline that embeds text, a k-NN index with a knn_vector field, and a neural query. The ingest pipeline’s text_embedding processor turns each document’s text into a vector at index time. The neural query embeds the search text with the same model and finds the nearest vectors through an HNSW graph. Hybrid search adds a keyword match query alongside the neural one, inside a hybrid query. A search pipeline with a normalization processor rescales and combines the two score sets. Memory follows the k-NN plugin’s estimate of 1.1 times (4 times dimension plus 8 times m) bytes per vector, about 3.52 GB for a million 768-dimension vectors with m of 16.

What is neural search in OpenSearch?

Neural search in OpenSearch is the set of features that convert text to vectors inside the cluster and run vector similarity queries, so applications can search by meaning without managing embeddings themselves. It builds on the k-NN plugin, which stores and searches the vectors.

OpenSearch is an open-source search engine that began in 2021 as a community fork of an Apache-licensed, Lucene-based engine. It is now run under the Linux Foundation, which hosts the OpenSearch Software Foundation. Like other Lucene-based engines, it started from keyword search with an inverted index and BM25 ranking. The k-NN plugin added vector fields and approximate nearest neighbour search, and the neural search plugin added model-driven embedding during ingest and query.

The difference between search engines and vector databases is covered in search engine vs vector database. The closely related Elasticsearch build is in semantic search with Elasticsearch.

Which pieces does OpenSearch neural search need?

OpenSearch neural search needs an embedding model, an ingest pipeline that applies it, a k-NN index whose default pipeline is that ingest pipeline, and a neural or hybrid query. Each is configured through its own API.

The model can run inside the cluster on machine learning nodes, or OpenSearch can call an external embedding API through a connector. Pretrained sentence-transformer models are available for local deployment. Whichever model is used, the same one must embed documents and queries.

The order matters. The model must be deployed before the pipeline can reference its id, and the pipeline must exist before the index names it as a default. Documents indexed before the pipeline was attached will have no vectors and need reindexing.

Keep the model id, pipeline name and index settings in version control, for example as a setup script. Rebuilding a cluster, or creating a new index for a new model, then repeats the same steps exactly. A mismatch between the model that embedded the documents and the one used at query time silently ruins results, so record the model id with the index.

How do you register an embedding model in OpenSearch?

You register an embedding model through the ML Commons API, deploy it, and keep the returned model id for the pipeline and queries. Registration and deployment run as tasks that report their status.

POST /_plugins/_ml/models/_register
{
  "name": "huggingface/sentence-transformers/all-MiniLM-L6-v2",
  "version": "1.0.1",
  "model_format": "TORCH_SCRIPT"
}

POST /_plugins/_ml/models/<model_id>/_deploy

The register call returns a task id; polling the task gives the model id. All-MiniLM-L6-v2 produces 384-dimension vectors, which sets the index’s dimension. Cluster settings may need to allow models on data nodes in small test clusters; production clusters usually add dedicated machine learning nodes.

For a hosted embedding API, create a connector with the provider’s endpoint and credentials, then register a remote model that uses it. The rest of the setup is identical. Model choice is covered in choosing an embedding model.

How do you create the ingest pipeline and k-NN index?

Create an ingest pipeline with a text_embedding processor that maps a text field to a vector field. Then create an index with knn enabled, a knn_vector field of the model’s dimension, and the pipeline as default. Documents then get vectors automatically.

from opensearchpy import OpenSearch

client = OpenSearch(hosts=[{"host": "localhost", "port": 9200}])
client.ingest.put_pipeline(id="embed", body={"processors": [
    {"text_embedding": {"model_id": MODEL_ID,
                        "field_map": {"text": "text_embedding"}}}]})
client.indices.create(index="articles", body={
    "settings": {"index.knn": True, "default_pipeline": "embed"},
    "mappings": {"properties": {
        "text": {"type": "text"},
        "text_embedding": {"type": "knn_vector", "dimension": 384,
            "method": {"name": "hnsw", "engine": "lucene",
                       "space_type": "cosinesimil"}}}}})
client.index(index="articles", body={"text": "Refunds are issued within 14 days."})

The method block chooses the HNSW algorithm, the engine and the distance. The Lucene engine stores graphs inside Lucene segments; the Faiss engine adds options such as IVF and product quantization. HNSW parameters such as m and ef_construction can be set in the method parameters. Their effect on recall and speed is covered in approximate nearest neighbour search.

Long documents should be chunked before indexing, or with a text chunking processor in the pipeline, so each vector covers one passage. The principles are in chunking for semantic search.

How do you run a neural query?

You run a neural query by naming the vector field, the query text, the model id and k, the number of nearest neighbours to retrieve. OpenSearch embeds the text with the model and searches the HNSW graph.

res = client.search(index="articles", body={
    "_source": {"excludes": ["text_embedding"]},
    "query": {"neural": {"text_embedding": {
        "query_text": "how long do refunds take",
        "model_id": MODEL_ID, "k": 10}}}})
for hit in res["hits"]["hits"]:
    print(hit["_score"], hit["_source"]["text"])

Excluding the vector field from the returned source keeps responses small. Filters can be added inside the neural clause or combined in a bool query; how filtering interacts with approximate search is covered in vector databases with metadata filtering.

The k value controls how many neighbours each shard retrieves before results are merged. Set it at least as high as the number of results needed, and higher if filters remove many candidates. The mechanics are covered in top-k retrieval.

How do you set up hybrid search in OpenSearch?

Set up hybrid search by creating a search pipeline with a normalization processor, then sending a hybrid query that contains a keyword match query and a neural query. The processor rescales each query’s scores and combines them with weights.

client.search_pipeline.put(id="hybrid", body={"phase_results_processors": [
    {"normalization-processor": {
        "normalization": {"technique": "min_max"},
        "combination": {"technique": "arithmetic_mean",
                        "parameters": {"weights": [0.3, 0.7]}}}}]})
res = client.search(index="articles", params={"search_pipeline": "hybrid"}, body={
    "query": {"hybrid": {"queries": [
        {"match": {"text": "refund 14 days"}},
        {"neural": {"text_embedding": {"query_text": "refund 14 days",
                                       "model_id": MODEL_ID, "k": 50}}}]}}})

BM25 and vector scores sit on different scales, so they must be normalized before combining. Min-max scaling maps each list to 0 to 1, and the weights set the keyword-vector balance, here 0.3 for keyword and 0.7 for vector. Newer OpenSearch versions also offer rank-based fusion. The methods are compared in score fusion methods and relative score fusion.

Weaviate offers the same pattern in one query, with BM25 built on its inverted index and fusion selected by parameter, as shown in the hybrid search tutorial. Whichever engine is used, tune the weights on judged queries.

How much memory does OpenSearch HNSW need?

The k-NN plugin documentation estimates HNSW memory as 1.1 times (4 times the dimension plus 8 times m) bytes per vector, so dimension dominates. Graphs for the Faiss engine live in native memory outside the Java heap.

For one million 768-dimension vectors with m of 16, the estimate is 1.1 times 3,200 bytes, about 3.52 GB. Each replica holds its own copy, so one replica doubles it. Plan node memory for the total across shards and replicas, plus the Java heap for everything else.

Quantization reduces the vector part. Recent versions offer byte and binary vectors, Faiss scalar and product quantization, and disk-based modes that keep compressed vectors in memory and full vectors on disk. Trade-offs are covered in embedding quantization.

How do you use OpenSearch for RAG?

OpenSearch can serve RAG either as the retriever behind application code or through its own retrieval-augmented generation search processor, which sends retrieved passages to a connected language model. Retrieval quality decides answer quality either way.

The simplest pattern runs a hybrid query, takes the top passages and builds a prompt in the application. Frameworks such as LangChain and LlamaIndex include OpenSearch vector store integrations, as outlined in semantic search with LangChain and LlamaIndex. The built-in processor instead calls an LLM through an ML Commons connector and returns the generated answer with the hits.

Evaluate retrieval separately from generation, with judged queries and recall at the passage depth sent to the model. The pattern and its pitfalls are covered in semantic search in RAG.

How should you evaluate OpenSearch for hybrid and semantic search?

Evaluate OpenSearch on your own judged queries, comparing keyword, neural and hybrid results, and measure latency, memory and indexing speed at production scale. Feature support alone does not predict quality.

OpenSearch suits teams already running it for logs or keyword search, since vectors can be added to existing clusters. It needs more configuration than a vector-native database, because model, pipelines and index are separate pieces. Recent releases have also added performance features; for example, GPU-accelerated vector index building has appeared as an experimental feature in the 3.x line. Check the current release notes for status.

Compare with Weaviate and other vector databases on the same data, using the guides for Weaviate, Qdrant, Milvus, Pinecone, Redis, Chroma and Postgres with pgvector. The decision process is in choosing a vector database for semantic search. The general method is in how to build semantic search, semantic search with Python, search index schema design, semantic search over documents and PDFs and migrating to a new vector database.

Semantic search in OpenSearch combines four pieces. Register and deploy an embedding model through ML Commons, and create an ingest pipeline with a text_embedding processor. Then build a k-NN index with a knn_vector field and HNSW method, and query it with a neural query. Hybrid search adds a match query inside a hybrid query and a search pipeline that normalizes and weights the two score sets. Plan memory with the k-NN estimate of 1.1 times (4 times dimension plus 8 times m) bytes per vector, about 3.52 GB per million 768-dimension vectors. For a vector-native alternative with built-in hybrid search, compare Weaviate on the same queries.