Home / Use cases / Semantic Search for Recommendations: Similar Items

Semantic Search for Recommendations: Similar Items

Semantic recommendations suggest items by comparing what the items mean, using embeddings of their text, images, or other content, rather than relying only on which items people clicked together. A vector database finds items whose embeddings sit close to an item a user is viewing, or close to a vector that summarizes what the user liked. Weaviate notes that recommendation and search use the same vector index; the difference is that recommendations also depend on the user. Semantic recommendations handle new items with no history, the cold start problem that limits collaborative filtering. They also have a blind spot: similarity finds substitutes, not complements, so “customers also bought” usually still needs interaction data. Most production systems blend both signals.

What are semantic recommendations?

Semantic recommendations are suggestions based on the meaning of items, captured as embeddings, so that a system can recommend things that are similar in content, style, or topic. “Semantic” here means meaning, the same sense in which language models represent meaning as vectors.

The mechanism is the one behind semantic search. Each item, such as a product, article, or song, is turned into a vector by an embedding model. Items with similar meaning get nearby vectors. Recommending then becomes a search: find the vectors closest to a starting point.

Weaviate’s explanation of recommendations frames the link clearly. Both search and recommendation use the approximate nearest neighbor index to find suitable objects. In search, relevance lives in the query. In recommendation, relevance also depends on the user, which makes it subjective. A user searching for “adidas shoes for the summer” needs those shoes, ranked by what that particular user likes.

The starting point can be an item, as in “more like this,” or a user, as in a personalized home feed. Either way, the content itself drives the result. That is the main contrast with the older approach to recommendations.

How are semantic recommendations different from collaborative filtering?

Semantic recommendations are different from collaborative filtering because they compare item content, while collaborative filtering compares patterns of user behavior, such as who bought or rated what. Each sees signals the other misses.

Collaborative filtering learns from interactions alone. Koren, Bell, and Volinsky’s 2009 overview of matrix factorization, written after the Netflix Prize, describes the idea. Users and items become vectors learned from a matrix of ratings or clicks, and items liked by similar users end up close together. The item’s own content plays no part.

The “finds” row matters most in practice. A content embedding of a phone sits near other phones, so similarity search returns alternatives. Collaborative data shows that phone buyers also buy cases and chargers, which are complements with very different descriptions. That is why “customers also bought” lists usually draw on interaction data, while “similar items” lists suit embeddings. The two approaches fit into a broader family of methods.

What are the different types of recommendations?

The different types of recommendations are content-based, collaborative, hybrid, knowledge-based, and, more recently, generative recommendations. Semantic recommendations are the modern form of content-based recommendation.

• Content-based: recommend items similar to ones the user liked, judged by item features. Embeddings made this approach far stronger, since they capture meaning rather than hand-picked tags.
• Collaborative filtering: recommend what similar users liked, from interaction data alone.
• Hybrid: combine content and behavior. YouTube’s 2016 system, described by Covington and colleagues, is a well-known two-stage design: candidate generation finds a few hundred videos by nearest neighbor search, then a ranking model orders them.
• Knowledge-based: use explicit rules or a knowledge graph, such as “lenses compatible with this camera.”
• Generative: a sequence model predicts the next item directly, often using semantic IDs.

The types differ mainly in what data they need. Content-based and knowledge-based methods work from item information alone, so they can serve a new catalog on day one. Collaborative and generative methods need interaction logs, which take time to collect but capture tastes that descriptions miss.

Most large systems are hybrids in the sense of the YouTube design. A fast retrieval step narrows millions of items to hundreds, and a heavier model ranks them. Vector search is a natural fit for the first step. The simplest version starts from a single item.

How do item-to-item semantic recommendations work?

Item-to-item semantic recommendations, often labeled “more like this,” work by using the vector of the item a user is viewing as the query, and returning the nearest other items in the vector index. No user history is needed.

In Weaviate, this is a near_object query. It takes an object’s ID, reuses that object’s stored vector, and finds the most similar objects. The item itself comes back as the closest match, so the query should exclude it, usually with a filter on its ID. Filters can also enforce business rules, such as in-stock items only or the same category.

A simple semantic search example shows the idea. A shopper viewing a waterproof hiking jacket sees other waterproof shells, even ones described as “rain coat” or “storm layer,” because their embeddings are close. Keyword matching on “hiking jacket” would miss them. Semantic search examples shows more cases. Personalizing the list requires representing the user too.

How do you recommend items for a user rather than an item?

You recommend items for a user by building a vector that represents the user’s tastes, most simply the average of the vectors of items they liked, and searching for items near it. Weaviate’s Ref2Vec module does this automatically.

Ref2Vec, short for reference-to-vector, arrived in Weaviate 1.16. A User object holds cross-references to Products the user liked. The ref2vec-centroid module sets the user’s vector to the centroid, the average, of those product vectors. Weaviate’s example is a user who bought boots, shorts, and a t-shirt, represented by the average of the three.

The user vector updates when references change, so a new like shifts recommendations right away. Weaviate’s demo shows a shopper clicking two school-style backpacks and getting more of that style, not duffel bags. A query then uses the user’s ID with near_object, which Weaviate calls “user-as-query,” the pattern behind home feeds.

A single average has limits. A user who likes both hiking gear and cookbooks gets a centroid between the two, which may match neither. Weaviate’s own roadmap notes point at the fixes: weighting recent interactions and using several centroids for diverse interests. The Weaviate Personalization Agent goes further. It stores personas and weighted positive or negative interactions, clusters by them, and can rerank results with a language model. Content vectors also solve a problem behavior data cannot.

How do semantic recommendations solve the cold start problem?

Semantic recommendations solve the item cold start problem because a new item gets an embedding from its content the moment it is added, so it can be recommended before anyone has interacted with it. Collaborative filtering has nothing to go on until interactions arrive.

The cold start problem has two sides. New items have no clicks, ratings, or purchases. New users have no history. Koren and colleagues’ overview of collaborative filtering names cold start as its central weakness, because the method learns only from interactions.

For new items, content embeddings fill the gap. A product added this morning has a title, description, and images, so it lands near similar products at once. For new users, the gap is smaller but real. A first click, a search query, or a few chosen interests can serve as a starting vector, then grow into a centroid as the user interacts. Research on generative recommenders has made the same point at scale.

What is generative recommendation with semantic IDs?

Generative recommendation with semantic IDs gives each item a short code derived from its content embedding, then uses a sequence model to predict the next item’s code. It turns recommendation into a generation task, much like a language model predicting the next word.

Rajput and colleagues introduced the idea in “Recommender Systems with Generative Retrieval,” published at NeurIPS 2023 and known as TIGER. Each item gets a Semantic ID, a tuple of codewords built from its content embedding. A transformer reads a user’s session as a sequence of Semantic IDs and generates the next one. The authors report gains over earlier models and better retrieval for items with no prior interactions.

A companion study shows why the codes matter. Singh and colleagues tested Semantic IDs in a YouTube ranking model in 2023. Replacing random item IDs with raw content embeddings lowered quality, because the model lost its ability to memorize specific items. Semantic IDs, learned from frozen content embeddings with a method called RQ-VAE, kept overall quality while improving results for new and long-tail items.

The lesson carries beyond research labs. Pure content similarity generalizes well but forgets what is special about popular items. Pure IDs memorize well but cannot handle new items. Semantic IDs, and hybrid systems in general, try to keep both. For most teams, the practical question is simpler: which database to build on.

What should a vector database offer for customers-also-liked recommendations?

A vector database for customers-also-liked recommendations should offer fast item-to-item search, filters, real-time updates, a way to represent users, and room to blend in behavior signals. Which one is best depends on your data, scale, and team, so the checklist matters more than a ranking.

Two caveats keep the choice honest. First, a “customers also liked” list built only from content similarity will show substitutes, not the complements that co-purchase data reveals. Plan to store interaction signals alongside vectors, or blend in a collaborative model. Second, test on your own catalog: measure whether people click and buy what you recommend. Semantic search tools compared covers the wider field of databases. Building the basic version takes only a few queries.

How do you build semantic recommendations in Weaviate?

You build semantic recommendations in Weaviate by embedding items in a collection, then querying with near_object for similar items or with a user vector for personalized ones. The example uses the Python v4 client.

import numpy as np
import weaviate
from weaviate.classes.query import Filter, MetadataQuery

client = weaviate.connect_to_local()
products = client.collections.use("Product")

# 1. Item-to-item: "more like this", excluding the item itself and out-of-stock items
viewed_id = "8b3a5e1c-6f1f-4c1e-9a0e-2a7d2f1b9c11"
similar = products.query.near_object(
    near_object=viewed_id,
    limit=6,
    filters=Filter.by_id().not_equal(viewed_id) & Filter.by_property("in_stock").equal(True),
    return_metadata=MetadataQuery(distance=True),
)

# 2. User-to-item: average the vectors of items the user liked, then search near it
liked_ids = ["...", "...", "..."]          # IDs from the user's history
liked = [products.query.fetch_object_by_id(i, include_vector=True) for i in liked_ids]
centroid = np.mean([o.vector["default"] for o in liked], axis=0).tolist()
candidates = products.query.near_vector(near_vector=centroid, limit=10 + len(liked_ids))
for_you = [o for o in candidates.objects if str(o.uuid) not in liked_ids][:10]

client.close()

The second pattern is what Ref2Vec automates inside Weaviate: with the ref2vec-centroid module, the user object carries the centroid as its own vector, and a near_object query on the user ID replaces the manual averaging. For richer personalization, the Personalization Agent on Weaviate Cloud adds weighted interactions and optional language model reranking.

The embedding model decides what “similar” means. A text model compares descriptions; a multimodal model can compare product images. Choosing an embedding model and multimodal embeddings help with that choice. Even a well-built system has limits worth planning for.

How do you measure whether semantic recommendations work?

You measure semantic recommendations offline against held-out interactions first, then confirm them online with A/B tests, because offline scores and real user behavior do not always agree. Both stages matter.

Offline evaluation hides some of each user’s later interactions, generates recommendations from the earlier ones, and checks how many hidden items appear in the top results. Recall at k and nDCG are common measures, as recall at k explains. This is fast and cheap, so it suits comparing embedding models, centroid strategies, or filters before anything reaches users.

Online evaluation shows whether people actually click, buy, or keep watching. Covington and colleagues, describing YouTube’s system, note that offline metrics guided their development but that final decisions relied on live A/B tests, since offline results did not always match what happened online. Weaviate’s guidance on changing embedding models follows the same order: test on a sample of your own data first, then run an A/B test on a small share of live traffic.

Two extra checks suit semantic recommendations in particular. Track coverage, the share of the catalog that ever gets recommended, since pure similarity can keep surfacing the same items. And track results for new items separately, because cold start performance is the main reason to use content embeddings at all. The measurements also expose the approach’s limits.

What are the limits of semantic recommendations?

The limits of semantic recommendations are that they find substitutes rather than complements, ignore popularity and quality, can trap users in narrow similarity, and depend on the embedding model’s notion of similar. Each has a known mitigation.

• Substitutes, not complements: similar items are alternatives. Accessories and add-ons need co-purchase data or rules.
• No sense of quality: a poorly reviewed item can be as similar as a bestseller. Blend in ratings, sales, or a ranking model.
• Narrow results: pure similarity shows more of the same. Diversity methods, such as maximal marginal relevance, spread results out.
• Model dependence: a text model may judge two products similar by description while their photos differ. Test which modality matches what users mean by similar.
• User averages blur: a single centroid can sit between unrelated interests. Use recent windows or several centroids.

These limits explain the hybrid designs used at scale, where vector retrieval proposes candidates and other signals decide the final order. Related use cases include semantic product search for e-commerce, image, video, and audio search, semantic search use cases, and similarity search.

Semantic recommendations compare item meaning through embeddings, so a vector database can suggest similar items from a single product or a summary of a user’s likes. They recommend new items immediately, which collaborative filtering cannot, but they find substitutes rather than complements and know nothing of popularity. Weaviate supports the core patterns with near_object, filters, Ref2Vec centroids, and the Personalization Agent, while research on semantic IDs shows how content and behavior can be combined. The practical step is to launch item-to-item recommendations from embeddings, then blend in interaction data as it accumulates.