Semantic search

What is semantic search?

Semantic search finds results based on what you mean, not the exact words you typed. Search for "cancel my plan" and it can return a help page titled "ending your subscription", even though the two share no words in common.

Traditional search matches strings of characters. If the page does not contain your search term, it does not come back. Semantic search works on meaning instead, so wording, synonyms, and phrasing stop getting in the way. This is why it feels closer to asking a knowledgeable colleague than querying a database.

How semantic search works

The idea is to represent meaning as numbers. An embedding model reads a piece of text and turns it into a list of numbers, a vector, that captures its meaning. Texts with similar meaning end up with vectors that sit close together, even when they use different words.

The flow runs in two phases.

1. Indexing. Every document, or a chunk of it, is passed through the embedding model and stored as a vector, usually in a vector database built for this job.

2. Querying. Your search is turned into a vector the same way. The system then looks for the stored vectors nearest to it and returns the text behind them.

Comparing your query against every stored vector one by one is too slow once you have millions of them. So vector databases use approximate nearest neighbour search, usually shortened to ANN. It trades a tiny amount of accuracy for a large gain in speed, returning results that are almost always the true closest matches in a fraction of the time.

Semantic search versus keyword search

Keyword search is exact and predictable. If you look up a specific product code, an error number, or a surname, you want the literal match, and keyword search gives it to you directly. It also needs no model and is cheap to run.

Semantic search wins when the same idea can be worded many ways, or when people do not know the exact term to look for. Its weakness is the mirror image of its strength: because it works on meaning, it can miss an exact code or name that a keyword index would have caught.

Because each covers the other's blind spot, many systems run both and merge the results. That combination has its own name, hybrid search, and it is often the most reliable option in practice.

Why semantic search underpins RAG

Retrieval-augmented generation, or RAG, is the common pattern for giving a language model access to your own documents. Before the model answers, the system fetches the most relevant passages and hands them over as context, so the answer is grounded in real material instead of the model's memory.

Semantic search is the fetching step. When you ask a question, it is the part that finds the handful of passages that actually address it, out of thousands you have stored. A RAG system is only as good as the passages it retrieves, so the quality of the semantic search decides the quality of the answer. Weak retrieval leads straight to a confident but wrong reply.

What to watch out for with semantic search

The embedding model sets the ceiling. A model trained on general web text may not grasp your industry's jargon. If two terms your business treats as different look similar to the model, search quality suffers. Feeding it domain-specific data can be worth the extra effort.

Chunking decides what can be found. Text is usually split into chunks before embedding. Chunks that are too large blur several ideas into one vague vector; too small and they lose the context that made them meaningful. How you split the text quietly shapes every result.

Exact matches can slip through. A pure semantic system can rank a close-in-meaning result above the exact code or name someone typed. Where precise matches matter, add keyword or hybrid search rather than relying on meaning alone.

It costs more to run. Embedding every document, storing the vectors, and running the search takes compute and memory that a keyword index does not. For a small, tidy set of documents, keyword search may be enough.