What Are Embeddings? A Builder's Guide to Vector Search
Embeddings are lists of numbers that represent the meaning of text. An embedding model reads a word, sentence, or paragraph and returns a fixed-length vector, and things that mean similar things get vectors that sit close together in space. That is what makes semantic search, RAG, and agent memory possible: once meaning is a set of coordinates, finding related text becomes finding nearby points. This guide covers what an embedding actually is, how similarity search uses it, and when you need one.
TL;DR
- An embedding is a vector, a fixed list of numbers, that encodes the meaning of a piece of text. Similar meaning lands at nearby coordinates.
- Vector search embeds your query the same way, then returns the stored vectors closest to it. Closeness is measured with cosine similarity.
- Embeddings power semantic search, RAG retrieval, deduplication, clustering, and long-term agent memory. They match on meaning, not exact words.
- Dimension count (384, 768, 1536, 3072) is fixed by the model you pick. Pick for quality on your own data, not for the biggest number.
- Use hybrid retrieval: embeddings for meaning, keyword search for exact terms like product codes. Merge both result sets.
What an embedding actually is
A computer cannot compare two sentences the way you can. "Cancel my subscription" and "how do I end my plan" share almost no words, yet they mean the same thing. Keyword matching sees two unrelated strings. An embedding fixes that by turning each sentence into a point in space.
An embedding model takes text in and gives a vector out: a fixed-length list of numbers, maybe 768 or 1536 of them. Each number is a coordinate along one axis of a high-dimensional space. You cannot picture 1536 dimensions, and you do not need to. What matters is the rule the model was trained to follow: text with similar meaning gets placed at nearby coordinates, and text with different meaning gets placed far apart.
So "cancel my subscription" and "how do I end my plan" end up close together, because the model learned they carry the same intent. "What time do you open" ends up far away. No keyword overlap required. The meaning is encoded in where the point sits, not in the words themselves.
How similarity search actually works
Storing meaning as points is only half of it. The other half is finding the nearby ones on demand.
When a question comes in, you embed it with the same model that embedded your content. Now the question is a point in the same space as everything you stored. Vector search finds the stored points closest to it and hands those back. That is the whole retrieval step.
Closeness is almost always measured with cosine similarity, which looks at the angle between two vectors rather than the raw distance. Two vectors pointing in nearly the same direction score close to 1.0, vectors at right angles score near 0, and opposite ones score near -1. Angle beats straight-line distance here because it ignores how long the text was and focuses on what direction the meaning points.
Doing that comparison against a handful of vectors is trivial. Doing it against ten million, on every query, in under 50 milliseconds, is not. That is the job a vector database does: it builds an index so it can find approximate nearest neighbors without scoring every single vector. You trade a sliver of accuracy for a search that stays fast as your data grows.
Where embeddings come from and what they cost
You do not write an embedding model. You call one. Providers like OpenAI, Cohere, Voyage, and Google offer embedding endpoints, and there are strong open-weight models you can run yourself. You send text, you get a vector.
Three practical facts shape which one you pick:
- Dimensions are fixed by the model. A model outputs 384, 768, 1536, or 3072 numbers, and you take what it gives. More dimensions can hold finer distinctions, but storage and search cost climb with them and the quality gain flattens fast.
- Both sides must match. The vector for your stored content and the vector for your query have to come from the same model. Mixing two models puts their points in different spaces, and the distances become meaningless.
- Cost is per token, and it is cheap to embed but not free to store. Embedding a large corpus is a one-time bill measured in cents per million tokens. The recurring cost is the vector database holding those millions of vectors in memory.
Pick a well-regarded general model, measure retrieval on your own questions, and only move to something bigger or domain-specific if the numbers say you should.
What you build on top of embeddings
Embeddings are infrastructure, not a feature. A short list of what they unlock:
- Semantic search. A search box that finds "how to end my plan" when the page says "subscription cancellation." This is the default upgrade over keyword search.
- RAG retrieval. The retrieval half of retrieval-augmented generation runs on embeddings. You embed your chunks, embed the question, and pull the closest chunks into the prompt. The upstream step that decides what gets embedded is covered in how to chunk documents for RAG, and it matters more than the model choice.
- Long-term agent memory. An agent that remembers past conversations stores them as vectors and retrieves the relevant ones when a new message arrives. See how to give your AI agent memory.
- Clustering and dedup. Group similar support tickets, flag near-duplicate documents, or spot outliers, all by looking at which vectors bunch together.
When you need embeddings, and when you do not
Embeddings are not free lunch. They add a model call, a database, and a class of failure where the search returns something that looks related but is not. Reach for them when your users phrase things in their own words and your content uses different words for the same ideas. That gap is exactly what semantic search closes.
Skip or delay them in two cases. First, when exact matching is the whole job: looking up an order number, a SKU, or a legal clause by its exact title, where a plain keyword index is faster, cheaper, and more precise. Second, when your corpus is tiny. If you have fifteen documents, you can often stuff the relevant ones straight into the prompt and let the model sort it out, no vector database required.
The strongest production setups rarely pick one lane. They run hybrid retrieval: embeddings catch meaning-based matches, keyword search catches exact terms and names, and a merge step combines both rankings. You get the recall of semantic search without losing the precision of exact matching. If you are still assembling the surrounding toolchain, my AI daily driver stack covers the pieces I actually run.
Common mistakes to avoid
Four traps catch most first builds:
- Different models on the two ends. Embed your content with one model and your queries with another and the distances mean nothing. Lock the model and version.
- Reaching for fine-tuning to add facts. Fine-tuning changes behavior, not knowledge. New facts belong in retrieval, which means embeddings. See RAG vs fine-tuning.
- Chasing dimension count. A 3072-dimension model is not automatically better than a 768 one for your task. Measure on your own questions before you pay for the bigger index.
- No keyword fallback. Pure vector search fumbles exact strings like error codes and product names. Add keyword search alongside it.
Fixing these is mostly about measuring retrieval on real questions rather than trusting a demo, the same discipline that makes chunking work.
The bottom line
Embeddings turn text into coordinates so that similar meaning sits close together, and vector search finds the nearby points on demand. That single idea powers semantic search, RAG retrieval, and agent memory. Pick one embedding model and use it on both ends, measure retrieval on your own questions instead of a demo, and run keyword search alongside it so exact terms still land. Do that and you have the retrieval backbone most AI features quietly depend on. For a new practical breakdown like this one every week, join the newsletter.
What are embeddings in simple terms?
An embedding is a list of numbers that represents the meaning of a piece of text. An embedding model reads a word, sentence, or paragraph and returns a fixed-length vector, often a few hundred to a few thousand numbers long. Those numbers are coordinates in a high-dimensional space where things that mean similar things land near each other. So the vector for return window sits close to the vector for refund policy even though they share no words, and both sit far from the vector for office hours. That is the entire point. Embeddings turn language, which computers cannot compare directly, into geometry, which they can. Once meaning is a set of coordinates, similar meaning is just a short distance, and finding related text becomes finding nearby points.
What is the difference between embeddings and vector search?
Embeddings are the numbers. Vector search is what you do with them. The embedding step converts each piece of text into a vector and stores it. Vector search takes a new query, embeds it the same way, and finds the stored vectors closest to it, usually by cosine similarity. Think of embeddings as the addresses and vector search as the lookup that finds every address near a given point. You cannot do one without the other in a real system: embeddings with no search is a pile of numbers nobody reads, and search with no embeddings falls back to exact keyword matching, which misses anything phrased differently. Vector databases like Pinecone, Weaviate, pgvector, and Qdrant exist to make that nearest-neighbor lookup fast across millions of vectors.
Do I need embeddings for a RAG system?
Almost always yes, and usually alongside keyword search rather than instead of it. RAG works by retrieving the most relevant chunks of your own content and handing them to the model at answer time. Embeddings are how retrieval understands that a question about canceling my plan should pull the chunk titled subscription termination even though the words do not match. Pure keyword search would miss that. The strongest setups run hybrid retrieval: embeddings catch meaning-based matches, keyword search catches exact terms like product codes and names, and the two result sets get merged. If your content is small, a few dozen documents, you can sometimes skip the vector database and embed on the fly, but you still use embeddings to rank.
How many dimensions should an embedding have?
You do not choose the number directly. It is fixed by the embedding model you pick, commonly 384, 768, 1536, or 3072 dimensions. More dimensions can capture finer distinctions but cost more to store and search, and the gain flattens out fast for most tasks. The practical move is to start with a well-regarded general model, measure retrieval quality on your own questions, and only reach for a larger model if the numbers say you need it. Some newer models support shortening the vector after the fact, so you can trade a little accuracy for a smaller, cheaper index without re-embedding everything. Dimension count matters far less than whether the model was trained on text like yours.
Are embeddings the same as fine-tuning?
No, and confusing them wastes money. Embeddings give a model access to your information at answer time by retrieving relevant text and putting it in the prompt. Nothing about the model changes. Fine-tuning changes the model's weights so it learns a style, format, or narrow behavior, and it does not reliably teach new facts. For a knowledge base, product docs, or anything that updates often, embeddings plus retrieval is the right tool because you can add or edit content instantly without retraining. Fine-tuning earns its place when you need a consistent output shape or tone that prompting alone cannot hold. Most teams reach for fine-tuning first when embeddings would have solved the problem for a fraction of the cost.
OpusJake is Jake Schincariol's operating system for building with AI: agents, workflows, prompts, and the free resources behind them. Get the next move every week.