Embeddings

An embedding is a list of numbers, a vector, that represents the meaning of a piece of text. The model that produces it is trained so that texts meaning similar things land near each other in this high-dimensional space, while unrelated texts land far apart. This single trick is what makes vector search work: searching for meaning turns into measuring distance.

From words to coordinates#

A dense vector representation captures meaning across hundreds or thousands of dimensions at once, with every dimension carrying a fraction of the signal. That is what "dense" means, as opposed to sparse keyword vectors that are mostly zeros: a bag-of-words vector for a document has a slot for every word in the vocabulary and almost all of them are empty, while a 1536-dimension embedding uses all 1536 numbers to encode one compressed summary of meaning.

The classic intuition is that "dog" and "puppy" sit close together, "dog" and "wolf" sit a bit further, and "dog" and "invoice" sit far apart, even though none of those words share letters. The model learned these positions from how words are actually used across billions of sentences, so paraphrases and synonyms cluster naturally without anyone hand-coding rules. The same holds at sentence and document scale: "How do I get my money back?" and "What is the refund procedure?" embed to nearly the same point despite sharing almost no words, which is exactly the failure mode of keyword search and exactly the strength of embeddings.

Explore the idea below: click any word and watch its nearest neighbors light up.

Cosine similarity, the distance that matters#

To compare two embeddings you almost always use cosine similarity, which measures the angle between the vectors rather than their raw length. Vectors pointing the same direction score near 1 (very similar), perpendicular ones score near 0 (unrelated), and opposite ones score near -1. Angle wins over straight-line distance because it ignores magnitude and focuses purely on direction, which is what semantic similarity actually tracks; most providers also normalize their vectors to length 1, at which point cosine similarity and Euclidean distance rank results identically.

In practice the numbers are unintuitive at first: two paraphrases might score 0.9, a related-topic pair 0.6, and two unrelated texts 0.2 rather than 0. The absolute values vary by model, so never hard-code a threshold from a blog post; calibrate it on your own data. Retrieval itself is then simple to state: embed the query, and return the stored vectors with the highest cosine similarity, typically via an approximate nearest neighbor index so you are not brute-forcing millions of comparisons per query.

Choosing a model and sizing the vector#

The embedding model you pick sets your retrieval quality ceiling, and the field moves fast. As of mid-2026, Google's Gemini embedding models lead the MTEB leaderboard among API offerings, with Voyage and Cohere close behind and OpenAI's text-embedding-3-small remaining the budget workhorse at around $0.02 per million tokens. Open-weight models are competitive too if you want to run inference yourself. Check the current leaderboard rather than habit, but weigh it against your own evaluation set: a model that wins general benchmarks can still lose on your legal contracts or your codebase.

Dimensionality is the second decision. Bigger vectors (1536 or 3072 dimensions) hold more nuance but cost more to store and search: at 4 bytes per float, ten million 3072-dimension vectors are about 120 GB before indexing overhead. Matryoshka representation learning softens this trade-off. Models trained this way, including OpenAI's text-embedding-3 family and Google's Gemini embeddings, pack the most important information into the earliest dimensions, like nested dolls, so you can truncate a vector to 256 or 768 dimensions and keep most of the quality. OpenAI reports that a text-embedding-3-large vector shortened to 256 dimensions still outperforms the older ada-002 model at its full 1536. Truncating from 3072 to 768 cuts storage and search cost by 75% for a few points of recall, a trade most production systems happily take.

The gotcha: switching models means re-embedding everything#

One hard rule with no exceptions: embeddings from different models are not comparable. Each model defines its own coordinate system, so a query embedded with model B is meaningless against documents embedded with model A, even if both vectors happen to have 1536 dimensions. The same applies across major versions of the same model, and to changing your truncation length.

Operationally this means a model switch is a migration, not a config change. You re-embed the entire corpus, rebuild the index, and re-calibrate any similarity thresholds. The API bill is usually the small part (ten million chunks averaging 400 tokens is 4 billion tokens, around $80 at small-model prices), but the pipeline run, the index rebuild, and the evaluation pass take real time. The standard playbook is to build the new index alongside the old one, shadow-test queries against both, compare retrieval metrics, then cut over. Budget for this when you choose a model, because the switching cost is exactly why teams stay on mediocre embeddings for years.

Beyond RAG: what else embeddings are for#

RAG made embeddings famous, but they are a general-purpose tool for "is this like that?" questions. Clustering groups support tickets, survey answers, or news articles by topic with no labels required. Near-duplicate detection flags pairs above a high similarity threshold, catching reworded copies that exact-match hashing misses. Classification embeds the text once and trains a tiny, cheap classifier on top of the vectors instead of fine-tuning a whole language model. Recommendation embeds items and users in the same space, so "articles like the ones you read" is a nearest-neighbor lookup. The same vectors can also feed anomaly detection and hybrid search. If you are already paying to embed your corpus for retrieval, these capabilities come nearly free.

The takeaway#

Embeddings are the bridge from language to geometry. Pick one strong model and validate it on your own data, compare by cosine similarity with thresholds you calibrated yourself, use Matryoshka truncation to keep vectors small, and never mix vectors from two models. Treat a model switch as the full re-embedding migration it is, and reuse the same vectors for clustering, dedup, and classification once you have them.