Embeddings & Vector Representations

The plain-language idea

An embedding is a fixed-length vector (an array of numbers, typically hundreds to a few thousand entries long) that a model produces to represent a piece of text's meaning. The specific numbers aren't human-readable — you can't eyeball a vector and know what it means — but the relationships between vectors are what matters: text with similar meaning produces vectors that land near each other in that number space, and unrelated text produces vectors that land far apart.

"Near" and "far" here are measured mathematically, most commonly with cosine similarity (how aligned two vectors' directions are) or a distance metric over the same space. Two pieces of text can be worded completely differently and still embed close together if they mean similar things — that's the entire point, and it's what makes embeddings useful for search: matching by meaning instead of matching by exact keyword overlap.

This is the mechanism underneath semantic search and the retrieval half of RAG (retrieval-augmented generation, covered in a later lesson): embed a user's query, embed a corpus of documents ahead of time, and find the documents whose vectors are nearest the query's vector. No keyword matching required — "how do I reset my credentials" can retrieve a document titled "password recovery steps" even though they don't share a single significant word.

See it in a real thing: matching a support query to a help article by meaning

// Conceptual shape, not this app's actual embedding call
const queryVector = await embed("how do I get back into my account");
const articleVectors = [
  { title: "Password recovery steps", vector: await embed("Password recovery steps") },
  { title: "Billing FAQ", vector: await embed("Billing FAQ") },
  { title: "Changing your email address", vector: await embed("Changing your email address") },
];

// Rank articles by cosine similarity to the query vector — no shared keywords required
const ranked = articleVectors
  .map((a) => ({ ...a, score: cosineSimilarity(queryVector, a.vector) }))
  .sort((a, b) => b.score - a.score);

Four short phrases, before embedding — just text, no notion of "similar" yet.

BackNext

Notice the query and the winning article share almost no words in common — "get back into my account" versus "password recovery." A keyword search would likely miss that match entirely; an embedding-based search finds it because the two phrases mean nearly the same thing, and that similarity shows up directly as the two vectors landing close together.

What people usually miss

Watch out

• An embedding is not a summary and not compressed text. You can't decode a vector back into the original words — it's a representation of meaning for comparison purposes, not a reversible encoding.
• Similar meaning, not similar wording, is what clusters together. Two sentences with zero words in common can embed very close together; two sentences sharing many words can embed far apart if their meanings actually differ.
• The embedding model matters, and vectors from different embedding models generally aren't comparable. A vector produced by one embedding model isn't guaranteed to be meaningfully comparable to a vector produced by a different one — mixing them in the same similarity search silently produces garbage results.
• Embeddings capture meaning, not truth. A confidently wrong statement and a correct one can still embed close together if they're topically similar — semantic similarity is not the same thing as factual agreement.
• "Fixed-length" doesn't mean the input length is unlimited. Embedding models still have their own token limits on the text they can accept per call, the same way chat models do.

Why this matters in interviews

"How would you build search that finds relevant results even when the user's wording doesn't match the source text?" is a direct embeddings question, and it's usually the entry point into a deeper RAG system-design conversation.