Vector Databases

The plain-language idea

A vector database indexes embeddings — numeric vectors representing meaning — not raw text. When a RAG pipeline searches for the chunks closest to a query, the vector database is the thing doing that search: given a query vector, find the stored vectors nearest to it.

What makes this hard at scale is the search algorithm. Comparing a query vector against every single stored vector one by one (brute-force search) works fine for a few hundred vectors, but falls apart as the collection grows into the millions — it's too slow for a real-time request. Vector databases instead use approximate nearest-neighbor (ANN) search: index structures that find very close to the truly nearest vectors, fast, by not exhaustively checking everything. The "approximate" is a deliberate tradeoff — a small chance of missing the single best match in exchange for search that stays fast as the dataset grows.

That's the whole value proposition: fast semantic search over large, high-dimensional vector collections, at a scale where naive comparison would be too slow to use in a live request.

See it in a real thing: deciding whether you even need one

// A few hundred FAQ entries, rarely changing — brute-force cosine similarity is fine
function bruteForceSearch(queryVector: number[], allVectors: number[][]) {
  // compare against every vector directly — fine at small scale, too slow at millions
}

// Millions of growing documents, queried live — this is what a vector database is for
// const results = await vectorDb.query(queryVector, { topK: 5 });

Not every "search my data" problem needs a dedicated vector database. If the dataset is small and mostly static (a few hundred FAQ entries, a fixed product catalog), a linear scan comparing the query against every stored vector can be fast enough and a lot simpler to run than standing up dedicated infrastructure. And if what users are actually typing is closer to "find this exact term or phrase" than "find something that means roughly this," plain keyword or full-text search is often the more correct tool, not just a simpler one — semantic search solves a different problem (matching meaning despite different wording) than exact-term lookup does.

What people usually miss

Watch out

• "Vector database" and "embedding model" are two different, separable pieces. The embedding model turns text into vectors; the vector database stores and searches those vectors. You choose them independently — swapping one doesn't require swapping the other.
• ANN search trades a small amount of accuracy for a large amount of speed. It is not guaranteed to return the single mathematically closest vector every time — it returns vectors that are very likely close enough, which is the entire reason it scales.
• A small, static dataset often doesn't need a vector database at all. Standing up dedicated vector infrastructure for a few hundred rarely-changing entries is usually unnecessary complexity — a simple linear scan can be fast enough.
• Semantic search isn't automatically better than keyword search. If users type exact product codes, error messages, or specific terms, full-text search can outperform a vector search built for fuzzy meaning-matching.

Why this matters in interviews

"When would you reach for a vector database versus something simpler?" tests whether a candidate defaults to trendy infrastructure or actually reasons about data scale and query intent — a well-calibrated answer names both the ANN-at-scale case and the "you probably don't need this yet" case.