RAG (Retrieval-Augmented Generation)

The plain-language idea

RAG (Retrieval-Augmented Generation) is a pattern, not a product: instead of asking a model to answer purely from what it learned during training, you first retrieve relevant material from your own data, then hand that material to the model as context and ask it to generate an answer grounded in what you just gave it.

The flow has a fixed shape:

1. A user asks a question.
2. That question gets turned into an embedding (a numeric vector representing its meaning).
3. The embedding is matched against a vector store holding embeddings of your own documents, chunked into pieces.
4. The top-k best-matching chunks come back — the ones whose meaning is closest to the question.
5. Those chunks get inserted into the prompt as context, alongside the original question.
6. The model generates an answer using that context, instead of relying only on whatever it happened to absorb during training.

Nothing here retrains the model. The model's weights never change — you're changing what it sees at request time.

See it in a real thing: answering a question about a lesson

// Rough sketch of the RAG steps for "explain closures" against this app's own content
const query = "How do closures relate to useState?";

// 1. Embed the query
// const queryVector = await embed(query);

// 2. Search the vector store for the closest lesson chunks
// const topChunks = await vectorStore.search(queryVector, { topK: 4 });

// 3. Assemble retrieved chunks + query into a grounded prompt
const prompt = `Answer using only this context:\n${"<retrieved chunks>"}\n\nQuestion: ${query}`;

A user asks a question — it needs to be turned into something searchable.

BackNext

This is the same shape this app's own tutor uses conceptually: lib/ai/prompts/tutor.ts instructs the model to answer only from the lesson context it's given, not from general training knowledge. That's the anti-hallucination guarantee RAG makes possible — the model's answer can be traced back to specific retrieved chunks instead of being an ungrounded guess.

What people usually miss

Watch out

• RAG doesn't make the model "know" anything permanently. Every request re-retrieves and re-injects context from scratch — nothing persists between calls unless you build that persistence yourself.
• Retrieval quality caps answer quality. If the top-k chunks that come back aren't actually relevant, the model is generating an answer grounded in the wrong context, not a knowledgeable one — "garbage in, garbage out" applies to the retrieval step just as much as the generation step.
• RAG isn't a fine-tuning substitute for everything. It's for giving the model access to specific facts it doesn't have, not for changing its tone, output format, or behavior — that's a different problem (covered in the fine-tuning-vs-RAG lesson).
• "Grounded" doesn't mean "guaranteed accurate." A model can still misread or overgeneralize from correct retrieved context — grounding reduces hallucination risk, it doesn't eliminate it, which is why the anti-hallucination system prompt still matters even with good retrieval.

Why this matters in interviews

"How would you build a feature that answers questions about our internal documentation?" is a standard system-design-adjacent question, and reaching for embed-retrieve-inject instead of "just fine-tune the model on our docs" is exactly the distinction that separates a candidate who's actually built with LLMs from one who's only read about them.