Fine-Tuning vs. Prompting vs. RAG

The plain-language idea

There are three distinct ways to get a model to do what you need, and they solve three different problems. Treating them as interchangeable is where most of the confusion comes from.

• Prompting — just asking better, with clearer instructions, examples, or structure in the request itself. Cheapest option, fastest to iterate (change a string, try again), and works for the large majority of real cases. Always try this first.
• RAG — when the model needs access to specific, current, or proprietary knowledge it wasn't trained on. You're not changing how the model behaves, you're giving it facts to work with at request time (covered in the RAG lesson).
• Fine-tuning — actually adjusting the model's weights through additional training, to change its behavior: consistent tone, a specific output format, a specialized skill the base model doesn't reliably do. This is genuinely expensive, slow to iterate (each change means a new training run, not a string edit), and requires real ML infrastructure and expertise to do well.

The decision tree in practice: start with prompting. If the model needs facts it doesn't have, add RAG. Only reach for fine-tuning when the problem is about how the model behaves — its style, format, or a skill — not what it knows.

See it in a real thing: diagnosing "the model doesn't know about our docs"

Symptom: "The model gives wrong answers about our internal API."

Wrong diagnosis: "We need to fine-tune it on our docs."
Right diagnosis: "The model needs access to our docs at request time — that's RAG."

This mistake is common enough to name explicitly: fine-tuning teaches a model to answer in a certain style more reliably, it doesn't reliably teach it new facts, and it definitely doesn't keep pace with docs that change. A team that fine-tunes on last month's documentation has baked stale facts into the model's weights — the exact problem RAG's "no retraining needed when data changes" property was built to avoid. If the actual gap is missing knowledge, not a style or behavior mismatch, RAG is the correct tool, and fine-tuning is both more expensive and less effective at closing it.

What people usually miss

Watch out

• "The model doesn't know about our internal docs" is almost never a fine-tuning problem. It's a knowledge-access problem, which is what RAG solves — this is the single most common mismatch between symptom and reached-for solution.
• Fine-tuning is for behavior, not facts. It changes how consistently a model writes in a certain tone, follows a certain format, or performs a specialized task — it's a poor, expensive way to inject new factual knowledge compared to just handing the model that knowledge in context.
• Prompting is underrated. A surprising amount of "the model isn't doing what I want" is solved by a clearer, more structured prompt (better instructions, few-shot examples, explicit constraints) — reaching for RAG or fine-tuning before exhausting prompting is usually premature.
• Fine-tuning's iteration loop is fundamentally slower. A prompt change is instant to test; a fine-tune change requires a new training run — this cost alone should push teams toward prompting and RAG whenever those can solve the problem.

Why this matters in interviews

"Our model doesn't know about our product's latest features — how do you fix that?" is a direct test of this decision tree, and naming RAG (not fine-tuning) as the fix, with the reasoning for why, is exactly the signal that distinguishes someone who understands these tools from someone repeating buzzwords.