Hallucination

The plain-language idea

A language model generates text one token at a time by predicting what's statistically likely to come next, given everything before it. It has no separate "is this true" step — there's no internal fact-checker running alongside generation, and no built-in mechanism for the model to know what it doesn't know. When you ask it something it has strong, well-represented patterns for, the plausible continuation and the correct answer usually line up. When you ask it something obscure, ask it to cite a specific detail it never actually saw, or push it past what it's confident about, it keeps generating the next plausible token anyway — confidently, fluently, and sometimes completely wrong. That's a hallucination: fluent, plausible-sounding output that isn't grounded in anything true.

This is why hallucination isn't a "bug to be patched away" in some future model version — it's an inherent property of how generation works. A model with no fact-checking mechanism can't be patched into having one just by making it bigger or newer; it can get better at recognizing when it's on shaky ground (so it hedges or declines more often), but "never generates a plausible-sounding falsehood" isn't a property this kind of system can guarantee. Treat it as a property to design around, not a defect to wait out.

See it in a real thing: grounding a tutor's answer

// lib/ai/prompts/tutor.ts — the system prompt constrains the model to
// only the lesson content it was actually given, not its general training.
const systemPrompt = `
Answer the learner's question using ONLY the lesson context provided
below. If the answer isn't in this context, say so explicitly instead
of guessing.

Lesson context:
${lessonContext}
`;

This is grounding in practice: instead of letting the tutor answer from whatever it generally "knows" about a topic (and risk generating a fluent-sounding but wrong explanation), the prompt hands it the actual lesson content and instructs it to answer only from that — and to say "not in this context" rather than fill the gap with a plausible guess. This is the same idea lib/ai/chat/handler.ts and the interview grader lean on: giving the model real, provided material to answer from (this is the core idea behind RAG — retrieval-augmented generation — covered later in this track) is worth far more than asking it to just try harder to be accurate.

Beyond grounding, three more mitigations actually move the needle:

• Ask the model to cite which part of the provided context supports its answer. This doesn't guarantee correctness, but it makes ungrounded answers easier to catch — an answer with no supporting citation, or one that cites something not actually in the context, is a visible red flag.
• Lower the temperature for factual tasks. Temperature controls how much the model favors the single most likely next token versus sampling more freely from less-likely ones; lower temperature narrows toward the most probable continuation, which reduces (but doesn't eliminate) confident departures from what's grounded.
• Add a structured verification step, like a second pass that checks the answer against the source material, rather than trusting the first generation outright.

What people usually miss

Watch out

• Hallucination isn't the model "lying." There's no intent to deceive — it's the same token-prediction process that produces correct answers, just running with nothing grounding it in a particular case.
• Confidence in the model's tone is not evidence of correctness. A hallucinated answer and a correct one are generated by the identical process and can sound equally fluent and certain.
• A bigger/newer model lowers the hallucination rate, it doesn't remove the mechanism that causes it. This is a probability shift, not a guarantee — features with real stakes still need grounding.
• Grounding only helps if the model is actually instructed to stay within it. Handing a model relevant context without an explicit instruction to answer only from it still leaves room for it to blend in outside "knowledge."
• Citations reduce risk, they don't eliminate it. A model can hallucinate a citation just as fluently as it hallucinates a fact — a cited answer is easier to verify, not automatically verified.

Why this matters in interviews

"How would you reduce hallucinations in an AI feature?" is a common question, and the strong answer names concrete mechanisms — grounding, citation, temperature, verification steps — instead of the vague, non-answer of "use a better model."