Evaluating LLM Output

The plain-language idea

"I tried it a few times and it looked good" feels like validation, but it isn't one — it's a handful of anecdotes against a system that can behave differently on the exact same input from one call to the next. A change that looks fine on the three prompts you happened to try can still be quietly worse on the hundred prompt shapes you didn't try, and there's no way to know which without actually checking.

An eval (evaluation) is what checking systematically looks like: a fixed, repeatable set of test cases run against the system, each scored against some standard, so a change to a prompt, model, or pipeline can be compared against a baseline instead of judged by feel. Two common ways to score each case:

• Automated scoring — exact match, or checking whether the output contains specific expected elements. Fast, cheap, fully deterministic, but only works when "correct" has a checkable shape (a specific value, a required keyword, valid JSON matching a schema).
• LLM-as-judge — a separate model call grades the response against stated criteria. Useful exactly where automated scoring falls short: open-ended output ("is this explanation clear and accurate?") that can't be reduced to an exact match.

Running the same fixed set of cases after every meaningful change is what turns "I think this prompt change helped" into "this prompt change improved the eval score from X to Y on the same 40 cases as before" — a claim that's actually checkable.

See it in a real thing: catching a prompt regression before it ships

// Rough sketch of an eval run — not this app's actual test harness
type EvalCase = { input: string; expectedContains?: string[] };

const cases: EvalCase[] = [
  { input: "Explain useState", expectedContains: ["closure", "re-render"] },
  { input: "Explain useEffect cleanup", expectedContains: ["unmount", "dependency"] },
  // ...a fixed set, run identically before and after a prompt change
];

async function runEval(systemPrompt: string) {
  let passed = 0;
  for (const c of cases) {
    const output = await callModel(systemPrompt, c.input);
    const ok = c.expectedContains?.every((kw) => output.includes(kw)) ?? true;
    if (ok) passed++;
    // For open-ended cases with no fixed keyword to check for,
    // an LLM-as-judge call would grade `output` against stated criteria instead.
  }
  return passed / cases.length;
}

Run this same set of cases against the current prompt and a candidate change, and the comparison is a real number, not a vibe. This is the same discipline this app's own anti-hallucination system prompts (lib/ai/prompts/tutor.ts, lib/ai/prompts/interview.ts) would need before any meaningful edit: a fixed set of test questions, run against both the old and new prompt, to check the change didn't quietly loosen the constraint against answering outside the given context.

What people usually miss

Watch out

• A few manual tries is not an eval. It's anecdotal sampling against a non-deterministic system — it can't tell you whether a change helped, hurt, or did nothing on the cases you didn't happen to try.
• Automated scoring only works when "correct" is checkable. Exact match or contains-expected-keyword works for structured or fact-based output; it can't judge tone, clarity, or open-ended quality, which is exactly the gap LLM-as-judge exists to fill.
• LLM-as-judge is not a source of ground truth. It's another model call, with its own failure modes and biases — a strong signal to run at scale, not an infallible grader that removes the need for human review.
• Evals earn their value on the second run, not the first. The whole point is comparability: running the same fixed set before and after a change is what makes "this got better" a checkable claim instead of an impression.
• This is especially important before shipping any change to a prompt or model that other people rely on — a subjectively "it seemed fine" check is a poor substitute for a real before/after comparison on a fixed case set.

Why this matters in interviews

"How would you know if a change to your prompt actually made things better?" is a real question that separates candidates who reach for a systematic eval (fixed cases, automated or LLM-as-judge scoring, a real before/after comparison) from those who'd just try it a few times and go with their gut.