Lesson 10 of 30

Structured Output & JSON Mode

Q: You need an LLM's response to reliably parse into a specific JSON shape your code can act on programmatically. What options do you have, and how would you rank them by reliability?

A strong answer covers: Names provider-enforced schema constraints as the most reliable option, because generation itself is constrained rather than checked afterward; Names prompted JSON with parsing, validation, and a retry or fallback as the middle option, correctly identifying it as reactive rather than preventive; Identifies 'asking nicely with no validation' as the least reliable, worth naming as an anti-pattern rather than a real strategy; Mentions a concrete fallback behavior (e.g. a safe default verdict) for when parsing fails, rather than assuming failure won't happen.

A model's raw output is free-form text — but your code usually needs a specific, parseable shape it can actually run logic against. Getting from one to the other reliably is its own problem, with solutions that differ a lot in how much you can trust them.

11 min read

Check your understanding

Every option explains itself — why the right one is right, and why each wrong one is wrong. Miss one and it comes back later on a spaced-repetition schedule.

Question 1 of 3

Why is provider-enforced schema output considered more reliable than simply prompting the model to 'respond in JSON'?

Because the constraint is enforced during generation itself, not just requested and checked afterwardBecause schema-enforced output runs on a completely different, more accurate modelBecause schema-enforced output is always faster to generate than free-form JSON

Stuck? Ask the AI tutor

Hints without spoilers, and a plain explanation of why a wrong answer was wrong — answered only from this lesson. Free account, no card.

Interview questions on Structured Output & JSON Mode

What interviewers actually ask about this, and what a strong answer has to cover.

You need an LLM's response to reliably parse into a specific JSON shape your code can act on programmatically. What options do you have, and how would you rank them by reliability?

A strong answer covers

Names provider-enforced schema constraints as the most reliable option, because generation itself is constrained rather than checked afterward
Names prompted JSON with parsing, validation, and a retry or fallback as the middle option, correctly identifying it as reactive rather than preventive
Identifies 'asking nicely with no validation' as the least reliable, worth naming as an anti-pattern rather than a real strategy
Mentions a concrete fallback behavior (e.g. a safe default verdict) for when parsing fails, rather than assuming failure won't happen

Likely follow-ups

If a response parses cleanly and matches your schema exactly, does that guarantee the data inside it is correct? Why or why not?
What would you do differently if the provider you're using doesn't support schema-enforced output at all?

Practice these with the AI interviewer →

The plain-language idea

An LLM generates free-form text by default — even when you ask for JSON in the prompt, what comes back is still fundamentally a text completion, and text completions can go wrong in text-shaped ways: an extra sentence before the JSON starts, a trailing comma, a field renamed to something close-but-not-exact. That's a real problem the moment your code needs to do something programmatic with the response — parse a field, insert it into a database, render it into a specific UI shape — rather than just display it as prose.

There are three real approaches, and they differ a lot in how much you can trust them:

Schema-enforced output — the provider itself constrains generation so that the output is guaranteed to match a JSON schema you supply. This is the most reliable option, because the constraint is enforced during generation, not checked afterward — the model structurally cannot produce a response that violates the schema.
Prompted JSON with validation and retry — you instruct the model to output JSON in your prompt, then parse the response and validate it against your expected shape. If parsing or validation fails, you retry (possibly with a follow-up telling the model what went wrong). This is less reliable than schema enforcement because nothing stops the model from producing malformed output in the first place — you're catching failures after they happen, not preventing them.
Asking nicely and hoping — putting "respond in JSON" in the prompt with no validation or retry logic at all. This is the least reliable approach: any malformed response reaches your code as-is and either crashes something downstream or silently corrupts data.

Treat schema-enforced output as the real fix whenever the provider supports it. Defensive parsing (option 2) is the honest fallback for cases where it isn't available or reliable enough on its own — not a substitute for it when a stronger option exists.

See it in a real thing: this app's own defensive fallback

// app/api/ai/interview/route.ts — the interview grader asks for strict
// JSON, but doesn't trust that blindly; it parses defensively and falls
// back to a safe default instead of crashing or passing bad data through.
const parsed = parseInterviewVerdict(out.content);

let result: EvalResult = parsed ?? {
  verdict: "gap",
  missingConcept: null,
  followUp:
    "I couldn't read that evaluation — the model didn't answer in the " +
    "expected format. Try rephrasing your answer, or pick a different " +
    "model in Settings.",
};

This is option 2 in practice, done honestly: the system prompt in lib/ai/prompts/interview.ts asks the model for strict JSON, but the route never assumes that request was honored. parseInterviewVerdict attempts to parse the response, and if that fails, the code falls back to treating the response as a "gap" verdict — a safe, defined default — instead of letting a malformed response crash the request or silently propagate broken data into the UI. This is worth internalizing as a pattern on its own: whenever a prompted-JSON approach is your only option (a provider without schema enforcement, a model that doesn't reliably honor it), the fallback path is not optional polish, it's the difference between a graceful degradation and a production incident.

What people usually miss

Watch out

"Respond in JSON" in the prompt is a request, not a guarantee. Nothing stops the model from wrapping it in prose, adding commentary, or drifting from the exact shape you asked for.
Schema-enforced output is enforced at generation time, not checked afterward. That's the actual reason it's more reliable than prompted JSON — the difference between "structurally cannot violate the schema" and "was asked nicely to follow one."
Defensive parsing needs a real fallback, not just a try/catch that swallows the error. Catching a parse failure and then doing nothing with it just moves the crash somewhere less visible.
Validating the shape isn't the same as validating the content. Even a response that parses cleanly and matches your schema can still contain a hallucinated or wrong value — structural correctness and factual correctness are two different problems.

Why this matters in interviews

"How do you make sure an LLM response is actually usable as data in your application?" tests whether a candidate reaches for real reliability mechanisms — schema enforcement, defensive parsing with a defined fallback — instead of assuming a prompt instruction alone is sufficient.