Handling AI Latency in UX

The plain-language idea

Most API calls a frontend developer builds around complete in well under a second, and a generic loading spinner is fine for that. An AI completion is a different order of magnitude: even with streaming, there's real latency before the first token arrives (the model has to process the whole prompt before it can start generating), and a non-streamed response can take several seconds to finish entirely. A spinner that's appropriate for a 200ms fetch reads as broken or frozen once the wait stretches past a second or two.

This is a UX problem with real patterns, not just an engineering one:

• Streaming is the single biggest lever. Showing tokens as they arrive turns "nothing happens for four seconds, then a wall of text" into continuous visible progress — the perceived wait drops even though the total time doesn't change.
• A distinct "thinking" indicator, separate from a generic spinner, sets the right expectation before the first token arrives — it signals "the model is working on this" rather than "something might be broken."
• Progressive disclosure shows partial results as they become available where the shape of the task allows it, instead of holding everything back until the very end.
• Explicit status for genuinely slow operations — a multi-step agent task, a tool call, a retrieval step — replaces one long silent wait with visible intermediate progress ("searching your documents," "reasoning about the result"), which matters even when total latency doesn't change, because uncertainty about whether it's still working is what actually erodes trust.

See it in a real thing: a multi-step assistant response

[Immediate]     "Thinking…"                          <- distinct from a generic spinner
[~1-2s later]   "Searching your documents…"           <- intermediate status, not silence
[~3s later]     "Found 4 relevant sections. Drafting…"
[~4s later]     "Based on the error logs, the issue |" <- streaming begins, text grows token by token

Each stage tells the user something true about what's happening right now. Compare that to a single spinner shown for the entire four seconds: functionally the same total wait, but the user has no way to distinguish "still working normally" from "hung." That distinction is the whole reason latency needs deliberate UX handling instead of being treated like any other loading state.

What people usually miss

Watch out

• Streaming's value is perceived latency, not actual latency. The total time to full completion doesn't necessarily change — what changes is that the user sees continuous progress instead of silence, which measurably changes how long a wait feels.
• A "thinking" state and an "error" state need to look genuinely different, not just different text on the same spinner — a user staring at an ambiguous loading state for 8 seconds with no distinction from a hung request will often refresh or abandon, undoing whatever latency work was done server-side.
• Progressive disclosure only works when the task's shape supports it. A response that has to be internally consistent end-to-end (e.g. a single verdict that depends on the whole answer) can't safely show a partial, possibly-about-to-change result — know which case you're in before reaching for this pattern.
• None of this is a substitute for actually reducing latency (smaller prompts, fewer round-trips, a faster model where quality allows) — UX patterns manage the experience of waiting, they don't replace the work of shortening the wait itself.

Why this matters in interviews

"A user says the AI feature feels slow — what would you check, and what would you change?" is a real question that separates candidates who only reach for "add a spinner" from ones who can name streaming, staged status, and the perceived-vs-actual latency distinction as the actual levers.