Multi-Turn Conversation State

The plain-language idea

An LLM API call is stateless. The model doesn't remember your last message, your last conversation, or that you talked to it five minutes ago — each request is evaluated fresh, with no memory of anything outside the text it's handed this time.

So how does a chat UI feel continuous? By cheating, in a completely standard way: the client keeps an ever-growing list of every message in the conversation (role: "user", role: "assistant", back and forth) and re-sends that whole list, or some processed version of it, on every single new turn. What looks like the model "remembering" you mentioned your name three messages ago is really just that message still being physically present in the array you sent this time. Delete it from the array, and the model has no name to recall — there's nothing else to forget it from.

This has two direct consequences that only show up once a conversation runs long:

• It eventually hits the context window's hard limit (the model's total per-request token budget, covered in its own lesson) — a conversation that grows forever without pruning will, at some point, simply no longer fit.
• It gets more expensive every single turn, because most APIs price by total input tokens, and resending the full history means resending everything that came before, again, every time (the cost mechanics live in their own lesson too).

Managing that growth deliberately is the actual engineering problem this lesson is about.

See it in a real thing: a chat array that has to be pruned before it breaks

type Message = { role: "user" | "assistant" | "system"; content: string };

// Every turn appends to this — nothing here is "remembered" by the model,
// it's just resent as part of the request body.
let history: Message[] = [
  { role: "system", content: "You are a helpful tutor for this lesson." },
];

function sendTurn(userText: string) {
  history.push({ role: "user", content: userText });
  // ...call the model with `history`, get back an assistant message...
  // history.push({ role: "assistant", content: reply });

  // Naive: never prune. Eventually `history` exceeds the context window
  // and every turn also costs more than the last one.
}

Two common, honest strategies for keeping this under control as a conversation grows:

• Sliding window — only send the most recent N messages (plus the system prompt), dropping older ones outright. Simple and cheap, but anything said before the window closes is genuinely gone from the model's view.
• Summarization — periodically compress older messages into a shorter summary ("user is debugging a React state bug involving stale closures"), and send that summary plus recent messages instead of the full transcript. Preserves more of the gist at a fraction of the token cost, but the summary is now a lossy, one-way compression — details it drops are just as gone as if they'd been dropped by a sliding window, and the summary itself has to be generated by another model call, which costs tokens too.

What people usually miss

Watch out

• The model has no memory of its own between requests. Every fact it appears to "remember" mid-conversation is only still visible because it's still physically present in the history you resent — there is no persistent state on the model's side to fall back on.
• "Just send the whole history forever" is not a real long-term strategy. It works fine for short conversations and then, without warning to anyone who didn't plan for it, starts failing once total tokens exceed the context window.
• Cost grows with every turn even if the user says nothing new that's expensive. Resending 40 prior messages to ask one more question means paying to reprocess all 40 again, not just the new one — this is exactly why pruning is a cost lever, not just a context-window one.
• Summarization is compression, not memory. A summary is generated once and then treated as fact going forward — if it mischaracterizes something, that mischaracterization now persists into every future turn just as durably as the original detail would have.
• Pruning strategy needs to match the use case. A sliding window is fine for casual chat; a support or interview-prep tool that needs to recall something said many turns ago needs summarization, explicit memory extraction, or retrieval instead — see the RAG lesson for the retrieval-based version of "give the model back something it can no longer see directly."

Why this matters in interviews

"How would you handle a chat conversation that runs for hundreds of messages?" is a standard system-design-adjacent AI question, and naming the statelessness, the context-window ceiling, and a concrete pruning strategy (sliding window vs. summarization, with an honest tradeoff) is what separates a candidate who's actually built a chat feature from one who's only used one.