When a model runs out of context

A language model seems to remember the whole conversation. It doesn't. For every reply it re-reads a limited slice — the context window — and answers based only on that.

The window has a fixed size

Think of the context window as a table that only holds so much paper. Add something new and something old has to make room. In a chat that means: the longer the conversation, the sooner the early messages slide out of the window. The model then can't refer to them any more — not because it "forgets", but because it simply no longer sees them.

How large that window is differs considerably between models and grows with every generation: current frontier models such as Claude Opus 4.8, GPT-5.5 or Gemini 3 take around a million tokens, more compact models such as Claude Haiku 200,000 (as of July 2026). But a bigger window only pushes the problem back; it doesn't abolish it.

"Lost in the middle": the middle suffers first

Even when everything still fits in the window, not every spot is equally present. Models use the beginning and the end of the context more reliably than the middle. An important detail buried in the middle of a long text is more easily missed than the same detail at the very top or bottom. Research describes this effect as lost in the middle.

The study that gave the effect its name (Liu et al. 2023) measured it: when GPT-3.5 had to pull the key passage from the middle of 20 documents instead of the start, accuracy fell from 75.8 to 53.8 percent — less than the model got with no documents at all (56.1 percent).

Four things that help

You don't need to understand the mechanics to work around them. Four habits go a long way:

1. New session for a new topic. When the conversation drifts or the model starts contradicting itself, a fresh chat often helps more than another correction. An empty window thinks more clearly.
2. Summarise as you go. Ask the model to capture the current state in a few sentences, then carry on from that summary. That actively puts what matters back into the window instead of hoping it's still there.
3. Important things last. What counts right now belongs in your final message — not ten steps earlier. The end of the context is the most present spot.
4. Let it compact. Some tools summarise the conversation so far and replace it with that summary — in Claude Code that's /compact. It usually saves what matters, but rarely every detail: take a quick look afterwards to check the essentials survived.

Something to touch

It's clearest when you can see it: a fixed-size window overflowing as the conversation grows. That's exactly what you can try hands-on in the Lab — with a selectable window size and a mode that shows what a summary rescues and what evaporates.