A 200K-token context window does not mean 200K tokens of reliable reasoning. Accuracy on retrieval and multi-step tracking starts falling long before the advertised limit, often by the time a few tens of thousands of tokens are in play. The gap between the number on the spec sheet and the size at which the model still reasons well has a name in the research literature, and it matters for anyone running long agentic coding sessions.
TL;DR — does a larger context window mean more usable context?
No. The advertised context window is a hard input ceiling, not a guarantee of accuracy across that span. Empirically, model performance on retrieval, reasoning, and variable-tracking tasks degrades steadily as the input grows, and the degradation begins far below the maximum. Recent benchmark work measures this directly and proposes the Maximum Effective Context Window (MECW) — the input length at which a model still meets an accuracy threshold — which is consistently a fraction of the advertised window. For long coding sessions this means the model gets less reliable as the session fills up, regardless of whether you have "hit the limit."
What is the difference between advertised context and effective context?
The advertised context window is the maximum number of tokens a model will accept as input before it refuses or truncates. It is a capacity figure: 128K, 200K, 1M. It tells you what the model can ingest, not what it can use well.
Effective context is the span over which the model still performs a task to an acceptable accuracy. The 2025 study Context Window Performance Degradation (Singh et al., arXiv:2509.21361) formalizes this as the Maximum Effective Context Window — the longest input at which the model's task accuracy stays above a chosen threshold. The headline finding is that the effective window is a fraction of the advertised one, and that it varies by task: a model with a large nominal window can have a far smaller effective window for, say, multi-fact retrieval than for a single lookup.
Here is the distinction in extractable form:
| Property | Advertised context window | Maximum Effective Context Window (MECW) |
|---|---|---|
| What it measures | Maximum tokens accepted as input | Input length where accuracy stays above threshold |
| Set by | Architecture / serving config | Empirical benchmark on a task |
| Typical magnitude | 128K – 1M tokens | A fraction of the advertised window |
| Task-dependent | No (fixed per model) | Yes (varies by retrieval, reasoning, tracking) |
| Degrades gradually | No (hard cutoff) | Yes (accuracy slopes down as input grows) |
| What you control | Which model you pick | How much you put in the window |
Why does accuracy degrade far below the limit?
The window is not a uniform medium. Tokens at different positions are not equally easy for the model to attend to, and the harder a task is to perform at a given position, the sooner accuracy slips.
How does attention dilution cause it?
A transformer attends across every token in context through the attention mechanism. As the number of tokens grows, the attention budget for any single relevant token is spread thinner against a larger field of competing tokens. The relevant fact does not disappear, but the signal that points to it has to compete with far more noise. The well-documented "lost in the middle" effect — models recall content at the start and end of a long input more reliably than content buried in the middle — is one visible symptom of this. The arXiv:2509.21361 results extend the picture: degradation is not only positional, it is cumulative with total length.
Why does variable tracking fail first?
Single-fact retrieval ("find the one sentence that mentions the API key") is the easy case, and it holds up the longest. The tasks that break early are the ones that require the model to track multiple pieces of state across the whole input — counting, following a chain of references, holding several variables and updating them as new information arrives. The Singh et al. benchmark separates these task types and shows that multi-variable tracking has a markedly smaller effective window than simple lookup. The intuition is direct: every additional variable the model must keep live is another thing competing for the same diluted attention, and the probability that all of them stay accurate drops as the input grows.
This is the finding that matters most for coding work, because a long agentic session is almost entirely variable tracking. The model is holding the file it just edited, the test that just failed, the decision it made twenty turns ago, and the plan it is working through — all at once, all as live state.
What does this mean for long agentic coding sessions?
A coding agent's context fills with exactly the kind of content that triggers the worst degradation: many interdependent facts that must all stay accurate, accumulated over a long session. The window does not have to be near full for quality to slip.
Concretely, as a session grows you tend to see:
- Earlier decisions get re-litigated. A constraint established near the start ("use the repository pattern here, not a direct query") sits in the diluted middle of a long context by hour two. The model stops weighting it, and re-proposes the thing you already ruled out.
- Cross-file reasoning weakens. Tracking how a change in one file affects three others is multi-variable tracking. It is precisely the task class with the smallest effective window, so it degrades while single-file edits still look fine.
- Tool results from early turns stop being used. The grep output from turn five is still technically in context, but its attention signal is buried under everything since. The model behaves as if it never read it.
- Instruction-following gets sloppy. Formatting rules, naming conventions, and "always do X" directives given early are long-range dependencies. They are among the first things to soften.
The trap is that none of this announces itself. The session has not errored, has not compacted, has not hit a limit. It is simply operating past its effective window, where output still looks confident but the underlying tracking has quietly decayed. Confident-but-degraded output is the failure mode that costs the most, because there is no banner telling you it happened.
If you run sessions long enough to live in this zone, it is worth having something watch for the symptoms rather than trusting the absence of an error. The watchdog we ship for session reliability when context has degraded does exactly that — more on that below.
How do you work around context degradation?
You cannot raise a model's effective window from the outside, but you can keep your working set inside it. The mitigations all reduce to one principle: put less in the window, and make what is there matter.
- Scope each task narrowly. A request that touches three files and a migration is three or four tasks. Run them as separate, focused exchanges so each one's relevant state fits comfortably inside the effective window rather than competing with unrelated context.
- Chunk large inputs and summarize between chunks. When you must feed a large file or log, process it in sections and have the model emit a compact, verified summary of each before moving on. The summary is far smaller than the source and keeps the live state inside the effective span. Verify each summary — a wrong summary is worse than a long input.
- Externalize durable state. Project conventions belong in a file the agent reloads every turn (
CLAUDE.md,AGENTS.md, a README), not in conversational memory that drifts to the diluted middle. Reloaded-every-turn content effectively sits at the high-attention edge of the window each time. - Start fresh sessions at task boundaries. The cheapest reset is a new session. When a sub-task finishes, open a clean window and seed it with a short, curated statement of where things stand. You trade a few minutes of re-priming for a model that is back inside its effective range.
- Prefer subagents for bounded sub-tasks. Let an isolated agent do the wide-context work in its own window and return one compact, high-signal result to the main session. The parent keeps its window small; the child's degradation is contained to a throwaway context.
- Pick the model by effective window, not advertised window. If your task is multi-variable tracking, a model with a smaller nominal window but a higher effective window on that task class is the better choice. The advertised number is a poor proxy for the thing you actually need.
None of these are exotic. They are the discipline of keeping the working set small — the same discipline good engineers already apply to function scope and module boundaries, applied to the context window.
What about reliability once a session is already degraded?
The mitigations above are preventive. The harder problem is the session that is already deep in the degraded zone: it has not crashed, it has not compacted, it is just getting quietly worse, and you may not notice until the output is wrong in a way that costs you an hour to unwind.
This is a monitoring problem, not a model problem. The signals that a session has drifted past its effective window are observable from outside the model — repeated questions, re-proposed decisions, ignored earlier results, instruction slippage — and they correlate with the same session-health markers that indicate a stalled or crashed process. A background watchdog that polls session state can surface "this session has been running long and is showing degradation symptoms — consider a fresh window with a curated handoff" before you have lost work to confident-but-wrong output. That is the reliability layer that complements scoping and chunking: the discipline keeps you inside the window; the watchdog tells you when you have drifted out of it anyway.
Frequently asked questions
What is the Maximum Effective Context Window?
The Maximum Effective Context Window (MECW) is the longest input length at which a model's accuracy on a given task stays above a chosen threshold. It is an empirical, task-specific measurement introduced in the arXiv:2509.21361 benchmark, and it is consistently smaller than the model's advertised context window. The advertised window tells you what the model accepts; the MECW tells you how much it can actually use well for a specific task.
Does a bigger context window always mean better long-document performance?
No. A larger advertised window raises the input ceiling but does not guarantee accuracy across that span. Two models can advertise the same window size and have very different effective windows, and a model with a larger nominal window can underperform a smaller one on long-input tasks. Evaluate models on their effective window for your task type — especially multi-variable tracking — rather than on the headline capacity figure.
Why does my coding agent forget earlier decisions even though context is not full?
Because forgetting is driven by effective context, not the hard limit. Earlier decisions sit in the diluted middle of a long context, where attention signal is weakest, so the model stops weighting them well before the window is full. The fix is to keep durable rules in a file the agent reloads each turn and to start fresh sessions at task boundaries, rather than relying on conversational memory to hold long-range state.
Is context degradation the same thing as auto-compaction?
No, though they interact. Auto-compaction is an explicit event where a tool summarizes the conversation to fit the hard limit. Context degradation is the gradual, silent accuracy loss that happens as the input grows, with no event and no banner. You can be deep in degradation long before any compaction fires, which is why an absence of a compaction warning is not evidence that the session is still reasoning reliably.
How do I tell if a session has passed its effective window?
Watch for behavioral symptoms rather than a number: the model re-asks questions you answered, re-proposes options you ruled out, ignores tool results from earlier turns, or softens rules it followed strictly at the start. These are the externally observable signs that long-range tracking has decayed. When they appear, the most reliable response is a fresh session seeded with a short, curated statement of current state.
References
- Singh et al., Context Window Performance Degradation in Large Language Models — the benchmark that introduces the Maximum Effective Context Window and measures degradation by task type. arXiv:2509.21361
- Liu et al., Lost in the Middle: How Language Models Use Long Contexts — the positional-recall effect referenced above. arXiv:2307.03172
- Vaswani et al., Attention Is All You Need — the attention mechanism underlying the dilution argument. arXiv:1706.03762
Closing
The advertised context window is a capacity figure, not a quality guarantee. The size at which a model still reasons reliably — its effective window — is smaller, task-dependent, and where the work you care about actually happens. Scope narrowly, chunk large inputs, externalize durable state, and start fresh at task boundaries to stay inside it.
For the long sessions that drift past their effective window anyway, the 14-day Premium Trial is $0 today and you can cancel anytime before day 15 — it adds the session watchdog that flags degradation symptoms and keeps a resumable record so a fresh window is a clean handoff, not a restart from memory. Prefer to try it without a card? Download free for a 7-day trial with full Pro access, no credit card.