A few weeks ago Garrit Franke published a short post that put a name to something I'd been working around for months without a vocabulary for it. He splits an LLM's context window into two zones: a smart zone, where the model is sharp, and a dumb zone, where attention thins out and the model starts forgetting what you told it earlier in the session. The cutoff sits somewhere around 100k tokens, and it doesn't move much no matter how large the advertised window is.
That framing is doing real work, because it reverses the default assumption. When a vendor prints "1M tokens" on the box, you read it as a working set: a million tokens you can fill and reason over. Franke's claim is that the usable part is closer to the first 100k, and everything past it is borrowed against falling attention. I think he's basically right, with one important qualification I'll get to. First, why the number on the box and the number you can use have drifted so far apart.
Where the window stops working
The architectures behind long context work. They just paper over a limitation in the attention mechanism that they don't actually solve, so the advertised window grows every release while the usable fraction lags behind. Franke points to two pieces of evidence: RULER, and Chroma's report on context rot. Both show that effective context is a fraction of the advertised number and that performance degrades gradually as the window fills, rather than holding flat and falling off a cliff at the limit.
This matters specifically for coding agents, because an agent fills context faster than a human chatting ever would. A few file reads, one long debug session, a sprawling test run, and you've crossed 100k before lunch. The agent walks you into the dumb zone on your behalf, without asking, and the symptoms (forgotten constraints, re-introduced bugs, drift away from the original task) look like the model getting dumber when really the model is fine and the context is bloated.
Tools have started compensating. Claude Code's auto-compaction summarizes the session history when it gets long and starts fresh from the summary. That helps. But it has two structural weaknesses Franke names cleanly: it kicks in after you've already spent time in the dumb zone, and the summary is produced by a model that's already degraded. You're asking a tired model to decide what mattered. It's better than nothing, and I'd still rather not lean on it.
The split is about how you fill the window, not whether rot exists
Here's where the discussion got interesting, and where I have to adjudicate, because the comment thread did not agree with the post.
Several builders reported no degradation at the scales Franke warns about. One said that since Anthropic shipped the 1M-token window on subscription plans, he routinely pushes past 500k tokens, sometimes up to 800k, without seeing the problem, and only notices something above 900k, and even then less severe than the post describes. [1] Another was blunter: "Hasn't been my experience at all — 1M window is a very clear upgrade working with Claude Code." [2] A third runs long tasks into the 200k zone and clears only as a precaution, based on anecdote rather than any bad experience he'd actually had. [3]
If you take those at face value, the post looks overstated. I don't think it is, and the reconciliation is in how each person fills the window. The builder running clean at 800k describes filling it with "a single task, or a series of tasks related enough to warrant the same context." [1] The cautious one says he always loads the core source files and the files under work up front. [3] That is curated, coherent context: high signal, low redundancy, all pointing at one goal. The dumb-zone failure mode Franke describes comes from the opposite, the context a coding agent accumulates on its own — duplicated file reads, dead debug output, abandoned branches of reasoning. Same token count, very different signal density.
So the disagreement is mostly not a contradiction. Context rot is real and measured; the people who don't feel it are unconsciously managing signal density and working on tasks coherent enough that the rot stays below their perception threshold. The post's "100k cutoff" is a heuristic, not a hardware limit, and treating it as a hard line is the one place I'd push back on it. What's robust is the underlying instruction: assume the usable fraction is small, and don't let an agent fill the rest with noise.
What builders actually said
Strip out the "works fine for me" replies and the thread converges on a single move: get state out of the live session and into something external.
The most structured version came from a builder who acts as the AI's product manager, insisting it write a short PRD for every feature before building it. Each feature gets its own conversation, and the PRDs give the model a durable reference to past decisions without carrying all of that history live in the window. He framed it as a happy medium: enough structure to stop the agent going off the rails, enough memory to reference earlier choices. [4]
“I'm getting a lot of mileage out of basically acting like the AI's Product Manager, and insisting that it writes up short PRDs for every feature we propose to build. That gives it a reference over time of everything that has been built, but also makes it less liable to drift with each one. Each one gets its own conversation. For me this is a happy medium between stopping it going off the rails but also making sure it can reference past decisions when it needs to. The one thing I dislike about Pocock's method (not to use PRDs so much but to have an in depth discussion to get alignment) first is”
Another described a more radical architecture he calls transposing the agent loop: instead of one long-running agent, run many short agent loops, each generating its prompt dynamically from structured data and advancing the state one small step toward the goal. [5] That's the same insight as the PRD approach taken to its conclusion. The durable state lives in structured artifacts outside any session, and each session is short-lived and disposable.
Then there's the minimalist camp. One commenter said he hits /clear constantly out of habit, wanting to finish each task with minimal context, partly because it lets him re-run a task from known seed conditions if he doesn't like the result. [6] Reproducibility, not just attention hygiene. Another goes further and stops reasoning about sessions at all: he treats LLMs and agents as black boxes, fires each feature request at several of them, compares outcomes, git reset --hards the ones he dislikes, and keeps an ELO score of which agents satisfy his demands best. [7]
“Considerations about what goes on in agents internally will probably not be part of software development for long. Personally, I already see LLMs and agents as blackboxes. I give each feature request to multiple LLMs and then compare the results. I don't manually use "sessions" at all. I just look at the outcome. When I dislike it, I "git reset --hard", change my prompts and restart the feature request. To have an ongoing sense of which agents perform best, I keep a log and calculate an ELO score of which agents meet my demands best. This score is imporant to me, not so much how the agent achi”
I find that last one clarifying even though it's the opposite of Franke's careful-curation approach. If your loop is cheap enough and your evaluation is good enough, you don't manage context, you discard and re-roll. It only works when restarting is cheap, which is exactly the condition the artifact approach is trying to create.
One more, on the compaction problem itself: a commenter suggested compacting context across multiple requests over smaller, overlapping chunks, so the summarization never has to run inside the dumb zone it's trying to escape. [8] I haven't seen a tool ship this, but it's the obvious fix for the "the summarizer is itself degraded" complaint, and I'd expect someone to build it.
The artifact discipline
Pull the threads together and they point the same direction. PRDs, transposed loops, habitual clearing, re-rolling, chunked compaction — these are all ways of refusing to let one session's window become the system of record.
Franke's own habit is the cleanest statement of it. Rather than rely on auto-compaction, he opens a new session and hands it a spec he wrote himself, because a human-authored spec is a higher-signal handoff than any summary a degraded model produces. He gets to decide what carries forward. He points at projects like obra/superpowers and mattpocock/skills that structure whole agent workflows around small named artifacts (PRDs, plans, skills, sub-agent handoffs), each one a way to move information out of the live session so attention has fewer things to fight over.
The mental model I've adopted from this is to treat the context window as a budget. Assume only the first chunk is really working for you. Everything you can move out of the live session and into a written artifact is one less thing competing for attention, and one more thing the next session (or the next person) can pick up cleanly. The artifact is the breadcrumb.
This is also why I think the "1M window is a clear upgrade" camp [2] and the "don't trust large context windows" camp are both right and not actually arguing. A bigger window is a genuine upgrade for the curated, single-task case. It does not change the discipline you need when an agent is filling that window for you. The number on the box went up. The part you should rely on did not move much.
If you're building on this today
Watch your token count the way you'd watch a memory budget, not as a limit you're allowed to approach but as a signal of declining quality you're already paying for. A few concrete things I'd take from the thread:
Treat anything past roughly the first 100k tokens of an agent session as suspect, not unusable. It's a place quality starts sliding, and the slide is gradual, so you won't get a warning.
Don't wait for auto-compaction to rescue a bloated session. Write the handoff yourself. A spec or a PRD you authored beats a summary written by a model that's already in the dumb zone, and you decide what survives the reset.
If your tasks are coherent and your context is curated, you can run much deeper than 100k without trouble, as several builders reported running clean past 500k. [1] The risk isn't the token count by itself, it's signal density. An agent left to read and re-read files unsupervised will rot a window long before you hit any advertised limit.
And keep restarts cheap. Whether you do it with PRDs, transposed loops, or git reset and a re-roll, the builders getting the most out of these tools have made starting over so cheap that context rot stops being a thing they manage and becomes a thing they route around. That's the part of this I'm most convinced by.
Sources
Primary source: Don't trust large context windows
- HN comment by kelnos — kelnos · community
- HN comment by mock-possum — mock-possum · community
- HN comment by PeterStuer — PeterStuer · community
- HN comment by kristianc — kristianc · community
- HN comment by afc — afc · community
- HN comment by mcapodici — mcapodici · community
- HN comment by mg — mg · community
- HN comment by da-x — da-x · community