When I started using these models in earnest, the context window was a closet. A few thousand tokens. You learned to pack it like a backpacker counting grams: cut the greeting, drop the examples, paraphrase the document you actually needed the model to read. You were always one paragraph from the door slamming shut.
A year later the closet is a cathedral. Hundreds of thousands of tokens, then a million, then more. You can pour an entire codebase in, a quarter of legal discovery, a novel and its three sequels, and still have nave to spare. The first time you do it there is a genuine, vaulted feeling, the kind you get walking into a tall stone room. So much space. So much that could happen here.
I spent the year working in that room and building things on top of it for a living, and the feeling never quite went away. Neither did the suspicion underneath it. The cathedral kept getting bigger. What I had to put inside it did not.
the awe is real, and it is a trap
It would be cheap to pretend the scale isn't impressive. It is. Watching a model hold a hundred thousand lines of code in working memory and answer a question that needs three files at opposite ends of it is not a parlor trick. It feels closer to the first time you saw a flying buttress and understood that the wall was being held up by something clever and invisible.
But the cathedral was never the point of the cathedral. Nobody quarried limestone for forty years so that tourists could admire the ceiling height. The room was built to hold something: a relic, a rite, people who showed up with a reason. The architecture serves a contents problem, not the other way around.
This is the thing I kept getting wrong, and kept watching other people get wrong with more money than me. We treated the size of the window as the achievement. Bigger context, the reasoning went, means a smarter system, the way more shelf space means a better library. So we filled it. We stuffed entire wikis and Slack histories and every document anyone had ever touched into the prompt, on the theory that more material in the room could only help.
It did not only help. Past a certain point it actively hurt, and the way it hurt is the most useful thing the year taught me.
what a full room actually does
Give a model forty documents when the answer lives in two, and you have not made it better informed. You have made it a person trying to find their keys in a house where every surface is covered in keys. The relevant fact is still there. It is just harder to attend to, sitting in a pew somewhere between two thousand irrelevant ones.
The failure is quiet, which is what makes it dangerous. The model does not error out. It does not tell you it is overwhelmed. It produces something fluent and confident and a little worse, anchored to whatever happened to be loud in the noise you gave it. You get an answer shaped like the right answer, built from the wrong attention. By the time you notice, you have already shipped it.
More room is not more thought. It is more room.
I started thinking of it as the difference between a research library and a desk. A library that contains everything is a triumph of storage and a catastrophe of focus. The scholar's actual work happens at a desk that holds six books, open to the right pages, chosen on purpose. The skill was never acquisition. It was selection, and selection means subtraction, which is the one thing more space quietly discourages you from doing.
the diminishing returns of more
There is a pattern here that goes well beyond machines, which is why I think it is worth your time. The first increment of capacity changes everything. The closet becoming a room lets you do things you simply could not do before. The room becoming a hall lets you do a few more. The hall becoming a cathedral lets you do almost nothing new, and it hands you a maintenance problem you did not have.
We know this curve from everywhere. The second monitor changes your life; the fourth is decoration. The first ten thousand words of research sharpen the argument; the next hundred thousand bury it. More money fixes poverty and then, past a surprisingly modest line, fixes very little and complicates a lot. The shape is always the same. Capacity is a tool that solves a scarcity, and once the scarcity is gone, more of it stops being an answer and becomes a question you have to manage.
The industry is currently in love with the steep part of the curve and refuses to look at the flat part. Every release is a larger window, a longer context, more, and the more is genuine, and the more is also, increasingly, beside the point. We keep extending the cathedral while the congregation thins out, and we mistake the size of the building for the strength of the faith.
the thing worth putting in
So here is what a year inside these rooms actually taught me, and it is not a lesson about machines.
The hard part was never the container. It was deciding what deserved to go in it. The bottleneck moved, the way bottlenecks do, from the thing that used to be scarce to the thing that suddenly is. Storage and recall used to be the constraint, so we obsessed over them. Now they are nearly free, and the constraint is something older and more stubborn: knowing what the question really is, and bringing only what answers it.
A cathedral with the wrong relic is just an expensive room. The architecture cannot rescue a contents problem. No window is large enough to make up for not knowing what you are looking for. A small, sharp, deliberate prompt from someone who understands their actual problem will beat an enormous one from someone who hopes volume will do the thinking. It did before the machines and it does now.
The space is extraordinary. I do not want to undersell the engineering, which is some of the most impressive of my lifetime. But the awe is a feeling about the room, and the work is a question about the contents, and the year I spent confusing the two was the year I learned to tell them apart.
Build the cathedral. Then walk in with one thing worth its silence.