The context window looks like a capacity. It behaves like a budget.
Everything competes for it: the system prompt, the tool definitions, the conversation so far, the passages you retrieved, and the answer the model still has to write. Add them up honestly and the number is usually larger than you thought — and the request that fails at the limit is the easy case. The hard case is the one that fits, costs three times as much, and answers worse than it did with half as much context.
Token Budget is about that allocation. Where the window is actually going, how to count it properly, what to cut first when it overflows, and why the same passage helps in one position and gets ignored in another.
Budget tables, not benchmarks. Formulas you fill in with your own numbers, because anything specific enough to quote is out of date by the next model release.
Latest posts
-
The context window is a budget
Five things compete for every request's window, including the answer. How to write down the allocation, and why the output reservation is the row people forget.
-
Counting tokens before you spend them
Why character-count estimates drift, what message formatting adds that you didn't count, and how to build a budget check that fails before the API does.
-
What to cut first when context overflows
A cut order for when the budget doesn't balance: which component gives up space first, why history usually loses to retrieval, and how to degrade visibly.
-
Where in the prompt things should go
Position affects whether the model uses what you put there. A default layout, why instructions get repeated at the end, and what prompt caching constrains.
-
Context budgets for multi-turn conversations
History grows every turn until it crowds out everything else. Window policies, structured state extraction, and why agent loops overflow fastest.