Tagged “budgeting”
-
Context budgets for multi-turn conversations
History grows every turn until it crowds out everything else. Window policies, structured state extraction, and why agent loops overflow fastest.
-
What to cut first when context overflows
A cut order for when the budget doesn't balance: which component gives up space first, why history usually loses to retrieval, and how to degrade visibly.
-
The context window is a budget
Five things compete for every request's window, including the answer. How to write the allocation down, and why the output reservation is forgotten.
-
What deserves a permanent seat in the window
Some material is needed on nearly every request. The test for making it a resident line item instead of fetching it, and the cost of getting it wrong.
-
Giving each agent its own window
One window shared by a long task overflows. Splitting work across separate windows with a handoff is an allocation strategy, and it has real costs.
-
How much of your window is earning its place
Every component costs tokens on every request. How to find which ones change the answer, using ablation and citation coverage rather than intuition.
-
What formatting overhead costs you
Delimiters, labels and serialisation are paid per item in the prompt. How to price the per-passage wrapper and keep the parts that earn their tokens.
-
Fixed shares or priority order?
Two ways to divide a window: give each component a percentage, or fill in priority order. Where each fails, and why request types need their own profiles.
-
Do your few-shot examples still earn their space?
Examples are often the largest block in a system prompt and the least examined. How to run the ablation, and what to do with each outcome.
-
The output format is a budget decision
Verbose response formats spend the window twice — once on the schema, once on the answer. How to price a format and pick a cheaper one.
-
Budgeting for tokens you never see
Reasoning traces, retries and discarded drafts consume window and billing without appearing in the answer. How to find them and reserve for them.
-
Reserving room for the answer
Input and output share the window on most models. How to size the output reservation, what an unset cap means, and why truncation looks like success.
-
Auditing a system prompt nobody trimmed
System prompts grow one patch at a time and nothing is ever removed. How to audit yours by section, delete safely, and keep it from regrowing.
-
Tool definitions are a line item
The tool block is paid on every request whether or not a tool is called. How to audit it per tool, what to trim, and what terser schemas cost you.