Tagged “overflow”
-
What to cut first when context overflows
A cut order for when the budget doesn't balance: which component gives up space first, why history usually loses to retrieval, and how to degrade visibly.
-
Size the budget for the tail, not the average
Average requests never overflow. The failures live at p99, where one oversized item or a long conversation breaks a budget built on means.
-
Fixed shares or priority order?
Two ways to divide a window: give each component a percentage, or fill in priority order. Where each fails, and why request types need their own profiles.
-
Reserving room for the answer
Input and output share the window on most models. How to size the output reservation, what an unset cap means, and why truncation looks like success.