Giving each agent its own window
A single agent working a long task accumulates everything: every tool result, every intermediate step, every dead end. By step twelve the window is mostly a record of steps one through eleven, and the actual task is competing with its own history.
There is an allocation answer to this that is not compaction. Use more than one window. Split the work into stages, give each stage a fresh window, and pass forward only a summary of what the next stage needs.
The arithmetic
A single window carrying everything, hypothetical numbers, for a twelve-step task:
| Step | Accumulated history | Prompt total |
|---|---|---|
| 1 | 0 | 4,400 |
| 4 | 6,200 | 10,600 |
| 8 | 14,800 | 19,200 |
| 12 | 26,300 | 30,700 |
Split into three stages of four steps, each with its own window and a handoff of roughly 800 tokens:
| Stage | Inherited summary | Own history at end | Peak prompt |
|---|---|---|---|
| 1 | 0 | 6,200 | 10,600 |
| 2 | 800 | 6,400 | 11,600 |
| 3 | 800 | 6,100 | 11,300 |
Peak prompt falls from 30,700 to 11,600. That is the mechanism: history is bounded by stage length rather than by task length, so the peak stops being a function of how long the task runs.
The total tokens consumed does not necessarily fall — you now pay a fixed cost per stage, plus the summarisation calls. What changes is the peak, and peak is what overflows.
single_window_peak ≈ fixed + Σ(all steps)
staged_peak ≈ fixed + handoff + Σ(steps in longest stage)
staged_total ≈ (fixed × stages) + Σ(all steps) + handoff_costsRead those two together: staging trades a somewhat higher total for a much lower peak. If you are not near your limit, that trade is a loss and you should not do it.
Where the boundaries go
The split is not arbitrary. Put a boundary where the handoff is small.
Good boundaries have a compact deliverable: a stage that reads twenty sources and produces a list of findings hands over the findings, not the sources. A stage that gathers data and produces a table hands over the table.
Bad boundaries are where the next stage needs everything the previous one saw. If your handoff is 6,000 tokens, you have moved the problem rather than solved it, and you are now paying an extra fixed cost for the privilege.
The test is direct: write the handoff format before you split. If you cannot specify it in a few hundred tokens, that is not a boundary.
Two shapes that work well:
Sequential stages. Gather, then analyse, then write. Each hands forward a structured artefact.
A parallel fan-out with a merge. Several workers each handle one source or one sub-question in their own window, and a final stage combines their outputs. The peak per worker is small, the merge stage carries only the outputs, and the workers can run concurrently. Note the merge stage is now the component at risk of overflow — budget it as the constrained one.
Design the handoff as a budget line
The handoff is the interface, and it should be structured rather than prose:
## Objective
<the unchanged task statement>
## Established
- <facts confirmed, with source identifiers>
## Decided
- <choices made, and why>
## Open
- <what remains, and what was already ruled out>
## Artefacts
- <identifiers of files, records, or results produced>Four properties worth insisting on:
Carry identifiers, not content. Reference the source, do not reproduce it. The next stage can retrieve what it needs. This is the largest single saving available in the handoff.
Carry what was ruled out. Otherwise the next stage re-explores the dead ends, which costs more than the tokens it saved.
Restate the objective verbatim. Task drift across stages is the characteristic failure of this design, and an objective that gets paraphrased at each handoff drifts fastest.
Fix a token ceiling for the handoff and enforce it. An unbounded handoff grows back into the problem you were solving.
What it costs
Say these out loud before building it, because staging is fashionable and its costs are real.
Lost detail at every boundary. The handoff is lossy by design. A later stage cannot notice something in material it never saw, and it cannot know that it cannot. This is the fundamental cost and no format fixes it.
Objective drift. Each stage interprets a summary written by the last one. Over several hops the task can quietly become a related but different task.
Fixed cost multiplied by stages. Every stage pays the system prompt and its tool block. On a three-stage design that is three times the fixed cost, which makes trimming those blocks more valuable than before — see tool definitions are a line item.
Cache fragmentation. Different stages with different system prompts and tool sets means several distinct prefixes rather than one, each needing enough traffic to be worth caching. See designing a prompt prefix that stays stable.
Debugging across windows. A wrong answer now has to be traced through several prompts and handoffs rather than read from one transcript. Log every handoff verbatim, with the stage identifier, or this becomes genuinely painful.
When to prefer compaction instead
Staging is not the default. Prefer bounding history within a single window when:
- the task is genuinely continuous and every step depends on the full detail of prior ones;
- your handoff cannot be made small;
- prompt caching is doing a lot of work for you and one growing prefix is cheap;
- conversations rarely approach the limit anyway.
Prefer staging when the work has natural phases with compact deliverables, when the peak prompt is what breaks, or when parts of the work can run in parallel. The two are compatible: stages internally, with a compaction policy inside each stage, per when compaction collides with your cache and context budgets for multi-turn conversations.
What to measure
- Peak prompt tokens per stage, and the whole-task peak. The number staging exists to reduce.
- Handoff size, distributed. Growth here is the leading indicator that the design is degrading.
- Total tokens per completed task, staged versus single-window. The cost you accepted; know its size.
- Stage count per task, if it is dynamic. Unbounded stage counts reintroduce unbounded cost.
- Re-exploration rate. How often a later stage repeats work an earlier one did. Directly measures handoff quality, and it is the metric that tells you the boundary is in the wrong place.
- End-to-end task success, against the single-window baseline. This design can reduce quality while every token metric improves, and that is the outcome to watch for.