You asked Tarsk to find a flaky test. Fifteen tool calls later the agent has grepped logs, read configs, and re-run suites. Call sixteen still carries every earlier dump. That re-send tax is where long agent turns dig into your bill and your context window.
Tarsk now runs microcompaction during the turn: older tool outputs shrink before the next model call, while the latest work stays full fidelity. You keep exploring. The window stops ballooning for free.
The re-send tax
An agent turn is a loop. The model calls tools, reads results, plans the next step, and calls again. Providers do not remember the previous step for free. Each request re-includes the transcript for this turn: your ask, every tool call, and every tool result so far.
By mid-task the bulk of that transcript is tool output you already used:
- Full file reads from three refactors ago
- Grep dumps that pointed at the wrong path
- Build logs from the first failing run
Those tokens bill again on every later call. Coding agents burn 5–30× more tokens than chat because of this loop. Unit prices fell; volume rose. Mid-turn tool results sit in the hottest part of that volume.
Prompt caching can discount stable prefixes. It does not free the window for new work. If a 12KB log still occupies the live context, the model has that much less room for the next read and for its answer.
What microcompaction does
Microcompaction is housekeeping inside the active turn. Before each model call, Tarsk estimates how full the context is. When usage sits near the effective window and enough older tool mass would free a meaningful chunk of tokens, older eligible results become short digests.
Defaults in plain language:
| Rule | Default | Why it exists |
|---|---|---|
| Keep newest eligible results | 3 | The work you are acting on stays exact |
| Protect newest tool tokens | ~40,000 | Recent raw material for multi-step fixes |
| Fire near the window | ~20,000 tokens under the effective window | No thrash on thin turns |
| Minimum savings | ~20,000 tokens | Skip if the clean-up is too small |
| Ignore tiny results | under ~2,000 characters | Not worth shrinking |
Errors stay full. Small results stay full. Tools outside the high-volume set (subagents, tasks, user prompts, observation retrieval, and similar) stay full.
Replacements prefer a stable digest with a retrieval id when Tarsk has stored the original as an observation. The model can call get_observation and pull exact bytes again. If storage is unavailable, the digest still names the tool and size, and tells the model to re-run the tool if it needs the full log.
Your chat UI still shows the pre-compact tool cards. Microcompaction changes what the model receives on the next request. It does not rewrite your audit trail.
Observations, microcompaction, auto-compact
Tarsk manages context at three scopes:
| Observations | Microcompaction | Auto-compact / /compact | |
|---|---|---|---|
| When | When a large tool result finishes | Before later model calls in the same turn | Between turns, or on demand |
| Targets | One bloated output (large text) | A stack of mid-size and aging results | Whole conversation history |
| Goal | Keep exact bytes off the live message | Stop mid-turn growth | Keep multi-turn chats slim |
Observations catch oversized single results at ingest and expose a preview plus id. Microcompaction ages older results as the same turn continues, including previews that would otherwise ride along forever. Auto-compact summarizes finished exchanges so tomorrow’s message does not re-send last week’s investigation.
You need all three for different jobs. Observations stop a single tome. Microcompaction manages the re-send tax inside long loops. Auto-compact manages the conversation after the turn ends.
Effect on token usage
Token savings show up as reduced input growth on later steps of a busy turn.
Exploration-heavy work. Early greps, finds, and file reads decide where to look. Once the agent locks onto two or three files, older search results matter less. Microcompaction collapses those older blobs toward short digests and leaves the recent reads intact.
Debug loops. First failing test logs are useful once. The fifth re-run still needs the latest log at full fidelity; the oldest ones become digests or observation refs. That frees room for the next command and for the model’s diagnosis.
Multi-file edits. Read-edit chains leave stale full files behind. Keeping the newest few results and roughly 40k of recent tool tokens mirrors how you debug by hand: care about the last outputs, consult the archive when something specific comes up.
Gates keep the feature quiet when savings would be small. Short turns with a handful of tools skip microcompaction. Only when costed mass past the protect budget looks larger than about 20k tokens does Tarsk apply digests. Replacements stay memoized for the rest of the run, so a compacted result does not thrash rewrite cycles and stays friendlier to prompt caching.
Light Q&A without tools barely moves the meter. Aggressive lost-in-the-codebase sessions, flaky-test hunts, and wide refactors benefit most. Do not expect a fixed percent off every session.
On dense turns you should see:
- Peak input track recent tool work instead of the sum of every prior tool
- More headroom for the model’s final answer inside the same window
- Fewer mid-task “context full” cliffs that force a restart
What the agent still knows
Compacted digests stay recoverable when observations back them. Large and mid-size results can keep an obs_ handle. The model retrieves pages of the original on demand, subject to normal observation limits.
Heuristic digests without storage are lossier. The model may re-run read, bash, or grep if it needs exact content again. Spending one fresh tool call beats carrying a 20KB corpse for the rest of the turn.
Error results never shrink. A failed permission, broken build, or stack trace stays complete so the agent does not invent a success path over a truncated failure.
When it skips
Microcompaction stays off the hot path when:
- The turn sits well under the window
- Clearing older results would free less than the minimum savings
- Eligible full results number three or fewer (or already fit under the protect budget)
- The model’s context window is unknown
- You disable it for debugging
Thin asks stay thin. Fat agent loops get the haircut.
How to use it
Nothing to toggle for normal use. Ship a multi-step coding task and let the agent run. Context holds out longer on heavy tool sequences. Use /compact or rely on auto-compact when you finish a large thread and want the next user message on a lean history. That is a different layer from mid-turn microcompaction.
Keep watching spend and context indicators you already use. Microcompaction is one slice of Tarsk’s token stack: compression and observations cut bloat at ingest; microcompaction manages age inside the turn; auto-compact cleans the conversation between turns.
Takeaways
- Every later model call used to re-invoice old tool output inside the same turn. That re-send tax is the mid-turn cost driver.
- Microcompaction keeps the newest results full and digests older eligible tool output once the window is near full and savings clear a high bar.
- Observations make most digests recoverable; auto-compact handles multi-turn history. Use all three for work that explores, edits, and debugs over many steps.
Open Tarsk, pick a capable coding model, and throw a multi-file investigation at it. The agent still searches hard; you stop paying full price for tool results the model already acted on.