A follow-up to Token Saving as Noise Reduction (23 June 2026), which was itself a follow-up to More Context Is Not Always Better. Literature cutoff: 25 August 2026.

What I Claimed in June

The June post made one sentence carry two claims: “every unnecessary token competes for attention, increases inference cost, and makes the next decision less precise.” It then described my agent harness as a sequence of filters: a proxy at the API boundary, a shell-output filter, a compressed file reader, symbol tools, policy files, and quality gates. Each layer was supposed to improve what I called the relevance density of the context before the model reasoned over it.

The first claim, about attention and precision, is a claim about quality. The second, about inference cost, is a claim about money. The June post put its weight on the first and called the argument “not only economic”. It never modelled the second. It treated the cost benefit as a free side effect that points the same way as the quality benefit, so that every token removed saves money in proportion. That assumption is where it went wrong.

The quality claim holds. The cost claim is true but much smaller than the June post implied, it reverses for some kinds of compression, and the filter stack had a cost I never counted.

The Part That Held: Length Itself Costs Accuracy

The June post supported the quality claim with four compression preprints. They show that compressed context can work, not that long context hurts. The direct evidence was already published, and I should have cited it. It is worth separating what each piece shows.

Irrelevant padding. Levy, Jacoby, and Goldberg built reasoning tasks whose relevant content stays fixed while irrelevant padding grows. Averaged over the models they tested, accuracy fell from 0.92 to 0.68 by an input length of about 3,000 tokens, far below any advertised window [[1]].

Retrieval without literal overlap. NoLiMa minimises literal overlap between question and answer. Of 13 models claiming at least 128K tokens of context, 11 fell below half their short-context baseline at 32K; GPT-4o, one of the exceptions, went from 99.3% to 69.7% [[2]].

Length with no distraction at all. This is the result that changed my mind the most, and the June post did not take it into account. Du et al. ensure that models can retrieve all the relevant evidence, then vary only input length. Performance still drops by between 13.9% and 85% across five models and three task types. The drop persists when the irrelevant tokens are replaced with whitespace, and even when they are masked so that the model attends only to the relevant tokens [[3]].

Breadth. Chroma’s technical report tests 18 models on deliberately simple tasks while holding task complexity constant. Performance “consistently degrades with increasing input length,” and distractors make it worse in model-specific ways [[4]]. It is a vendor report rather than a peer-reviewed paper. Its value is breadth, not mechanism.

These studies measure different things: padding, lexical distance, pure length, and distractors. They should not be merged into one law. They do share a direction, and it is the direction the June post assumed. The Du et al. result goes further. My June ratio treated irrelevant material as harmful because it diluted relevant material. Their masking experiment suggests that length can cost accuracy even when nothing is diluted. If that holds beyond their models, the June post was too mild about length.

Anthropic’s context-engineering guidance makes the same practical point: good context engineering means “finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome” [[5]].

The Part That Did Not: A Cached Token Is Not a Full Token

The proportional-saving assumption holds only if every token in the context is paid for at the full input price on every request. In an agent loop with prompt caching, that is false.

Anthropic’s standard prompt-caching multipliers, relative to the base input price, are 1.25 for a five-minute cache write, 2 for a one-hour write, and 0.1 for a cache read [[6]]. Some models list a lower read multiplier, and I give \(r = 0.05\) as a sensitivity case below. Output costs five times the base input price for every model on Anthropic’s price list. Measure every price in units of the base input price per token, and let

$$ w = 1.25, \qquad r = 0.1, \qquad o = 5 $$

be the write, read, and output prices. A token that enters the context once and stays for \(m\) later requests costs

$$ c_{\text{cached}}(m) = w + r\,m, \qquad c_{\text{uncached}}(m) = 1 + m . $$

This assumes the token sits in a cacheable prefix and that requests come closer together than the cache lifetime, which is five minutes by default and refreshed on every hit. For \(m = 20\), that is 3.25 against 21. With a warm cache, a token carried through twenty further requests costs about 15% of what it would cost without caching, and about 11% at \(r = 0.05\). So caching has already removed most of the per-token saving that the proportional assumption promised. The rest still matters. But the cost argument for trimming a stable context is roughly six to nine times weaker than the proportional assumption suggests.

This does not make input tokens a negligible part of the bill. The Manus team reports an average input-to-output ratio of about 100:1 in their agent, and calls the KV-cache hit rate “the single most important metric for a production-stage AI agent” [[7]]. Apply their ratio to Anthropic’s prices, with an illustrative cache hit rate \(h\) (the cache misses are written to the cache). The input cost per output token is then \(100\,[h r + (1-h) w]\). At \(h = 0.9\) that gives 21.5 against 5 for the output, so input is still about 81% of the bill. Without caching, it is 100 against 5, or 95%, and the total is about four times higher. From \(h = 0.9\), halving the input tokens brings the bill to about 59% of what it was; raising the hit rate to 0.99 brings it to about 61%. The two levers are comparable in size. What matters is that they are not independent.

That would be a footnote if token reduction and caching were independent. They are not.

Compression That Breaks the Cache

Caching is a prefix match. Anthropic’s documentation states the hierarchy plainly: tools, then system, then messages, and “changes at each level invalidate that level and all subsequent levels” [[6]]. Anything that rewrites earlier context, such as a summary replacing old turns or a compressed version of a file read three steps ago, turns cheap cache reads into fresh cache writes from that point on.

Take compaction, where a history of length \(L\) is replaced by a summary of length \(L' < L\). Compacting costs one summarisation call, which reads \(L\) (at best from cache, \(rL\)) and generates \(L'\) output tokens (\(oL'\)). The next request must then write the new prefix (\(wL'\)). After that, each request reads \(L'\) instead of \(L\) and saves \(r(L - L')\). Assume the cache stays warm, count the request that writes the new prefix as the first of \(n\) later requests, and note that new turns cost the same on both paths, so they cancel. Setting the two cost paths equal gives the number of later requests needed to break even:

$$ n^{*} = \frac{rL + oL' + (w - r)L'}{r\,(L - L')}. $$

For \(L = 100{,}000\) and a tenfold compression to \(L' = 10{,}000\), this gives \(n^{*} \approx 7.9\). At fivefold compression (\(L' = 20{,}000\)) it is about 16.6. If cache reads cost half as much (\(r = 0.05\)), the tenfold case rises to about 14.9. These are derived scale estimates, not measurements of any product. They also flatter compaction, because they assume the summarisation call itself hits the cache.

Most of that cost is the summary itself. A mask that replaces old tool outputs with short placeholders has no summarisation call, so \(rL + oL'\) drops out and the break-even becomes \((w - r)L' / [r(L - L')]\). That is about 1.3 requests in the tenfold case, and less in practice, because the prefix before the first masked output stays cached.

The arithmetic says something the June post did not: summarising history is a long-run investment. It pays off only if the session keeps going for many more requests, and it is a loss if the session ends soon after. The two exceptions are a cache that would go cold anyway, because a pause outlasts the cache lifetime and the rewrite is paid either way, and a context about to exceed the window, where the question is no longer cost.

The best empirical study of this I know of points the same way, though it does not model cache economics directly. Lindenbauer et al. compare context strategies for SWE-agent on SWE-bench Verified across five model configurations [[8]]. Their costs come from list prices, and for one model those prices do not separate cache hits from misses. So the study tests cost per task, not my cache argument. In four of the five configurations, both strategies cut cost by 41–57% against the unmanaged agent: simple observation masking over a rolling window of recent turns, and LLM summarisation. Masking was the cheaper of the two in four of five, with solve rates within about a percentage point of summarisation’s in all but one configuration. It also started saving sooner, even on short trajectories, and the authors end up treating summarisation as a last resort for bounding very long runs. Where summarisation lost, they give two reasons:

  1. The summary calls themselves were up to 7.2% of total instance cost, and they were expensive partly because each one processes a unique sequence of turns, “limiting cache reuse.”
  2. The summaries lengthened trajectories. With Gemini 2.5 Flash, the mean rose to 52 turns, against 44 under masking. With Qwen3-Coder 480B it was 15% longer than the raw agent’s. The authors’ reading is that summaries “mask failure signals that would otherwise prompt earlier termination.”

The second point is about quality as much as cost. A summary that says the work is going well removes exactly the signal that would have stopped the agent. In the June article’s terms, a summary can raise relevance density on paper while deleting the one token that mattered. The paper is a workshop paper and the pattern is not uniform. With Qwen3-32B, masking rather than summarisation lengthened the trajectories, and summarisation was the cheaper strategy. With Gemini 2.5 Flash in thinking mode, both strategies lowered the solve rate against the raw agent, summarisation by nine percentage points. The result still matters because it measures cost per task, not tokens per request.

Filtering at the Door Still Wins

Not all token reduction is compaction. A shell filter like RTK, whose repository describes itself as reducing “LLM token consumption by 60-90% on common dev commands” (a vendor claim) [[12]], works at a different point. It shrinks a command’s output before that output ever enters the context. Nothing that has already been cached changes. The filtered tokens are written once and read cheaply afterwards, like any other.

The obvious risk is that the filter drops the line that mattered, so the agent has to re-run the command unfiltered. Let a raw output have \(x\) tokens, the filtered version \(x'\), and let each token in the context cost \(\varphi = w + r\,m\) over its lifetime. A re-run costs an extra request that reads the current context \(C\) (\(rC\)), a short tool call of \(a\) output tokens (\(oa\)), and the raw output at full lifetime cost. Assume one extra request suffices, and that the raw output then stays in the context as long as the filtered one would have. If a re-run is needed with probability \(p\), filtering is cheaper whenever

$$ p < \frac{(x - x')\,\varphi}{x\,\varphi + rC + oa}. $$

For a 5,000-token test log filtered to 1,000 tokens, \(m = 20\), \(C = 60{,}000\), and \(a = 100\), the threshold is \(p \approx 0.57\). More than half of all filtered runs would have to need a re-run before the filter lost money. For a short 2,000-token output filtered to 400 tokens, in a 100,000-token context with the same \(m\) and \(a\), the threshold falls to about 0.31, because the re-run’s context read now dominates.

So, on cost, filtering at insertion nearly always wins, and the June post was right to make it the default. The real risk is not cost. It is a silent miss: the filter drops the decisive line and the agent never notices. The formula does not price that. The Manus team states the design rule I should have written in June: compression should be “restorable,” so that dropped content can be fetched again from a path or URL that stays in the context [[7]].

The Filters Were Part of the Noise

The second mistake is less flattering. The June post listed what each layer of the harness removed. It never listed what each layer added.

A filter delivered as an MCP server, as several in the June stack are, puts its tool definitions into the prefix of every request, whether or not it is used. That overhead is measurable. Anthropic’s tool-search announcement gives a five-server example: GitHub’s 35 tools at about 26K tokens, Slack’s 11 at about 21K, and three smaller servers, “58 tools consuming approximately 55K tokens before the conversation even starts.” Internally, they had seen tool definitions reach 134K tokens [[9]]. GitHub’s average works out to about 740 tokens per tool. On top of the definitions, the API adds a tool-use system prompt of a few hundred tokens whenever any tools are present [[10]].

On cost, definitions behave like any other stable prefix. A block of \(T\) definition tokens over \(N\) requests with a warm cache costs \(T\,[w + r(N - 1)]\). For \(T = 10{,}000\) and \(N = 50\) that is 61,500 base-price token equivalents. To pay for itself, the server must remove at least \(S\) tokens per session, with

$$ S\,(w + r\,\bar m) \;\ge\; T\,[w + r(N-1)], $$

where \(\bar m\) is the average number of later requests that a saved token would have stayed for. With savings spread evenly over a 50-request session, \(\bar m = 25\) and \(S \ge 16{,}400\) tokens. A test-heavy session can clear that easily, a configuration session might not. It depends on the workload, which is exactly why it has to be measured rather than assumed.

The attention cost is the part the June ratio should have caught. A tool definition is relevant to the current step only when that tool is a plausible next action. With 58 tools loaded and one or two plausible at any step, nearly all definition tokens sit in the denominator of my own relevance ratio. I measured the filters by what they removed from the transcript and left their own schemas out of the count. The empirical picture fits this. Anthropic reports that deferring definitions behind a search tool raised accuracy on its internal MCP evaluations from 49% to 74% for Opus 4 and from 79.5% to 88.1% for Opus 4.5 [[9]]. In the RAG-MCP preprint, retrieving the single best-matching MCP server description instead of presenting all of them raised selection accuracy from 13.62% to 43.13% [[11]]. The first is a vendor’s internal evaluation and the second an unreplicated preprint with a narrow test bed, but they point the same way.

There is a trap in the obvious fix. Loading and unloading tools as the task changes looks like noise reduction, but tool definitions sit at the front of the prefix, so every change invalidates the whole cache. Manus learned this in production: “avoid dynamically adding or removing tools mid-iteration,” because any change “will invalidate the KV-cache for all subsequent actions and observations,” and because earlier turns that refer to vanished tools confuse the model [[7]]. A tool set should be stable for the session. Discovery should be append-only: Anthropic’s tool search loads only the definitions it finds, and the deferred tools are kept out of the prefix entirely, so the prefix stays cacheable [[9]].

A Corrected Decision Rule

The June post’s section on failure modes gave a rule: compress only if \(\Delta R > 0\), where \(R\) is the relevance-density ratio. That rule is necessary but not sufficient. It is a quality criterion, and I used it as if it settled cost too. A context intervention, whether a filter, a summary, a mask, or a new tool server, should pass two tests:

$$ \Delta R \ge 0 \quad\text{and}\quad \Delta K < 0, \qquad K = \frac{\mathbb{E}[\text{total cost of a task}]}{\Pr(\text{task solved})}. $$

\(K\) is cost per completed task. It counts cache writes as well as reads, summarisation calls, re-runs, and extra turns. Measured per request, these interventions look cheaper than they are. The quality test is relaxed from \(> 0\) to \(\ge 0\) on purpose: a change that leaves relevance untouched but lowers cost, such as freezing a tool list, should qualify.

In practice, the two tests reorder the June stack:

  • Filter at the door. Reduce tool output before it enters the context. This saves on both counts and leaves the cache alone.
  • Freeze the prefix. Keep tools, system prompt, and policy files stable for the whole session. Add instructions as appended messages, not edits.
  • Mask early, summarise late. Masking edits history too, so it costs a cache rewrite from the masked point onward, but it pays back within a request or two and invents nothing. Summaries are a long-run investment and can hide the signal to stop.
  • Make compression restorable. Keep the path, URL, or command that recovers what was dropped.
  • Count your own overhead. Every MCP server, hook, and policy file is context. Measure it the same way you measure the logs it filters.

None of this changes the June conclusion that selection matters more than capacity. It changes where selection should happen. Do it as early as possible, before material reaches the cache, and leave material alone once it is there.

Right and Wrong at the Same Time

The noise was real. Longer inputs cost accuracy, sometimes even when every extra token is blank, and a smaller working set is still the better default. What I got wrong was the price. At standard rates a cached read costs a tenth of an uncached token, a rewritten token costs more than either, and the filters I recommended put their own definitions into every request.

The cheapest token is still the one that never enters the context. The most expensive is the one you change after it has.

Literature cutoff: 2026-08-25. Prices and multipliers are Anthropic’s standard published rates; some models list lower cache-read multipliers, which lengthen the compaction break-even and leave the other conclusions intact. All worked examples are derived scale estimates in units of the base input price, not measurements of my own setup.

References

[1] Levy, M., Jacoby, A., & Goldberg, Y. (2024). Same task, more tokens: The impact of input length on the reasoning performance of large language models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 15339–15353. https://doi.org/10.18653/v1/2024.acl-long.818

[2] Modarressi, A., Deilamsalehy, H., Dernoncourt, F., Bui, T., Rossi, R. A., Yoon, S., & Schütze, H. (2025). NoLiMa: Long-context evaluation beyond literal matching. Proceedings of the 42nd International Conference on Machine Learning (ICML 2025), PMLR 267, 44554–44570. https://arxiv.org/abs/2502.05167

[3] Du, Y., Tian, M., Ronanki, S., Rongali, S., Bodapati, S., Galstyan, A., Wells, A., Schwartz, R., Huerta, E. A., & Peng, H. (2025). Context length alone hurts LLM performance despite perfect retrieval. Findings of the Association for Computational Linguistics: EMNLP 2025. arXiv:2510.05381. https://arxiv.org/abs/2510.05381

[4] Hong, K., Troynikov, A., & Huber, J. (2025, July 14). Context rot: How increasing input tokens impacts LLM performance (Technical report). Chroma. https://www.trychroma.com/research/context-rot

[5] Rajasekaran, P., Dixon, E., Ryan, C., & Hadfield, J. (2025, September 29). Effective context engineering for AI agents. Anthropic Engineering. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

[6] Anthropic. Prompt caching (documentation; accessed for this post). https://platform.claude.com/docs/en/build-with-claude/prompt-caching. Prices: Pricing. https://platform.claude.com/docs/en/about-claude/pricing

[7] Ji, Y. (2025, July 18). Context engineering for AI agents: Lessons from building Manus. Manus blog. https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus

[8] Lindenbauer, T., Slinko, I., Felder, L., Bogomolov, E., & Zharov, Y. (2025). The complexity trap: Simple observation masking is as efficient as LLM summarization for agent context management. 4th Deep Learning for Code Workshop (DL4C), NeurIPS 2025. arXiv:2508.21433v3. https://arxiv.org/abs/2508.21433

[9] Wu, B. (2025, November 24). Introducing advanced tool use on the Claude Developer Platform. Anthropic Engineering. https://www.anthropic.com/engineering/advanced-tool-use

[10] Anthropic. Tool use with Claude (documentation; tool-use system prompt token table). https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview

[11] Gan, T., & Sun, Q. (2025). RAG-MCP: Mitigating prompt bloat in LLM tool selection via retrieval-augmented generation. arXiv:2505.03275 [cs.AI], preprint. https://arxiv.org/abs/2505.03275

[12] rtk-ai. rtk (GitHub repository; description and README are vendor claims). https://github.com/rtk-ai/rtk

Changelog

  • 2026-10-10: Added current prices. Anthropic now lists cache reads at 0.05× base input for Claude Opus 5.5 and Claude Sonnet 5.5 and at 0.025× for Claude Fable 5.1, with writes unchanged at 1.25× and output still at 5× input [[6]]. Recomputed at \(r = 0.05\) and \(r = 0.025\): a token kept for 20 further requests costs 10.7% and 8.3% of its uncached price; the tenfold compaction break-even rises to 14.9 and 28.8 requests (32.2 and 63.5 at fivefold), and masking to 2.7 and 5.4; the filter re-run thresholds rise to 0.61/0.36 and 0.65/0.43; the break-even saving for a 10,000-token tool block falls to 14,800 and 13,200 tokens. Cheaper reads strengthen every conclusion: keeping a token costs less, so rewriting history needs a longer session to pay off. I also no longer use the tool stack described in the June post; the argument does not depend on it.