Tooling
2 articlesRSS feed all topics
2026
2 articles- Token Saving, Revisited: Right About the Noise, Wrong About the Bill In June I argued that token saving in agent systems is noise reduction. The quality half of that argument holds, and the evidence for it is stronger than the June post showed. The cost half rested on an assumption I never wrote down: that fewer tokens always means a smaller bill. With prompt caching, a token you keep is far cheaper than a token you change, so some compression costs more than it saves. And the filters I recommended put their own tool definitions into every request. This post does the arithmetic.
- Token Saving as Noise Reduction Token saving in agent systems is often framed as an efficiency concern. A better framing is signal-to-noise control: every unnecessary token competes for attention, increases inference cost, and makes the next decision less precise. Current work on context compression, KV-cache compression, and long-running interactions points in the same direction.