LLM
12 articlespage 1 of 2RSS feed all topics
2026
8 articles on this page- The Wealth of the Stimulus By the age of thirteen a child has heard something like sixty million words; Llama 3 was trained on the equivalent of roughly ten trillion words. The gap is often read as settling Chomsky’s poverty-of-the-stimulus argument, but that argument is about what a child-sized input contains, and the experiments that bear on it run at child scale, with mixed results. What children get and a corpus lacks is contingency: speech that answers them. In a language model the nearest thing is post-training on human feedback. In the published datasets of 2022–23 it is three to four orders of magnitude smaller than the same model’s pretraining, yet in absolute terms it is comparable to a childhood’s linguistic input, within about an order of magnitude either way, whether counted in words or in replies. The part of training that looks like upbringing is roughly the size of an upbringing.
- Pelican on a Bicycle is the New Carwash In the same February week, the car-wash prompt went viral and Google’s Jeff Dean posted an animated pelican riding a bicycle. Seven months later the car wash has been absorbed into proper benchmarks, and as a single pass/fail prompt it has little left to say. The pelican has become the launch-day ritual instead. This post looks at why a generated artefact outlasts a question with one right answer: information theory, paired comparisons, SVG geometry, the psychology of drawing bicycles, and what happens when the pelican starts to move and has to obey kinematics. It also looks at why the pelican is heading for the car wash’s fate.
- Token Saving, Revisited: Right About the Noise, Wrong About the Bill In June I argued that token saving in agent systems is noise reduction. The quality half of that argument holds, and the evidence for it is stronger than the June post showed. The cost half rested on an assumption I never wrote down: that fewer tokens always means a smaller bill. With prompt caching, a token you keep is far cheaper than a token you change, so some compression costs more than it saves. And the filters I recommended put their own tool definitions into every request. This post does the arithmetic.
- Token Saving as Noise Reduction Token saving in agent systems is often framed as an efficiency concern. A better framing is signal-to-noise control: every unnecessary token competes for attention, increases inference cost, and makes the next decision less precise. Current work on context compression, KV-cache compression, and long-running interactions points in the same direction.
- More Context Is Not Always Better The intuition that feeding a language model more information improves its outputs is wrong often enough to matter. Here is why, and what to do about it.
- If You Think This Is Written by AI, You Are Both Right and Wrong AI detectors can flag human writing, including historical and highly formulaic prose. Evidence exists for a false-positive risk affecting some non-native English writers; claims about autistic or physics-trained writers need direct validation. The detector score is not authorship evidence.
- Car Wash, Part Three: The AI Said Walk In early 2026, variants of a prompt about a nearby car wash spread online after some assistants recommended walking instead of taking the car. This is neither the tokenisation failure from the strawberry post nor the grounding failure from the rainy-day post. It is a pragmatic inference failure: the response followed the travel wording but missed the likely purpose of the trip, assigning its advice to the wrong interpretation of the question. A third and more subtle failure mode, with Grice as the theoretical handle.
- Should I Drive to the Car Wash? On Grounding and a Different Kind of LLM Failure A viral video this month showed an AI assistant confidently answering “should I drive to the car wash today?” without knowing it was raining outside. The internet found it funny. The failure mode is real but distinct from the strawberry counting problem — this is not a representation issue, it is a missing-context issue. The words posed no special tokenisation difficulty. What the system lacked was access to the state of the world the question was about.
2025
2 articles on this page- Constraining the Coding Agent: The Ralph Loop and Why Determinism Matters In late 2025, agentic coding tools went from impressive demos to daily infrastructure. The problem nobody talked about enough: when an LLM agent has write access to a codebase and no formal constraints, reproducibility breaks down. The Ralph Loop is a story-driven harness intended to bound that variability through one call to the coding agent per attempt, scoped writes, and atomic state. These constraints make runs easier to inspect; they do not make an unpinned model deterministic.
- The Papertrail: AI PDF Renaming and the Tokens That Make It Interesting Everyone has a Downloads folder full of “scan0023.pdf” and “document(3)-final-FINAL.pdf”. Renaming them by content sounds trivial — read the file, understand what it is, give it a name. The implementation reveals something useful about how LLMs actually handle text: what a token is, why context windows matter in practice, why you want structured output instead of prose, and why heuristics should go first. The project, now called Folionym, is at github.com/sebastianspicker/folionym.