Three Rs in Strawberry: What the Viral Counting Test Actually Reveals
Abstract
The viral “three Rs in strawberry” test exposes a real mismatch between token-level input and character-level operations. It does not, by itself, identify one tokeniser boundary or explain any particular model response.
Contents
The Setup
When OpenAI released o1-preview in September 2024, “how many r’s are in strawberry?” was already a familiar demonstration of language-model failure. The coincidence was irresistible. It was also reported too neatly: OpenAI’s own release page [[1]] shows o1-preview decoding a cipher whose answer is “THERE ARE THREE R’S IN STRAWBERRY.” That is evidence of one successful, scaffolded task, not a benchmark of character counting. I wrote earlier that the page rules out the stronger joke, that the model could not count the letters in its supposed codename. It does not: the “three” is already in the prompt’s ciphertext, which the chain of thought on the page decodes one letter pair at a time. Decoding is not counting.
Before the opinions: the answer is three. s-t-r-a-w-b-e-r-r-y. One in the str- cluster, two in the -rry ending. Some model versions and prompt formulations have answered this incorrectly. Without a recorded model snapshot, interface, prompt, settings, date, and repeated trials, “most models return two” is not a result I can defend.
The question worth asking is not “why is the model stupid.” It is not stupid, and “stupid” is not a useful category here. The question is: what does this specific error reveal about the structure of the system?
The answer involves tokenisation, and it is actually interesting.
How You Count Letters (and How the Model Doesn’t)
When you count the r’s in “strawberry,” you do something like this: scan the string left to right, maintain a running count, increment it each time you see the target character. This is a sequential operation over a character array. It requires no semantic knowledge about the word — it does not matter whether “strawberry” is a fruit, a colour, or a nonsense string. The characters are the input; the count is the output.
A GPT-style language model does not ordinarily receive a character array. It
receives token IDs produced by a model-specific tokeniser. Byte Pair Encoding
(BPE) and related subword methods often group several characters or bytes into
one token. The split is not universal: OpenAI’s documentation explicitly lists
different encodings for different model families and does not identify the
o1-preview encoding [[2]]. The release page says nothing about encodings
either. The nearest thing to a statement is a lookup table in OpenAI’s
tiktoken library, which has mapped model names beginning o1- to the newer
o200k_base encoding since version 0.8.0 of October 2024 [[3]]. A cl100k_base
demonstration therefore cannot establish the token sequence seen by o1-preview.
For a known encoding and tokeniser version, the split can be measured rather
than guessed. I ran tiktoken 0.14.0 (Fig. 1). On its own, strawberry
becomes str|aw|berry in cl100k_base, three tokens holding one, zero and two
of the r’s, and st|raw|berry in o200k_base, holding zero, one and two.
Neither split is what the question delivers when it is typed as “How many r’s
are in strawberry?” There the word follows a space, and space and word
together have an entry of their own in both vocabularies. The whole
word, all three r’s included, arrives as one integer: 73700 in cl100k_base,
101830 in o200k_base.
tiktoken 0.14.0 encodes the word on its own or after a space (the open-box
mark), as in “How many r’s are in strawberry?” [[3]]. Numbers are token IDs;
the r’s are marked.Those integer IDs are looked up in an embedding table. The model is then operating on one vector per token, not on a conveniently indexed array with one element per letter.
That mismatch matters. It is not yet a causal account of one answer.
What BPE Does (and Doesn’t) Preserve
BPE began as a greedy compression algorithm. Gage’s version finds the most frequent pair of adjacent bytes, replaces every occurrence with a byte value the data does not contain, and repeats [[4]]. Sennrich, Haddow and Birch adapted it to build a subword vocabulary: start from single characters and iteratively merge the most frequent pair of adjacent symbols into a single new token [[5]]:
$$\text{merge}(a, b) \;:\; \underbrace{a \;\; b}_{\text{separate}} \;\longrightarrow\; \underbrace{ab}_{\text{single token}}$$Applied to a large text corpus until a fixed vocabulary size is reached, this produces a vocabulary of common subwords. Frequent words and common word-parts become single tokens; rare sequences stay as multi-token fragments.
The tiktoken encodings start from the 256 possible byte values rather than
from characters, and they apply BPE only inside chunks that a regular
expression cuts out first: roughly, a run of letters with at most one leading
space or punctuation mark. Merges never cross a chunk boundary, and new text is
encoded by applying the learned merges in a fixed order of priority [[3]]. Both
details show in Fig. 1. After a space, space and word form a single chunk,
which has an entry of its own. At the start of a text, or after a space and a
quotation mark (which then form a chunk of their own), the chunk is the bare
strawberry, which has no entry and is assembled from smaller ones. And
o200k_base has a str token too; its merge order simply joins r and a
first.
What BPE optimises for is a compact, useful vocabulary, not
character-level transparency. A multi-character token still preserves its
bytes in the tokeniser: OpenAI’s tiktoken documentation demonstrates both
decoding a token sequence back to text and recovering the bytes represented by
each token [[2]], and the library’s README calls BPE “reversible and lossless”
[[3]]. Tokenisation is therefore not lossy in the simple
information-theoretic sense.
The harder distinction is architectural. After embedding lookup, the transformer does not receive the tokeniser’s byte table as an indexed scratch pad. It may learn spelling regularities from training, and tools or generated intermediate text can make character operations easier, but exact access is not guaranteed merely because the original string is reversibly tokenised.
What an Error Would Establish
An incorrect answer would establish that this particular run failed an exact string operation. Repeated, versioned tests across controlled prompt variants could establish a pattern. Tokenisation supplies a plausible pressure: the operation requested by the prompt and the units presented to the transformer do not line up.
It does not follow that a model answering “two” must have counted the two Rs
in berry and missed the one in str. That story depends on a token split that
may not apply — typed without quotation marks, the question gives neither
encoding in Fig. 1 a str token to miss — and it attributes a hidden
computation from the final answer. Different models may spell, retrieve a
memorised answer, call a tool, or use other intermediate strategies. The output
alone does not distinguish them.
Scaffolding Can Help
Prompting a model to spell the word before counting can expose intermediate
mistakes and may improve a given model’s answer. What it puts into the context
depends on the format the model chooses, since it writes the spelling in tokens
from the same model-specific vocabulary. Written as s-t-r-a-w-b-e-r-r-y, the
spelling encodes as ten tokens in both cl100k_base and o200k_base, one
letter each, every letter after the first fused to the hyphen before it (-r
is a single token). Glued forms merge again: straw-berry encodes as str,
aw, -b, erry in cl100k_base. Even the ten-token form only brings the
input closer to a character array; it does not make the count correct. No
cross-model error-rate estimate is available here.
For a result that must be exact, the robust intervention is simpler: use a deterministic string operation. A model can write or call that operation and report the result. Visible spelling remains useful as a check, not a proof of the model’s internal representation.
The Right Frame
The “how many r’s” test is sometimes cited as evidence that language models don’t “really” understand text, or that they are sophisticated autocomplete engines with no genuine knowledge. These framing choices produce more heat than light.
The more precise statement is this: next-token prediction does not directly train an indexable character array or a guaranteed exact-count procedure. Character counting can therefore expose a mismatch between representation and operation. How often it does so is an empirical question about a specified model and evaluation, not a timeless property of all language models.
“Strawberry” sits squarely in that class. A wrong answer is neither proof of general stupidity nor evidence that tokenisation deleted a letter. It is a small test of whether the system can bridge token-level processing and an exact character-level request.
Literature, product documentation and token splits checked through
2026-10-25. The article does not claim an error rate for current models; the
only o1 encoding it reports is the tiktoken library’s lookup.
A follow-up post covers a structurally different failure mode: Should I Drive to the Car Wash? — where an incompletely documented prompt family illustrates a different boundary: missing situation-specific world state.
References
[1] OpenAI. (2024, September 12). Learning to reason with LLMs. Release page. https://openai.com/index/learning-to-reason-with-llms/ (archived copy, 13 September 2024)
[2] Sanders, T. (2022). How to count tokens with tiktoken. OpenAI Cookbook, archived recipe dated 16 December 2022 and since updated; accessed 25 October 2026. https://developers.openai.com/cookbook/examples/how_to_count_tokens_with_tiktoken
[3] OpenAI. (2026). tiktoken (Version 0.14.0)
[Computer software]. README; tiktoken/model.py; tiktoken_ext/openai_public.py,
which defines the encodings’ chunking patterns and the URLs and SHA-256 hashes
of their rank files (cl100k_base.tiktoken, o200k_base.tiktoken); and the
CHANGELOG, whose 0.8.0 entry (released 3 October 2024) adds “Support for o1-
and chatgpt-4o- models”.
https://github.com/openai/tiktoken
[4] Gage, P. (1994). A new algorithm for data compression. The C Users Journal, 12(2), 23–38.
[5] Sennrich, R., Haddow, B., & Birch, A. (2016). Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1715–1725. DOI: 10.18653/v1/P16-1162. arXiv:1508.07909
Changelog
2025-12-01: Corrected the tokenisation of “strawberry” from two tokens (
straw|berry) to three tokens (str|aw|berry), matching the actual cl100k_base tokeniser used by GPT-4. The directional argument (token boundaries obscure character-level information) is unchanged; the specific analysis was updated accordingly.2026-07-11: Removed the unsupported o1 tokenizer attribution and cross-model failure-rate claims; corrected the claim that tokenisation loses character information; and separated a plausible representational pressure from an unobserved causal account of any model answer.
2026-10-25: Measured the split with
tiktoken0.14.0 (Fig. 1). On its own, “strawberry” isstr|aw|berryincl100k_base(r’s 1, 0, 2) andst|raw|berryino200k_base(0, 1, 2); after a space it is one token in both. The 2025-12-01 split therefore holds only for the bare word, and the question typed without quotation marks has nostrtoken to miss. Addedtiktoken’s mapping ofo1-names too200k_base(since 0.8.0) and the chunking and merge order behind the splits. Replaced “adjacent characters can still share tokens” with measured spellings. Retracted the claim that the release page rules out the joke that o1-preview could not count: the count is in the ciphertext. Corrected the Cookbook citation to Sanders (2022); citations are now numbered.