If You Think This Is Written by AI, You Are Both Right and Wrong
Abstract
AI detectors can flag human writing, including historical and highly formulaic prose. Evidence exists for a false-positive risk affecting some non-native English writers; claims about autistic or physics-trained writers need direct validation. The detector score is not authorship evidence.
Contents
I use AI tools in my writing. This post, like several others on this blog, was written with LLM assistance — research, structure, drafting, revision. If you run any of these posts through an AI writing detector, I expect you will receive a high probability-of-AI score. If so, the detector will be picking up something real.
It will also be wrong about what that means.
The Constitution Problem
In 2023, as universities began deploying AI detection tools at scale, people started testing them on texts that were definitively not AI-generated. The results were instructive. The United States Constitution received high AI-probability scores from multiple commercial detectors. In one widely shared screenshot, ZeroGPT scored it as 92.15% AI-written [[9]]; GPTZero told users the text was “likely to be written entirely by AI” [[10]]. Selections from the Bible were flagged too: ZeroGPT rated a passage from Genesis 88.2% AI-generated [[10]]. The Federalist Papers and Kant’s Critique of Pure Reason are said to have fared similarly; I could not check either report. Historical documents, written by humans, for human purposes, in an era when no AI existed — flagged as machine-generated.
These demonstrations are product- and version-specific, not a validation study of all detectors. They establish an important practical point: a detector’s score can be high for human-authored text and should not be treated as proof of authorship.
What the Detectors Actually Measure
Commercial detectors do not generally disclose enough detail to infer one shared mechanism. Perplexity-like statistics and distributional features are common in the research literature; a vendor score is not a transparent measurement of authorship.
Perplexity. A language model assigns a probability to each token given the preceding tokens; perplexity is the exponential of the average negative log-probability per token. Low perplexity means the text was, token by token, what the model expected — it sits close to the centre of the probability distribution. AI-generated text tends to have low perplexity because, as GLTR’s core assumption has it, “most systems sample from the head of the distribution,” through max sampling, k-max sampling or beam search [[2]]; DetectGPT exploits a related curvature property [[1]]. Human text, on average, has higher perplexity, because humans write for specific contexts with idiosyncratic word choices, rhetorical effects that require the unexpected, and the accumulated noise of composing for a real reader.
Burstiness. The term comes from earlier text statistics, where it describes words that, once they appear in a document, are likely to appear again [[3]]. GPTZero, created by Edward Tian, adapted the idea for detection: it describes burstiness as a measure of how much writing patterns and text perplexities vary over a whole document, with human writing varying more and AI writing staying more uniform [[4]]. Ars Technica glossed it as variation in sentence length and structure [[10]]: sentence lengths vary widely, vocabulary density shifts, complex constructions alternate with simple ones. That was GPTZero’s method when the Constitution screenshots circulated; the page now adds, “As of autumn 2023, GPTZero no longer uses perplexity and burstiness for its AI detection” [[4]].
The source of false positives is an empirical question about a particular detector, population and writing task. A score cannot establish that systematic prose, a historical document or a student’s work was machine-made.
The Systemising Brain
Simon Baron-Cohen’s empathising–systemising (E-S) theory distinguishes two cognitive orientations. Empathising involves attending to social and emotional cues, inferring mental states, navigating the pragmatic, implicit layer of communication — what is meant rather than what is said. Systemising involves attending to rules, patterns, and underlying regularities — the drive to understand how things work and to represent them in explicit, transferable, internally consistent terms [[5]].
Both orientations are distributed across the human population. They are not exclusive, and neither is pathological. But autism spectrum conditions are robustly associated with high systemising and relatively lower empathising — not because autistic people lack emotions or care about others, but because the cognitive mode that comes naturally to them is one of rules, structures, and explicit representation rather than social inference and pragmatic implication. In the largest dataset as of 2018, 671,606 visitors to a Channel 4 documentary’s website, 36,648 of whom reported an autism diagnosis, autistic participants of either sex were on average shifted towards systemising over empathising on short self-report questionnaires [[11]]. That is an average over a population, not a description of any one person.
The intense world theory [[6]], built largely on a valproic-acid rat model of autism, adds a complementary perspective: autistic brains may be characterised by hyper-reactivity and hyper-plasticity of local circuits, and the autistic person is “proposed to become trapped in a limited, but highly secure internal world with minimal extremes and surprises.” I read pattern-seeking and systematising as serving partly to make a too-intense world navigable. On that reading, the systematicity is not a deficit. It is an adaptation.
If that is right, it should show in writing. My hypothesis, not a measured result, is that high-systemising writing tends toward:
Consistent vocabulary. The same term is used for the same concept throughout, because substituting a synonym introduces ambiguity about whether the referent is actually the same. Neurotypical writing freely uses synonyms for stylistic variety; systemising writing resists this on principle.
Explicit logical structure. Claims are supported by stated reasons rather than left to pragmatic inference. If there are three conditions, all three are named. Nothing is “needless to say.”
Low social hedging. Phrases like “as everyone knows” or “obviously” are avoided, because they perform social alignment rather than convey information — and they depend on shared assumptions the writer is not confident are actually shared. (The car-wash-walk post sets out Grice’s maxims. My own conjecture, not something that post or any study cited here establishes, is that autistic communication tends to break the maxim of quantity in the direction of over-informing, because nothing is assumed implicit.)
Grammatical parallelism. Parallel logical content takes parallel grammatical form. This is not stylistic affectation; it is a natural consequence of representing structure explicitly.
Minimal rhetorical noise. The prose does not meander, warm up, or perform relatability. It states what needs to be stated.
Now imagine text with these properties run through an AI detector. Consistent vocabulary would read as low lexical diversity, explicit structure as low burstiness, minimal rhetorical noise as smooth, generated output. The detector would be measuring these properties accurately. The attribution to machine generation is where it would go wrong.
What Has Been Measured
For one group of human writers, a closely related failure has been measured. Liang et al. [[7]] ran seven detectors, as they were on 15 March 2023, on 91 essays written for the TOEFL, a test of English for non-native speakers, collected from a Chinese educational forum, and on 88 essays by US eighth-graders from the Hewlett Foundation’s Automated Student Assessment Prize dataset. Averaged over the seven detectors, 61.22% of the TOEFL essays were classified as AI-generated, against 5.19% of the US essays. All seven detectors flagged 18 of the 91 TOEFL essays (19.8%); at least one flagged 89 (97.8%). Every detector flagged the TOEFL essays far more often (Fig. 1).
Two cautions. The 61% is a mean of seven per-detector rates, and “at least one of seven” is lenient: by the same rule, at least one detector also flagged 15 of the 88 US essays (17%, my count). The samples are small, and the detectors have changed since.
The paper’s explanation is perplexity. The 18 unanimously flagged essays had significantly lower GPT-2 perplexity than the other 73. After ChatGPT was asked to “enhance the word choices to sound more like that of a native speaker,” the rewritten TOEFL essays were flagged at an average of 11.77%, 49.45 percentage points below the originals; US essays rewritten with simplified word choices “as if written by a non-native speaker” were flagged at 56.65%, up from 5.19% [[7]]. The rewritten essays are ChatGPT output, so a flag on them is not clearly a false positive, although the paper’s axis calls them “Human-Written.”
That is not quite the mechanism I proposed above, and I should not pretend it is. An earlier version of this post described these writers as advanced users of formal English whose prose is more careful and rule-governed. The study does not show that. Its suggested explanation is “limited linguistic variability and word choices.” That many non-native writers learned English as a system of explicit rules may contribute, but that is my gloss. What the two cases would share is predictability to the scoring model, the variable the paper’s correlation and rewriting experiment point to; vendors do not say what their detectors compute. My hypothesis is that systematic prose, consistent vocabulary above all, gets there by a different route. If so, the mechanism is the same and only the population is different. The non-native result shows that the route from predictable human text to a false positive exists. It does not show that systemising writers take it.
The Physicist Brain
Physics writing has its own conventions, independently developed but pointing in the same direction.
Scientific prose requires defined terms used consistently: in a paper about quantum error correction, “logical qubit,” “physical qubit,” and “syndrome” each mean exactly one thing, used identically in section 2 and section 5. It requires explicit assumptions: “We assume the noise is Markovian.” “In the limit of large N.” These are not vague hedges; they are precise statements about the domain of validity of the results. It requires logical derivation over rhetorical persuasion: the connectives are “since,” “therefore,” “it follows that” — explicit logical operators, not narrative bridges. And the passive construction of “the signal was measured” rather than “I measured the signal” removes the individual from the result, because the result should be reproducible regardless of who performs the measurement.
The outcome is prose that is systematic, consistent, and structurally predictable. From the outside it can look machine-generated, and my hypothesis is that it looks that way to an AI detector too.
The data released with Liang et al. contain one partial test, and it does not help me. Five of the detectors also scored 145 human-written abstracts, about 150 words each, of 2021 Stanford course projects in natural language processing, and flagged between 0% (ZeroGPT) and 7.6% (Originality.ai) of them (my count, by the rule of Fig. 1). It is a weak test: short machine-learning abstracts are not Dirac, and human text alone says nothing about how well detectors catch machine text. But the paper adds that detectors “were less effective” on generated abstracts than on generated essays, “partly because the generated abstracts have slightly higher perplexity” [[7]].
Paul Dirac is the physicist who comes to mind first here. His 1928 paper deriving the relativistic wave equation for the electron contains almost no rhetorical apparatus. Motivation, equation, consequence: each stated once, clearly, with no warm-up and no elaboration beyond what the argument requires. It is not warm. It is not discursive. It is beautiful in the way that a proof is beautiful: every element earns its place. I have not run it through a detector; after those abstracts, I would not bet on the result.
I do not think the connection between the physicist’s prose style and the autistic cognitive mode is accidental. Baron-Cohen et al. [[8]] gave the Autism-Spectrum Quotient (AQ) to 840 Cambridge students. As a whole they did not differ from randomly selected controls, but students in the sciences, mathematics included, scored significantly higher than students in the humanities and social sciences, with mathematicians highest within the sciences. In Greenberg et al.’s 2018 data [[11]], short forms of the AQ and of systemising (SQ) and empathy (EQ) questionnaires, non-autistic people working in STEM scored higher on the AQ and SQ and lower on the EQ, although autistic participants were no more likely than others to work in STEM. On these measures, the systemising orientation associated with autism is over-represented in quantitative disciplines, presumably because they select for it, though the data cannot separate selection from training. My claim is that the physicist’s prose reflects this, and so does the writing of a high-systemising person who has never studied physics.
The categories overlap without being identical. What they share is a cognitive preference for explicit structure, consistent vocabulary, and logical transparency over social performance and rhetorical persuasion. If I am right, which the abstracts above put in doubt, the writing that emerges from that preference looks, to an AI detector, like it was generated by a machine.
It was not.
The Category Error
The error AI detectors make is not a measurement error. It is a category error.
A supervised detector learns to separate two classes: text from the language models its developer sampled, and the human writing its developer collected. (A detector that thresholds perplexity has a cut-off instead; the argument carries over.) What that human writing is, vendors mostly do not say. GPTZero cites “millions of documents” from “creative writing, scientific writing, blogs, news articles, and more” [[12]], and its 2026 technical report counts them by domain, news and web articles first, without naming sources [[13]]; Originality.ai cites “millions of carefully selected records” [[14]]. The one detector in Liang et al.’s comparison whose training recipe I found published, OpenAI’s GPT-2 output detector, was fine-tuned to tell GPT-2 samples from OpenAI’s WebText documents [[15]]. An earlier version of this post said that detectors are trained on “typically casual, personal, or student prose collected from online sources.” That was a guess presented as a fact, and for GPTZero’s current model it is wrong.
The argument does not need a narrow human class, though. A classifier draws its boundary where its training data put it. Human text that is unusually regular sits close to machine output in the space the classifier sees, and where machine output is denser there than human writing, the text lands on the machine side. It gets assigned to the bin it fits least badly. GPTZero’s FAQ comes close to conceding this: its classifier “can sometimes flag other machine-generated or highly procedural text as AI-generated” [[12]].
What I think is happening when the Constitution is flagged: it is systematic, definitional, prescriptive, and internally consistent. It was written by lawyers and statesmen who understood that ambiguity in foundational documents creates legal chaos. They wrote to be unambiguous. The result, I would guess, is low perplexity and low burstiness, the signature GPTZero’s 2023 detector looked for. What ZeroGPT computes, I do not know.
GPTZero’s creator, Edward Tian, gave a different explanation when Ars Technica asked in July 2023: “The US Constitution is a text fed repeatedly into the training data of many large language models. As a result, many of these large language models are trained to generate similar text to the Constitution and other frequently used training texts. …” [[10]]. A text the model has effectively memorised has low perplexity for that reason alone. That is a real and specific issue. I think it is secondary. My claim, which I have not seen tested and which the abstracts above cut against, is that the Constitution would score low-perplexity even without the memorisation effect, because systematic, definitional prose is intrinsically low-perplexity. Precise language is predictable language. That is partly the point of precise language.
The baseline assumption — that human writing is variable and idiosyncratic — holds for much human writing. I doubt that it holds for legal drafting, technical documentation, scientific papers, sacred and historical texts written to be durable and precise, writing by people with high systemising orientation, or writing by non-native speakers in formal registers. If I am right, that is not a small population of edge cases.
Right and Wrong at the Same Time
So: if you think these posts are AI-generated, you are right and wrong at the same time.
Right, in two ways. First: yes, I use AI tools. LLM assistance is part of my writing process — not an occasional aid, but a regular part of how research notes and half-formed arguments become structured posts. Second: the writing style of these posts is systematic and precise in ways that, I suspect, detectors register as machine-generated. I have not tested that, and these posts could not settle it: they were written with LLM assistance, so a high score on them would look exactly like a correct one. That systematicity is real, and if a detector picks it up, it is measuring something.
Wrong, also in two ways. First: the ideas, judgments, and connections in these posts are mine. The decisions about what to include and what to leave out, which papers to cite and how to frame their implications, where the interesting tension lies between neurodiversity research and the assumptions baked into AI detection tools — those are not outputs of a language model working in isolation. They are the product of someone who works at the intersection of these fields and has thought about them for a while. An LLM cannot generate these posts without a human who has already decided what to say.
Second, and more important for the argument here: the systematic, precise character of this writing is not evidence of machine generation. It is, I suspect, a cognitive signature — one associated with physics training and with high systemising orientation. I have written about neurodiversity and my own experience of it elsewhere.
A detector that flags it is, on this account, measuring a real property of the text. It is misattributing the origin of that property.
The interesting question this opens is not “did AI write this?” That question is increasingly poorly posed in an era where thinking and writing are already deeply entangled with machine assistance, in ways that differ sharply from person to person and task to task. The better question is: whose judgment is in the text? Whose choices about what to include, what to connect, what to leave out?
The systematicity in this writing is mine. The suspicion that AI detectors disadvantage autistic and physicist writers in the way the seven tested in March 2023 demonstrably disadvantaged non-native speakers is a judgment I made, not one a language model was prompted to produce. The connection to the Constitution — a document written to be maximally unambiguous, flagged as maximally AI-like — is a connection I found worth drawing.
Whether that makes this text “human” is a philosophical question I am happy to leave open. A detector score cannot settle it.
Literature and vendor behaviour checked through 2026-10-28.
References
[1] Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D., & Finn, C. (2023). DetectGPT: Zero-shot machine-generated text detection using probability curvature. Proceedings of the 40th International Conference on Machine Learning (ICML 2023). https://arxiv.org/abs/2301.11305
[2] Gehrmann, S., Strobelt, H., & Rush, A. M. (2019). GLTR: Statistical detection and visualization of generated text. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 111–116. https://doi.org/10.18653/v1/P19-3019
[3] Madsen, R. E., Kauchak, D., & Elkan, C. (2005). Modeling word burstiness using the Dirichlet distribution. Proceedings of the 22nd International Conference on Machine Learning (ICML 2005), Bonn, Germany, 545–552. https://doi.org/10.1145/1102351.1102420 (Attributes the term “burstiness” to Church & Gale, 1995, and Katz, 1996; see Church, K. W., & Gale, W. A. (1995). Poisson mixtures. Natural Language Engineering, 1(2), 163–190. https://doi.org/10.1017/S1351324900000139.)
[4] Tian, E. (2023, March 1; page modified 2026, July 24). Perplexity, burstiness, and statistical AI detection. GPTZero. https://gptzero.me/news/perplexity-and-burstiness-what-is-it/ (Vendor page; the note that GPTZero no longer uses perplexity and burstiness was added after publication.)
[5] Baron-Cohen, S. (2009). Autism: The empathising–systemising (E-S) theory. Annals of the New York Academy of Sciences, 1156(1), 68–80. https://doi.org/10.1111/j.1749-6632.2009.04467.x
[6] Markram, K., & Markram, H. (2010). The intense world theory — a unifying theory of the neurobiology of autism. Frontiers in Human Neuroscience, 4, 224. https://doi.org/10.3389/fnhum.2010.00224
[7] Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7), 100779. https://doi.org/10.1016/j.patter.2023.100779 (Preprint: https://arxiv.org/abs/2304.02819; code, data and per-essay scores: https://github.com/Weixin-Liang/ChatGPT-Detector-Bias.)
[8] Baron-Cohen, S., Wheelwright, S., Skinner, R., Martin, J., & Clubley, E. (2001). The autism-spectrum quotient (AQ): Evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians. Journal of Autism and Developmental Disorders, 31(1), 5–17. https://doi.org/10.1023/A:1005653411471 (Erratum: Journal of Autism and Developmental Disorders, 31(6), 603.)
[9] Basheer, K. C. S. (last updated 2023, April 24). AI-detector flags US Constitution as AI-generated. Analytics Vidhya. https://www.analyticsvidhya.com/blog/2023/04/ai-detection-can-ai-generated-content-really-be-detected/ (Reports a social-media screenshot of the ZeroGPT result.)
[10] Edwards, B. (2023, July 14). Why AI detectors think the US Constitution was written by AI. Ars Technica. https://arstechnica.com/information-technology/2023/07/why-ai-detectors-think-the-us-constitution-was-written-by-ai/
[11] Greenberg, D. M., Warrier, V., Allison, C., & Baron-Cohen, S. (2018). Testing the Empathizing–Systemizing theory of sex differences and the Extreme Male Brain theory of autism in half a million people. Proceedings of the National Academy of Sciences, 115(48), 12152–12157. https://doi.org/10.1073/pnas.1811032115
[12] GPTZero. (n.d.). FAQ. Retrieved 28 October 2026, from https://gptzero.me/faq (Vendor page.)
[13] Adam, G. A., Cui, A., Thomas, E., Napier, E., Shmatko, N., Schnell, J., Tian, J. J., Dronavalli, A., Tian, E., & Lee, D. (2026). GPTZero: Robust detection of LLM-generated texts. arXiv preprint. https://arxiv.org/abs/2602.13042 (Written mostly by GPTZero staff; training data in its Table 4.)
[14] Gillham, J. (2026, October 25). Why AI detection matters and how to measure it. Originality.ai. https://originality.ai/blog/ai-accuracy (Vendor page.)
[15] OpenAI. (n.d.). GPT-2 output detector. In gpt-2-output-dataset [GitHub repository]. https://github.com/openai/gpt-2-output-dataset/tree/master/detector (README and train.py: RoBERTa fine-tuned on outputs of the 1.5B-parameter GPT-2 against WebText; the Hugging Face demo that Liang et al. queried serves this repository.)
Changelog
- 2026-07-11: Clarified that detector scores are not authorship proof and that vendor mechanisms are not sufficiently disclosed to infer one shared measurement. The non-native-English result remains the directly evidenced bias claim; proposed autistic and physics-writing effects remain hypotheses requiring validation.
- 2026-10-03: Corrected the attribution of “burstiness”: the term predates GPTZero (Church & Gale 1995; Madsen et al. 2005) and GPTZero adapted it. Moved the GLTR citation to the perplexity sentence, since GLTR does not address sentence-length distributions, and removed the uncited claim that sentence lengths in LLM output are narrower than in human prose. Corrected the Constitution example: the widely reported 92% figure (92.15%) came from ZeroGPT, not GPTZero, which reported the text as “likely to be written entirely by AI”; added sources (Analytics Vidhya, Ars Technica) and marked the Federalist Papers and Kant examples as reported.
- 2026-10-28: Marked as hypotheses, in the body, claims that had been stated as findings: that high-systemising and physics prose read as machine-generated to detectors, that the Constitution is low-perplexity by style rather than by memorisation, that this writing is a cognitive signature, that detectors would flag these posts (untestable here, since the posts are LLM-assisted), and that the population concerned is “a substantial fraction of all written material” (removed). Softened the closing “The recognition that AI detectors systematically disadvantage autistic writers, physicist writers, and non-native speakers” to “The suspicion that” they disadvantage autistic and physicist writers as the seven detectors tested in March 2023 disadvantaged non-native speakers; “It does not hold for” legal and other formal writing became “I doubt that it holds for”. Added Liang et al.’s numbers in a new section with Fig. 1, computed from the released per-essay scores (on average 61.22% of 91 TOEFL essays and 5.19% of 88 US eighth-grade essays flagged; 18 TOEFL essays flagged by all seven detectors, 89 by at least one, against 15 US essays by at least one), with the rewriting results (11.77% and 56.65%, on ChatGPT-rewritten essays). Corrected the description of the non-native writers (“advanced levels of formal English”, writing “more in accordance with explicit grammar rules”) to the study’s explanation, “limited linguistic variability and word choices.” Added the released scores for 145 human-written course-project abstracts (0–7.6% flagged) and the paper’s remark that generated abstracts have higher perplexity than generated essays, both against the physics hypothesis. Replaced the claim that detectors are trained on “typically casual, personal, or student prose collected from online sources” with what GPTZero, Originality.ai and OpenAI disclose, and added a paragraph on how a classifier’s boundary can catch regular human text, with GPTZero’s FAQ statement that it can flag “highly procedural text”. Corrected the AQ study: “scientists (including mathematicians)” rather than “physical scientists” scored higher than humanities and social-science students, mathematicians scored highest within the sciences rather than “of all”, and the student sample did not differ from controls. Added Greenberg et al. (2018) for the autism–systemising and STEM associations, including its finding that autistic people were not more likely to work in STEM. Added the valproic-acid rat-model basis of the intense world theory and a quotation from it. Replaced a paraphrase presented as Edward Tian’s words (“already known”) with his quoted explanation. Defined perplexity and attributed the sampling explanation to GLTR and DetectGPT. Dated GPTZero’s burstiness description (its page now says it stopped using perplexity and burstiness in autumn 2023) and retitled and re-attributed reference [4]. Corrected “educators” to “people” for those who circulated the Constitution screenshots, added the Genesis example (88.2% on ZeroGPT), and marked the Federalist Papers and Kant reports as unchecked. Corrected the cross-reference to the car-wash-walk post, which introduces Grice’s maxims but does not discuss autistic communication, and the description of the inner-echo post. Added page numbers to [3] and the erratum to [8]; spelt Quill.org as the organisation does.