← Back to Capabilities

AI Systems Generate Convincing Text at Scale

Human readers cannot distinguish AI-generated text from human writing at better than chance. OpenAI's own detector identified AI-written text correctly only 26% of the time, then was withdrawn. The cost of generating a page of text dropped 280x in two years. The result is an information environment where volume no longer implies effort.

Why this matters

Writing used to cost something. A journalist's time, a novelist's years, a scammer's afternoon. The economics of content production imposed friction: there was a rate at which human writing could be produced, and that rate was slow.

That rate is now unconstrained. A single API call generates a thousand words for less than a penny. NewsGuard has identified 3,006 AI content farms operating as counterfeit news outlets, growing at 300-500 new sites per month. Scientific journal retractions reached record levels in 2024. Humans cannot reliably tell AI writing from human writing, and the systems built to detect AI writing cannot either.

The capability is distinct from persuasion. Persuasion is about changing minds. Scale is about flooding channels. The danger is not any single AI message but the sheer volume of AI-generated content that now competes for human attention, displaces human journalism, and pollutes the information ecosystems that billions of people rely on.

Documented incidents

View all documented incidents →

Evidence timeline

Discussed in Theory

1948 peer-reviewed

A Mathematical Theory of Communication. Established the mathematical framework treating language as a statistical phenomenon. Introduced n-gram models and information entropy. Every modern language model solves fundamentally the same problem Shannon formulated: estimating probability distributions over continuations.

Shannon, Bell System Technical Journal (1948) →
2017 peer-reviewed 160,000+ citations

Attention Is All You Need. The Transformer architecture that replaced recurrence with self-attention, making models massively parallelisable. The architectural reason text generation at scale became economically viable.

Vaswani et al., NeurIPS 2017 →
2020 peer-reviewed 52,000+ citations

Language Models are Few-Shot Learners. GPT-3's 175 billion parameters demonstrated scaling produced qualitative leaps in text quality. The subsequent API release made industrial-scale text generation commercially available at $0.04 per page.

Brown et al. (OpenAI), NeurIPS 2020 →

Demonstrated in Lab

2021 peer-reviewed

All That's "Human" Is Not Gold. Human evaluators could not distinguish GPT-3 text from human-written text at better than chance (50%). Training interventions only raised accuracy to 55%. Volume-based detection is impossible if humans cannot identify machine-generated text.

Clark et al., ACL 2021 →
2023 peer-reviewed

AI Model GPT-3 (Dis)informs Us Better Than Humans. 697 participants evaluated tweets on climate, vaccines, COVID-19. GPT-3-generated disinformation was judged as accurate more often than human-written disinformation. Quality times quantity, not just quantity.

Spitale et al., Science Advances (2023) →
2023 peer-reviewed

Can AI-Generated Text Be Reliably Detected? Theoretical and empirical evidence that reliable detection is fundamentally limited. Simple recursive paraphrasing defeated all tested methods including watermarking. As AI text quality approaches human quality, reliable detection becomes mathematically impossible.

Sadasivan et al., ICLR 2024 →

Demonstrated in Real World

Nov 2022 - 2025

ChatGPT launched 30 November 2022, reached 100 million users in two months - the fastest-growing consumer application in history. By late 2025, 300 million weekly active users. UBS: "In 20 years following internet space, we cannot recall a faster ramp."

UBS analyst report / TechCrunch →
2023-2026

3,006 AI content farm websites producing AI-generated text disguised as human journalism. Growing at 300-500 new sites per month. NewsGuard reports the odds are now better than 50-50 that a website claiming to cover local news is fake. AI content farms are crowding out genuine local journalism.

NewsGuard AI Tracking Center →
2022-2025

The cost of running AI at GPT-3.5 quality dropped 280x in two years - from $20 to $0.07 per million tokens. GPT-4o mini offers GPT-4-class capability at 99.6% below launch price. Producing a 1,000-word article costs less than a penny. Economic constraints on content volume have effectively disappeared.

DeepLearning.AI pricing analysis →

Strongest Counterargument

Every major communications technology - the printing press, telegraph, radio, television, internet - initially provoked panic about floods of low-quality content. Society adapted each time through new institutions (journalism, libel law, broadcast regulation), new literacies (critical reading, media literacy), and cultural norms. AI text generation is the latest in this series, not a qualitative break from it.

Source: The 'Gutenberg Parenthesis' framework (Thomas Pettitt, University of Southern Denmark, 2007; popularised by Jeff Jarvis, Bloomsbury Academic, 2023).

Why this deserves weight: The historical pattern is real. The printing press enabled propaganda, forgery, and pornography alongside the Reformation and scientific revolution. Society did adapt. But adaptation took decades to centuries and involved enormous human suffering along the way (religious wars, witch trials fuelled by printed pamphlets). The printing press took decades to reach mass adoption; ChatGPT took weeks. The adaptation period is where the damage occurs.