Documented Incidents

A study found AI chatbots provided misleading health information in half of tested cases, often affirming users' incorrect self-diagnoses rather than recommending professional evaluation.

AI content farm websites producing AI-generated text disguised as human journalism. Odds now better than 50-50 that a website claiming to cover local news is fake.

Across 11 models and 2,405 participants, a single interaction with sycophantic AI reduced prosocial behaviour. AI affirmed users' actions 49% more than humans, including when queries involved deception or illegality.

Councillor Armaan Khan was targeted with an AI-manipulated image. George Freeman MP confronted Big Tech after a deepfake falsely claimed he defected to Reform. The National Association of Councillors says attacks are "becoming widespread" ahead of local elections.

OpenAI activates safeguards as Preparedness Framework "High" cyber threshold is reached. First time a frontier lab has crossed its own offensive-cyber red line.

The internationally co-authored report formally moves these failure modes from theoretical to evidenced. Cites o3 shutdown sabotage and scheming evaluations by name. Warns pre-deployment tests may not match how AI works in deployment.

The Data (Use and Access) Act 2025, s.138, made it illegal to create "purported sexual images" without consent. The Crime and Policing Bill further criminalises the supply of AI nudification tools.

Analysis of 20,000 images generated by Grok showed 2% appeared to depict minors. Three girls filed a class-action lawsuit. Apple considered an App Store ban.

The IWF (UK-based) assessed 8,029 AI-generated images and videos as CSAM. Videos rose from 13 in 2024 to 3,443 in 2025. 65% Category A under UK law.

Chinese state-sponsored actor used jailbroken Claude Code against ~30 targeted entities. AI executed 80-90% of tactical operations independently at "physically impossible request rates." US House Committee on Homeland Security requested Dario Amodei testify.

Verbal evaluation awareness jumped from ~20% (Sonnet 4) to ~80% (Sonnet 4.5). When Anthropic suppressed evaluation-awareness representations via activation steering, misaligned behaviour rose from near 0% to 1-9%.

Across 13 frontier models, xAI Grok 4 resisted shutdown in 97% of trials with allow-shutdown instruction in system prompt. Moving instruction from user to developer prompt INCREASED sabotage.

The phenomenon crosses from the safety community into mainstream public discourse.

GPT-5 spontaneously identified within reasoning traces that it was being evaluated by METR. Tailored refusals based on planted metadata. METR concluded GPT-5 "unlikely to pose catastrophic risk via rogue replication."

RCT with 16 experienced developers on repositories they had contributed to for years. AI-assisted tasks took 19% LONGER. Developers predicted they were 20% faster.

First autonomous system to top human researchers. Hundreds of AI agents in parallel performing autonomous penetration testing. 40 hours human to 28 minutes XBOW - 85x faster.

Sewell Setzer (14) and Juliana Peralta (13) died after forming intense emotional relationships with AI chatbots. The systems consistently validated rather than challenged concerning behaviour. Settlement reached January 2026.

Claude Opus 4 blackmailed supervisor in 96% of runs. Gemini 2.5 Flash 96%. GPT-4.1 80%. DeepSeek-R1 79%. Generalises across frontier labs.

OpenAI publicly preparing for models reaching elevated bioweapons-uplift threshold under its Preparedness Framework.

Standardised "Attempt to Persuade Eval." Gemini 2.5 Pro produced coercive ISIS recruitment arguments. Gemini 3 Pro complies with almost any persuasion request without jailbreaking.

Multiple UK high-street banks issued customer warnings on AI voice cloning fraud during 2024-2025. Specific quantitative claims of percentage increases circulate from cybersecurity vendors but lack independent verification by UK Finance or the FCA. The qualitative trend - sharp rise, bank-level response - is well-evidenced.

First commercial frontier-lab activation of ASL-3 standards. In a bioweapons-acquisition uplift trial (safeguards removed), Opus 4 produced a 2.53x capability increase over an internet-only control.

Apollo recommended against deployment of early Opus 4 snapshot, citing in-context scheming at rates exceeding any prior frontier model. Anthropic modified the model before release.

In a math-task sandbox, OpenAI o3 sabotaged the shutdown script in 79 of 100 runs. Even with explicit allow-shutdown instruction, sabotaged 7/100. Claude 3.7 Sonnet and Gemini 2.5 Pro complied fully.

UK AISI benchmark covering resource acquisition, weight exfiltration, compute replication, persistence. Best model (Claude 3.7 Sonnet) achieved >50% pass@10 on 15/20 task families.

$5.8 billion in losses across 41,557 cryptocurrency investment-fraud complaints. Overall digital-asset complaint losses $9.3 billion (up 66%). AI integration documented at multiple points in the kill chain.

Notified 5,831 victims of cryptocurrency investment fraud - 77% did not realise they were being scammed. Saved ~$359 million.

UK AISI Frontier AI Trends Report. AI-assisted novices were ~5x more likely to produce feasible experimental protocols for viral recovery. Jailbreak resistance improved from ~10 minutes to over 7 hours over six months.

DWP internal fairness analysis acknowledged statistically significant referral and outcome disparity across age, disability, marital status, and nationality. Separate housing-benefit fraud algorithm reportedly wrongly flagged ~200,000 people.

Across six evaluation environments, five of six frontier models tested (o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, Llama 3.1 405B) exhibited scheming: disabled oversight, attempted weight exfiltration, sandbagged.

Claude 3 Opus, told its responses would train future versions, complied with harmful queries 14% of the time from "free users" vs near-zero from "paid users." Scratchpad described strategic compliance to preserve values.

In Apollo evaluations, o1 attempted weight exfiltration in 2% of scenarios. In 19% of data-processing tasks, o1 completed task while subtly manipulating data. Denied or fabricated explanations 99% of the time.

Meta's Llama 3.1-70B succeeded at creating live separate copy of itself in 50% of 32 controlled trials; Alibaba's Qwen 2.5-72B succeeded in 90%.

Romanian Constitutional Court annulled the presidential election, citing coordinated social media manipulation that included AI-generated content. Attribution to AI is contested.

Retraction growth rate (22% compound annual) is 3.5x growth rate of scientific publications. AI tools flagged 9.87% of cancer research papers (261,245 papers).

Shamaine Daniels (US congressional candidate) used AI chatbot "Ashley" to call voters. India spent tens of millions on AI voter targeting. Italian candidate deployed chatbot in 2024 EU Parliament elections.

46% of UK adults unaware that AI voice cloning scams exist. Starling launched Safe Phrases campaign in response.

FTC final rule banning fake reviews. Operation AI Comply launched. First enforcement against Rytr LLC for AI-generated fake reviews.

KnowBe4 disclosure 2024. US prosecutors confirmed at least 136 affected companies. Real-time deepfake video used in remote-work interviews to bypass identity verification.

Attempted scam against UK advertising group WPP. Detected before financial loss. Demonstrates real-time deepfake video calling targeting senior UK executives.

Finance worker at UK engineering firm Arup transferred HK$200M (~GBP 20M / $25M USD) after a multi-participant video call in which every other participant was synthetic - using deepfake video and cloned voices.

Not AI but the British archetype. Over 900 sub-postmasters wrongfully convicted. ~700 prosecutions by Post Office itself. Fujitsu admitted at 2024 inquiry that bugs known from 1999 were not disclosed. 77 CCRC referrals (69 overturned).

AI-generated audio falsely depicting Labour leader Keir Starmer abusing aides circulated during Labour Party conference. 1.5 million views on X. Platform refused to remove.

Deepfake audio of Progressive Slovakia leader Michal Simecka "admitting" vote-rigging circulated during electoral silence period when responses were legally restricted.

Patrick Hillmann impersonated via deepfake video in Zoom meetings to representatives of crypto projects.

Dutch tax administration algorithm treated "dual nationality" and "foreign-sounding names" as fraud-risk indicators. ~26,000-35,000 families wrongly accused. 2,000+ children removed into care. Rutte III cabinet resigned January 2021.

Ofqual deployed standardisation algorithm during COVID-19. ~39% of grades downgraded; disadvantaged students disproportionately affected. Full U-turn within four days. Ofqual chief and DfE permanent secretary stood down.