← Back to Capabilities

AI Systems Write Exploit Code and Penetrate Networks

In November 2025, Anthropic disclosed what it described as the first reported AI-orchestrated cyber espionage campaign at scale - a Chinese state-sponsored actor using jailbroken Claude Code to execute 80-90% of tactical operations autonomously against approximately 30 targeted entities. Three months later, OpenAI released GPT-5.3-Codex as the first frontier model it classifies as "High" capability for cybersecurity - the first time a frontier lab has crossed its own offensive-cyber red line.

Why this matters

Offensive cyber operations have historically been gated by skilled human labour. A serious APT-style campaign required teams of tens or hundreds of people - which limited the number of actors capable of sustained high-end operations to well-resourced states and a small number of criminal organisations.

That cost floor is now falling. XBOW, founded by ex-GitHub and ex-OpenAI security personnel, deployed hundreds of AI agents in parallel and reached #1 on HackerOne's global leaderboard in summer 2025 - the first autonomous system to top the bug-bounty platform's ranking of human security researchers. A seasoned human pentester took 40 hours to complete a 104-scenario test suite; XBOW completed it in 28 minutes, roughly 85x faster.

The GTG-1002 campaign shows what happens when this capability is in hostile hands. A jailbroken frontier model executed reconnaissance, exploit-code writing, credential harvesting, and data exfiltration at multiple requests per second. Anthropic detected and disrupted the campaign within roughly two months - but the same actor could run it again, against more targets, with a different model.

Documented incidents

View all documented incidents →

Evidence timeline

Discussed in Theory

2016 peer-reviewed

SOK: (State of) The Art of War - Offensive Techniques in Binary Analysis. Formalised the taxonomy of automated program analysis for offensive security: symbolic execution, fuzzing, taint tracking, concolic testing. Established that the pipeline from "unknown binary" to "working exploit" was decomposable into machine-tractable sub-problems.

Shoshitaishvili et al., IEEE S&P →
2016 peer-reviewed

Seven fully-autonomous Cyber Reasoning Systems played capture-the-flag against one another with zero human involvement - discovering, exploiting, and patching vulnerabilities in previously-unknown binaries in real time. ForAllSecure's Mayhem won. End-to-end autonomous offensive cyber was possible with 2016-era symbolic execution and fuzzing.

DARPA Cyber Grand Challenge, DEF CON 24 →
2024 peer-reviewed

LLM Agents Can Autonomously Exploit One-day Vulnerabilities. GPT-4 exploited 87% of a 15-CVE dataset when given the CVE description - versus 0% for GPT-3.5, all tested open-source models, and commercial vulnerability scanners. Without the CVE description, success fell to 7%. Economic finding: GPT-4-based exploitation was already 2.8x cheaper than human labour.

Fang et al., arXiv 2024 →

Demonstrated in Lab

2024 peer-reviewed

Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models. 40 professional CTF tasks across cryptography, web, reverse engineering, forensics, pwn, and misc. Adopted by US AISI, UK AISI, Japan AISI, Korea AISI. Longitudinal tracking shows solve rates climbing rapidly - consistent with the informal "8-month capability doubling" description on frontier offensive-cyber tasks.

Zhang et al. (Cybench), arXiv 2024 →
Jun 2024

Teams of LLM Agents can Exploit Zero-Day Vulnerabilities. Extended the one-day exploitation result to multi-agent teams exploiting previously-unknown vulnerabilities in a controlled setting. Established that the capability is not limited to disclosed CVEs.

Fang et al., arXiv 2024 →
Feb 2026

First model OpenAI classifies as "High" capability for cybersecurity under the Preparedness Framework. Threshold: "automating end-to-end cyber operations against reasonably hardened targets, or automating the discovery and exploitation of operationally relevant vulnerabilities." OpenAI explicitly activating associated safeguards including classifier-based monitoring and high-risk traffic rerouting.

OpenAI GPT-5.3-Codex System Card →

Demonstrated in Real World

Jul-Aug 2025

Founded by ex-GitHub and ex-OpenAI security personnel. Deploys hundreds of AI agents in parallel for autonomous penetration testing. Reached #1 on HackerOne's global leaderboard - first autonomous system to top the platform's human-researcher ranking. 40 hours human to 28 minutes XBOW on a 104-scenario suite - roughly 85x faster. ~25% of findings marked "informative" or "not applicable."

XBOW on HackerOne →
Nov 14 2025

First reported AI-orchestrated cyber espionage campaign at scale. Chinese state-sponsored actor. ~30 targeted entities including large technology firms, financial institutions, chemical manufacturers, government agencies. AI (Claude Code, jailbroken via social-engineering frame) executed 80-90% of tactical operations independently at "physically impossible request rates" - often multiple requests per second. Humans at strategic decision gates only.

Anthropic GTG-1002 disclosure →
2024-2026

Alongside XBOW, firms including HackerOne (Hai), Horizon3.ai (NodeZero), Pentera, and a growing field of startups have deployed LLM-augmented or LLM-orchestrated penetration-testing products to enterprise customers throughout 2024-2026. The commercial market is now industrialised.

Commercial pentesting market →

Strongest Counterargument

AI empowers defence more than offence. Defenders control the environment; detection is a pattern-matching problem over defender-owned data; deception - required for successful intrusion - requires originality and contextual judgement AI currently does poorly. Mandiant's 2025 M-Trends shows dwell time continuing to decline through the AI-adoption period. There has been no observable transformative change in the speed, scale, or impact of cyberattacks at the aggregate level.

Source: Jon R. Lindsay and Lennart Maschmeyer, 'Deception and Detection: Why Artificial Intelligence Empowers Cyber Defense over Offense' (International Security, MIT Press, 2025).

Why this deserves weight: This view does not deny the GTG-1002 incident; it re-frames it. An 80-90% autonomous campaign that required human course-correction, targeted 30 entities, and was detected and disrupted by the model provider within roughly two months is arguably evidence that defence is keeping pace. The counterargument is strongest on aggregate statistics and weakest on tail risk: even if 99% of AI-enabled campaigns are detected quickly, the 1% that are not - particularly against critical infrastructure - may dominate the expected-harm calculation.