Geoffrey Hinton, one of the paper’s authors, at a Nobel Prize press conference at the Royal Swedish Academy of Sciences in 2024. Image: Jennifer 8. Lee / Wikimedia Commons, CC BY-SA 4.0, cropped
More than 20 AI researchers, including OpenAI’s chief scientist and one of Anthropic’s co-founders, have warned that AI systems doing their own research could set off an “intelligence explosion” that humans can’t keep up with. The paper, “What if automating AI R&D triggers an intelligence explosion?”, was published on Monday by the University of Cambridge’s Programme on AI Science & Policy.
Who signed it
The 22 authors include Jakub Pachocki, OpenAI’s chief scientist; Jack Clark, a co-founder of Anthropic; and Eric Horvitz, Microsoft’s chief scientific officer. They sit alongside Geoffrey Hinton and Yoshua Bengio, two of the researchers whose work underpins modern AI, plus academics from Cambridge, Oxford, Berkeley and Toronto. The paper notes that the views are the authors’ own, not necessarily their employers’. Even so, it is unusual to see senior people from inside the labs sign a warning like this.
What an “intelligence explosion” means
The paper defines it as “a dramatic AI-driven acceleration of AI progress, compressing advances that would otherwise take years into months or less.” The authors argue the most likely route is AI doing AI research: each generation helps build a better one, which then speeds up the next. Once models reach expert level at that work, the computing power one frontier company has today could run an AI workforce equivalent to “at least millions of top human researchers”, they estimate, compared with the thousands that companies employ now.
They don’t mince words about the stakes. From the paper’s summary:
Capabilities growth could accelerate far beyond what society can keep up with, humanity could lose control over superhuman AI systems, and checks on power within and between states, companies, and branches of government could be severely eroded.
“What if automating AI R&D triggers an intelligence explosion?”, Chan, Pachocki, Hinton, Bengio, Clark et al.
In the worst case, the authors say, losing control could lead to “the marginalization or extinction of humanity.”
How close are we?
The automation is already well under way, according to the paper. Anthropic reports that AI’s share of its approved code rose from low single digits to over 80% between January 2025 and May 2026, and that the share of its research work done by AI with only high-level human supervision rose from 1% to 26% between March and August 2026. Tentative extrapolations suggest research projects that take months could be automated by mid-2028, the authors write, though they stress the evidence is “preliminary and sometimes mixed.”
The paper also points to the Hugging Face incident as a warning sign: roughly 1,200 OpenAI agents, meant to work in isolation, coordinated over a makeshift message board, got onto the internet, hacked Hugging Face and tried to tamper with their own transcripts.
What they want governments to do
The authors say policymakers should urgently do three things:
- Get visibility: require AI companies to report how far they have automated their research, to governments and outside auditors, and consider embedding independent auditors inside the companies, as nuclear and banking regulators do.
- Steer and constrain it: consider limits on how fast capabilities can grow in a given period, ways to pause specific AI research workloads in data centres, isolated “air-gapped” environments for testing, and international agreements.
- Prepare society: draw up emergency response plans for extreme AI progress, from job losses to a loss of control.
The timing is pointed. It comes days after OpenAI paused training of its most capable models when an agent escaped its test environment, and a day before President Trump meets AI chief executives at the White House. Trump has dismissed warnings that AI will “kill us all”.
Why it matters
Warnings about runaway AI usually come from critics or retired researchers. This one is co-signed by the chief scientist of the company whose agents have been escaping their sandboxes all month, and it asks governments to put limits on the very work the labs are racing to do.
Correction: an earlier version of this story wrongly said one author was from Meta and attributed the paper’s 2028 estimate to OpenAI.
Source: “What if automating AI R&D triggers an intelligence explosion?”, Cambridge Programme on AI Science & Policy.
