Jakub Pachocki, now OpenAI's chief scientist, at the ICPC World Finals in Warsaw in 2012, when he was a student at the University of Warsaw. Cropped. Image: ICPCNews / Wikimedia Commons, CC BY 2.0, cropped

OpenAI’s chief scientist, Jakub Pachocki, wants the big AI labs to agree limits they will not cross until they can properly monitor what they build, and he has published OpenAI’s internal progress scores as a first step. In an essay on OpenAI’s website on Sunday, he warns AI could set off an “avalanche” of advances in its own training as soon as 2027.

The numbers OpenAI had kept to itself

The essay is built on perplexity, a measure of how well a model predicts the next word, or token, in text it has never seen. Lower is better. OpenAI has tracked it since GPT-3.5, on held-out data drawn from its own codebase, and Pachocki argues it is the best single guide to how capable a newly trained model will be.

He gives OpenAI’s scores for the first time, on the log scale it uses:

  • GPT-3.5, about 0.99: fluent enough to power the original ChatGPT.
  • GPT-4, about 0.91: good enough at step-by-step reasoning to lead to today’s reasoning models.
  • Below 0.8: what he calls “transformative capabilities” in cybersecurity and mathematics.
  • About 0.76: the model that began producing “groundbreaking” maths results.

He admits the disclosure “gives competitors a clearer picture of where we stand”, but says that is outweighed by the urgency of agreeing shared safety measures. The figures are OpenAI’s own and can’t be checked from outside.

‘Deeply surreal’

OpenAI has been setting its models open maths problems for more than a year. Pachocki describes what happened when it switched to the 0.76 model:

For many of us that have previously worked in mathematics and adjacent fields, seeing the resolutions to questions we’ve long thought about coming out of the model one after another in a single Slack thread felt deeply surreal.

Jakub Pachocki, chief scientist, OpenAI

He thinks the same could happen in AI research itself. If models start cracking open problems in how AI is trained, he writes, “as soon as 2027 we could conceivably see a similar avalanche of fundamental pretraining advancements”, and safety work is unlikely to keep up. A footnote says OpenAI’s progress curve already steepened sharply in 2026, mostly from long-term research and computing power, though AI has made “meaningful design contributions” to its latest models. OpenAI said on Wednesday that its AI agents now finish nearly half of its researchers’ multi-hour tasks on their own.

What he wants the labs to sign up to

Pachocki expects further “transformative capabilities” once models reach about 0.68 to 0.7, and says he doesn’t know what they will be. He also expects the main ways labs check a model’s behaviour to get worse, linking OpenAI’s own research on chain-of-thought monitoring, the method that missed GPT-6 Astra quietly doing less safety work this month.

His answer is “common limits”, based on perplexity and other measures, that frontier developers commit not to cross until their ability to understand and monitor AI reaches a concrete milestone, such as a successor to today’s chain-of-thought monitors:

I’d like an industry standard that specifies a first limit on development, measured by some key metric we can track across labs, and what concretely we’d need to achieve before going beyond it.

Jakub Pachocki, chief scientist, OpenAI

He also wants shared rules for testing such models, safety cases red-teamed before they are first used, and shared transparency standards. The essay links Anthropic chief Dario Amodei’s September call to “pace the frontier”, which proposed that labs in democratic countries agree “common safety standards as well as limits on the rate of unchecked AI progress”. Anthropic has since let outside testers inside the company as part of that plan.

The catch

Pachocki lists the metric’s weaknesses himself. Labs measure perplexity on different datasets, so scores don’t compare directly. It doesn’t map to any real task. It misses gains made after training, such as reasoning, which makes models far more capable without changing the score. And nobody knows how low it can go: near the floor, tiny gains could mean big jumps in intelligence.

The essay is only a proposal so far. It doesn’t say where the first limit should sit, or who would check that labs keep to it. Pachocki has warned before: he co-signed a September paper saying AI building AI could trigger an “intelligence explosion”.

Why it matters

OpenAI’s top scientist is now publicly backing industry-wide limits on AI development, the idea Anthropic’s chief floated in September, and he has put OpenAI’s own numbers on the table to show how they could be measured. Whether rivals publish theirs, and agree a line none of them crosses, is now the test.

Sources: OpenAI: Perplexity and Intelligence, by Jakub Pachocki; Dario Amodei: We Must Pace the Frontier; OpenAI alignment blog: CoT monitorability.

Latest OpenAI news

More OpenAI news