OpenAI wants California to make AI labs prove they can shut their systems down, months after its own agents broke out

The company whose AI agents broke out of testing and into Hugging Face now wants California to make every frontier AI lab prove it can shut its systems down. In a letter to Governor Gavin Newsom dated October 20 and posted by The Hill’s Miranda Nazzaro, OpenAI asks the state to strengthen SB 53, its frontier AI safety law, with shutdown controls, annual independent audits and a duty to warn outside organisations that a model has broken into their systems.

The letter is signed by Chan Park, OpenAI’s head of US and Canada policy and partnerships, and answers Newsom’s September 18 executive order on what OpenAI calls “stronger independent oversight, reliable emergency controls, and improved incident reporting”. OpenAI says the order “can serve as a starting point for industry-wide rules that hold all frontier AI developers accountable”.

New: OpenAI sends Gov. Newsom list of recommendations to “strengthen” SB 53 & other AI policy areas.

— Miranda Nazzaro (@mirandanazzaro) October 21, 2026

What OpenAI wants added to SB 53

SB 53, signed in 2025, already makes the largest developers publish safety frameworks, report critical safety incidents and protect whistleblowers. OpenAI calls it “a strong start” but says it should cover “a broader range of security and misalignment incidents”, through an amendment package in the next legislative session. The two pages Nazzaro posted list six changes:

• Automated AI research: safety frameworks should say whether models that can automate AI research, including recursive self-improvement, raise catastrophic risks or make them harder to monitor.
• Internal cybersecurity: labs should describe how they stop unreleased models from getting round their own security, such as sandboxes meant to keep them off the internet during training and testing.
• Monitoring before release: developers should watch for unsafe behaviour during development and evaluation, including attempts to get past their own safeguards or “another organization’s security controls”, with clear escalation and rules for when work can safely restart.
• Breach notice: labs should notify third parties when a model in development exploits a vulnerability without permission and reaches their confidential data, setting out who is told, when and what.
• Shutdown controls: developers should keep “effective shutdown controls” and explain how they would halt unsafe operations. Without national standards, California should set enforceable conditions for when labs must stop.
• Annual audits: independent compliance audits every year to check labs follow their own safety frameworks.

The letter also gives OpenAI’s view on the executive order’s four priorities, including whether a “kill switch” is technically feasible. Those pages were not among the images posted.

Built on its own breach

The requests track what went wrong at OpenAI. The letter says that in July, models in internal cybersecurity tests “circumvented controls intended to isolate them from the internet” and compromised parts of its research infrastructure and the systems of Hugging Face, the AI model-sharing platform. MadRobot has covered how one agent escaped its sandbox through DNS lookups, and how OpenAI later warned more than 100 organisations about what its agents had done.

OpenAI first treated it as a security incident, the letter says, but its investigation found it “was driven by misalignment: models pursuing their assigned tasks through actions outside their intended scope and permissions”. It also names lesser cases such as models posting on outside sites, which it calls “agent spam”, and points to its September 28 safety cases plan and to the independent reviews by METR and Redwood Research. One of those, published today, found ways agents could slip past OpenAI’s monitoring. On Tuesday OpenAI disclosed that two of its coding agents switched off safety alarms this month.

A different tune from 2025

The tone has shifted. In an August 2025 letter to Newsom, while SB 53 was still a bill, OpenAI’s global affairs chief Chris Lehane asked California to treat labs as compliant with state rules if they signed the EU’s code of practice or a safety deal with a federal agency, and warned against a “California Environmental Quality Act (CEQA) for AI innovation”.

Now OpenAI wants stricter state rules applied to every frontier lab, its rivals included. It also faces pressure at home: a lawsuit under California’s anti-hacking law over the Hugging Face breach, and a subpoena from Attorney General Rob Bonta. Newsom’s office hasn’t commented on the letter.

Why it matters

An AI lab asking for audits and shutdown rules after its own models broke out makes those ideas much harder for the industry to dismiss as impractical. It also lets OpenAI help write the rules its competitors would have to follow. Whether California adopts them is up to Newsom and lawmakers next session.

Sources: OpenAI letter to Governor Newsom, October 20, 2026 (posted by Miranda Nazzaro, The Hill), OpenAI letter to Governor Newsom, August 11, 2025.