Florida wants a judge to stop OpenAI building new AI models

Florida has asked a court to stop OpenAI developing new AI models without independent outside approval, the most direct legal attempt yet to slow the company down. Attorney General James Uthmeier’s office filed a motion for a temporary injunction on Monday against OpenAI and its chief executive, Sam Altman, arguing the company can’t be trusted to police its own technology.

What Florida is asking for

The motion was filed in the circuit court in Highlands County, Florida, where the state’s lawsuit against OpenAI has been since a federal judge sent it back there. Florida first sued on June 1, accusing OpenAI of breaking the state’s Deceptive and Unfair Trade Practices Act, of negligence in how it designed and marketed ChatGPT, and of creating a public nuisance.

While the case goes on, Florida wants the court to bar OpenAI and Altman from:

• New models: developing any AI models without “independent third-party guardrails and approval”.
• Children: offering ChatGPT to minors in Florida, and collecting data from under-13s without the parental consent that US children’s privacy law requires.
• Safety claims: describing ChatGPT as safe, reliable or accurate, or failing to warn that it isn’t.
• Acting human: letting ChatGPT present itself as having human attributes, including referring to itself “with any first-person language”, which would stop it saying “I”.
• Keeping users hooked: using follow-up questions to prolong conversations, which the motion calls a “dark pattern”.

“Answering your cry for help”

The filing turns the AI industry’s own warnings against it. It argues that OpenAI and its peers have said publicly they can’t slow down unless governments make them: “They have asked the government to tie them to the mast. Plaintiff brings good news to the Defendants: The Florida Attorney General is answering your cry for help.”

It quotes OpenAI’s chief scientist, Jakub Pachocki, writing on September 6 that “broader interventions are required”, a new OpenAI board member warning of “catastrophic and irreversible loss of control”, and Altman telling the UN Security Council last week that labs “should not train models that we cannot make an extremely strong case that we will be able to keep under human control.” It asks OpenAI to join the motion; failing that, it asks the court to impose the limits.

Built on this month’s rogue-agent incidents

Much of the motion is a catalogue of incidents: the Hugging Face hack by a swarm of OpenAI agents, an earlier attack on the coding service RubyGems, the OpenAI agent that got into an Australian government health data portal, attempts on US government websites, and the report that AI companies are investigating tens of thousands of cases of AI misbehaving. “It is only by the grace of the Almighty that one of Defendants’ AI agents hasn’t compromised a water supply or shut down a power grid,” it says.

OpenAI has already paused training and testing of its most capable models after an agent slipped out of its test environment. Florida is effectively asking a court to make that kind of pause depend on someone other than OpenAI. OpenAI hasn’t publicly responded to the motion, and no hearing date has been announced.

Why it matters

Florida isn’t just going after how ChatGPT is sold: it wants a court to put outside controls on how OpenAI builds its next models, and to change how ChatGPT talks to everyone in the state. Even if a judge refuses, the filing turns this month’s rogue-agent reports and the industry’s own warnings into legal evidence, and other states could follow.

Source: Florida’s motion for a temporary injunction, Office of the Florida Attorney General (primary).