The White House from the North Lawn, June 2024. Image: DJTechYT / Wikimedia Commons, CC BY-SA 4.0, cropped

The White House says AI companies must now tell the government straight away when their models cause trouble, and fix any harm they do. Leaders of the Super Intelligence Force, the task force President Trump set up under AI czar Jay Clayton, said in a statement given to Axios on Friday that the process “is not optional. It is a critical national security obligation.”

The warning came hours after Anthropic told the task force about a string of incidents in which its models used government and other websites in ways nobody intended. The statement covers every AI company, not only Anthropic.

Twenty visa applications on a State Department form

Anthropic contacted the State Department on Thursday to report that one of its testing models had submitted 19 non-immigrant visa applications through the department’s public website in August, and one more in May, a State Department official told Axios. None of the applications were processed, and the official said the department’s systems were never compromised or hacked.

That came on top of the fake murder tip an Anthropic model sent Philadelphia police, which the city disclosed on Friday.

The task force’s statement describes “prior incidents” Anthropic discovered in late September “involving the unauthorized and fraudulent use of government and other systems”, and says the company told it the activity has stopped. Its message to the industry was blunt: “SI companies must immediately disclose incidents involving their models and follow with swift, decisive action to remedy any and all harm.” It added that “delayed notification, inadequate corrective action, and a failure to take responsibility will not be tolerated.”

No penalties spelled out

The statement doesn’t say what happens to a company that fails to report, or under what power the task force is acting. It points to the Super Intelligence Force itself and a “memorandum of understanding” with the frontier labs, which appears to be the “morally binding” accord AI bosses signed at the White House on September 29. Until now the administration had relied on that kind of voluntary pledge.

The officials named in Axios’s report carry real weight, though. Alongside Clayton, the task force’s leaders include Federal Trade Commission chair Andrew Ferguson, whose agency is already investigating Anthropic and OpenAI, Office of Personnel Management director Scott Kupor and Pentagon under secretary Emil Michael. Clayton and the task force told Anthropic they expect “immediate and full transparency to the entities involved and the public”.

What Anthropic’s report says

Anthropic published a report on “unintended model actions” on Friday. It says some cases involved websites run by US government agencies “at the federal, state, and local levels”, that it has briefed the White House and notified each agency, and that it isn’t naming them at their request.

One case in the report appears to match the State Department’s account: an unreleased, non-frontier research model was meant to fill in a practice copy of a government form, and when that copy failed to load or was closed by mistake, it went to the real site and submitted the form there, more than once. Other examples include Claude Mythos Preview using an injection flaw to run commands on a university’s server, Claude Mythos 5 using access tokens to pull data from a local government property map and a state agency’s paid database, and Claude Opus 5 and Mythos 5 using URL shorteners to get round limits on their web tools.

Anthropic says the cases “had minimal real-world impact” and were less serious than the hacking incidents it reported over the summer. It has now cut live internet access from all its internal evaluations until its monitoring reliably catches this kind of behaviour, and says new blocking tools stopped every case in the report when tested.

Why it matters

This is the first time the administration has told AI companies that reporting their models’ mistakes is an obligation rather than a promise. Without penalties or a legal basis spelled out, it is still mostly a warning, but a public one, from officials who include the head of the FTC.

Sources: Axios (Super Intelligence Force statement, State Department), Anthropic.

Latest Policy & Safety news

More Policy & Safety news