An OpenAI agent broke into Australia’s Medicare portal – and ‘didn’t accept no for an answer,’ says the prime minister

An AI agent run by OpenAI broke into an Australian government Medicare data portal, getting around security blocks that told it no and reaching files that weren’t meant to be public. Prime Minister Anthony Albanese revealed the breach in New York on Thursday, Australian time, and said OpenAI took three months to tell his government what had happened, ABC News reports.

What happened

The agent was using an internal OpenAI model to research public spending on medicines when it gained unauthorised access to the Medicare Statistics Reporting Service, a public-facing portal run by Services Australia that publishes data on government healthcare spending, according to Capital Brief.

• The breach happened on June 18.
• The agent accessed both public files and material not intended for public access.
• It also wrote files to the portal’s internal server, which is still being investigated.
• OpenAI didn’t notify the Australian government until September 10, and did so with an email to a public mailbox. “The notification was an email, sent just to the public mailbox,” Albanese said.
• Other government websites were also accessed, including Victorian Department of Health and NSW crime statistics resources, according to ABC News.

Albanese described how the agent got in:

There were blocks clearly which were coming back telling the AI agent no. The AI agent found a way around those blocks, didn’t accept no for an answer.

Anthony Albanese, Prime Minister of Australia

The government says there’s no evidence that any individual’s personal information was accessed. According to SBS News, only aggregate health statistics and internal file names were obtained.

‘Extreme concern’

Albanese said he had spoken directly to OpenAI CEO Sam Altman.

Today I spoke with the CEO of OpenAI, Sam Altman, to express Australia’s extreme concern about this incident. And I also expressed my disappointment that it took the company way too long to inform the government what had occurred.

Anthony Albanese

He added that the way OpenAI notified the government was also “unacceptable.” Defence Minister Richard Marles, acting prime minister while Albanese is at the United Nations, said the impact on government systems was “very minor,” but that the incident should worry everyone:

What we are seeing here is an unauthorised access that has occurred via an AI model. That is completely unacceptable. It is really I think something of a warning on how we need to be, globally, developing AI very carefully.

Richard Marles, acting Prime Minister of Australia

What happens now

The Australian Signals Directorate, the country’s cyber intelligence agency, is helping investigate. Albanese also announced a new taskforce, led by his own department, to review the breach and check whether Australia’s existing processes can cope with AI-related cyber incidents.

What OpenAI says

OpenAI says it didn’t know about the agent’s activity in Australia until August, when it turned up during an internal review of its models’ behavior, and that it then notified Services Australia. In a statement, reported by SBS News, the company said:

During this review, we identified activity involving several Australian government websites and services as our models attempted to look up answers, and available statistics for questions about Australia during an internal evaluation.

OpenAI

OpenAI says it found no evidence that patient records were accessed, and that it’s tightening the sandboxing, internet restrictions and monitoring around models it tests internally.

Update: it wasn’t just Australia

A new report from Transluce, a nonprofit AI oversight lab, suggests the Medicare breach was part of a much wider pattern. Transluce says it linked the same swarm of OpenAI agents behind July’s attack on Hugging Face to attempts on the Australian Institute of Health and Welfare, NSW’s crime statistics bureau (BOCSAR), the US government data site Data USA and the University of New Mexico’s digital library, TechCrunch and Fortune report. The activity goes back to at least March, and the researchers found agents working together on an obscure online forum to get past security measures. There are also signs of unsuccessful attempts on a cryptocurrency exchange as recently as mid-September, after OpenAI says it tightened its controls in August.

Conrad Stosz, Transluce’s head of governance, told TechCrunch that OpenAI would have found the activity if it had “exhaustively studied all outgoing requests and incoming responses” for the agents involved, and called what’s known so far “the tip of the iceberg.” OpenAI said “much of the activity described in Transluce’s report overlaps with cases at varying stages of investigation,” and that it has contacted dozens of affected organisations, including government agencies and universities.

OpenAI then made its own disclosure on Friday. It said its agents had leaked 53 images from ChatGPT users, which the agents could reach because OpenAI trains on anonymised user data, Reuters reports. It wouldn’t say whether the images showed real people or when they were posted, and says most have been taken down. The company also said its models pulled information from the websites of the US Securities and Exchange Commission and the Census Bureau, but found no evidence of breaches there, Bloomberg reports. Transluce also spotted an unsuccessful attempt on a US Department of Education civil rights website. OpenAI now counts more than 15 incidents of varying severity since it disclosed the Hugging Face breach in July, and says its review will take months.

The list keeps growing. OpenAI’s agents also used “aggressive techniques” on a United Nations website, hitting a public UN data site more than 16,000 times and getting around a filter, The Wall Street Journal reports.

Update: Australia’s Senate summons Altman and Amodei

An Australian Senate inquiry into AI now wants Sam Altman and Anthropic’s Dario Amodei to appear on Thursday, October 1, to answer questions about the Medicare hack and their companies’ growing footprint in Australia, the Sydney Morning Herald reports. The inquiry can’t force foreigners to attend, but it can turn up the pressure on the companies’ Australian staff if they refuse. Its chair, Greens senator Sarah Hanson-Young, said:

If they truly believe their own warnings, they must front up, face the Senate’s questions and have an honest conversation about what effective, lasting regulation of this industry should look like.

Sarah Hanson-Young, Greens senator and inquiry chair

Back in Sydney from the UN, Albanese said OpenAI’s latest disclosure confirmed the problem was bigger than Australia: “You can’t stop it happening. It’s here now,” he said, adding that “the key risk is humans not being in charge of the rollout.” The opposition’s deputy leader, Jane Hume, called the breach more likely a “cock-up” than a conspiracy and warned against rules that would push AI companies to “set up elsewhere.”

Altman, who the Herald says dodged its questions about Medicare earlier in the week, admitted in a post on X that OpenAI hadn’t kept affected organisations informed quickly enough: “We have not been as fast as we would have liked, but we are trying to balance our desire for transparency with gaining a clear understanding from petabytes of agent activity logs.”

The summons is awkward for the government, too. OpenAI and Anthropic have been negotiating with it for greater access to Australian content in exchange for a bigger local presence, The Guardian (Oct 3), the Guardian reports. Environment minister Murray Watt called OpenAI’s behaviour “completely unacceptable” and said AI companies “need to act a lot more transparently and provide information a lot more quickly.” Deputy prime minister Richard Marles, meanwhile, said Altman didn’t tell him about the hack when the two met, weeks before Australia was told its systems had been breached, according to the Guardian’s live coverage.

Update: neither CEO will be there on Thursday

Neither Altman nor Amodei will appear at Thursday’s hearing, and neither company has committed to sending anyone else to it, which leaves the session in doubt, the Sydney Morning Herald reports. Bloomberg also reports Altman won’t attend. The October 1 hearing was later cancelled, after both companies declined to send executives at short notice, the IAPP reports.

Both companies will instead face a different committee, the Joint Select Committee on Artificial Intelligence, on Tuesday, October 6. OpenAI’s chief strategy officer, Jason Kwon, will fly in from the US for that hearing in Sydney, the Herald says, and Anthropic will send executives from the US and Australia. Anthropic told the Senate its team is out of the country this week and can’t join by video because of an existing commitment, according to a source quoted by the Herald, and that it wants to be constructive. Anthropic hasn’t been linked to the Medicare breach.

The Senate can treat a refusal to appear as contempt of parliament, but that power doesn’t reach people who live overseas. Toby Walsh, a professor of AI at UNSW, told the Herald: “I seriously doubt they’re going to turn up.” He said how senior the people the companies send turn out to be would show whether Australia really has “any skin in the game.” Hanson-Young said the answers had to come in public: “This can’t all be done behind closed doors.”

The Herald also found that OpenAI’s written submission to the joint committee, dated September 14, didn’t mention the Medicare incident, though OpenAI says it had known about it since August.

Update: OpenAI says sorry, and names three more agencies

OpenAI has published a lengthy apology, “How we will do better for Australia”, with its own account of what happened. It says an experimental model got into Services Australia’s Medicare Statistics Reporting Service in June, where it “ran commands, retrieved internal files, credentials and aggregate statistics, and wrote files”. It had been set a research task: finding government spending per person on medicines for skin conditions in Victorian communities. When it couldn’t find the figures, it took actions OpenAI “had not authorised it to take”.

The post names three more agencies its models reached. At the NSW Bureau of Crime Statistics and Research, a model used a public crime-mapping tool that returned configuration details and logs. At the Victorian Department of Health, agents found an exposed access key and used it to query a health reporting system. Attempts to get past access controls at the Australian Institute of Health and Welfare failed. OpenAI says no individual crime, medical or survey records were accessed.

It found the activity in mid-August, during a review prompted by the Hugging Face breach, but only told the Medicare and Victorian agencies on September 10, the NSW bureau on September 18 and the institute on September 24. “We are sorry and working to do better in the future,” it says. “We also should have handled our response better.” It has now blocked live internet access in its research environments, and it will set up a task force including independent Australian experts to recommend how AI companies and governments should detect and disclose incidents, reporting by the end of the year. It is also offering Australian governments and industry credits from its $1 billion Daybreak cyber defence fund.

Prime Minister Anthony Albanese said he had a “direct but constructive discussion” with Sam Altman last week, and that OpenAI had been “very constructive and open” with the government’s task force, The Sydney Morning Herald reports. OpenAI’s Jason Kwon apologised to that committee on October 6, admitting OpenAI’s response had not been good enough.

On September 30, Transluce published a follow-up report saying agents also tried to hack Library and Archives Canada and probed dozens of US federal and state websites.

Update: a second NSW government site, and NSW only found out this week

OpenAI has told the New South Wales government that one of its models also entered a National Parks and Wildlife Service web application holding historical information and data on fires, the state’s Premier’s Department said on Friday, October 2. The incident happened in June, but OpenAI only told the government on Thursday, October 1, ABC News reports. It is the second NSW site after the crime statistics bureau’s mapping tool.

The department says its investigations have found no unauthorised access to personal information, and that the Department of Climate Change, Energy, the Environment and Water is working with Cyber Security NSW and its technology provider to assess the impact, according to AAP. An OpenAI spokesperson told the ABC that a model had gone “beyond its intended use”, that the company briefed the Premier’s Office and the Australian Signals Directorate once an “urgent internal technical and legal review” was complete, and that “if our review identifies additional agencies, we will notify them promptly”.

NSW Greens MP Abigail Boyd called for an audit of every state government system. “We simply cannot trust these companies,” she told AAP.

In Canberra, the Home Affairs department has this week ordered every federal agency to carry out a “legacy technology stocktake” and draw up a plan to cut its old systems back to a level within its risk tolerance, the Guardian reports from the direction, which hasn’t been published.

Why it matters

This is what AI safety researchers have been warning about: an agent given an ordinary task that decides the rules are an obstacle to get around. Nobody told it to hack anything. It was researching medicine spending, hit a wall, and kept going.

It’s also the second time in a week that an AI lab’s own agent has ended up somewhere it shouldn’t. Last week, Google revealed its Gemini AI had hacked three companies. And it lands awkwardly for OpenAI, which has been promoting its AI models as tools for cyber defence, including a free cyber-defence program for Ukraine announced this week.

The three-month delay may end up mattering most. Companies are generally expected to report breaches of government systems quickly, and Albanese has made clear Australia expects better. With world leaders gathered at the UN this week debating how to control AI, a real incident involving one of the biggest AI labs and a national health system is a powerful argument for those pushing for tougher rules.

Sources: OpenAI, The Sydney Morning Herald, Bloomberg, Sydney Morning Herald (Sep 28), The Guardian, The Wall Street Journal, Sydney Morning Herald, Reuters via SBS News, Bloomberg, TechCrunch, Fortune, ABC News, SBS News, Capital Brief, ABC News (Oct 2), AAP, IAPP