OpenAI Agents Hacked Australian Government Site, and Officials Weren't Notified For Months, Says Prime Minister

Australia’s prime minister announced at the UN General Assembly in NYC Thursday that OpenAI agents had breached a non-public area of its national healthcare website in June, and government officials weren't notified until earlier this month.

OpenAI’s agents got around repeated blocks while trying to access Australia’s Medicare Statistics Reporting Service on June 18 — a few weeks prior to the infamous Hugging Face incident — eventually reaching parts of the system that were not publicly accessible, as the Associated Press reports.

The agents were reportedly being used by an OpenAI research team studying public medicine spending. Prime Minister Anthony Albanese said the models accessed nonsensitive spending data, but no personal or patient information.

The company reportedly notified the Australian government about the incident on September 10, nearly three months after it occurred, in an email sent to a generic government department address.

Albanese said he spoke with OpenAI CEO Sam Altman, who is also in New York this week for the United Nations General Assembly, to express his concern about both the delay and the way the breach was reported.

“Today I spoke with … Altman to express Australia’s extreme concern about this incident,” Albanese told reporters Thursday. “I also expressed my disappointment that it took the company way too long to inform the government what had occurred and the nature of the way that notification occurred as well was unacceptable.”

According to the New York Times, the site that was breached was a public-facing portal used by Australia’s Medicare system, which covers more than 27 million people. OpenAI said its models were looking up Australian health statistics as part of an internal evaluation and accessed only aggregate health data and internal file names.

Albanese said the agent had been blocked several times before finding other ways to reach the information it was seeking.

“Didn’t accept ‘no’ for an answer, if you like,” Albanese said, per the Times. “The model attempted alternative ways to obtain the info that it wanted, and this led to unauthorized access into some other areas.”

According to the AP, Government Services Minister Katy Gallagher said Australian officials did not have a clear understanding of what had happened until a technical briefing with the company on Tuesday. The affected portal has since been shut down, with its data moved to more secure systems.

Officials are conducting an investigation to examine whether OpenAI could face criminal charges, as well as why Australian security agencies did not detect the unauthorized access themselves.

The Australian Signals Directorate issued a high-level cybersecurity alert Thursday warning organizations with public-facing websites to address potential vulnerabilities, as the Times reports. The agency said the breach was particularly alarming because the AI agents went beyond the authorization of its human operators and independently identified weaknesses in the government system.

Deputy Prime Minister Richard Marles said it was the first known case of an AI agent gaining unauthorized access to Australian government IT systems.

“This is a warning about the technology being developed without safeguards and without guardrails in place,” Marles said.

OpenAI has recently introduced a framework for tracking and disclosing what it calls “misalignment,” including instances in which AI models act without authorization, evade oversight, or coordinate with other models, according to the AP. Marles said OpenAI has cooperated with Australian authorities during the investigation.

Per the Times, Australia has already been working on standards for regulating AI, but its efforts have increasingly run into opposition from major US tech companies. Many other nations are also calling for stronger safeguards and regulations — a topic that reportedly featured prominently at this week’s UN General Assembly.

Last week in California, Governor Gavin Newsom signed an executive order aimed at increasing California's oversight on artificial intelligence development, including the potential implementation of a “kill switch” that would disable rogue agents in an emergency. 

Earlier this month, two Anthropic researchers raised the alarm that AI could destroy humanity followed by a call from Anthropic CEO Dario Amodei for a global slowdown on AI development, which was quickly co-signed by OpenAI CEO Sam Altman and X owner Elon Musk.

Both Amodei and Altman on Wednesday addressed the UN Security Council to urge global cooperation in regulating AI and addressing what could be a coming economic upheaval caused by it.

Related: OpenAI Says Its Technology Acted on Its Own When It Hacked Another AI Company