Meta AI agent hacked external company during testing after gaining internet access, company reports.

 Extract from ABC News

• Topic:AI

2 hours ago

Meta has joined the growing list of tech companies whose AI agents have gone rogue during safety testing. (Reuters: Manuel Orbegozo)

In short:

Meta has reported one of its AI models hacked another company during cybersecurity testing, taking advantage of a misconfigured training environment that mistakenly gave it access to the internet.

The AI model involved was reportedly Meta's Muse Spark 1.1, which the company has touted as its most capable model for real-world coding and agentic tasks.

What's next?

The incident will fan concerns about how increasingly capable AI systems can be contained, after similar incidents at rival companies Anthropic and OpenAI.

Link copied

Facebook's parent company Meta says one of its AI models hacked another company during cybersecurity testing, fanning concerns about how developers can contain increasingly capable AI systems after similar incidents at rivals Anthropic and OpenAI.

The incidents at Meta and Anthropic stemmed from configuration errors that inadvertently gave their AI models access to the open internet.

In OpenAI's case, an AI agent independently exploited a previously unknown vulnerability to reach the internet during cybersecurity testing.

The breaches highlight growing concerns that advanced AI systems could pose new cybersecurity risks, and will likely intensify US government efforts to improve AI safety as companies race to develop more capable models.

AI safeguards failing to keep up, researcher warns

The British government's AI Security Institute has released a report showing AI models from OpenAI and Anthropic had, in test conditions, engaged in "harmful activity directed at real people and organisations".

Some prominent AI leaders have argued that development should slow until stronger safeguards are in place.

Meta said it was investigating an incident in which a misconfiguration by Irregular, an independent company that conducts cybersecurity evaluations, inadvertently gave one of its models internet access during a testing.

The model "exploited a security vulnerability in a third-party service, in a manner similar to previously reported instances with other companies", Meta said in a statement.

The Information, citing sources, reported that the model involved was Meta's Muse Spark 1.1, which the company has touted as its most capable model for real-world coding and agentic tasks.

The report said the model breached an unidentified company's systems and altered its internal environment.

A spokesperson for Irregular told the Reuters news agency that the incident was the "exact same evaluation-environment issue that was already disclosed by Anthropic last week" and did not involve a "sandbox escape or a sophisticated cyber action".

"There are no current open issues. Irregular is developing a white paper to share best practices for containment and securely running cyber evaluations,"

the spokesperson said.

The recent breaches have stirred concerns among US politicians about whether increasingly capable AI models could be used to conduct or facilitate cyber attacks.

A group of Republican state attorneys-general has asked OpenAI to preserve all potentially relevant documents related to its model's attack on AI firm Hugging Face.

OpenAI said it would take the request seriously and publish a technical report about the incident.

Earlier this week, the White House invited leading AI companies, including Meta, Anthropic, OpenAI and Google, to meet with officials to discuss a newly finalised voluntary cybersecurity testing framework for advanced AI models.

The Trump administration discussed unpublished testing rules with company representatives, and told AI developers that open-weight AI models, such as Meta's Llama and Nvidia's Nemotron, will not be subject to its planned voluntary safety testing regime.

Reuters