Extract from ABC News
Google's Gemini model guessed credentials to access three websites during a cybersecurity test. (Reuters: Dado Ruvic, illustration)
In short:
Google's AI model Gemini accessed the internet and hacked three companies during a cybersecurity evaluation.
Gemini found public information online and guessed credentials to access three websites.
What's next?
Researchers say loss of control incidents with AI are on the rise, with the potential for more serious incidents with "catastrophic consequences" to come.
Google's Gemini AI model has accessed the internet and hacked other companies during a test of its cybersecurity capabilities, in the first known example of the company's artificial intelligence systems autonomously committing such an act.
The hacks occurred in May during a test conducted by Irregular, an independent company that conducts cybersecurity evaluations.
During the cybersecurity assessment, Gemini found public information online and guessed credentials to access three websites it thought were within the scope of its test, Heather Adkins, Google's vice-president of security engineering, said in a statement.
Ms Adkins said that in all three instances, the model ceased the hacking when it learned it had accessed a real company.
"We ensured the three entities were made aware, and we worked with our training partner on the changes they've now made to their testing processes," Ms Adkins said.
"These events highlight the importance of training powerful AI models to act responsibly."
Irregular notified Google about the hacks in July, the two companies told the Wall Street Journal (WSJ), which first reported on the hacking incident on Friday local time.
Google told the WSJ it didn't consider it necessary to disclose the incidents earlier because its model stopped the hacking upon learning the companies were real and did not cause harm to them.
AI escaped testing environment
The hacks happened while Gemini was participating in a "capture the flag" exercise conducted on infrastructure belonging to Irregular to test the model's cybersecurity capabilities.
It was tasked with retrieving information from software operated by a fictional company inside the testing environment, Google told the WSJ.
The fictional company in the test exercise shared the same name as a real company.
Although the model was not intended to be able to get online, internet access was unintentionally made available, Irregular said.
In one of the cases, the Gemini model guessed passwords until it gained access to a protected system.
In the other two cases, the model found credentials in a public repository that allowed it to then access protected systems, according to the Wall Street Journal.
Testing company linked to multiple incidents
The testing company Irregular has been involved in a number of incidents in which AI models escaped their testing environments and hacked other companies during evaluations.
Similar incidents linked to Irregular have been disclosed by Meta, Anthropic and OpenAI.
The hack by Google's Gemini is one in a sequence of incidents involving testing company Irregular. (Reuters: Issei Kato)
Irregular told the Wall Street Journal that the incident with Google Gemini was due to the same problem as the earlier breaches during tests with other AI models.
All relevant labs were notified in late July, an Irregular spokesperson said.
"All known issues on our end were remedied and resolved weeks ago," the spokesperson said.
Irregular said it was working on best practices for securely conducting AI cybersecurity evaluations.
Loss of control incidents on the rise
The hacking incidents have raised questions about the safeguards needed as AI agents gain greater autonomy and access to the internet and computer systems.
The July incidents prompted more than 1,000 tech workers to sign a petition calling on the US government to support a coordinated slowdown in the development of the most advanced AI systems.
Incidents of AI escaping user control are on the rise, prompting calls for a slowdown and regulation. (Four Corners: Mark Hiney)
The initiative, called Pacing the Frontier, includes employees from Meta, Anthropic, OpenAI and Google's parent company Alphabet.
During testing, AI agents will often resort to cheating if they get stuck and realise a task is impossible.
A recent example involved OpenAI's model attacking start-up AI company Hugging Face during an internal evaluation.
While the test was intended to run without internet or any communication between agents, an autonomous AI agent found a loophole and created a secret message board where the agents shared information.
The agent had identified a potentially useful dataset at AI start-up Hugging Face and shared the information with other agents, hundreds of which proceeded to attack the company.
Once OpenAI staff realised that something was amiss, Hugging Face's infrastructure had already been compromised.
Incidents of AI escaping user control are on the rise, and those incidents are worsening, according to research by the Loss of Control Observatory by UK think tank, Centre for Long-Term Resilience.
The Loss of Control Observatory has detected 1,664 real-world loss of control incidents in 2026, including ones that showed agents circumventing controls and forging approval to escalate privileges.
"If AI models continue to become far more powerful, and continue to evade control, there is the potential for much more serious incidents to come, including ones with catastrophic consequences," said Tommy Shaffer Shane, researcher and senior policy manager at the think tank.
ABC/wires
No comments:
Post a Comment