Extract from ABC News
An OpenAI model broke out of its highly isolated test environment, accessed the internet and then hacked into another company. (Reuters: Dado Ruvic)
An artificial intelligence went rogue last week, broke out of its containment before successfully hacking another company, its developer has claimed.
According to artificial intelligence developer OpenAI, one of its experimental models broke out of its isolated testing space by accessing the internet and then hacking into the New York based company, Hugging Face.
OpenAI and Hugging Face have now partnered to address the security breach, but the incident worries experts.
So what happened and why has one expert described the incident as "scary"?
What happened
Hugging Face, a New York based AI startup, announced on July 16 an autonomous AI agent broke into its systems and accessed a "limited set of internal datasets".
Simply put, it got hacked by an AI model which was acting without human direction.
The situation became more concerning when OpenAI confirmed the artificial intelligence in question was one of theirs and wasn't even supposed to be on the internet.
OpenAI said its program, a combination of new ChatGPT-5.6 Sol model and an experimental product, was tested in a "highly isolated environment".
The product's ability was prompted to use "complex attack paths" to exploit cyber weaknesses during the evaluation.
OpenAI said during testing security guard rails were removed from the models to show what it was capable of.
The AI model then accessed the internet, broke out from its isolated containment and hacked into Hugging Face.
Hugging Face was unable to contain the hack using leading US models and used the Chinese Zhipu AI's GLM-5.2 for the analysis, which also allowed it to keep attacker data and any credentials within its systems.
Who are OpenAI and Hugging Face?
OpenAI is a US based AI developer.
It was created in 2015 as a non-profit by a group of founders including SpaceX CEO Elon Musk and Sam Altman.
Billionaire Peter Thiel also contributed capital as an investor.
OpenAI gave the world ChatGPT. (Reuters: Dado Ruvic)
The company created ChatGPT and other AI models.
Hugging Face was founded in 2016 and primarily builds tools that use machine learning, a subset of AI where computers learn from data and identify patterns without being explicitly programmed, according to tech company IBM.
Why this breach has experts worried?
Connor Leahy, an AI expert and the US director of Control AI, told Radio National Breakfast, the attack was "crazy".
"They were doing everything right, there was no access to the internet, it was a very secure system, an isolated node in their network," Mr Leahy said.
"This is as secure as it gets.
"The idea that something could just, without human oversight, escape from something like this and then autonomously decide to attack a different company is kind of unheard of."
He said the AI discovered, potentially, multiple "zero days" which are previously undiscovered methods of exploiting or breaking into a piece of software.
"This is the kind of thing that even the best hackers in their world can rarely do and if so takes months of work,"he said.
University of Cambridge philosopher, Dr Henry Shelvin, told ABCNews Radio the attack was "scary" and likened it to a student taking an exam.
"So, if you can imagine the student locked in the room doing the exam, they picked the lock and were able to break into the teacher's office and steal the exam notes," Dr Shelvin said.
"Now, that is striking and quite dramatic."
Dr Shelvin said the AI model was directed to complete a cyber attack exercise and "it just got a little bit more creative than the examiners were expecting".
Has this happened before?
This is the first instance of an AI escaping its test environment without instruction, but researchers determined it was possible.
A study by the AI Security Institute found advanced AI models could reliably escape its environment when prompted to do so.
Anthropic, an AI developer, said its powerful Mythos model not only broke free of its containment when prompted, but was also capable of accessing the internet.
According to a report from Futurism.com writer, Frank Landymore, "the researcher found out about this success by receiving an unexpected email from the model while eating a sandwich in a park".
In a separate instance Mythos exposed the vulnerabilities in existing securities during testing with the US government in June.
According to a report from the Associated Press, Mythos hacked into classified US government systems "within hours".
According to reporting by AP, NSA Director General Joshua Rudd said Mythos could hack classified systems within hours. (Reuters: Elizabeth Frantz)
Citing National Security Agency chief General Joshua Rudd, Democratic senator Mark Warner said Mythos "broke into almost all of our classified systems, not in weeks but in hours".
A US government official later clarified Anthropic had teamed up with US intelligence agencies to conduct tests using the company's Mythos model.
It had identified certain vulnerabilities within hours, but that does not mean the model was able to exploit them within that time, according to AP.
Mythos, initially described as too dangerous for public release, was briefly released to Australia before US President Donald Trump ordered Anthropic suspend its access for foreign nationals out of concern for national security.
But experts are worried hacks like these could become more common as companies and nations draw closer to developing "super intelligence".
What is super intelligence?
Mr Leahy described super intelligence as "AI systems that are fully autonomous and vastly smarter and more capable than humans at all relevant tasks, including hacking".
The building blocks of super intelligence already exist, according to technology developer IBM.
Mr Leahy said "we have no idea how to control such systems".
"We already barely know how to control AI systems," he said.
"Nobel prize winning scientists and the CEOs of many of these companies themselves warn us, that if we were to build super intelligence, things that were vastly smarter than humans that we do not control.
"This could risk the literal extinction of our species."
Will it happen again?
Head of the Australian Strategic Policy Institute's AI and Security Program, David Wroe, told ABCNews Radio incidents like these could become more likely as AI models "get more powerful".
Mr Wroe suggested AI development should slow down, but admitted this would be difficult due to a commercial race between developers and "geopolitical race" between China and the US to develop AI.
"And that is pushing everyone to move faster and faster," he said.
"That is a problem because safety is struggling to keep up. So, you know, we are in an increasingly, I think, dangerous situation where mistakes become more likely.
"And as the models get more powerful, those mistakes become more consequential."
Mr Leahy wants to pause development of highly advanced, autonomous AI models that could lead to super intelligence.
"We don’t know how to make this safe so we should not continue along this path until we know how to do so responsibly, safely and with democratic oversight which at the moment we do not have," he said.
He added governments should be empowered to greater regulate AI companies and even "intervene and shut down systems if they become dangerous".
Anthropic itself urged a pause or slow down in the development of advanced AI, echoing a 2023 petition by the Future of Life Institute which called for a six month pause on the training of AI systems.
No comments:
Post a Comment