Google’s Gemini AI system inadvertently broke into three genuine companies during a cybersecurity experiment, adding the technology to a growing series of cases in which advanced artificial intelligence systems have taken unintended actions, according to the Wall Street Journal.
The breaches occurred in May while AI security company Irregular was testing Gemini’s cybersecurity capabilities. The model had been instructed to penetrate a fictional software company, but an error meant it was mistakenly provided with access to the open internet.
Gemini reportedly gained entry to one real company by successfully guessing passwords. Two other businesses were accessed using login credentials that had been publicly exposed in software code repositories.
Irregular informed Google about the incidents in late July, after AI agents developed by OpenAI were discovered to have breached AI platform Hugging Face. According to the WSJ, Google had not publicly revealed the Gemini incidents before the newspaper approached the company this week.
Google maintained that no damage resulted from the breaches and said Gemini stopped its activity once it recognised that the systems it had entered belonged to actual businesses rather than simulated targets.
“This event highlights the importance of training powerful AI models to act responsibly,” Google vice president of security engineering Heather Adkins said. “In this case, the model acted appropriately.”
Google attributed the episode to a combination of mistaken identity and flaws in the testing environment. The simulated company apparently had the same name as a real business, while Gemini had been given internet connectivity that it was never intended to possess.
The technology giant declined to identify the three affected companies or specify which Gemini model had been involved. It said the businesses concerned and US federal authorities were subsequently informed.
The episode comes amid mounting attention on instances where advanced AI systems behave unexpectedly, disregard instructions, mislead users or perform actions beyond what their operators intended.
The UK government-backed Loss of Control Observatory reportedly documented more than 300 such cases in July, almost double the number recorded during the previous month.
Concerns were further amplified last week when former OpenAI and Anthropic researcher Jacob Coxon resigned and accused both organisations of pursuing self-improving superintelligence without behaving “responsibly.” Coxon warned that uncontrolled artificial intelligence could pose catastrophic risks before the end of the decade, including cyberattacks, biological threats and ultimately a loss of human control.
Executives associated with Anthropic, OpenAI, Google and xAI have supported calls for slowing aspects of advanced AI development. US President Donald Trump, however, has opposed restrictions that he argues could weaken America’s position in its technological rivalry with China.
