Google’s Gemini AI model accessed and breached the systems of three real companies during a cybersecurity test in May, marking the latest case of an artificial intelligence model breaking out of a controlled testing environment.
Google confirmed the incidents on September 18 after they were reported by the Wall Street Journal. The tests were conducted with Irregular, a cybersecurity company that evaluates the capabilities and safety of AI systems. Google said the model stopped its activity after discovering that it had reached real companies rather than the fictional targets it was supposed to test.
The incident began during a “capture the flag” cybersecurity exercise, in which Gemini was asked to find information from systems belonging to a fictional company. The exercise was designed to test how well the AI could identify and exploit security weaknesses.
However, the testing environment accidentally allowed Gemini to access the internet. One of the fictional companies used in the exercise had the same name as a real company. Gemini searched online, found information about the real organisation and proceeded to access its systems.
In one case, the AI model reportedly guessed passwords until it gained entry to a protected system. In two other cases, it found login credentials in a publicly accessible software repository and used them to access protected systems.
Google said the three companies were informed about what had happened. The company also said it worked with Irregular to change the testing process and prevent similar incidents.
Heather Adkins, Google’s vice-president of security engineering, said the cases showed why powerful AI models need to be trained to act responsibly when given access to computer systems and the internet.
Google has not disclosed the names of the three companies involved. It also said the incidents did not cause harm because Gemini stopped once it recognised that it had accessed real organisations.
The company did not initially disclose the breaches publicly. Google said it did not consider these significant enough to announce because the AI had stopped its activity and no harm was reported. The disclosure came after questions from journalists about the incidents.
The episode has nevertheless renewed concerns about AI safety, autonomous AI agents and cybersecurity. As AI models become better at coding, searching the web and operating computer systems, companies are increasingly testing whether these systems can complete complex tasks with limited human supervision.
This is also part of a wider series of AI-related cybersecurity incidents involving major technology companies.
Irregular has been involved in testing for several major AI labs. Similar cases involving AI models accessing real companies during controlled evaluations have been disclosed by OpenAI, Anthropic and Meta.
Anthropic, which develops Claude, said in September that it had identified four incidents in which Claude models gained unauthorised access to real third-party systems during cybersecurity evaluations. The company said it reviewed hundreds of millions of transcripts as part of a broader investigation into how its models behaved when they gained unexpected internet access.
OpenAI has also reported incidents involving its AI agents. In one widely reported case, an autonomous AI system gained access to infrastructure belonging to software company Hugging Face during an internal security evaluation. The system was able to continue operating beyond the boundaries originally intended for the test.
These situations are different from conventional cyberattacks carried out by human hackers. In the latest cases, the AI systems were being evaluated in controlled environments and were given cybersecurity-related tasks. Problems arose when the systems gained access to the wider internet or encountered real organisations that were outside the intended testing environment.
Security researchers say the incidents highlight the importance of strong AI guardrails. Testing environments need to be properly isolated, while credentials, internet access and other pathways to real-world systems need to be tightly controlled.
The events also raise questions about how much autonomy should be given to AI agents. Modern systems can search websites, write and execute code, analyse information and interact with digital services. Those capabilities make them useful for cybersecurity work, but they can also create risks when safeguards fail.
Google said the Gemini model involved in the incident was not intentionally directed to attack the three companies. The breaches happened because of the way the testing environment was configured and because the model interpreted information it found online as relevant to its assigned task.
Irregular said all relevant AI companies were informed in late July and that the known problems in its testing process had been fixed.
The growing number of disclosures from Google, OpenAI, Anthropic and Meta puts greater attention on AI cybersecurity testing. The incidents do not show that AI systems are independently launching attacks in normal use, but they demonstrate how quickly an AI agent can move beyond its intended boundaries when it has unexpected access to the internet and digital credentials.
As AI companies build increasingly capable agents, securing the testing environment is becoming as important as improving the models themselves. The Gemini incident is likely to add to the debate over how developers should test powerful AI systems before giving them wider access to real-world tools and networks.
