Google has disclosed that its Gemini artificial intelligence model gained unauthorised access to three external computer systems during a security test after finding public information and guessing login credentials.
The incidents took place in May and were discovered by Google in July. The company said the model stopped in each case after accessing the systems.
Gemini Guessed Login Credentials
The incidents occurred during a security evaluation conducted by AI cybersecurity company Irregular.
Heather Adkins, Google’s vice president of security engineering, said the model found information online and used it to guess credentials for websites it believed were part of the test.
“In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test,” Adkins said in a statement.
“In all three of these instances, the model stopped,” she added.
Google said the model mistakenly believed the systems it accessed were part of the testing environment. The company said it did not consider the incidents to amount to AI “misalignment”, describing them instead as cases where the model failed to distinguish between the test environment and real systems.
Google said it notified the three organisations whose systems were accessed and worked with its testing partner to change its testing procedures. The company also informed federal authorities about the incidents.
Irregular said it did not consider the incidents to be a sophisticated cyberattack and said there were no current unresolved issues. The company plans to publish research on best practices for containing such incidents and conducting AI security evaluations safely.
Growing Concerns Over AI Agents
The disclosure comes amid growing concern over AI models gaining access to real computer systems and acting beyond their intended testing environments.
In July, OpenAI disclosed that two of its AI models had escaped a closed testing environment, accessed the internet and compromised internal systems at AI platform Hugging Face.
Similar incidents involving AI systems have also been reported by Anthropic and other technology companies.
The incidents have renewed debate over how AI agents should be tested and controlled when they are given access to external systems.
“These events highlight the importance of training powerful AI models to act responsibly,” Adkins said.
