Hackers Just Found a Wild Way to Trick Google Gemini Into Phishing You

A flaw has been detected in Google Gemini, a flaw that allows attackers to hijack email summaries for phishing purposes. This flaw has been spotted in Google Gemini for Workspace.

Gemini flaw allows attackers to phish users via Google’s AI

So, what exactly is going on here? Google Gemini can be exploited to generate email summaries that appear legitimate, but they’re not. They include malicious instructions or warnings that direct users to phishing sites.

Some of you may recall that similar attacks were reported last year. Safeguards have been put into place by Google, but it seems like the same technique still works.

This was disclosed through Odin, Mozilla’s bug bounty program for generative AI tools. A researcher by the name Marco Figueroa has disclosed it, who is GenAI Bug Bounty Programs Manager at Mozilla.

How does it work? Well, the attacker creates an email with an invisible directive for Gemini. The attacker can hide the malicious instruction in the body text at the end of the message. He can do that by using HTML and CSS that sets the font size to zero, and also its color to white.

That instruction will not be rendered in Gmail due to that fact. Considering that there are no attachments or links present, the changes are quite high that the message will reach the potential target’s inbox, and not end in spam or get flat-out blocked.

If the person who received the email asks Gemini to generate a summary of the email, Google’s AI tools will parse the invisible directive and obey it.

Google says it has seen no evidence of Gemini being manipulated in such a way

BleepingComputer reached out to Google for a comment, and a spokesperson pointed in the way of a Google blog post on security measures against prompt injection attacks. He also said the following: “We are constantly hardening our already robust defenses through red-teaming exercises that train our models to defend against these types of adversarial attacks.”

He also said that Google has seen no evidence of incidents manipulating Gemini in the way Figueroa demonstrated.