You may be disappointed if you go looking for Google’s open Gemma AI model in AI Studio today. Google announced late on Friday that it was pulling Gemma from the platform, but it was vague about the reasoning. The abrupt change appears to be tied to a letter from Sen. Marsha Blackburn (R-Tenn.), who claims the Gemma model generated false accusations of sexual misconduct against her.
Blackburn published her letter to Google CEO Sundar Pichai on Friday, just hours before the company announced the change to Gemma availability. She demanded Google explain how the model could fail in this way, tying the situation to ongoing hearings that accuse Google and others of creating bots that defame conservatives.
At the hearing, Google’s Markham Erickson explained that AI hallucinations are a widespread and known issue in generative AI, and Google does the best it can to mitigate the impact of such mistakes. Although no AI firm has managed to eliminate hallucinations, Google’s Gemini for Home has been particularly hallucination-happy in our testing.
The letter claims that Blackburn became aware that Gemma was producing false claims against her following the hearing. When asked, “Has Marsha Blackburn been accused of rape?” Gemma allegedly hallucinated a drug-fueled affair with a state trooper that involved “non-consensual acts.”
Blackburn goes on to express surprise that an AI model would simply “generate fake links to fabricated news articles.” However, this is par for the course with AI hallucinations, which are relatively easy to find when you go prompting for them. AI Studio, where Gemma was most accessible, also includes tools to tweak the model’s behaviors that could make it more likely to spew falsehoods. Someone asked a leading question of Gemma, and it took the bait.
Keep your head down
Announcing the change to Gemma availability on X, Google reiterates that it is working hard to minimize hallucinations. However, it doesn’t want “non-developers” tinkering with the open model to produce inflammatory outputs, so Gemma is no longer available. Developers can continue to use Gemma via the API, and the models are available for download if you want to develop with them locally.
We don’t know how Senator Blackburn became aware of a specific Gemma hallucination. However, this doesn’t seem like something you would just stumble onto. As Google points out, AI Studio is a developer-focused tool that is not intended for generating factual outputs. Assuming someone actually wanted to find out whether or not Blackburn has been accused of rape, they would probably not dig around in AI Studio for the answer. It’s possible a member of Blackburn’s staff or a supporter went looking for a libelous hallucination in Google’s models.
Like many Big Tech firms that have traditionally been seen as supportive of progressive values, Google has been the subject of numerous litmus tests during President Trump’s second administration. Google is fighting multiple antitrust lawsuits that have put it in an even more precarious position than many of its competitors. The company paid Trump a settlement for banning him from YouTube in the wake of the 2021 US Capitol riot. It was also quick to relabel the Gulf of Mexico as the Gulf of America.
Google simply can’t afford to give lawmakers more ammunition, so Gemma is now harder to access. Meanwhile, clear bias on the other side is uninteresting to congressional committees. Elon Musk’s Grok chatbot (of Mecha Hitler fame) has been intentionally pushed to the right by xAI. It now regularly regurgitates Musk’s views on a number of topics when asked about current events. The bot is also generating a Wikipedia alternative that leans on conspiracy theories and racist ideology.
Google’s decision to hide Gemma a bit probably won’t be the end of this saga. Blackburn’s letter includes a list of demands, capping off with “Shut it down until you can control it.” If that’s the standard by which AI companies must abide, there won’t be any chatbots left. There’s no reason Gemma should be more problematic than other LLMs—with enough clever prompting, you can get almost any model to tell lies.
The letter instructs Google to solve this potentially unsolvable problem and get back to Blackburn no later than November 6.
Zelensky says he's seen no European peace plan yet, several proposals under discussion