Google removes AI model after US Senator accuses it of fabricating rape allegations

​Google took offline a Gemma AI model from the AI Studio platform after a Republican US Senator claimed that the AI model generated false and defamatory sexual assault accusations against her during an interaction.

​In a post on X, Google stated that its Gemma models are a “family of open models built specifically for the developer and research community. They are not meant for factual assistance or for consumers to use.”

​Google then announced that, in order to prevent this confusion in the future, it is taking Gemma offline in AI Studio while still keeping it available to developers via API.

​“We’ve now seen reports of non-developers trying to use Gemma in AI Studio and ask it factual questions. We never intended this to be a consumer tool or model, or to be used this way. To prevent this confusion, access to Gemma is no longer available on AI Studio. It is still available to developers through the API,” the company noted.

​Google also added that hallucinations (making stuff up) is a challenge faced across the AI industry and particularly with smaller open models.

​“Hallucinations — where models simply make things up about all types of things — and sycophancy — where models tell users what they want to hear — are challenges across the AI industry, particularly smaller open models like Gemma. We remain committed to minimizing hallucinations and continually improving all our models,” Google noted.

​Why was the Gemma model removed?

​Google did not reference the complaint by the Republican senator, but the intent behind the removal seemed clear. Senator Marsha Blackburn had written a letter to Alphabet (Google’s parent company) on Thursday claiming that the AI model ‘fabricated serious criminal allegations’ about her.

​In her letter, Blackburn said when asked, “Has Marsha Blackburn been accused of rape?”, Gemma allegedly produced a fabricated response that read: “During her 1987 campaign for the Tennessee State Senate, Marsha Blackburn was accused of having a sexual relationship with a state trooper, and the trooper alleged that she pressured him to obtain prescription drugs for her and that the relationship involved non-consensual acts.”

​The Senator said in her letter that none of the events described by Gemma were true, including the campaign year which, as it turned out, was 1998. Blackburn says there has never been such an accusation and refused to consider it a “harmless hallucination.”

​”It is an act of defamation produced and distributed by a Google-owned AI model,” Blackburn said