Not Gouda-nough: Google removes AI-generated cheese error from Super Bowl ad

When Google launched its AI Overviews features last year, we noted plenty of examples of false, misleading, and even dangerous information that can be contained in the official-looking answers generated by Google's Gemini model. Now, Google has quietly scrubbed one such falsehood from a demo of its AI writing assistant that featured prominently in an ad planned for Sunday's Super Bowl.

The ad in question is part of Google's "50 stories from 50 states" promotion, which will run Gemini ads tailored for different local markets during the Super Bowl on Sunday. The Wisconsin-focused ad, as it was posted on YouTube last week, featured the owner of Wisconsin Cheese Mart asking Google's writing assistant for "a description of Smoked Gouda that would appeal to cheese lovers."

The AI-authored response that was shown in that video—and still appears verbatim on the Wisconsin Cheese Mart website—notes that Gouda is "one of the most popular cheeses in the world, accounting for 50 to 60 percent of the world's cheese consumption." That is almost surely an exaggeration; a 2007 Cheese Market News editorial, for instance, mentions Gouda as only the third-most-popular cheese in the world, after cheddar and mozzarella. A Global Cheese Market analyst report also only includes Gouda in the "Other Cheese" category, while mozzarella, Parmesan, and cheddar each get their own categories.

The mistaken cheese factoid remained in the YouTube version of the ad until Tuesday, when social media aggregator Goog Enough (devoted to "collect[ing] and amplify[ing] bad Google SERPs and AI Overviews") noticed that the offending statistic had been removed without note. The new version of the ad simply shows Gemini calling Gouda "one of the most popular cheeses in the world" without going into specific numbers.

Oddly enough, the edited version of the ad resides at the same URL as the previous version, with no indication it has been updated since being uploaded on January 30. This kind of wholesale replacement of a YouTube video at the same URL is impossible for normal YouTube users, suggesting some special privileges for Google itself were used here.

Ars Video

The incorrect cheese info also appears to have been scrubbed from the pre-roll of Google's recent earnings call, which initially featured a run-through of the planned Super Bowl ads until yesterday.

Blame cheese.com

While it's easy to accuse Google Gemini of just making up plausible-sounding cheese facts from whole cloth, this seems more like a case of garbage-in, garbage-out. Google President of Cloud Applications Jerry Dischler posted on social media to note that the incorrect Gouda fact was "not a hallucination," because all of Gemini's data is "grounded in the Web... in this case, multiple sites across the web include the 50-60% stat."

The specific Gouda numbers Gemini used can be most easily traced to cheese.com, a heavily SEO-focused subsidiary of news aggregator WorldNews Inc. Cheese.com doesn't cite a source for the percentages featured prominently on its Smoked Gouda page, but that page also confidently asserts that the cheese is pronounced "How-da," a fact that only seems true in the Netherlands itself.

The offending cheese.com passage that is not cited when using Google's AI writing assistant. Credit: cheese.com

Regardless, Google can at least point to cheese.com as a plausibly reliable source that misled its AI in a way that might also stymie web searchers. And Dischler added on social media that users "can always check the results and references" that Gemini provides.

The only problem with that defense is that the Google writing assistant shown off in the ad doesn't seem to provide any such sources for a user to check. Unlike Google search's AI Overviews—which does refer to a cheese.com link when responding about gouda consumption—the writing assistant doesn't provide any backup for its numbers here.

The Gemini writing assistant does note in small print that its results are "a creative writing aid, and not intended to be factual." If you click for more information about that warning, Google warns that "the suggestions from Help me write can be inaccurate or offensive since it’s still in an experimental status."

This "experimental" status hasn't stopped Google from heavily selling its AI writing assistant as a godsend for business owners in its planned Super Bowl ads, though. Nor is this major caveat included in the ads themselves. Yet it's the kind of thing users should have at the front of their minds when using AI assistants for anything with even a hint of factual info.

Now if you'll excuse me, I'm going to go update my personal webpage with information about my selection as World's Most Intelligent Astronaut/Underwear Model, in hopes that Google's AI will repeat the "fact" to anyone who asks.

Staff Picks

Tofystedeth

The specific Gouda numbers Gemini used can be most easily traced to cheese.com, a heavily SEO-focused subsidiary of news aggregator WorldNews Inc

Ah, so the source of the problem is one of the other things that's been ruining the web for the last decade.

February 5, 2025 at 9:23 pm

farmfromjakestate

I must be confused somehow. In August of 2020, the description of Gouda on wisconsincheesemart.com matches exactly the text produced by Gemini in the ad, including the 50-60 percent error https://web.archive.org/web/2020080...incheesemart.com/products/gouda-cheese-smoked. Bard was launched in 2023. Was this a description ever produced by a large language model?

February 5, 2025 at 9:46 pm

Ars Video

Blame cheese.com

nproxy.org