But there really are errors in the operation of AI Overviews
Google has always lied? Leaked internal company documents show how the search engine really works
Google recently added an artificial intelligence-based AI Overviews feature to its search engine, and users have begun to notice sometimes strange responses. Some of them were attributed to AI hallucinations, but Google stated that it was not them and tried to explain the situation.
More precisely, she explained it a few days ago, and now she decided to explain in more detail how AI Overviews works, and why the strange results are not AI hallucinations.
Over the past week, people on social media have been sharing strange and erroneous reviews (as well as a lot of fake screenshots). We know that people trust Google Search to provide accurate information, and they've never been shy about pointing out oddities or errors when they encounter them — in our ratings or other search functions. We hold ourselves to high standards, as do our users, so we expect, value and take feedback seriously. Given the attention the AI Overviews feature has received, we wanted to explain what happened and what steps we took.
AI Overviews works very differently from chatbots and other LLM products people may have tried. They don’t simply generate output based on training data. While AI Overviews’ responses are based on a custom language model, that model is integrated with our core web ranking systems and is designed to perform traditional “search” tasks, such as identifying relevant, high-quality results from our index. That’s why AI Overviews doesn’t just provide text output, but also includes relevant links so people can explore further. Because accuracy is paramount in search, AI Overviews is built to only display information that is backed by the best web results.
This means that AI Overviews generally doesn’t “hallucinate” and doesn't make things up the way other LLM products might. When AI Overviews gets it wrong, it's usually for other reasons: misinterpreting queries, misreading the nuances of online language, or missing a lot of useful information. (These problems arise with other search features, too.)
This approach is very effective. Overall, our tests show that AI Overviews' accuracy level is on par with another popular search feature, featured snippets, which also uses AI systems to identify and display key information with links to web content.
Google also notes that there have been many fake examples of AI Overviews malfunctioning, complete with fake screenshots.