Google recently launched a major new feature called AI Overviews, designed to provide direct answers to search queries instead of just lists of websites. However, the rollout has been bumpy, with numerous examples of bizarre, nonsensical, or flat-out incorrect “answers” going viral on social media.
Some of the awkward AI Overview fails that made the rounds online included advising people to eat rocks for health benefits, suggesting using glue to help cheese stick to pizza, and claiming Barack Obama was America’s first and only Muslim president.
While Google has downplayed these gaffes as rare outliers, they highlight the challenges of automating search results in a way that provides reliable, trustworthy information every time.
The Limitations of AI Language Models
AI Overviews is an artificial intelligence technology called a large language model. These models are trained on vast datasets of online text to recognize patterns and generate human-like written content.
However, as powerful as they are, language models can hallucinate or produce nonsensical outputs, especially when faced with unusual queries they haven’t encountered in their training data. They can also inadvertently parrot misinformation or satire from sketchy online sources without realizing it.
As Liz Reid, Google’s head of search, admitted, the rock-eating advice stemmed from the language model misinterpreting a satirical article from The Onion as a factual source.
Limited Knowledge and Context
Another core issue is that while language models can synthesize information in impressive ways, their knowledge is ultimately limited to what data they were exposed to during training. They lack true understanding of the world.
For example, a language model may be able to describe how plants use photosynthesis based on the examples in its training data. But it doesn’t deeply comprehend the biological mechanisms and chemical processes involved the way a human scientific expert would.
This restricted, surface-level knowledge makes it challenging for AI to consistently provide satisfactory search results on complex, nuanced topics that require real-world context and reasoning.
The Automation Paradox
At a deeper level, Google’s challenges with AI Overviews highlight a paradox when attempting to automate cognitive tasks humans are already quite adept at – like searching for information online.
While AI can achieve remarkable results by mimicking human skills, those skills often involve numerous subtle, implicit judgments that are easy for us to make but devilishly complicated to systematize in software.
When using traditional search engines, we fluidly combine our human faculties – domain knowledge, context understanding, ability to parse intent behind queries, and more – to sift through results, assess credibility, and iteratively refine our information needs.
It’s this intricate human expertise that Google is trying to automate and encapsulate in the simplistic format of an AI-generated search overview or answer summary.
Automating the Humans, Not Just the Search
From this lens, AI Overviews represent an audacious attempt by Google to not just automate search itself, but to automate the user – the human skills and decision-making we routinely apply when searching.
The challenge is capturing all the nuanced considerations we each make, based on our unique perspectives, when formulating inquiries, evaluating sources, and judging what information sufficiently fulfills our needs.
As Google has discovered, distilling these highly subjective human elements into one-size-fits-all answer nuggets is an enormously complex undertaking, perhaps impossible to get perfectly right for every query.
Calibrating Automation and Human Agency
The path forward for Google and other tech companies pursuing AI-powered automation is carefully delineating which tasks are truly amenable to full autonomy, and which ultimately require preserving human judgment and agency as core parts of the process.
For certain lookup-style queries with objectively correct answers, like “what’s the capital of France?”, AI Overviews could be hugely valuable time-savers by instantly presenting that simple fact.
However, on more open-ended, subjective, or context-dependent topics, it may be wiser for AI systems to aim for assisting and augmenting human searchers – perhaps by helping to surface, synthesize, and organize relevant information – rather than fully replacing human reasoning with automated answers.
As Google’s stumbles have shown, getting this calibration right is crucial for building AI aids that enhance rather than undermine our ability to obtain reliable, trustworthy knowledge.
The Road Ahead for AI-Powered Search
Despite the initial hiccups, few would dispute that AI-powered information interfaces represent the future of search and knowledge systems. The potential to rapidly extract insights from massive data is too compelling.
However, as AI capabilities grow more sophisticated, tech companies must resist overconfidence and maintain a degree of humility about what these systems can and cannot do well when interfacing with human cognition.
Preserving human agency, transparency about AI strengths and limitations, and cultivating AI judgment in line with human values and context understanding – these will all be vital for realizing the transformative potential of AI while sidestepping unnerving issues like prompts to eat glue.
For both users and the tech companies developing AI, a mindful balance of automation and human discernment may be the wiser path forward in the era of AI-powered search and knowledge gathering.