What Is an LLM Hallucination, and Why Does It Occur?
An LLM hallucination occurs when the model “perceives patterns or objects that are nonexistent or imperceptible to human observers, creating outputs that are nonsensical or altogether inaccurate,” according to an IBM blog post.
Generative AI LLMs predict patterns and generate outputs based on vast amounts of training data, notes Huzaifa Sidhpurwala, senior principal product security engineer at Red Hat, who has written about hallucinations. LLMs, he says, “excel at mimicking human-like communication and producing contextually relevant responses,” but hallucinations are a “critical limitation” of the technology.
“Distinguishing between verified information and hallucinations can be challenging for users because generative AI often produces responses that sound plausible and well-constructed,” Sidhpurwala says. “Understanding this limitation is crucial, especially when using AI for decision-making, research or other applications where accuracy is paramount.”
LLM hallucinations occur because of the fundamental design of the models and the data that they rely on, Sidhpurwala says.
“Their primary training objective is to predict the next word or token in a sequence, regardless of whether the output aligns with reality or the question’s context,” he says. “This means that even when the model doesn’t have reliable knowledge on a topic, or an incorrect data set, it may confidently fabricate details to maintain the flow of communication.”
RELATED: Some companies are building artificial intelligence centers of excellence.
How Are Businesses Impacted by LLM Hallucinations?
The business impacts of LLM hallucinations can be significant, Sidhpurwala says, especially when inaccuracies put an organization at risk.
“When an AI hallucination produces a plausible but false statement, the reputation of the organization utilizing the LLM can suffer, potentially leading to market-share losses,” he says.
Hallucinations can also create legal risk for enterprises, especially in industries with strict regulatory requirements, such as finance and healthcare, he notes. In such cases, an AI hallucination “could result in noncompliance and legal penalties.”
Enterprises should also be aware of the potential impacts of AI hallucinations on finances and productivity, according to Sidhpurwala.
“When critical business decisions are made based on output from flawed AI models without human oversight, they can have detrimental financial repercussions,” he says.
LLM hallucinations can also produce code with errors and omissions. “If these mistakes are caught, developers must spend time reviewing and correcting the LLM’s output, which can be more laborious than producing the code themselves,” Sidhpurwala says. “Ultimately, these inaccuracies can negate any initial cost or time savings and lead to higher operational costs.”
DIG DEEPER: Train your artificial intelligence bot with chain-of-thought prompting.
What Are Some Examples of LLM Hallucinations?
Numerous AI hallucinations have gone mainstream in recent years, including a 2023 incident in which a judge sanctioned two attorneys who used ChatGPT to write a legal brief that cited six nonexistent legal cases.
In another instance, Air Canada was ordered to compensate a passenger who received incorrect information about refund policies from the airline’s LLM-powered chatbot.
“The chatbot provided inaccurate details, leading to customer dissatisfaction and a legal ruling against the airline,” Sidhpurwala says. “This case demonstrates the potential legal and reputational consequences of relying on LLMs for customer-facing applications without adequate oversight.”
EXPLORE: What is asymmetric information, and how does it affect your IT?
Best Practices to Prevent LLM Hallucinations
Any company using an LLM to power services should ensure that the model was built using the key pillars of responsible AI, including transparency, explainability, inclusivity and sustainability, according to Sidhpurwala.
Additionally, companies can turn to small language models, or SLMs, which “are trained with domain-specific information, rather than the larger bucket of information, correct or otherwise, that is the internet,” he says.
IT leaders may also want to use retrieval-augmented generation, or RAG, to ground models and reduce hallucinations. RAG essentially allows models to train on a company’s verified business information to generate more accurate and up-to-date outputs for customers.
“Above all else, using trustworthy data with a strong provenance is vital,” Sidhpurwala says.
