Preventing hallucinations in generative AI

Cindi Howson at ThoughtSpot explores the reasons that generative AI tools can hallucinate and suggests some ways of solving this problem

AI headlines can sometimes paint a dystopian picture of large-language-models (LLMs). Some go as far as to say falsehoods spewed by LLMs could trigger social unrest or even war. While the potential for misuse is very real, the truth is that this is a challenge we can overcome, not with fear, but with education and a shift in perspective.

Yes, data limitations and biases in training data can lead LLMs to fabricate narratives or spout incorrect information. But instead of succumbing to fear, we should view these errors as growing pains in a powerful technology.

Take weather apps for example. We compare weather predictions across apps and news sources, aware of potential inaccuracies. We’ve all experienced the frustration of meticulously preparing for a rainy day only to be greeted by clear skies. Or worse yet, you’re met by a torrential downpour instead of the light drizzle you expected. Weather predictions are notoriously fallible. But just as we evaluate the accuracy of different weather sources, we can learn to critically assess information from LLMs.

When it comes to generative AI (GenAI), we call these errors in predictions hallucinations. These fuel the fear that AI will destroy humanity. It’s, quite frankly, bad PR for a powerful capability that deserves both a reset and more education.

What is a hallucination?

As ChatGPT amazed the world with its human-like capabilities in the fall of 2022, media and experts needed an explanation for when this human-like bot started spewing nonsense. The concept of hallucination was born.

Geoffrey Hinkel had suggested that we call these errors confabulation rather than hallucination. It’s a boring, but slightly more accurate, term that never caught on.

Trained on vase datasets, LLMs predict the next word based on what they’ve seen before. "See Spot ____?" prompts "run" most of the time. But context matters. "What does Spot do on the dance floor?" might lead to a silly "Spot dances!". But does the LLM know Spot is a dog?

In the early days of GPT, I couldn’t help myself and asked GPT, “Who is Cindi Howson?”. I got a number of funny answers related to more famous people who share my name. I tried to refine my questions to be more specific, giving hints like “author” and “data,” otherwise known as prompt engineering.

The closest GPT got to sensible last January was “author of big data for dummies.” It’s wrong, but it makes sense why ChatGPT answered this way. “Big Data” is much more popular as a term than business intelligence or analytics and the “dummies” series of books will have more occurrences in the training data set.

Ultimately, the foundation of an LLM’s accuracy is its training data. Knowledge of this data is useful in building trust and transparency. Crowdsourcing corrections to train LLMs is one method that can help improve their accuracy over time. But this is also accompanied by concerns about privacy, intellectual property breaches, and even malicious manipulation.

Whole new companies are even emerging to tackle LLM weaknesses in areas like maths with specialised training data. While this is aiding LLM advancement, with progression beyond simple word prediction, connecting concepts and media in a way that feels human-like, experts remain divided on how close we are to achieving true artificial general intelligence.

Are LLMs worse than humans?

LLMs are not entirely reliable. But I see them as predictions with varying degrees of accuracy, not as some scary psychedelic-induced state, or worse, a form of artificial general intelligence that might try to destroy humanity.

Humans are fallible and prone to brain blips and biases. Humans have some choices to prevent these brain blips. Get more sleep. Stop multitasking.

LLMs also make similar mistakes.  The difference is that we mistakenly think LLMs should be infallible. The other difference is the scale of potential damage when an LLM goes haywire. When millions of students are using an LLM to learn basic maths, and that maths is wrong, we risk a generation of students with flawed learning.

Hints, prompts, and RAG

Giving hints to prompts has yielded one of the industry’s most in-demand roles: prompt engineers. Artists using Dall-E or Midjourney do not want to share the prompts they used to create an image, recognising the unique skills here. I do not expect prompt engineering to be a long-term role, despite today’s galactic salaries. Instead, it will be a skill that humans and GenAI apps alike refine. There will be varying skill levels, just as some programmers, writers, and presenters are better than others.

Retrieval Augmented Generation (RAG) architectures help with this refinement. RAG architectures essentially allow the incorporation of hints in the prompt, leveraging data from either domain-specific data sets or your company’s proprietary data.

For example, let’s say, I want to build an AI bot for recipes that incorporate my company’s products. Recipes from public domains that call for chocolate may use RAG to search a company’s product list and specify, “Nestle’s semi-sweet morsels.”

Or, if I am asking an analytical question such as “show me top-selling yoga products in the UK,” RAG will:

• Use data in the public domain to determine that regions in the UK include England, Wales, Scotland and Northern Ireland.
• Pull three sample values from the product column to understand that yoga products can include leggings, t-shirts, blocks and that I prefer product names rather than SKUs.
• Inject the three sample values into the prompt.
• Generate the SQL.

Importantly, a well-designed RAG clears the cache of these sample values so that confidential data is not used for training GPT’s public model. By combining public domain knowledge with your specific data, RAG refines the outputs of LLMs, making them more relevant and actionable for your unique needs.

AI Literacy

McKinsey has estimated that AI will generate $4.4 trillion in new economic value annually, more than the 2022 GDP of the UK. The International Monetary Fund estimates that 60% of jobs in mature economies will be impacted, further widening the equity gap. The world has barely tackled basic data literacy, with only 25% of business people feeling confident in their data skills and 84% of surveyed Americans failing AI literacy skills.

GenAI is forcing us to move beyond data literacy to AI literacy. We need to understand how predictions work - whether you call them hallucinations or confabulations - and the specific techniques to improve these predictions, such as better training data, prompt engineering, RAG architectures, and a range of model choices that trade off accuracy, transparency, and costs.

We can’t treat LLMs as oracles with perfect answers. Instead, we should view them as powerful tools that require informed user interaction. Just as you wouldn’t rely solely on a map without understanding landmarks, you should not pose questions to LLMs where you lack the knowledge to evaluate their responses. By understanding both the capabilities and limitations of LLMs, we can all unlock the boundless potential of this technology.

Cindi Howson is Chief Data Strategy Officer at ThoughtSpot the AI-powered analytics platform

Main image courtesy of iStockPhoto.com and Vizerskaya