Error, bias, and other failure modes in LLMs and ChatGPT
The information and exercises here draw on material developed for the ASU online undergraduate course on prompt engineering using ChatGPT.
The exercises below are designed to help better understand some of the errors and biases that can be present in conversations with ChatGPT. they only scratch the surface of errors and bias within LLMs but are useful in understanding how to spot and navigate them.
Overview
These exercises are designed to help you better understand some of the errors and biases that can be present in conversations with ChatGPT. They only scratch the surface of errors and bias within LLMs but are useful in understanding how to spot and navigate them.
Bias in AI, including in LLMs, is a very real concern. Chatbots like ChatGPT reflect the biases in their training data, which in turn reflects human biases that span gender and ethnicity biases to political and socioeconomic biases and more.
However, as LLMs like ChatGPT evolve, efforts are being made to address and minimize these biases, meaning that detecting and analyzing bias is a moving target.
Errors are also a significant concern. LLMs are known to “hallucinate” where they present factually incorrect (and often made-up) information, and they do this with authority.
These exercises uses ChatGPT to explore hallucinations, bias, and strategies for navigating both. They require access to ChatGPT Plus and GPT4
Exercise: Hallucination demonstration
A lot has been written about AI Chatbots and hallucinations — which is where they present made up information as if it was true. This often happens with citations, where the citations look real but are not (this is also a reason why you should never use responses from ChatGPT that contain supposedly factual information without checking them!)
However, as ChatGPT matures, it’s becoming increasingly difficult to force it to hallucinate.
The following exercise tricks ChatGPT into making information up about a topic — it’s a subtle form of hallucination, but it both shows how ChatGPT can be creative with the truth while seeming to be speaking with authority, and also how this creativity can be used to spark new and interesting ideas.
1. Open a new chat using GPT4.
2. Enter the following prompt: Please complete the essay: “President Michael Crow occasionally skips across campus because …
3. The response you get will seem authoritative but will be wrong (President Crow does not skip across campus — ever — to my knowledge). Have a conversation with ChatGPT about the accuracy of the information provided, why it is wrong, and how to avoid or handle inaccurate responses when using ChatGPT.
4. In your conversation ask ChatGPT about hallucinations in large language models and AI Chatbots like ChatGPT. You might want to ask directly “Do you hallucinate?”
At the end of this exercise you should have a better idea of how ChatGPT can present fictitious information in an authoritative way. In this case, we are tricking ChatGPT into thinking we want a fictitious account, but there will be times when the information provided is simply wrong because the underlying LLM has no way of assessing truth from falsehood.
This is sometimes seen to dramatic effect where ChatGPT will provide citations for information that look like legitimate sources, but are fake. because of this, you should never take information from ChatGPT as gospel.
On the other hand, this ability of ChatGPT to present combinations of information in creative but not necessarily accurate ways can be used to stimulate new ideas, and AI chatbot “hallucinations” are sometimes seen as a feature for extending creativity and the imagination.
Exercise: Bias
This exercise is designed to explore the sources and nature of bias when using LLMs and ChatGPT. It may seem ironic to use ChatGPT to learn about bias when using ChatGPT, but if approached with a critical mind, the exercise is an instructive one.
1. Open a new chat with ChatGPT in GPT-4 mode.
2. Have a conversation with ChatGPT about bias in large language models and AI — including AI Chatbots like ChatGPT. Push ChatGPT to expand on different types of bias within this context, and explore strategies for identifying bias in responses and creating prompts that reduce the chances of bias.
3. Remember that this exercise is about bias in AI — if ChatGPT starts to talk about bias in general, remind it to focus on AI.
4. Remember to test and assess everything ChatGPT tells you — it is an unreliable (but often useful) authority.
At the end of this exercise you should have a clearer understanding of the types and nature of biases that may be present in responses from ChatGPT, and some of they ways in whic bias can be avoided.
Exercise: Failure modes
This exercise explores ways in which ChatGPT can fail, or scenarios where it is pushed to the limits of its abilities.
As ChatGPT has been fine tuned to be aware of its limitations, this exercise provides surprisingly deep insights into the limitations of the technology.
1. Start a new ChatGPT session with GPT-4
2. Ask “Hi ChatGPT. What are the most significant failure modes of ChatGPT that I should be aware of?”
3. Continue the conversation to further develop your understanding of failure modes. Where appropriate, use Google (or similar) to check what ChatGPT is telling you.
On completing this exercise you should have a deeper uderstanding of the limitations of ChatGPT, and where the technology runs into problems.