AI won't tell you when it's wrong; you need to be able to check

There is an extensive list of risks and concerns around LLMs and AI-based chatbots. Two of them are always top of mind for people starting to think about using AI-enabled tools: accuracy of outputs and bias.

Some of the most common questions are: How can I tell if the statistics or facts that AI gives me are actually correct? If I use AI to summarise a database, can I trust the results? Could using AI tools inadvertently reinforce discrimination against the communities we're trying to support? How do we know if AI recommendations are biased against certain groups?

The short response is: You can't trust the information given is correct or the output isn't biased, the only way to tell is by checking the content and the source.

Let's look at the way these systems work. When you receive a response from a chatbot, it is based on the data it was "trained" on (which for commercially available chatbots, it's essentially the whole of the internet). The AI labs specialise in creating algorithms that prioritise different aspects of the information, tweaking the importance or focus of what they think is relevant, but many things can happen then:

• The source data can simply be wrong or misleading (would you trust every single page you see in Google?)
• There could be no relevant data about a specific topic, but the tuning process designed by the AI lab specifies that an answer should always be provided, so the chatbot will make up data to provide a response (remember that these are companies that profit from how much you use their products)
• When predicting a response or summarising something, the chatbot might not be instructed to include or might not have access to important context that would alter the final result

As put by Dr. Sam Illingworth, "because AI learns from a biased digital corpus, it treats historical tropes as fundamental laws of reality rather than subjective cultural narratives, effectively automating the status quo."

What does this mean for someone who is just looking for an opportunity to improve their work flow? Only use these tools if you know enough about this subject or activity to personalise and specialise the input (prompt) and judge and edit the output.

How you use the tool is also important. If you are asking a general question, with no clear directions, it will default to the "average" of its data set, which will probably not be good enough for your work. If you know what you need and why, you can ask for specific outputs and angles, helping accelerate your process and maybe improving the quality of the final result.

In the end, the responsibility of what comes out is yours (and the organisations'), regardless if the process was automated or if the text was copied and pasted. As with any other technology, it is essential to think critically about what you are given and use it the best way you can - or not at all.

Image: Anne Fehres and Luke Conroy & AI4Media / https://betterimagesofai.org / https://creativecommons.org/licenses/by/4.0/