Getting things wrong: hallucinations

‘AI hallucination’ is a vague term used to describe an output presented by a tool which seems completely false and not supported by its training data. The analogy is with a human hallucination, where the brain presents something which doesn’t exist. It is probably not a good term to use, because it makes the system seem more human, but as with ‘artificial intelligence’ it has got into common use rather quickly and so you should be aware of it. ‘Something which is false’ is a more accurate description in plain language.

These outputs which are factually incorrect are not very surprising if you think that the software is just picking something which is statistically probable for the next word or number – whilst it should stay in context, for instance only giving you information about Ingmar Bergman if you ask it, but actually giving you something which confuses him with Ingrid Bergman, because they share the context of movies, you can see how a model might make that happen.

There could be serious consequences if the tools generate untrue information about individuals or organisations. According to the New York Times Links to an external site., the Netherlands politician Marietje Schaake was described as a terrorist by one tool (BlenderBot) – it is completely false. The company behind ChatGPT, OpenAI, is being sued Links to an external site. after the tool generated an untrue statement saying a man had been accused of fraud and embezzlement. The outcome of this case may help to determine liabilities and how tools are promoted and used.

Early versions of GenAI tools generated plausible-looking but wrong academic references. If you missed this video Links to an external site. on an earlier page, Lund University librarian Fredrik Eriksson describes students coming to libraries for help tracking down these fictitious references.  As with the challenges about training data, you need to know what is missing in order to identify random untruths in the outputs from GenAI tools. We don’t expect students to have this expertise yet, so we need to help them to find ways to check on the outputs from the tools. On 28 July 2023, a Google spokesman told BBC news Links to an external site. that the company recommends people “Google the information they find” from what was then their own tool, Bard, to make sure that it is accurate. That could be very confusing for people. (Bard was shortly after withdrawn from use and has now been replaced). 

Summary: we need to check the accuracy of outputs from GenAI tools. They are not designed to do the same task as a search engine.