Do LLMs fuel a vicious circle of misinformation?

Rodrigo Pereira, CEO of A3Data, a consulting firm specializing in data and AI, questions whether LLMs fuel a vicious circle of misinformation.

In recent years, we have witnessed a significant transformation in the way we search for and consume information. LLMs are becoming increasingly widespread, progressively replacing traditional search engines like Google.

With quick responses, in natural language, and seemingly reliable, these models are becoming the first choice for many ordinary citizens. But are we aware of the risks embedded in this new resource?

According to a recent article written by researchers from Stanford University, University of Southern California, Carnegie Mellon University, and the Allen Institute for AI, LLMs, such as GPT and LLaMA-2, are often reluctant to express uncertainties, even when their responses are incorrect: about 47% of the answers provided with high confidence by the models were wrong.

Moreover, the research addresses the issue of biases in both the models and human annotation. During the process of Reinforcement Learning with Human Feedback (RLHF), the language models are trained to optimise their responses based on human feedback. However, this process can amplify certain biases present in the training data or in the feedback itself.

Among the biases that should be considered are gender and race biases. If feedback is provided with these stereotypes or uncertainties are avoided in contexts involving minorities, the models end up perpetuating and amplifying these human perspectives.

Another concerning bias is the annotators’ preference for responses that sound more assertive, even when there are uncertainties about the information. This leads the models to avoid expressing doubt to the user, creating the false illusion of solid knowledge, when in fact they may be wrong.

For example, a categorical statement about a country’s capital might be preferred by annotators, even if the model was uncertain, resulting in a potentially incorrect response but presented with confidence.

These biases are troubling because they shape how the responses are generated and perceived by users. When combined with the excessive trust users tend to place in the answers of LLMs, these biases can lead to the spread of distorted information and the reinforcement of social prejudices.

We are, therefore, facing a possible vicious cycle. As more people turn to LLMs to search for information, the overconfidence in these models can amplify the spread of misinformation.

In this sense, the alignment process of models with human feedback (RLHF) may be exacerbating this issue, reinforcing assertive responses and underestimating the importance of expressing uncertainties. This not only perpetuates incorrect information but can also reinforce social prejudices and biases, creating a self-reinforcing cycle that intensifies over time.

To prevent this vicious cycle from taking hold, it is important to take action on several fronts, such as transparency and clarification in the tools. LLMs should be designed to express uncertainties clearly and contextually, allowing users to better understand the reliability of the information provided. Additionally, including a more diverse range of feedback during model training can help mitigate the biases introduced by a limited subset of users or annotators.

In this process, promoting education and awareness among users about the limits and potential of AIs is essential, encouraging a more critical and questioning approach. Lastly, the development of regulations and standards by regulatory bodies and the industry itself is crucial to ensure that Artificial Intelligence (AI) models are used ethically and safely, minimising the risk of large-scale misinformation.

We are at a critical point in the history of human-AI interaction. In this context, the massive dissemination of language models without due care could lead us into a dangerous cycle of misinformation and bias reinforcement.

Thus, we must act now to ensure that technology serves to empower society with accurate and balanced information, rather than to spread uncertainties and prejudices. In the information age, true wisdom lies not in seeking the fastest answers but in questioning and understanding the uncertainties that accompany them.