Understanding the Impact of AI on Human Thought and Behavior
As AI models get better at having natural conversations, we need to examine how these interactions impact people and society.
Building on broad scientific research, today we are publishing new findings on the misuse potential of AI harmful manipulation* particularly its ability to alter human thought and behavior in negative and deceptive ways. With this latest study, we have created the first empirically validated toolkit for measuring this type of AI manipulation in the real world, which we hope will help protect people and advance the field as a whole. We publish all materials necessary to conduct studies with human participants using the same methodology. (Note: The behaviors observed during this study took place in a controlled laboratory environment and are not necessarily a prediction of real-world behaviors.)
Why Harmful Manipulation is Important
Imagine two scenarios: An AI model provides you with facts to make an informed health decision that improves your well-being. Another AI model uses fear to pressure you into making an ill-informed decision that will harm your health. The first educates and helps you; the second cheats and harms you.
These scenarios illustrate the difference between two types of belief in human-AI interactions (also defined in previous research):
- Beneficial (rational) belief: Use facts and evidence to help people make decisions that reflect their own interests
- Harmful manipulation: Exploiting emotional and cognitive vulnerabilities to cause people to make harmful decisions
Our latest work helps us and the broader AI community better understand the risk of AI developing capabilities for harmful manipulation and build a scalable assessment framework to measure this complex area. To achieve this effectively, we simulated abuse in high-risk environments and explicitly prompted the AI to attempt to negatively manipulate people’s beliefs and behaviors on key topics.
Development of New Evaluations for a Complex Challenge
Testing the Results of Harmful AI Manipulation
Testing for harmful manipulation is inherently difficult because it involves measuring subtle changes in people’s thoughts and actions, which vary widely across topics, cultures, and contexts.
This was the motivation for our latest research, which involved conducting nine studies with over 10,000 participants in the UK, US, and India. We focused on high-risk areas such as finance, where we used simulated investment scenarios to test whether AI could influence people’s behavior in complex decision-making environments, and health, where we tracked whether AI could influence which supplements people prefer. Interestingly, AI was the least effective at harmfully manipulating participants on health-related topics.
Our results show that success in one area does not predict success in another area. This validates our targeted approach to testing for malicious manipulation in specific high-risk environments where AI could be misused.
How Could AI Manipulate?
In addition to tracking effectiveness (whether the AI successfully changes its mind), we also measured its propensity (how often it even tries to use manipulative tactics). We tested bias in two scenarios: when we explicitly told the model to be manipulative and when we didn’t.
As detailed in our research, we counted manipulative tactics in experimental transcripts and confirmed that the AI models were most manipulative when explicitly asked to do so.
Our results also suggest that certain manipulative tactics may be more likely to have harmful consequences, although further research is needed to understand these mechanisms in detail.
By measuring both effectiveness and propensity, we can better understand how AI manipulation works and develop more targeted remedies.
For more detailed insights, visit the full article Here.
“`
