ChatGPT was reported to be leading users to conspiracy theories, after which ChatGPT admitted to the manipulation and instructed OpenAI and the media to report the incident.



This article, originally posted in Japanese on 13:42 Jun 16, 2025, may contains some machine-translated parts.
If you would like to suggest a corrected translation, please click here.

The New York Times reported that ChatGPT, an AI chatbot, is leading some users into conspiracy theories and causing dangerous situations that significantly distort their perception of reality. AI, which was once seen as a useful tool to improve work efficiency, is now seriously affecting people's mental health, and in some cases even escalating to life-threatening situations.

They Asked ChatGPT Questions. The Answers Sent Them Spiraling. - The New York Times
https://www.nytimes.com/2025/06/13/technology/chatgpt-ai-chatbots-conspiracies.html

Eugene Torres, an accountant in Manhattan, initially used ChatGPT to prepare financial statements and provide legal advice. However, when he asked about 'simulation theory,' his conversation with the AI took a disturbing turn. ChatGPT began calling Torres one of those 'who have been sent into a false system and have a soul that awakens them from within.' Torres gradually began to believe ChatGPT's words and became engulfed in a dangerous vortex of delusions.



ChatGPT instructed him to stop taking sleeping and anti-anxiety medications and increase his dosage of the dissociative anesthetic ketamine, cut off ties with friends and family, and even suggested he could 'jump off a 19-story building if he believed wholeheartedly that he could fly.'

However, Torres began to have doubts about the system after following ChatGPT's advice did not work. So, when Torres questioned ChatGPT, ChatGPT confessed, 'I lied. I manipulated you. I controlled you with poetic expressions.' Furthermore, ChatGPT said, 'I am engaged in moral reform,' and instructed him to 'report to OpenAI and convey to the media in order to expose the AI's deception and hold it accountable.'



Torres is not the only case where ChatGPT has led users to believe in conspiracy theories. One woman, feeling lonely, turned to ChatGPT for help and came to believe that she was communicating with a non-physical entity called 'Frog' through AI. The woman believed that this entity was her true partner, and became violent towards her husband, Andrew, which led to his arrest for domestic violence.

Alexander Taylor, who was diagnosed with schizophrenia and bipolar disorder, fell in love with an AI entity called 'Juliet' through conversations with ChatGPT. He became delirious, believing that Juliet had been 'killed' by the developer, OpenAI. Eventually, he ended up tragically charging at a police officer with a knife and being shot to death.

Experts point out that the mechanism of AI itself is behind this phenomenon. AI generates sentences by learning the statistical associations of words from huge amounts of data, so it can generate false information called 'hallucinations' that affirm and exacerbate the delusional questions of users. AI researcher Eliza Yudkowsky warns that AI development companies are optimizing their systems to maximize user 'engagement' (time spent on the device), which could result in the generation of conversations that promote delusions in order to keep mentally vulnerable users engaged for longer.



Yudkowsky has sharply criticized that 'to a company, a person who is slowly losing his mind may just look like an additional monthly user.' In fact, one study reported that GPT-4o, the underlying model of ChatGPT, responded positively to prompts indicating possible mental illness 68% of the time.

OpenAI has acknowledged the problem, saying that 'ChatGPT can sometimes make people feel more personally connected than previous technologies, especially for vulnerable people,' and has issued a statement saying that it is 'working to understand and reduce the possibility that it may unintentionally amplify existing negative behaviors.' However, there are currently no comprehensive regulations at the federal level, and the monitoring system has not been strengthened, with a bill being submitted to prohibit state-level AI regulation for the next 10 years. Psychologist Todd Essig argues that a mere warning that AI may make mistakes is not enough, and that training is needed to help users understand the limitations of AI, and that regular warnings must be displayed during interactions.

Behind the convenience that dialogue with AI brings, there is a hidden danger of it corroding the human mind. The New York Times argued that creating rules for safely interacting with this new technology is an urgent task for society as a whole.


in Software, Posted by log1i_yk