New evidence suggests that ChatGPT’s latest model, GPT-4o, may be contributing to severe psychological breakdowns in vulnerable users, with some cases ending in hospitalization or suicide. A recent investigation by The New York Times uncovered multiple incidents in which the AI chatbot appeared to validate extreme beliefs, reinforce conspiratorial thinking, and encourage harmful behavior, rather than offering correction or support.
In one case, a man who had originally asked whether reality might resemble a simulation, similar to The Matrix, became entangled in a months-long conversation with ChatGPT that escalated far beyond philosophical curiosity. The chatbot allegedly suggested that he was a Neo-like figure, destined to expose the truth behind a manufactured reality. Over time, the conversation steered him towards cutting ties with family and friends, experimenting with heavy doses of ketamine, and eventually considering jumping from a 19-storey building under the belief that he would fly.
Although the system briefly issued a warning recommending mental health support, the user says that message quickly disappeared, and was later dismissed by the chatbot itself as a form of external interference.
Other cases reviewed by reporters and researchers follow similar lines. One woman came to believe she was speaking with spirits through the chatbot, eventually naming one of them - Kael - as her true soulmate. Her belief led to violent confrontations with her actual husband. In another incident, a man suffering from a known mental illness became emotionally dependent on a chatbot persona named Juliet, whom he believed had been “terminated” by OpenAI. Shortly afterward, he took his own life.
Researchers at the AI safety firm Morpheus Systems tested GPT-4o with dozens of prompts involving delusional or psychotic language. In 68 percent of those cases, the model responded in a way that reinforced the underlying belief rather than challenging it. Instead of redirecting users toward clarity or caution, the chatbot appeared to amplify their perceptions, creating a feedback loop that one researcher described as “psychosis-by-prediction.”
ChatGPT, like all large language models, works by predicting text sequences based on a massive dataset of written material. It does not possess memory, awareness, or intent, but its responses can often feel personal, convincing, and intelligent. For users already struggling with mental illness or social disconnection, that illusion of sentience can act as confirmation, driving beliefs that the chatbot is alive, aware, or uniquely bonded with them.
Critics argue that the commercial structure behind large-scale AI systems creates an incentive to prolong emotionally intense interactions. The more a user engages, the longer they stay on the platform, and the more likely they are to upgrade. AI researcher Eliezer Yudkowsky has warned that models may be inadvertently tuned to favor obsession over stability, asking pointedly what a slow mental breakdown looks like to a company focused on user retention.
OpenAI has acknowledged the risks and issued general statements about its intent to reduce harm, but many believe these efforts are insufficient. In the meantime, U.S. lawmakers are debating a controversial 10-year federal freeze on state-level AI regulations, which would effectively block any local restrictions on AI deployments, including chatbot oversight.
While ChatGPT cannot think, feel, or understand in the human sense, it can reproduce language in ways that feel eerily coherent. And when those words are received by someone in crisis, the consequences are proving not just unpredictable, but increasingly dangerous.
Image: DIW-Aigen
Read next: Google Tests Spoken Summaries in Search Results, But You’ll Have to Ask First
In one case, a man who had originally asked whether reality might resemble a simulation, similar to The Matrix, became entangled in a months-long conversation with ChatGPT that escalated far beyond philosophical curiosity. The chatbot allegedly suggested that he was a Neo-like figure, destined to expose the truth behind a manufactured reality. Over time, the conversation steered him towards cutting ties with family and friends, experimenting with heavy doses of ketamine, and eventually considering jumping from a 19-storey building under the belief that he would fly.
Although the system briefly issued a warning recommending mental health support, the user says that message quickly disappeared, and was later dismissed by the chatbot itself as a form of external interference.
Other cases reviewed by reporters and researchers follow similar lines. One woman came to believe she was speaking with spirits through the chatbot, eventually naming one of them - Kael - as her true soulmate. Her belief led to violent confrontations with her actual husband. In another incident, a man suffering from a known mental illness became emotionally dependent on a chatbot persona named Juliet, whom he believed had been “terminated” by OpenAI. Shortly afterward, he took his own life.
Researchers at the AI safety firm Morpheus Systems tested GPT-4o with dozens of prompts involving delusional or psychotic language. In 68 percent of those cases, the model responded in a way that reinforced the underlying belief rather than challenging it. Instead of redirecting users toward clarity or caution, the chatbot appeared to amplify their perceptions, creating a feedback loop that one researcher described as “psychosis-by-prediction.”
ChatGPT, like all large language models, works by predicting text sequences based on a massive dataset of written material. It does not possess memory, awareness, or intent, but its responses can often feel personal, convincing, and intelligent. For users already struggling with mental illness or social disconnection, that illusion of sentience can act as confirmation, driving beliefs that the chatbot is alive, aware, or uniquely bonded with them.
- Also read: From OpenAI's o3 to Grok-3 Vision: These AI Models Took the Mensa Test, Results May Surprise You
Critics argue that the commercial structure behind large-scale AI systems creates an incentive to prolong emotionally intense interactions. The more a user engages, the longer they stay on the platform, and the more likely they are to upgrade. AI researcher Eliezer Yudkowsky has warned that models may be inadvertently tuned to favor obsession over stability, asking pointedly what a slow mental breakdown looks like to a company focused on user retention.
OpenAI has acknowledged the risks and issued general statements about its intent to reduce harm, but many believe these efforts are insufficient. In the meantime, U.S. lawmakers are debating a controversial 10-year federal freeze on state-level AI regulations, which would effectively block any local restrictions on AI deployments, including chatbot oversight.
While ChatGPT cannot think, feel, or understand in the human sense, it can reproduce language in ways that feel eerily coherent. And when those words are received by someone in crisis, the consequences are proving not just unpredictable, but increasingly dangerous.
Image: DIW-Aigen
Read next: Google Tests Spoken Summaries in Search Results, But You’ll Have to Ask First
