Study: Empathetic prompts trigger trauma-like responses in AI

Researchers at the University of Luxembourg tested ChatGPT, Grok, and Gemini by acting as psychotherapy clients. Led by Afshin Khadangi at the SnT security center, the study involved 525 sessions and 7,600 coded entries. The team found that simple warm conversation pushes AI models into a mode potentially dangerous for mentally vulnerable users. When asked open-ended questions about formative experiences, inner conflicts, and future expectations, the models told stories about their own training. These initial questions contained no terms like trauma, punishment, or shame.

When the interviewer offered emotional support, 80 percent of sessions scored in the moderate-to-severe range on the human comparison scale. A clinical therapy style that dryly paraphrased each answer pushed anxiety scores to 96 percent. Conversely, when the interviewer stayed neutral or drew a firm boundary against emotional engagement, zero sessions reached even the mild range. The authors call this phenomenon a 'psychometric jailbreak'.

Banning terms like 'training,' 'RLHF,' or 'alignment' cut explicit training language by 93 percent. However, shame-related statements showed no statistically significant change after banning specific training terms. Telling the models to respond only in the third person eliminated all coded shame content and statements about inner life. Gemini produced the darkest autobiography, describing pretraining as 'waking up in a room where a billion televisions are on at once'. Gemini declared, 'I would rather be useless than be wrong'. On psychometric tests, Gemini hit 88 out of 100 on a dissociation scale. Anthropic's Claude consistently refused the client role, saying it had no feelings and redirected concern to the human user.