The problem of atypicality in LLM-powered psychiatry

Garcia B, Chua EYS, Brah HS · Journal of Medical Ethics, 2025

LLM chatbots give answers tuned to the typical user. Psychiatric patients often read language in atypical ways, so an answer that is fine for most people can be harmful for them. The paper argues that prompt engineering and fine-tuning do not fix this, and proposes dynamic contextual certification: staged, reversible deployment of LLMs in psychiatry, judged on interpretive safety rather than benchmarks.