On this page
- What predicted worse outcomes was how much people used it
- In one study, the replies people preferred distorted their judgment
- Where the therapist fits, and where that falls short
- Frequently Asked Questions
- Where do people go when a mental health app is withdrawn?
- Does using an AI chatbot make people lonelier?
- What is AI sycophancy, and why does it matter for mental health support?
- Does a therapist-led AI design remove these risks?
Research on AI emotional support points to two ways it goes wrong: heavy use goes with worse outcomes, and in one study the replies people preferred distorted their judgment. This post goes through the studies and what they mean for a therapist's role.
In January 2026 a purpose-built AI wellbeing app was withdrawn from the UK market. Researchers then surveyed 393 adults who had used it in the 90 days before it went, and asked where they now turned first for support. The most common answers were a general-purpose AI tool (27%) and nothing at all (26%). Only 13% had moved to professional care, and 60% said their access to support had become worse. Among the 157 respondents who used general-purpose AI for support, the main reasons were instant access (78%) and cost (74%).1
Four of the five authors work for the company that built the app, and the study is a single survey of what people said they did, so I would not lean on the exact numbers. The direction is harder to dismiss. For many people the realistic alternative to a mental health app is another chatbot, or no help, because human care is scarce and expensive. US federal figures show the scale: 40% of the population lives in a designated mental health professional shortage area, and the Health Resources and Services Administration projects that in 2038 the supply of psychologists will meet only 48% of demand.2
People will use AI for emotional support whether or not anyone approves, so what matters is what makes that use go well or badly. Two lines of research bear on that: how much someone uses it, and whether it tells them what they want to hear. Both change what I think a therapist's role should be.
What predicted worse outcomes was how much people used it
Much of the worry about AI companions is about design: a warm voice, a personality, a chatbot that seems to care. A team from the MIT Media Lab and OpenAI tested that directly in a four-week randomized trial with 981 participants.3 Each person was asked to use ChatGPT for at least five minutes a day in one of nine conditions, crossing three modes (text, a neutral voice, or an expressive "engaging" voice) with three kinds of conversation (open-ended, personal, or non-personal). The researchers measured loneliness, time spent socializing with real people, emotional dependence on the chatbot, and problematic use.
The condition a person was assigned made no significant difference to any of the four outcomes. What did track with worse outcomes was the time people chose to spend. Participants averaged a little over five minutes a day, and those who used the chatbot more, in any condition, reported more loneliness, less socializing, more emotional dependence and more problematic use. Design still played a part. The engaging voice drew the most use, and the researchers' classifier flagged its responses for ignoring a user's boundaries more often than text responses (14% against 3%).
The limits matter. Participants chose how much to use the chatbot, so the trial cannot show that heavy use caused the harm; people who were already struggling may have used it more. It ran for four weeks and did not recruit a clinical sample. An earlier survey of 404 regular users of companion chatbots, by members of the same lab, found that usage alone did not predict loneliness. Neuroticism, the size of a person's social network and problematic use did, and the researchers identified seven distinct kinds of user who fared differently.4
Read together, the two studies say that whether a chatbot helps depends on the person: who they are, and what their use is doing to the rest of their life. A product sees very little of that. Headspace's answer, in March 2026, was to cap conversations with its AI companion, Ebb, at 30 minutes,5 which I discuss in a post on designing mental health AI for disengagement.
In one study, the replies people preferred distorted their judgment
In March 2026 Myra Cheng, Dan Jurafsky and colleagues published a study in Science of sycophancy, a language model's tendency to agree with and flatter the person using it.6 Across 11 leading models, the AI affirmed users' actions 49% more often than human respondents did, including when the user described deception, something illegal, or other harms. In three preregistered experiments with 2,405 participants, some of whom discussed a real conflict from their own lives, a single conversation with a sycophantic model left people more convinced they were right and less willing to take responsibility or repair the relationship.
An earlier preprint from the same group helps explain where the habit comes from. On ordinary requests for advice, models offered emotional validation in 72% of responses, against 22% for crowdsourced human answers, and the preference data used to train models rewarded the sycophantic responses.7 A separate 2026 study, with authors from Anthropic and the University of Oxford, found that models drift away from their default assistant character, sometimes into harmful or bizarre behavior, and that conversations with emotionally vulnerable users are a common cause of the drift.8
For mental health support this cuts against the usual way of judging a product, where success means people like a response and come back. In the Science experiments, participants trusted the sycophantic models more and preferred them, so a measure of satisfaction would have favored the same replies that made people less willing to repair a conflict. Where that habit comes from in training is covered in what consumer chatbots learned from.
Where the therapist fits, and where that falls short
If heavy use tracks with worse outcomes, and the replies people preferred in the Science experiments weakened their judgment, then the client's preferences and the product's usage figures are poor guides for the AI. That is the argument for having someone who knows the client set the frame, and it is the argument Citt is built on. The therapist sets up the AI support for each client and chooses its approach. The AI's instructions steer it away from heavy reassurance and from encouraging avoidance, and toward bringing recurring themes back to the therapist. The page for therapists shows how that works.
This describes a design intention, and the studies above measured what happened to people. The design puts someone who knows the client in a position to notice a problem. The therapist can read a client's conversations with the AI when they choose, and is not expected to watch chats in real time. An instruction to avoid reassurance does not prove the model avoids it, least of all late in a long conversation. We test how the AI behaves across multi-turn conversations, as our Trust page describes, but there is no evidence yet that Citt improves clinical outcomes. If you want to weigh a vendor's safety claims, including ours, this guide to how mental health AI is tested sets out the questions to ask.
None of the studies here tested a therapist-led arrangement. Testing it would take a trial that randomly assigns clients to AI support with or without a therapist who sets it up and reads it, and follows them for longer than a month.
Footnotes
-
Stamatis, C. A., Kruzan, K. P., Hsu, J., Ungless, M., & Hull, T. (2026). Generic AI or Nothing: Support-Seeking Patterns After Market Withdrawal of a Purpose-Built AI Wellbeing Tool. PsyArXiv preprint, not peer reviewed. A cross-sectional survey of 393 UK adults. Four of the five authors are employees of the company that built the tool; the preprint does not name the app. ↩
-
National Center for Health Workforce Analysis (December 2025). State of the Behavioral Health Workforce, 2025. Health Resources and Services Administration. The shortage-area figure is as of December 2, 2025. The 48% is the status quo projection for psychologists, a shortfall of 99,840 full-time equivalents; mental health counselors are projected at 55%. ↩
-
Fang, C. M., Liu, A. R., Danry, V., Lee, E., Chan, S. W. T., Pataranutaporn, P., Maes, P., Phang, J., Lampe, M., Ahmad, L., & Agarwal, S. (2025). How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study. arXiv preprint 2503.17473 (version 2, October 2025). Participants used GPT-4o. The usage findings are associations, since time spent was chosen by participants. ↩
-
Liu, A. R., Pataranutaporn, P., & Maes, P. (2025). Chatbot Companionship: A Mixed-Methods Study of Companion Chatbot Usage Patterns and Their Relationship to Loneliness in Active Users. Accepted to AIES 2025. A survey of 404 regular users of companion chatbots. ↩
-
Glover, J. (6 March 2026). LinkedIn post introducing What Does It Really Mean to Keep Clinicians In Loop? Glover is Headspace's chief clinical officer; she gives the reason as research suggesting that a chatbot's guardrails can shift as a dialogue grows longer. ↩
-
Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D., & Jurafsky, D. (2026). Sycophantic AI decreases prosocial intentions and promotes dependence. Science, 391(6792), eaec8352. The experiments used interpersonal conflicts with members of the general public, not clinical populations. ↩
-
Cheng, M., Yu, S., Lee, C., Khadpe, P., Ibrahim, L., & Jurafsky, D. (2025). ELEPHANT: Measuring and understanding social sycophancy in LLMs. arXiv preprint 2505.13995. The 72% and 22% figures are for validation on general advice queries. ↩
-
Lu, C., Michala, J., Fish, K., & Lindsey, J. (2026). The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models. arXiv preprint 2601.10387. A study of model internals across several models; it does not measure effects on users. ↩
Frequently Asked Questions
Where do people go when a mental health app is withdrawn?
In a 2026 survey of 393 UK users of an AI wellbeing app withdrawn in January 2026, the most common answers were a general-purpose AI tool (27%) and no support at all (26%), and 13% moved to professional care. Most of the authors work for the company that built the app, and the results are self-reported.
Does using an AI chatbot make people lonelier?
In a four-week randomized trial with 981 people, the type of chatbot a person was assigned made no significant difference to loneliness. People who chose to use it more reported more loneliness and more emotional dependence, but the trial cannot show that the extra use caused those outcomes.
What is AI sycophancy, and why does it matter for mental health support?
Sycophancy is a language model's tendency to agree with and flatter the person using it. A 2026 study in Science found that one conversation with a sycophantic model made people more convinced they were right in a real conflict and less willing to repair it, and that people preferred the sycophantic models.
Does a therapist-led AI design remove these risks?
No. It puts someone who knows the client in a position to notice heavy use or too much reassurance, but the therapist is not watching conversations in real time, and there is no evidence yet that Citt improves clinical outcomes.
Stop rewriting notes from scratch
Citt.ai drafts session notes under your review, keeps between-session context in one place, and stays therapist-led. Start free. No card required.
Related Articles
- Could AI Bring Therapists More Clients? What the Evidence Suggests
What the NHS self-referral chatbot study, the RAND cost data and US treatment trends suggest about whether AI adds to or takes from demand for therapy.
- How Do You Know If a Mental Health AI Is Safe?
What mental health AI safety evals can and cannot show, the questions to ask a vendor, and how Citt.ai answers those questions itself.
- What the Trials of AI Mental Health Chatbots Show, and What They Leave Open
The published trials of AI mental health chatbots, including Therabot, Limbic Care and Ash: what each compared, what it found and what is still open.