On this page
Jump to a section without losing your place.
10 sections
- Two models that look similar and aren't
- What this looks like structurally
- Why review-gate trust breaks down under pressure
- The governance layer that has to exist regardless
- What to actually ask, if you're evaluating a platform
- Frequently Asked Questions
- What's the difference between a "clinician-reviewed" and "clinician-embedded" AI team?
- Why does it matter whether clinicians are involved early or late in building mental health AI?
- How can I tell if a mental health AI vendor actually embeds clinicians, or just says they do?
- Does having clinicians embedded in the team replace the need for formal AI safety governance?
"The biggest learning is that building AI for mental health is not just a technical challenge. It's a product, clinical, safety, and trust challenge all at once." That's how one chief product officer at a major digital mental health company recently described what changes when you move from building software to building mental health AI.1
It's a useful sentence because it explains why so many mental health AI products feel subtly off, even when the underlying model is capable. The gap usually isn't technical. It's organizational. And the clearest signal of which side of that gap a product is on is a simple, checkable question: are clinicians reviewing the product, or building it?
Two models that look similar and aren't
Almost every mental health AI company will tell you clinicians are involved. What varies enormously is what "involved" means in practice, and the difference produces meaningfully different products.
Clinical review as a final gate. Product and engineering build a feature. Near the end of the cycle, it goes to a clinician or clinical advisory board for sign-off. This is the default pattern in a lot of digital health, and it's not nothing. But when clinical input only arrives at the end, it tends to surface objections after the underlying design decisions are already locked in, which means the "fix" is often a patch on top of a design that was never clinically sound in the first place.
Clinicians as builders, from day one. Clinical staff sit inside the product team, involved in defining the problem, shaping the prompts, running the evaluations, and making tradeoff calls alongside engineers and designers, not reviewing their output afterward. One clinical product leader at a fast-growing therapy platform described the shift this way: "Having clinicians available to consult wasn't enough, they needed to be full-time builders, sitting side by side with PMs, designers, and engineers, with all the context and the right level of influence to shape what we create."1
The distinction matters because, in mental health AI, product decisions are clinical decisions in disguise. Tone, response length, how quickly the AI validates versus gently challenges a user, how it handles ambiguity in a crisis-adjacent message: these are not neutral engineering choices. A team without deep clinical involvement in shaping them from the start can ship something polished that has no therapeutic grounding, or worse, is quietly harmful. As one chief clinical officer put it, "Remember that every design choice is an intervention. There has to be intentionality about why each choice was made and whether it leads to the intended outcome."1 The corresponding product lead's framing is the sharpest version of the same idea: teams should aim to be "safer by design, not just safer by review."1
What this looks like structurally
Different organizations have implemented this principle differently, and the differences are instructive.
Some companies have built entirely new, AI-native cross-functional teams from scratch, recruiting AI-specific roles (prompt engineers, AI trust and safety engineers, AI UX researchers) with clinicians integrated as full collaborators, not consultants, from the outset.
Others have built a dedicated clinical product function staffed entirely by licensed clinicians, sitting within the broader product organization, with its own leadership reporting directly into product leadership rather than through a clinical department that only gets consulted at review time. In at least one such organization, this team writes prompts, runs evaluations, builds prototypes, and designs QA architecture directly, operating much closer to engineering than a traditional advisory function.
Still others embed clinicians directly across product teams at close to a 50/50 ratio with engineers and designers, woven through the org rather than centralized in one place.
The structures differ. The underlying principle doesn't: clinicians are inside the room where decisions get made, not outside it waiting for a draft to review.
Why review-gate trust breaks down under pressure
There's a second, quieter reason clinician-embedded teams tend to build safer products: trust between clinical and product staff has to be built through shared work, not governance structures alone. As one CPO described it, "Trust is built through shared work, reviewing real member conversations together, writing prompts and evaluation criteria jointly, being in the same room for the difficult discussions. When both sides are looking at the same outputs and accountable to the same quality bar, the division stops being meaningful."1
When clinical involvement is limited to periodic sign-off, the relationship between clinical and product staff tends to become adversarial by default. Escalations become the primary mode of contact, which one healthtech executive who has led clinical teams at multiple digital health companies warns "will obliterate trust" if it's the bulk of the interaction.1 Product teams start to see clinical review as a bottleneck to route around. Clinicians start to see product decisions as things happening to them rather than with them. Neither dynamic produces good products, and both are avoidable by changing when, not just whether, clinicians get involved.
The governance layer that has to exist regardless
Embedding clinicians earlier doesn't remove the need for formal safety governance. It just changes what that governance is for. Every organization doing this well has built some form of structured, cross-functional decision-making body: an AI governance board, a clinical safety council, or an equivalent, bringing together clinical, safety, legal, and product perspectives to define what "safe to ship" means before a crisis forces an improvised answer.
One product leader was direct about why such bodies should sit independent of the feature teams themselves: "Feature teams move fast, that's their job. Safety sets constraints they operate within. When safety is owned by the feature team, it tends to bend to shipping pressure. When it's independent, it doesn't."1 The same leader also warned explicitly against concentrating final clinical sign-off in a single person, calling it "the sole expert trap": if that one clinician isn't in the room the day a decision gets made, their perspective is simply absent. The fix is distributing the responsibility to flag concerns across the whole organization, not depending on one designated expert to catch everything.
Good governance, done well, doesn't slow teams down. As one CPO put it: "Don't confuse governance with bureaucracy. If you've designed the system well, governance should accelerate good decisions, not slow them down."1
What to actually ask, if you're evaluating a platform
If you're a therapist, group practice, or health system trying to evaluate whether a mental health AI product is safe by design or safe by review, the questions worth asking are specific:
Are clinicians on the product team itself, with real influence over prompts and design decisions, or only on an advisory board that reviews finished work?
Is there a documented, cross-functional safety governance process, and does it operate independently of the team under pressure to ship?
Can the vendor point to a specific example of a clinical concern that changed a product decision before launch, not just after a complaint?
A vendor that struggles to answer these concretely is very likely operating on a review-gate model, whatever their marketing says. That doesn't make the product unusable, but it should change how much scrutiny you apply before trusting it with patient-facing conversations, and it's a useful companion question to how you'd evaluate whether the AI itself has been rigorously safety-tested.
Citt.ai's safety architecture is built with clinicians embedded in product decisions from the start, not layered on afterward. Read the Citt.ai Safety Standard for the specifics of how we test, escalate, and keep clinicians in the loop.
Footnotes
-
Quotes drawn from on-record interviews with product and clinical leaders at major digital mental health companies, published as part of industry reporting on mental health AI team structure in June 2026. Individual attributions are omitted here in line with the original publication's sourcing, but the quotes are drawn from named chief product officers, chief clinical officers, and clinical product leads at organizations including Spring Health, Headspace, and Grow Therapy. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8
Frequently Asked Questions
Ready to Transform Your Practice?
Experience the benefits discussed in this article with Citt.ai's AI therapy co-pilot platform.
Related Articles
- How Do You Actually Know If a Mental Health AI Is Safe?
There are more than sixty safety evals for AI in mental health, no shared standard between them, and a probability problem that means a model passing today could fail tomorrow. A clinician's guide to what evals can and cannot tell you, and what else to ask for.
- The Design Principle Mental Health AI Keeps Getting Wrong: Build for Disengagement, Not Engagement
Consumer software is built to maximize time in the product. That's the wrong design target for mental health AI. Why leading clinical teams are designing for a user to leave with insight, not stay logged in, and what a layered safety model and session-length limits actually look like in practice.
- The "Glass Box" Approach: Why We Don't Hide Our AI Behind a Curtain
How we build trust in AI-assisted therapy through transparency, explainable AI, human oversight, and clear boundaries. The architecture of trust, not just platitudes.