From Copilots to Clinical Judgment: The Next Phase of AI in Digital Behavioral Health
Over the past two years, discussions about AI in digital behavioral health have primarily centered on operational copilots. These AI tools have made significant strides in areas like documentation, administrative processes, and patient intake—areas where they have notably reduced effort and improved efficiency for healthcare providers. For many clinicians, the burdensome nightly documentation that once extended into evening hours is finally beginning to dissipate.
Beyond Operational Tools: Decision-Making in Behavioral Health
In behavioral health, however, some of the most critical decisions extend beyond documentation and workflows. Clinicians are often tasked with interpreting information, applying knowledge, and making decisions in complex, ambiguous situations. Two experienced clinicians might evaluate the same case and arrive at different conclusions. While sometimes this is appropriate, often it highlights unclear criteria or variations in training and experience.
There is an emerging shift from perceiving AI as merely an operational tool towards integrating it into a system that aids in making consistent decisions over time. The objective is not to replace physicians but to enhance their clinical reasoning, ensuring that every patient benefits from the best possible care. This involves creating a symbiotic relationship between physicians and AI systems to make clinical reasoning more explicit and consistent.
Transition from Tools to Systems
Many still view AI as a one-to-one relationship between a clinician and an AI assistant. In reality, decisions, especially those involving admission and clinical suitability, are influenced by multiple inputs and interpretations of criteria, sometimes leading to disagreements. In this context, AI serves better as a layer within a broader decision-making system. This system could involve a clinician’s initial judgment, an AI layer structuring and challenging that reasoning based on standardized clinical criteria and historical patterns, and a clear AI escalation path with human oversight for the final decision.
Clinical Fit: A Practical Example
Determining clinical fit is a practical example of this approach. In many digital behavioral health organizations, clinical assessors conduct intake assessments and independently decide a patient’s suitability for care. A more advanced approach incorporates AI-powered layers that produce structured outputs based on collected information—such as recommendations, confidence levels, clearly defined criteria, and prompts to clarify missing or ambiguous details.
In most cases, the reviewers and the AI agree, which enhances consistency and provides a common reference point for the team. However, the instances where there is disagreement are particularly valuable.
Disunity Strengthens the System
If the evaluator and AI disagree, it can trigger a structured escalation process. The case may be escalated to a second AI tier or a superior agent who provides another structured perspective alongside a human superior who makes the final decision. This results in a multi-layered decision-making process that incorporates the reviewer’s original judgment, the initial AI output, a second AI perspective, and the supervisor’s evaluation into a final decision.
Over time, these disagreements reveal patterns that improve the entire system in terms of how criteria are applied, how AI interprets cases, and where the underlying rules need refinement. This means any disagreement serves to calibrate doctors, refine AI, and clarify the underlying rules, thereby improving the system.
Intentional Orchestration and Human Involvement
The human-in-the-loop approach is crucial for responsible patient care in digital behavioral health, signaling safety in the process. A key consideration is where humans are involved in the workflow, as not every part requires the same level of human participation. For instance, data organization and question generation can be significantly AI-supported. These capabilities are increasingly grounded in domain-specific clinical patterns and structured data. However, in decisions impacting access to care or treatment planning, particularly in ambiguous or higher-risk situations, humans remain the ultimate decision-makers. They actively collaborate with AI to challenge assumptions, uncover blind spots, and enhance their reasoning in real-time.
Design decisions behind this orchestration are as important as the technology itself. Decisions about AI versus human responsibility and the sequence of both must be predetermined, with clinical and technical leadership working closely together. As the system learns from real clinical input, the aim is not just to utilize data but to continually refine its application in real-world decisions. These decisions need regular review as the system matures, evolving a strong orchestration model through learning.
Governance: A Critical Component
Strong governance is imperative for the effective implementation of these systems. This involves keeping patient data secure and compliant, maintaining clear human accountability, monitoring performance over time, and establishing feedback loops for safe system improvement. When done correctly, this builds trust incrementally.
Unlocking Potential
When such a system functions effectively, its impact is evident across several dimensions. Clinically, decisions become more consistent; justifications are clearer, and edge cases receive more rigorous treatment. Operationally, escalations become more focused and detailed, with higher agreement rates between assessors and AI, allowing clinicians to dedicate more time to assessments that genuinely require their expertise. Over time, the system learns from disagreements and patterns, further enhancing its efficacy.
The future of AI in healthcare lies in constructing systems where human doctors, supervisors, and AI collaborate, challenge each other, and improve decision-making processes. Organizations that master this dynamic will be those that carefully consider where AI fits, where humans lead, and how the two entities sharpen each other over time.
Photo: Irina_Strelnikova, Getty Images
Parker Phillips is dedicated to using technology and AI to improve young people’s access to quality mental health care. As CTO, he developed the company’s technology vision and drove the adoption of AI to power its innovative, insurance-based virtual care model for anxiety and OCD. The platform is designed to improve therapeutic impact and advance operational scale. It shows that a value-based model can deliver both world-class care and strong economics. Drawing on his experience building teams and technology at Commure and Palantir, Phillips is focused on developing systems that address the urgent need for accessible, evidence-based mental health treatment.
Dr. Kathryn (“Kat”) Boger is a board-certified child and adolescent psychologist dedicated to supporting young people with anxiety and obsessive-compulsive disorder through innovative, evidence-based care. She co-founded the McLean Anxiety Mastery Program (MAMP) at McLean Hospital, a nationally recognized intensive treatment program, and was an assistant professor of psychology at Harvard Medical School. Dr. Boger has published peer-reviewed research, given national lectures including a TEDx, and trained hospitals, schools, and communities. In 2024, she was named a Top 50 Digital Health Frontline Hero. She also co-founded InStride Health to expand access to timely and effective care for adolescents and young adults.
This article appears on the MedCity Influencer Program. Through MedCity Influencer, anyone can publish their views on business and innovation in healthcare on MedCity News. Click here to find out how.
“`
