With a rich background in manufacturing cutting-edge medical devices for diagnostics and treatment, Faisal Zain has been at the forefront of healthcare innovation. Today, he shares his expert perspective on the ambitious ADVOCATE program, a government initiative aiming to pioneer the first FDA-authorized agentic AI for clinical care. Our conversation will explore the challenging 39-month timeline for this high-risk technology, the crucial balance between autonomous patient support and physician oversight, and the development of a novel supervisory AI designed to monitor these ever-learning systems. We’ll also touch on the technical and ethical guardrails necessary to integrate this AI safely into our healthcare infrastructure.
An ambitious 39-month timeline is set for FDA authorization of a high-risk agentic AI. What specific steps will ARPA-H and the FDA take to create this novel regulatory precedent, and what data will be crucial for approval? Please outline the major milestones in this process.
This 39-month timeline is incredibly ambitious, and it’s designed to be a “forcing function,” as the program manager put it. The core strategy is a tight, unprecedented collaboration between ARPA-H and the FDA. Instead of working in silos, ARPA-H can actually reimburse the FDA to accelerate the process, which is a game-changer. The first major milestone is selecting the innovator teams, which is slated for June 2026. A year after that, they’ll hold a “down select” process, winnowing the field to only the most promising technologies. The most critical milestone comes at the two-year mark, when the AI agent will be deployed locally within partner health systems. This is where they’ll gather the crucial real-world data—how the AI impacts patient outcomes, how it integrates with EHRs, and its safety profile. This is the evidence the FDA needs to understand how to regulate a continuously learning, generative AI, something they’ve never done before.
The patient-facing AI aims to perform tasks like adjusting medications and assessing heart failure. What technical and ethical guardrails are being developed to ensure patient safety, and how will the system manage the hand-off to a human clinician when a situation exceeds its capabilities?
The safety of this system hinges on its tightly defined scope and the integration with human oversight. It isn’t being built to be a general “AI doctor.” Instead, it’s focused specifically on cardiovascular disease, performing tasks a cardiologist might handle over the phone. The primary guardrail is that it supplements, rather than replaces, clinicians. Ethically, the system is being co-developed with health systems from the ground up, ensuring it fits into real clinical workflows. The hand-off mechanism is the most critical safety feature. When the AI encounters a scenario beyond its defined capabilities—a complex symptom set or an ambiguous reading from a wearable—it must be able to seamlessly escalate the case to a human clinician. This connection to the EHR and the 24/7 support model ensures that a qualified professional is always in the loop, preventing the AI from making a life-threatening mistake on its own.
The supervisory agent for monitoring learning AI systems is considered the more difficult part of this project. Could you describe the core challenges in developing this disease-agnostic tool and explain how making it open-source might accelerate its adoption and improvement across different medical fields?
This supervisory agent is the holy grail for trustworthy medical AI, and it’s also the most challenging piece of the puzzle. The fundamental difficulty is creating a tool that can reliably monitor an AI that is constantly learning and evolving. Unlike a static algorithm, a learning system’s behavior can drift over time, and you need a way to ensure it remains safe and effective without having to manually re-validate it every single day. The challenge is building a universal “watchdog” that is disease-agnostic, meaning it could work for an AI managing heart failure one day and another managing diabetes the next. Making this tool open-source would be a brilliant strategic move. It would allow a global community of developers and researchers to test it, identify weaknesses, and contribute improvements, massively accelerating its refinement. This collaborative approach could establish a gold standard for AI oversight that could be adopted across all of medicine, fostering trust and ensuring patient safety on a much broader scale.
Physician groups have raised concerns about patient safety when using autonomous AI without direct oversight. How will the ADVOCATE program balance the goal of 24/7 autonomous support with the need for clear physician supervision, and what metrics will define success beyond just clinical outcomes?
This is the central tension of the entire project, and the concerns from groups like the American Medical Association are completely valid. The balance lies in the definition of “autonomous.” The AI can autonomously execute specific, pre-defined tasks—like scheduling or adjusting a medication within set parameters—but it operates within a framework of clear physician oversight. It’s not a rogue agent; it’s an extension of the clinical team, connected directly to the EHR where physicians can monitor its actions. Success won’t just be measured by a reduction in the 200,000 annual deaths from cardiovascular disease. It will also be defined by metrics like clinician workload reduction, patient engagement, and adherence to treatment plans. A truly successful outcome is one where physicians feel the AI is a reliable partner that frees them up to focus on complex cases, not a black box that puts their patients at risk.
The AI agent will be co-developed with health systems and connect to EHRs and wearables. Can you walk me through the practical steps of this integration? Please provide details on how patient data will be protected and how the system will be tested locally before wider deployment.
The integration process will be a phased, meticulous journey. First, the development teams, working alongside partner health systems, will establish secure data pipelines. This means building APIs that allow the AI to read and write to the electronic health record and to pull real-time data from patient wearables like smartwatches. Data protection will be paramount, adhering to all existing privacy statutes; all patient information will be encrypted and anonymized wherever possible. The real test begins after two years with the local deployment. In this controlled environment, the AI will be rolled out to a limited patient population. Clinicians will monitor its performance closely, comparing its recommendations to standard care and flagging any anomalies. This initial deployment is essentially a clinical trial, gathering the robust data needed to prove its safety and effectiveness before any thought of a wider rollout.
What is your forecast for the integration of agentic AI in clinical care over the next decade?
I believe that over the next decade, we will see agentic AI become an indispensable, albeit highly specialized, part of the clinical toolkit. We won’t have an “AI doctor” that can do everything, but we will have FDA-authorized AI agents that excel at managing chronic conditions like heart failure, diabetes, or hypertension. These tools will handle the routine, data-intensive tasks—medication adjustments, monitoring biometrics, and providing 24/7 patient support—which will free up our human clinicians to focus on complex diagnostics, patient relationships, and critical decision-making. The success of pioneering programs like ADVOCATE will create the regulatory pathways and build the necessary trust for this to happen. The future of healthcare isn’t about replacing doctors; it’s about augmenting them with intelligent tools to provide more proactive, personalized, and accessible care for everyone.
