New technologies are frequently hailed as transformative in healthcare, yet AI has exceeded even the loftiest expectations, emerging as a focal point of both hope and concern. The U.S. Food and Drug Administration (FDA) has been actively preparing for AI’s integration into healthcare and biomedical innovation, recognizing both the opportunities and unique challenges that this technology brings.
Artificial intelligence (AI) involves systems capable of making predictions, recommendations, or decisions based on predefined objectives, using both human and machine inputs to interpret environments and provide actionable insights. Its applications range from straightforward algorithms to complex models such as machine learning and generative AI, which demand flexible, risk-informed regulatory frameworks.
Early Approvals and Current Applications of AI in Medicine
The FDA approved its first AI-enhanced medical device, PAPNET, in 1995—a neural network-based tool designed to reduce cervical cancer misdiagnoses during Pap tests. Despite its higher accuracy than human pathology, clinical adoption was hindered by cost-effectiveness concerns. Since then, the FDA has approved about 1,000 AI-driven medical devices, particularly in radiology and cardiology.
AI is also increasingly integral in drug development, with the FDA receiving 132 applications for AI-related drug innovation in 2021 alone—a notable rise from prior years. Oncology is the primary field leveraging AI for purposes such as drug discovery, dosage optimization, trial design, and postmarket surveillance, with mental health following closely.
The FDA’s 5-Point AI Action Plan
The FDA oversees nearly a fifth of the U.S. economy and must continually update its strategies to balance safety and innovation in AI. In 2021, the FDA launched a forward-looking 5-point action plan to guide the safe use of artificial intelligence (AI) in healthcare. Recognizing the rapid pace of AI innovation, this plan seeks to create a balanced regulatory framework that safeguards patient safety while supporting continuous improvements in AI technology.
AI tools in healthcare can often be updated or improved to perform more efficiently or accurately. However, without a flexible regulatory system, developers would need to seek FDA re-approval for every minor adjustment—a process that could slow down innovation. To address this, the FDA’s action plan focuses on creating a more adaptable regulatory approach that allows AI to evolve without sacrificing safety.
Here’s a closer look at the five key areas the FDA is concentrating on to integrate AI effectively and safely into healthcare:
1. Collaboration for Public Health
The FDA’s first priority is to build partnerships across the healthcare sector to ensure that patient safety remains central as AI technology advances. The FDA collaborates with healthcare providers, developers, researchers, and other regulatory bodies to share knowledge and set clear expectations. By working together, the FDA aims to stay ahead of potential issues that might arise from using AI in healthcare, such as risks related to data privacy, algorithm bias, or over-reliance on AI tools for critical decisions. Through these partnerships, the FDA can create an environment where AI tools are tested and vetted by a broader network of healthcare experts, ensuring that they meet the highest standards of safety and reliability before reaching patients.
2. Developing Standards for AI in Healthcare
The second focus is on setting universal standards for AI technology used in healthcare. Standards provide clear, consistent guidelines for developers to follow, ensuring that all AI tools meet certain safety, quality, and performance benchmarks. By establishing these guidelines, the FDA helps developers understand what is expected from AI tools in different applications, whether it’s analyzing medical images, managing patient records, or assisting in surgical procedures.
For example, standards may specify how an AI tool should be trained to prevent biased outcomes, how it should handle sensitive patient data, or how often it should undergo testing to ensure it remains accurate. By setting these standards, the FDA ensures a baseline of quality and safety across all AI products, fostering trust among patients, healthcare providers, and developers alike.
3. Encouraging Innovation in AI Development
The FDA recognizes that the healthcare sector must embrace new ideas to fully benefit from AI technology. Therefore, its action plan encourages innovation by creating regulatory pathways that are less burdensome for developers, especially small companies and start-ups. This involves building more flexible, risk-based frameworks that allow for faster approval processes for low-risk AI applications while maintaining rigorous testing for higher-risk technologies.
The goal is to enable developers to push boundaries and explore new possibilities with AI—whether it’s predicting disease outbreaks, personalizing treatments, or reducing administrative burdens—without unnecessary regulatory hurdles.
By fostering an environment that welcomes fresh ideas and cutting-edge technology, the FDA aims to accelerate the adoption of AI in ways that can improve healthcare outcomes and streamline clinical processes.
4. Research and Monitoring of AI Tools Over Time
As AI tools are increasingly integrated into healthcare, ongoing monitoring and evaluation are essential to ensure they continue to work as expected in real-world settings. AI systems can adapt or learn from new data, meaning that their performance can change over time. The FDA’s action plan emphasizes the importance of continuous oversight to catch any potential issues early, whether it’s a drop in accuracy, a shift in performance, or unintended bias in results.
For example, an AI tool used in diagnosing heart conditions might need periodic testing to ensure it remains accurate as new data is introduced. By prioritizing ongoing monitoring and research, the FDA can help identify and address issues as they arise, ensuring that AI tools stay safe and effective throughout their lifecycle.
This approach also includes studying the broader impacts of AI in healthcare, such as how it affects clinical workflows, patient experiences, and the doctor-patient relationship. Through these insights, the FDA can better understand the real-world applications and implications of AI, adjusting regulations as necessary to keep pace with technological and clinical advancements.
5. Supporting Transparency and Clear Communication
An essential aspect of the FDA’s action plan is to foster transparency in AI development and usage. This means that AI developers should clearly communicate how their technology works, what data it uses, and what limitations it may have. By promoting transparency, the FDA aims to build trust among healthcare providers and patients, ensuring they understand how an AI tool reaches its conclusions or recommendations. Clear information can help doctors use AI as a support tool rather than a replacement for human judgment, enabling better-informed decisions in patient care.
Transparency also extends to data privacy and patient consent, especially as AI often relies on vast amounts of personal health data to operate effectively. The FDA encourages developers to adopt secure data practices and to be clear about how patient data is protected. Transparent communication helps mitigate privacy concerns and empowers patients to feel more comfortable with AI technologies that involve their personal information.
Aligning with Global Standards
FDA’s regulatory model aligns with U.S. government and international regulatory standards to maintain global compatibility. It co-leads the International Medical Device Regulators Forum’s AI working group and engages in initiatives to modernize data management within clinical trials, crucial for AI inclusion.
As AI evolves rapidly, its regulatory framework must adapt to handle the increasing volume of FDA submissions for AI-enabled products. Effective AI regulation requires science-backed adaptability to mitigate risks and promote beneficial innovation. The FDA’s total product life cycle approach to medical devices allows for the assessment of emerging AI models. Programs like the Software Precertification Pilot illustrate the agency’s openness to flexible pathways for novel technologies, though expanding these pathways may necessitate new legislative powers.
Risk-Based Regulation: Tailoring Oversight Based on AI’s Role
FDA’s risk-based approach for AI in healthcare differentiates between applications based on risk, from administrative AI tools, which are minimally regulated, to clinical decision support and traditional medical devices, which require more stringent oversight. Recent examples include the Sepsis ImmunoScore, which uses AI to assess sepsis risk while ensuring safety through specialized controls, such as ongoing performance monitoring and detailed clinical testing.
AI in Drug Development: Enhancing Research and Clinical Trials
In drug development, the FDA recognizes AI’s potential to enhance numerous facets of medical product development. Every product is rigorously assessed in clinical trials to confirm that benefits exceed risks for its intended use, requiring FDA reviewers to possess AI expertise.
Generative AI and large language models (LLMs), while promising, are complex due to emergent outcomes and potential risks in healthcare applications. Proactive collaboration among developers, clinicians, and regulators is essential to responsibly implement LLMs, especially in high-risk fields like cardiology and oncology.
Continuous Monitoring of AI Performance
AI life cycle management is essential, as AI models are sensitive to operational contexts and require continuous performance monitoring post-deployment. Effective management in clinical settings necessitates health systems capable of tracking AI performance akin to patient monitoring systems in intensive care units. Emerging solutions like external assurance labs or localized validation show promise, yet more diverse approaches are essential to address potential risks.
Responsibility of Regulated Industries
FDA-regulated industries bear the responsibility for compliance with agency standards and ethical AI development practices. Industry-driven studies reviewed by the FDA, alongside voluntary adherence to quality management standards, form the backbone of AI regulation. Ensuring that AI provides tangible health benefits requires local, continuous assessment within health systems, an area where clinical information systems currently fall short. Real-world model performance assessment parallels the rigor of premarket evaluation, necessitating thorough follow-up with patients, especially those facing adverse outcomes.
AI’s evolution presents unique challenges. Many AI models demand ongoing evaluation of operational characteristics, which could strain existing regulatory systems. Unlike traditional medical products, AI’s sensitivity to contextual factors means that robust studies cannot always guarantee universal safety and effectiveness. This complexity underscores the need for vigilant oversight as AI becomes more common.
Supply Chain Management in the Age of AI
Supply chain management is another crucial area, as AI is integral yet potentially vulnerable within complex supply networks. AI can help predict and mitigate shortages in essential supplies, though cybersecurity risks and technology outages must be proactively addressed.
Supporting Innovation Across Diverse Settings
Although big tech companies often lead in AI development, the FDA also supports small businesses and academic research to encourage diversity and safety throughout the AI product life cycle. Initiatives tailored for start-ups and academic institutions help ensure that AI solutions are accessible and safe across various healthcare environments.
Patient Outcomes vs. Financial Returns
Balancing financial incentives with patient care remains a concern. While healthcare institutions may prioritize profitability, the FDA emphasizes public health to prevent financial considerations from compromising patient care. AI has the potential to streamline clinical workflows, freeing clinicians to focus on patient interactions. However, a patient-centric approach is essential to prevent excessive reliance on AI from overshadowing the human connection in healthcare.
The FDA’s mission to protect public health intersects with the collective responsibility of stakeholders across the healthcare landscape. Through collaboration, standards development, continuous monitoring, and support for innovation, the FDA aims to integrate AI safely and effectively into healthcare. As AI continues to evolve, the FDA’s balanced approach will be essential in realizing AI’s potential to transform healthcare without compromising quality, safety, or patient trust.
Are you interested in how AI is changing healthcare? Subscribe to our newsletter, “PulsePoint,” for updates, insights, and trends on AI innovations in healthcare.
