Published on This Week in AI

AI in Healthcare: Legal and Regulatory Considerations

Artificial intelligence applications in healthcare represent both significant promise and real legal complexity. From diagnostic imaging analysis to clinical decision support to administrative automation, AI is being integrated into care delivery in ways that raise questions about regulatory classification, liability, and patient rights.

FDA Regulation of Software as Medical Device

The FDA regulates AI/ML-based software that meets the definition of a medical device under the Federal Food, Drug, and Cosmetic Act. Software that provides clinical decision support and influences clinical decisions is subject to FDA oversight; purely administrative software is not. The FDA's Digital Health Center of Excellence has published guidance on its approach to AI/ML-based software as a medical device (SaMD), including a framework for managing changes to algorithms after deployment.

Clinical Liability Questions

When an AI system contributes to a clinical error, liability questions become complex. Physicians remain responsible for clinical decisions even when assisted by AI tools — the learned intermediary doctrine and professional standard of care still apply. But manufacturers of defective AI tools may face product liability claims if the tool causes harm. The interplay of professional liability and products liability in AI-assisted medicine remains unsettled in most jurisdictions.

Patient Privacy

Healthcare AI systems are trained on patient data, raising HIPAA compliance questions. Business associate agreements are required for AI vendors with access to protected health information. De-identification of training data must meet the standards set out in HIPAA's Privacy Rule — either expert determination or safe harbor methods. Patients generally have no right under HIPAA to object to uses of their de-identified data.

FDA Regulatory Framework for AI/ML-Based Medical Devices

In the United States, AI systems used in clinical decision support, diagnostic imaging, and patient monitoring typically qualify as medical devices subject to FDA oversight under the Federal Food, Drug, and Cosmetic Act. The FDA has taken a risk-based approach: software that merely organises information for clinician review may fall outside device regulation, while software that provides treatment recommendations or analyses medical images for diagnostic conclusions typically requires premarket review. The FDA's AI/ML Action Plan and its proposed framework for "predetermined change control plans" attempt to address the unique challenge of AI systems that continue to learn after deployment — a feature that has no analogue in traditional device regulation.

HIPAA and AI-Processed Health Data

The Health Insurance Portability and Accountability Act imposes strict obligations on covered entities and their business associates regarding the use and disclosure of protected health information (PHI). AI systems trained on patient data or processing PHI in clinical settings are subject to HIPAA's privacy and security rules. Particular issues arise around: using patient data to train AI models (whether this constitutes a permissible treatment, payment, or operations use, or requires authorisation); deploying third-party AI tools that receive PHI as business associates; and ensuring AI-generated outputs containing patient information are protected appropriately. The HHS Office for Civil Rights has begun issuing guidance specifically addressing AI and HIPAA, and enforcement actions in this space are expected to increase.

Liability When AI Systems Cause Harm

Allocating liability when an AI diagnostic or treatment recommendation leads to patient harm is one of the most contested issues in health AI law. Potential defendants may include: the healthcare provider that deployed the system and relied on its output; the manufacturer of the AI device under products liability theory; the hospital or health system that procured and implemented the system; and, in some frameworks, the developers of the underlying model. Courts have not yet developed settled doctrine on how traditional medical malpractice standards apply when a clinician relies on an AI recommendation — whether following AI guidance constitutes reasonable professional practice or creates a separate liability exposure is currently unresolved.

The EU AI Act and High-Risk Healthcare AI

The EU AI Act (Regulation 2024/1689) classifies AI systems used as safety components of medical devices, or AI systems themselves constituting medical devices, as high-risk under Annex III. High-risk healthcare AI systems face the Act's most demanding requirements: conformity assessments, technical documentation, human oversight obligations, transparency requirements, and registration in the EU database. The Act's requirements overlap with and supplement the EU Medical Devices Regulation (MDR), creating a layered compliance framework that healthcare AI developers and deployers must navigate simultaneously. The provisions for high-risk AI took effect in August 2026.

Algorithmic Bias and Health Equity

A growing body of research documents that AI systems trained predominantly on data from certain patient populations may perform less accurately for others — with documented disparities in diagnostic AI performance across racial and ethnic groups, sex, age, and socioeconomic status. The legal dimensions of algorithmic bias in healthcare are developing rapidly: the HHS Office for Civil Rights has issued guidance on how civil rights laws — including Section 1557 of the Affordable Care Act, which prohibits discrimination in covered health programmes — apply to the use of AI tools that produce disparate outcomes. Healthcare organisations deploying AI face an obligation to evaluate and monitor for algorithmic bias as part of their civil rights compliance.