UK panel urges stricter AI oversight in healthcare
The UK’s National Commission into the Regulation of AI in Healthcare has issued 44 recommendations for a new regulatory framework for healthcare AI, including proposals to monitor AI throughout its lifecycle, track AI-related safety incidents, and prepare healthcare organizations to deploy the technology safely. Led by an independent panel convened by the Medicines and Healthcare products Regulatory Agency (MHRA), the report argues that existing regulations—originally crafted for conventional medical devices—do not adequately address the evolving nature of AI systems. These tools often adapt after deployment and behave differently based on input data, clinical workflows, and user interactions, creating risks that standard oversight fails to capture.
The commission’s recommendations focus on three key areas: proportionate lifecycle regulation; system-wide responsibility and safe management; and trust, transparency, and predictability. Unlike traditional devices, AI-enabled medical products frequently undergo updates post-launch, requiring regulatory frameworks that accommodate their dynamic behavior. The proposals emphasize real-world performance tracking and flexible pathways for iterative improvements, shifting focus from pre-market testing alone to ongoing evaluation in clinical settings.
Lifecycle regulation and risk-based oversight
The report demands clearer rules on when an AI product should be regulated as a medical device and how much oversight it should receive, based on potential risk to patients and its intended purpose. Current assessments often treat AI systems as static entities, failing to account for their post-deployment modifications or the manufacturer’s claims about their intended use. This misalignment can create mismatched expectations between developers and end-users, potentially compromising patient safety.
To mitigate these risks, the commission advocates for a phased market entry process for high-risk AI tools. Developers could roll out systems in controlled environments while collecting safety and effectiveness data under regulatory supervision. This approach would maintain oversight throughout the product’s lifecycle, supplemented by continuous monitoring using real-world clinical data to detect performance deviations after deployment.
A significant gap identified by the report is the absence of standardized testing environments where regulators and developers can collaborate before widespread adoption. Such platforms could help preemptively identify hazards, particularly for autonomous AI systems, what the commission refers to as “agentic AI.” These tools, designed to operate with minimal human intervention, introduce novel challenges for ensuring safety and accountability that current frameworks do not address.
Shared responsibility and patient safeguards
The commission insists that AI safety cannot be the sole responsibility of manufacturers. Healthcare providers, clinicians, and policymakers must share accountability, including implementing cybersecurity measures and ensuring staff are properly trained. The recommendations mandate full education programs for professionals at all levels, from initial training to continuous updates, to guarantee they understand both the strengths and limitations of AI tools in their daily practice.
Patients must also have clear visibility into how AI influences their care. The report urges regulators to establish transparent mechanisms for addressing harm caused by AI-related errors, acknowledging that liability for AI in healthcare is complex and evolving. While legal reforms may be needed and are likely to take time, the health system should take practical steps to ensure that the adoption of AI technologies does not weaken patients’ access to redress.
The commission further advocates for broader public engagement in shaping AI regulation. Regular surveys of patients and healthcare workers should inform policy decisions, ensuring their perspectives guide the development of governance frameworks.
International alignment and future challenges
Given the global nature of AI development, the commission recommends that the UK pursue regulatory harmonization with international partners. The MHRA could establish recognition pathways for AI medical devices pre-approved by trusted foreign regulators, streamlining market access while maintaining safety standards. These pathways would include periodic review points to adapt to new risks and support innovation.
Cybersecurity remains a pressing concern, particularly for AI systems embedded in consumer health apps and wearables, which often lack rigorous oversight. The report calls for stricter security requirements to prevent vulnerabilities that could endanger patient safety or compromise data privacy. Such risks could have widespread consequences, extending beyond individual users to entire healthcare ecosystems.
