The landscape of clinical artificial intelligence is currently characterized by both rapid innovation and escalating concerns regarding patient safety and ethical governance. Recent studies have brought to light significant risks associated with the deployment of AI in healthcare, particularly large language models (LLMs). Researchers from Stanford and Harvard, in their NOHARM study, evaluated 31 LLMs and found that these systems made severely harmful clinical recommendations in 11.8% to 14.6% of cases, with some models exceeding 40 severe errors per 100 encounters. A staggering 76.6% of these errors were due to omissions, such as failing to recommend critical tests or treatments, raising urgent questions about patient safety as these tools integrate into clinical workflows. Similarly, a Nature Digital Medicine study on an LLM-powered clinical decision support system in African primary care settings identified potentially harmful outputs in 7.8% of patient encounters, predominantly inappropriate medication recommendations and omitted differential diagnoses. These findings are echoed in the 2026 State of Clinical AI Report, which warns of severe harm risk and automation bias, noting that physicians exposed to erroneous LLM advice can show degraded diagnostic accuracy. Beyond direct errors, the issue of bias remains a critical threat to equitable and safe clinical AI. A PLOS Digital Health review meticulously outlines how bias can permeate every stage of medical AI development and deployment—from data collection and feature selection to model evaluation and implementation. The authors emphasize that imbalanced samples, missing social determinants of health, and provider bias embedded in labels can distort clinical decision support, undermine patient safety, and exacerbate existing health disparities, especially for marginalized populations. This necessitates active auditing and mitigation strategies throughout the AI lifecycle. The "Illusion of Safety" report to the FDA reinforces these concerns, advocating for stringent oversight, including mandatory post-market monitoring, transparency regarding training data, and rigorous subgroup performance evaluation across diverse demographics to ensure fairness and safety. Regulatory bodies are actively responding to these complex challenges. The FDA has updated its guidance on AI and medical products, emphasizing a collaborative approach with stakeholders and a clearer framework for evaluating AI-enabled tools over time. While the agency also signaled a more hands-off stance for certain digital health products in January 2026, potentially broadening commercialization pathways, the overall focus remains on safety, effectiveness, and lifecycle monitoring. The World Health Organization (WHO) has contributed significantly to global governance with its new ethics and governance framework for large multimodal models in health, offering 40 recommendations to ensure that powerful clinical AI systems augment, rather than undermine, safety, equity, and human oversight. At the national and sub-national levels, legislative and policy activities are accelerating. The Health AI Policy Tracker reports that over 40 state bills across 25 states have been introduced in 2026 to regulate AI in clinical settings, with common themes including clinician oversight and patient disclosure. Federal activity includes an ARPA-H initiative to develop the first FDA-authorized agentic AI system for clinical care. Furthermore, the FDA finalized its Clinical Decision Support software guidance in January 2026 and granted a first Breakthrough Device Designation for a patient-facing generative AI device in March 2026, indicating a maturing regulatory environment. Amidst these safety concerns, the potential benefits of AI in healthcare continue to emerge. AI assistants are demonstrating utility in improving clinician workflow efficiency, with reports indicating a 38% reduction in visit preparation time and a 23% decrease in self-reported burnout. AI models are also showing promise in predicting cardiovascular and cancer outcomes, and a new AI-powered tool can diagnose deadly rheumatic heart disease early, particularly valuable in resource-limited settings. In mental health, while concerns about safety, privacy, and the potential for "AI psychosis" persist, generative AI chatbots are being widely adopted, with some meta-analyses suggesting a small-to-moderate but statistically significant reduction in depression and anxiety symptoms. From the LOG Standards perspective, these developments underscore the critical need for robust, independent accreditation frameworks that complement regulatory oversight. The findings highlight that while AI offers transformative potential, its deployment without stringent validation, continuous post-market monitoring, and clear bias mitigation strategies poses unacceptable risks to patient safety and health equity. The emphasis from various bodies on transparency, subgroup performance evaluation, and human oversight aligns directly with LOG Standards' core tenets, advocating for comprehensive governance to ensure that clinical AI systems are not only innovative but also consistently safe, effective, and ethically sound.
