Safety, Accountability, and Regulation of Artificial Intelligence in Diagnostic Medicine: A Systematic Review

Authors & Affiliations

* Corresponding Author. Author contact details are available in the published PDF article as per journal policy.

Abstract

Importance: The Artificial Intelligence (AI) and machine learning (ML) technologies are majorly utilized in radiology and other clinical decision support systems. However, in spite of the regulatory approval of many AI diagnostic systems, their clinical use has progressed before the development of safety frameworks and regulatory standards. Objectives: To review the existing literature on safety, accountability, and the challenges in the regulation of AI diagnostic systems, and the evidence-based measures that could ensure the safe clinical use of the systems. Evidence Review: A literature search was conducted in PubMed/MEDLINE, Embase, Scopus, and Web of Science databases, utilizing artificial Intelligence, machine learning, diagnostic medicine, safety, regulation, accountability, liability, ethics, and implementation as the keywords. Only the studies that covered clinical AI diagnostic systems and analyzed safety, ethics, legal, and regulatory aspects were considered. After a review by two independent reviewers, fifty-three studies were included. The grading of evidence was performed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework. Findings: This systematic review of 53 studies identified empirically documented patient harm arising from algorithmic bias, producing measurable health inequities in deployed commercial systems; explainability deficits that prevented clinicians from exercising independent judgment; legal liability ambiguity that left 89% of jurisdictions without clear accountability frameworks; regulatory oversight failures, as no major jurisdictions had implemented mandatory post-market surveillance for adaptive AI; data privacy violations in cross-border deployments; and implementation barriers that disrupted clinical workflow in 73% of deployment studies. Effective safeguards, including obligatory human-in-the-loop oversight, demographically representative training datasets, specific explainability requirements, explicit institutional liability frameworks, and harmonized post-market surveillance, were identified but implemented inconsistently. Conclusions and Relevance: The AI-assisted diagnosis tools produce documented patient harm through algorithmic bias and unsolved errors. The study found that AI safety is primarily a governance failure and not a technical problem. The compulsory pre-deployment audit of bias, explainability standards modified to clinical stakes, explicit legal accountability frameworks, and harmonized international post-market surveillance are not mere aspirational goals but requirements for preventing foreseeable harm. A clinical deployment without these safeguards constitutes a predictable patient safety risk.

Keywords & Indexing Terms

Download Center

Indexing & Verification

G Google Scholar Check indexing visibility by DOI Crossref / DOI Verify permanent DOI record ORCID Author identifier support Open Access Free reader access Indexing Ready Scholar metadata compatible OAI-PMH Metadata harvesting ready Article Metrics Views & download analytics

Article Metrics

Metrics are indicative and may update periodically after indexing, downloads and citation tracking are enabled.

Article Integrity & Transparency

License & Copyright

Copyright © 2026 UnivColl International Multidisciplinary Research Journal. This work is licensed under a Creative Commons Attribution 4.0 International License . Authors retain copyright and grant the journal the right of first publication.

How to Cite

PDF Preview

If the PDF preview does not load on your device, open the PDF directly or use the Download PDF button. Open PDF in new tab

Declarations

Publisher's Note

References

Showing first 3 references. Click “Show All References” to view complete list.

1. Duggal M, Chauhan A, Gupta V, et al. Real-world evaluation of AI-driven diabetic retinopathy screening in public health settings: validation and implementation study. JMIR Medical Informatics. 2025;13(2):1-9. doi:10.2196/67529
2. Guo L, Zhou C, Xu J, Huang C, Yu Y, Lu G. Deep learning for chest X-ray diagnosis: competition between radiologists with or without Artificial Intelligence Assistance. Journal of Imaging Informatics in Medicine. 2024;37(3):922-934. doi:10.1007/s10278-024-00990-6
3. Nadour N, Duguet T, Zahedi S, Figoni H, Liard R. Diagnostic accuracy of artificial Intelligence compared to family physicians and dermatologists for skin conditions: a systematic review and meta-analysis. BMC Primary Care. 2025;26(1):1-9. doi:10.1186/s12875-025-03073-9
4. Ogut E. Artificial Intelligence in clinical medicine: challenges across diagnostic imaging, clinical decision support, surgery, pathology, and drug discovery. Clinics and Practice. 2025;15(9):169-182. doi:10.3390/clinpract15090169
5. Binder D, Bourgeois M. Direct and indirect effects of group discussion on consensus. Social Influence. 2026;1(4):249-264. doi:10.1080/15534510600867098
6. Ardic N, Dinc R. Artificial Intelligence in healthcare: current regulatory landscape and future directions. British Journal of Hospital Medicine. 2025;86(8):1-21. doi:10.12968/hmed.2024.0972
7. Cajueiro D, Celestino V. A comprehensive review of artificial intelligence regulation: weighing ethical principles and innovation. Journal of Economy and Technology. 2026;4(2):77-91. doi:10.1016/j.ject.2025.07.001
8. Liu F, Niu Y, Zhang Q, et al. A foundational architecture for AI agents in healthcare. Cell Reports Medicine. 2025;6(10):10-23. doi:10.1016/j.xcrm.2025.102374
9. Owens K, Walker A. Those designing healthcare algorithms must become actively anti-racist. Nature Medicine. 2020;26(9):1327-1328. doi:10.1038/s41591-020-1020-3
10. Jia L-L, Zhao J-X, Pan N-N, et al. Artificial intelligence model on chest imaging to diagnose COVID-19 and other pneumonias: a systematic review and meta-analysis. European Journal of Radiology Open. 2022;9(4):1-9. doi:10.1016/j.ejro.2022.100438
11. Suleman MU, Mursaleen M, Khalil U, et al. Assessing the generalizability of Artificial Intelligence in radiology: a systematic review of performance across different clinical settings. Annals of Medicine & Surgery. 2025;87(12):8803-8811. doi:10.1097/ms9.0000000000004166
12. Page MJ, Moher D, Bossuyt PM, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021;13(4):1-9. doi:10.1136/bmj.n160
13. Page M, Shamseer L, Tricco A. Registration of systematic reviews in Prospero: 30,000 records and counting. Systematic Reviews. 2018;7(1):1-9. doi:10.1186/s13643-018-0699-4
14. Gusenbauer M, Gauster S. How to search for literature in systematic reviews and meta-analyses: a comprehensive step-by-step guide. Technological Forecasting and Social Change. 2025;212(4):12-38. doi:10.1016/j.techfore.2024.123833
15. Valizadeh A, Moassefi M, Nakhostin-Ansari A, et al. Abstract screening using the automated tool Rayyan: results of effectiveness in three diagnostic test accuracy systematic reviews. BMC Medical Research Methodology. 2022;22(1):1-9. doi:10.1186/s12874-022-01631-8
16. Almarie B, Gonzalez-Gonzalez LF, dos Santos Barbosa LA, Lutz A, Grosse U, Fregni F. Machine learning-enabled medical devices authorized by the US Food and Drug Administration in 2024: regulatory characteristics, predicate lineage, and transparency reporting. Biomedicines. 2025;13(12):1-9. doi:10.3390/biomedicines13123005
17. Mitani AA, Freer PE, Nelson KP. Summary measures of agreement and association between many raters’ ordinal classifications. Annals of Epidemiology. 2020;27(10):1-9. doi:10.1016/j.annepidem.2017.09.001
18. Choi G, Kang H. Heterogeneity in meta-analyses: an unavoidable challenge worth exploring. Korean Journal of Anesthesiology. 2025;78(4):301-314. doi:10.4097/kja.25001
19. Stoll CRT, Izadi S, Fowler S, Green P, Suls J, Colditz GA. The value of a second reviewer for study selection in systematic reviews. Research Synthesis Methods. 2019;10(4):539-545. doi:10.1002/jrsm.1369
20. He Z, Yang L, Li X, Du J. Discrepancies in reported results between trial registries and journal articles for AI clinical research. eClinicalMedicine. 2025;80(2):1-9. doi:10.1016/j.eclinm.2024.103066
21. Alqattan H, Morrison Z, Cleland JA. A narrative synthesis of qualitative studies conducted to assess patient safety culture in hospital settings. Sultan Qaboos University Medical Journal. 2019;19(2):1-9. doi:10.18295/squmj.2019.19.02.002
22. Iseko A. Diversity as ethical infrastructure: reimagining AI governance for justice and accountability. International Journal of Science, Technology and Society. 2025;13(5):190-204. doi:10.11648/j.ijsts.20251305.13
23. Abbott EE, Rehman T, Rosania A, et al. Understanding and addressing bias in artificial intelligence systems: a primer for the emergency medicine physician. JACEP Open. 2025;7(1):10-32. doi:10.1016/j.acepjo.2025.100311
24. Ahmad A, Vallès Y, Idaghdour Y. Bias in AI systems: integrating formal and socio-technical approaches. Frontiers in Big Data. 2025;8(6):1-9. doi:10.3389/fdata.2025.1686452
25. Goktas P, Grzybowski A. Shaping the future of healthcare: ethical clinical challenges and pathways to trustworthy AI. Journal of Clinical Medicine. 2025;14(5):1-12. doi:10.3390/jcm14051605
26. Di Palma G, Scendoni R, Ferorelli D, De Benedictis A, Tambone V, De Micco F. AI-induced cybersecurity risks in healthcare: a narrative review of blockchain-based solutions within a clinical risk management framework. Risk Management and Healthcare Policy. 2025;Volume 18(2):3479-3497. doi:10.2147/rmhp.s544523
27. Algarni A, Thayananthan V. Cybersecurity for analyzing Artificial Intelligence (AI)-based assistive technology and systems in digital health. Systems. 2025;13(6):439-452. doi:10.3390/systems13060439
28. Bernstein MH, Atalay MK, Dibble EH, et al. Can incorrect Artificial Intelligence (AI) results impact radiologists, and if so, what can we do about it? A multi-reader pilot study of lung cancer detection with chest radiography. European Radiology. 2023;33(11):8263-8269. doi:10.1007/s00330-023-09747-1
29. Kocak B, Klontzas ME, Stanzione A, et al. Evaluation metrics in medical imaging AI: fundamentals, pitfalls, misapplications, and recommendations. European Journal of Radiology Artificial Intelligence. 2025;3(2):1-22. doi:10.1016/j.ejrai.2025.100030
30. Obuchowicz R, Lasek J, Wodziński M, Piórkowski A, Strzelecki M, Nurzynska K. Artificial Intelligence-empowered radiology—current status and critical review. Diagnostics. 2025;15(3):282-291. doi:10.3390/diagnostics15030282
31. Gaffney A, Himmelstein DU, Woolhandler S. Covid-19 and US health financing: perils and possibilities. International Journal of Health Services. 2020;50(4):396-407. doi:10.1177/0020731420931431
32. Glocker B, Jones C, Roschewitz M, Winzeck S. Risk of bias in chest radiography deep learning foundation models. Radiology: Artificial Intelligence. 2023;5(6):1-9. doi:10.1148/ryai.230060
33. Cherezov D, Fu P, Madabhushi A. Quantitative assessment of impact of technical and population-based factors on fairness of AI models for chest X-ray scans. Computers in Biology and Medicine. 2025;198(4):1-22. doi:10.1016/j.compbiomed.2025.111147
34. Ramadan OM, Alruwaili MM, Alruwaili AN, Elsehrawy MG, Alanazi S. Facilitators and barriers to AI adoption in nursing practice: a qualitative study of Registered Nurses’ perspectives. BMC Nursing. 2024;23(1):1-9. doi:10.1186/s12912-024-02571-y
35. Sanchez P, Del Ser J, van Gils M, Hernesniemi J. A design framework for operationalizing trustworthy artificial intelligence in healthcare: requirements, trade-offs and challenges for its clinical adoption. BMC Nursing. 2025;13(6):1-9. doi:10.2139/ssrn.5249603
36. Rosenbacke R, Melhus Å, McKee M, Stuckler D. How explainable artificial intelligence can increase or decrease clinicians’ trust in AI applications in health care: systematic review. JMIR AI. 2024;3(2):1-12. doi:10.2196/53207
37. Ghassemi M, Oakden-Rayner L, Beam AL. The false hope of current approaches to explainable Artificial Intelligence in health care. The Lancet Digital Health. 2021;3(11):1-21. doi:10.1016/s2589-7500(21)00208-9
38. Zhang J, Chao H, Dasegowda G, Wang G, Kalra MK, Yan P. Revisiting the trustworthiness of saliency methods in radiology AI. Radiology: Artificial Intelligence. 2024;6(1):1-9. doi:10.1148/ryai.220221
39. Nazir S, Dickson DM, Akram MU. Survey of explainable artificial intelligence techniques for biomedical imaging with deep neural networks. Computers in Biology and Medicine. 2023;156(2):10-36. doi:10.1016/j.compbiomed.2023.106668
40. Cestonaro C, Delicati A, Marcante B, Caenazzo L, Tozzo P. Defining medical liability when Artificial Intelligence is applied to diagnostic algorithms: a systematic review. Frontiers in Medicine. 2023;10(9):1-21. doi:10.3389/fmed.2023.1305756
41. Nji N. Navigating liability in automated healthcare: designing multi-stakeholder framework for responsibility in AI-powered health care systems. International Journal of Research and Innovation in Social Science. 2025;IX(VIII):5726-5734. doi:10.47772/ijriss.2025.908000466
42. Smith H. Artificial Intelligence for clinical decision-making: gross negligence manslaughter and corporate manslaughter. The New Bioethics. 2024;30(3):228-242. doi:10.1080/20502877.2024.2416862
43. Park H. Patient perspectives on informed consent for medical AI: a web-based experiment. DIGITAL HEALTH. 2024;10(2):1-9. doi:10.1177/20552076241247938
44. Vallenas N, Parent B. Informed consent in the age of value-laden medical AI. Journal of Ethics in Entrepreneurship and Technology. 2025;13(2):1-12. doi:10.1108/jeet-06-2025-0044
45. Hurley ME, Lang BH, Kostick-Quenet KM, Smith JN, Blumenthal J. Patient consent and the right to notice and explanation of AI systems used in health care. The American Journal of Bioethics. 2024;25(3):102-114. doi:10.1080/15265161.2024.2399828
46. Kuzior A. Navigating AI regulation: a comparative analysis of EU and US legal frameworks. Materials Research Proceedings. 2024;45(2):258-266. doi:10.21741/9781644903315-30
47. Carvalho E, Mascarenhas M, Pinheiro F, et al. Predetermined change control plans: guiding principles for advancing safe, effective, and high-quality AI-ML technologies. JMIR AI. 2025;4(2):1-9. doi:10.2196/76854
48. Mehta V, Komanduri A, Bhadouriya RS, et al. Evaluating transparency in AI/ML model characteristics for FDA-reviewed medical devices. npj Digital Medicine. 2025;8(1):1-9. doi:10.1038/s41746-025-02052-9
49. Aboy M, Minssen T, Vayena E. Navigating the EU AI Act: implications for regulated digital medical products. npj Digital Medicine. 2024;7(1):1-9. doi:10.1038/s41746-024-01232-3
50. Bignami E, Russo M, Semeraro F, Bellini V. Balancing innovation and control: the European Union AI Act in an ERA of global uncertainty. JMIR AI. 2025;4(2):10-19. doi:10.2196/75527
51. Malgieri G. Automated decision-making in the EU member states: the right to explanation and other “suitable safeguards” in the national legislations. Computer Law & Security Review. 2019;35(5):1-9. doi:10.1016/j.clsr.2019.05.002
52. Holzinger A, Zatloukal K, Müller H. Is human oversight to AI systems still possible? New Biotechnology. 2025;85(4):59-62. doi:10.1016/j.nbt.2024.12.003
53. Mennella C, Maniscalco U, De Pietro G, Esposito M. Ethical and regulatory challenges of AI technologies in healthcare: a narrative review. Heliyon. 2024;10(4):1-9. doi:10.1016/j.heliyon.2024.e26297
54. Kim MO, Coiera E, Magrabi F. Problems with health information technology and their effects on care delivery and patient outcomes: a systematic review. Journal of the American Medical Informatics Association. 2019;24(2):246-250. doi:10.1093/jamia/ocw154
55. Hadian M, jabbari KM, Mohammadi M, Jafari A. AI-driven innovations in NCD management: challenges and solutions based on expert perspectives in Iran’s health system. BMC Nursing. 2025;13(2):1-12. doi:10.21203/rs.3.rs-7225629/v1
56. Stogiannos N, Cuocolo R, Akinci D’Antonoli T, et al. Recognising errors in AI implementation in radiology: a narrative review. European Journal of Radiology. 2025;191(2):11-23. doi:10.1016/j.ejrad.2025.112311
57. McGaughey J, Fergusson DA, Van Bogaert P, Rose L. Early warning systems and rapid response systems for the prevention of patient deterioration on Acute Adult Hospital wards. Cochrane Database of Systematic Reviews. 2021;20(1):1-9. doi:10.1002/14651858.cd005529.pub3
58. Zhou SK, Greenspan H, Davatzikos C, et al. A review of deep learning in medical imaging: imaging traits, technology trends, case studies with progress highlights, and future promises. Proceedings of the IEEE. 2021;109(5):820-838. doi:10.1109/jproc.2021.3054390
59. Singh V, Cheng S, Kwan AC, Ebinger J. United States Food and Drug Administration regulation of clinical software in the era of Artificial Intelligence and Machine Learning. Mayo Clinic Proceedings: Digital Health. 2025;3(3):10-23. doi:10.1016/j.mcpdig.2025.100231
60. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. npj Digital Medicine. 2020;3(1):1-9. doi:10.1038/s41746-020-0221-y
61. Pietrobon R, Machiavelli A, Paulsen Rodrigues L, et al. The bias assessment method: evaluating racial inequities in clinical risk prediction models (motivated by Obermeyer et al.’s analysis of cost-driven labeling in healthcare algorithms). SSRN Electronic Journal. 2025;13(2):10-21. doi:10.2139/ssrn.5861522
62. Aldhafeeri F. Governing Artificial Intelligence in radiology: a systematic review of ethical, legal, and regulatory frameworks. Diagnostics. 2025;15(18):1-22. doi:10.3390/diagnostics15182300
63. Ekundayo F, Nyavor H. Ai-driven predictive analytics in cardiovascular diseases: integrating big data and machine learning for early diagnosis and risk prediction. International Journal of Research Publication and Reviews. 2024;5(12):1240-1256. doi:10.55248/gengpi.5.1224.3437
64. Neri E, Aghakhanyan G, Zerunian M, et al. Explainable AI in radiology: a white paper of the Italian Society of Medical and Interventional Radiology. La radiologia medica. 2023;128(6):755-764. doi:10.1007/s11547-023-01634-5
65. Chen JH, Asch SM. Machine learning and prediction in medicine — beyond the peak of inflated expectations. New England Journal of Medicine. 2022;376(26):2507-2509. doi:10.1056/nejmp1702071
66. Wachter S, Mittelstadt B, Floridi L. Why a right to explanation of automated decision-making does not exist in the General Data Protection Regulation. International Data Privacy Law. 2020;7(2):76-99. doi:10.1093/idpl/ipx005
67. Gu X, Zhou Y, Zhao J, et al. Diagnostic performance and generalizability of Deep Learning for multiple retinal diseases using bimodal imaging of fundus photography and optical coherence tomography. Frontiers in Cell and Developmental Biology. 2025;13(4):1-9. doi:10.3389/fcell.2025.1665173
68. Garcia Santa Cruz B, Bossa MN, Sölter J, Husch AD. Public COVID-19 X-ray datasets and their impact on model bias – a systematic review of a significant problem. Medical Image Analysis. 2021;74(2):10-22. doi:10.1016/j.media.2021.102225
69. Amann J, Blasimme A, Vayena E, Frey D, Madai VI. Explainability for Artificial Intelligence in healthcare: a multidisciplinary perspective. BMC Medical Informatics and Decision Making. 2020;20(1):1-9. doi:10.1186/s12911-020-01332-6
70. Ahmed MI, Spooner B, Isherwood J, Lane M, Orrock E, Dennison A. A systematic review of the barriers to the implementation of Artificial Intelligence in healthcare. Cureus. 2023;13(2):9. doi:10.7759/cureus.46454
71. Bailo P, Nittari G, Pesel G, Basello E, Spasari T, Ricci G. Governing healthcare AI in the real world: how fairness, transparency, and human oversight can coexist: a narrative review. Sci. 2025;8(2):36-45. doi:10.3390/sci8020036
72. Hanna MG, Pantanowitz L, Jackson B, et al. Ethical and bias considerations in Artificial Intelligence/Machine Learning. Modern Pathology. 2025;38(3):1-12. doi:10.1016/j.modpat.2024.100686
73. Blackman J, Veerapen R. On the practical, ethical, and legal necessity of clinical Artificial Intelligence explainability: an examination of key arguments. BMC Medical Informatics and Decision Making. 2025;25(1):1-9. doi:10.1186/s12911-025-02891-2
74. Beauchemin M, Cohn E, Shelton RC. Implementation of clinical practice guidelines in the health care setting. Advances in Nursing Science. 2019;42(4):307-324. doi:10.1097/ans.0000000000000263
75. Chee ML, Ong ME, Siddiqui FJ, et al. Artificial intelligence applications for covid-19 in intensive care and emergency settings: a systematic review. SSRN Electronic Journal. 2021;13(2):1-23. doi:10.1101/2021.02.15.21251727
76. Subasri V, Krishnan A, Kore A, et al. Detecting and remediating harmful data shifts for the responsible deployment of clinical AI models. JAMA Network Open. 2025;8(6):1-12. doi:10.1001/jamanetworkopen.2025.13685
77. Mann S, Berdahl CT, Baker L, Girosi F. Artificial intelligence applications used in the clinical response to COVID-19: a scoping review. PLOS Digital Health. 2022;1(10):1-9. doi:10.1371/journal.pdig.0000132

Related Articles