Risk Management Guidance for Medical Devices Incorporating Machine Learning (ML) or Artificial Intelligence (AI)
by Lezlie Hynes
Software continues to play an expanding role in both medical devices (SiMD) and standalone medical devices (SaMD). The latest versions of ISO 14971:2019 Medical devices – Application of risk management to medical devices and ISO/TR 24971:2020 Medical devices – Guidance on the application of ISO 14971 place increased emphasis on software-related risk management, particularly in cybersecurity.
To address the unique challenges of identifying, assessing, and mitigating risks for AI/ML-enabled medical devices, the Association for the Advancement of Medical Instrumentation (AAMI) and the British Standards Institution (BSI) collaborated on a Technical Information Report (TIR), AAMI TIR 34971:2023 Application of ISO 14971 to machine learning in artificial intelligence – Guide. This guidance document is intended to serve as a companion to ANSI/AAMI/ISO 14971:2019.
Considerations for AI/ML Medical Devices
While foundational risk management processes are covered under ISO 14971:2019, AI/ML-enabled devices present unique safety-related hazards that need to be addressed.
Data Management – Data sets used in the development, training, and testing of algorithms need to be accurate, complete, and consistent.
Bias in Data – The population affected by the device needs to be clearly defined and understood to reduce bias that could be incorporated and adversely affect the use and performance of the device.
Data Storage, Security, and Privacy – Areas of vulnerability need to be assessed to prevent malicious attacks and the potential loss or corruption of data.
Over-Trust of Data – An over-reliance on technology can lead to incorrect or misuse of the information provided by the device due to a change in circumstances and potential data drifts.
Example from AAMI TIR34971:2023:
“during the 2017 wildfires in California, the Los Angeles Police Department had to ask drivers to avoid navigation apps, because they were steering users onto less trafficked roads, but these routes were in neighborhoods that were on fire.”
Adaptive Systems – The performance of AI/ML models that continuously learn from new data after they are deployed (Machine Learning) can change depending on the quality of the new data incorporated. Incorporating high-quality data may improve performance over time, while low-quality data may negatively impact the performance of the device.
Upcoming Updates to AAMI TIR34971
At the November 2024 AI Summit held in Cincinnati, OH, it was announced that AAMI TIR 34971:2023 will be updated to include risks related to Large Language Models (LLM) and additional clarifications. The ISO/IEC parent committee also intends to rename the document 24971-2.
Stay Vigilant As Expectations Evolve
As AI/ML-enabled technology continues to evolve, medical device manufacturers must proactively assess and mitigate risks to align with regulatory expectations and ensure patient safety. Following guidance from AAMI TIR34971:2023 and its upcoming updates can help manufacturers maintain compliance and build safer AI-driven devices.
Why MDC Associates?
- Accelerated Path to Market: Over 35 years of IVD expertise across a broad range of technologies and applications, guiding you through regulatory and clinical processes efficiently.
- Regulatory Expertise with Continuous FDA Engagement: Our established, positive relationship with the FDA helps you navigate submissions like Pre-subs, 510(k), De Novo, and PMAs smoothly.
- Practical Quality Expertise: We help you achieve your quality objectives without overwhelming your team with too much too soon. From eQMS to MDSAP and ISO audits, get expert advice and hands-on support.
- Extensive Clinical Site Network: Access a trusted network experienced in supporting high-complexity assays, CLIA-waived tests, and OTC diagnostics, providing fast and reliable performance validation.
- Your Trusted IVD CRO: Beyond strategy, we work as part of your team to execute critical steps in regulatory submissions, clinical trials, and quality systems compliance.
