En 2020, une analyse a révélé que 70 % des systèmes d’IA diagnostique basés sur des données visuelles s'appuient sur des données provenant de seulement trois États américains. Ce manque de diversité pose un vrai problème dans le secteur des technologies de santé.
Avec l’évolution des exigences en matière d’étiquetage des données médicales, la Food and Drug Administration (ou FDA) a pris des mesures concrètes. En janvier 2021, elle a publié son "AI/ML-Based Software as a Medical Device Action Plan", établissant pour la première fois un cadre fédéral aux Etats-Unis, pour réguler l’IA et le Machine Learning dans les dispositifs médicaux.
To ensure that these technologies remain safe and effective, for patients and healthcare professionals alike, it is essential to understand why data labeling is an important link in the chain, and what standards need to be met.
💡 In this guide, we'll look in detail at the FDA's requirements, its approach to using training data for AI, and the key steps to implementing an FDA-compliant labeling process within your organization.
-hand-innv.png)
The fundamentals of medical data labelling
Dans le domaine médical, nous sommes confrontés à un défi majeur : 80% des données de santé ne sont pas structurées, ce qui les rend difficiles à exploiter. C'est précisément là que le Data Labeling ou la mise en place d'un processus LabelOps ou DataPrepOps médical entre en jeu.
Medical Data Labeling is the meticulous process of annotating images, videos and other medical data with precise, relevant information (also known as "metadonenes"). Indeed, this practice enables AI algorithms to understand and interpret medical images, in particular to identify :
- Anatomical structures
- Specific pathologies
- Potential anomalies
- Important clinical signs
To ensure the accuracy of medical diagnoses, data annotation specialists (such as Innovatiana) use various annotation methods, including :
- Bounding boxes to delimit areas of interest
- Polygons to mark precise contours
- Benchmarks for identifying specific structures
In addition, the quality of labeling is crucial, as it directly impacts the performance of artificial intelligence models, and therefore diagnostics. At Innovatiana, we call on experts in the medical field who can accurately identify symptoms, diseases and treatments. These specialists use specialized software such as V7, Encord, RadiAnt DICOM Viewer or OsiriX to guarantee annotations in line with clinical standards.
FDA regulatory requirements for Medical Data Labeling
To ensure data labeling compliance in medical artificial intelligence, it is essential to follow the FDA's strict guidelines for labeling and data management. The aim is to ensure that datasets used to train AI models comply with safety, traceability and quality requirements applicable to medical devices.
📌 Classification of medical devices and impact on Data Labeling
The FDA classifies medical devices into three categories according to their level of risk, which directly influences the level of Data Labeling requirements:
- Class I(low risk): General controls, suitable for non-critical annotation systems (e.g. diagnostic tools without automated decision-making).
- Class II(moderate risk): Stricter requirements, requiring rigorous validation of the annotations used to train AI models (e.g. automated detection of anomalies on medical images).
- Class III(high risk): Requires prior FDA approval, as these systems can have a direct impact on medical decision-making (e.g. AI interpreting scans to diagnose serious pathologies).
✅ Implementation of a quality assurance program for Data Labeling
The FDA requires industry players to set up a quality assurance program incorporating good practices specific to Data Labeling, including:
- Data annotation and structuring standards : Compliance with DICOM (medical imaging), HL7 FHIR (data interoperability) and GxP (good manufacturing and data management practices).
- Annotation validation: Rigorous process including cross-reviews by medical experts and AI specialists to ensure the accuracy of applied labels.
- Full documentation and traceability: All Data Labeling stages must be recorded in a validation file to prove compliance with regulatory requirements.
📑 Controls and audits to ensure compliance
To meet FDA standards, the Data Labeling process must include strict controls, including :
- Vérification systématique de la cohérence des annotations via des métriques de qualité (ex. inter-annotateur agreement, analyses statistiques).
- Recording changes : Any changes to labels or annotation algorithms must be documented and approved before implementation.
- Keeping version histories: Each dataset used to train a model must be stored with clear versioning to trace the origin and evolution of annotations.
🔍 FDA validation and acceptance of datasets
Before using an annotated dataset to train medical AI, the FDA requires a formal evaluation guaranteeing that it meets quality and safety criteria. This validation includes:
- Performance tests on annotated samples to ensure model robustness.
- Checks for compliance with FDA standards and guidelines on AI/ML-Based Software as a Medical Device.
- Audit of Data Labeling methods to avoid bias and guarantee data representativeness.
💡 By integrating these regulatory requirements right from the start of the Data Labeling process, we contribute to the compliance of medical AI models and facilitate their approval by the FDA, ensuring reliable and safe solutions for patients and healthcare professionals.
Setting up a compliant Labeling process
To ensure Data Labeling complies with regulatory requirements in the development of medical AI, it is essential to structure a rigorous process integrating both human expertise and advanced technological tools.
La première étape consiste à définir des protocoles d’annotation précis, alignés avec les recommandations de la FDA et les standards de l’industrie, tels que DICOM pour les images médicales et HL7 FHIR pour l’échange de données. Nous utilisons des plateformes d’annotation spécialisées comme V7 ou Encord qui permettent d’assurer une cohérence et une qualité élevée grâce à des validations croisées par des experts médicaux et des spécialistes en machine learning.
The integration of multi-level annotations is also key:
- Automatic pre-annotation using AI models to speed up processing.
- Human validation by radiologists or clinicians to guarantee accuracy.
- Audit and correction via iterations based on reliability metrics.
To process large volumes of data while respecting patient confidentiality, we implement pseudonymization protocols and apply HIPAA or GDPR requirements. In addition, the use of synthetic data is an interesting alternative to overcome the limitations of real medical datasets, particularly in terms of diversity and protection of sensitive data.
💡 A la recherche de jeux de données pour expérimenter et développer des modèles IA médicaux ? N'hésitez pas à consulter notre Top 15 !
Finally, a continuous monitoring system ensures that annotation models and datasets evolve in line with the latest FDA regulations and industry best practices, guaranteeing reliable and exploitable labeling for training AI models.
Conclusion
FDA requirements for medical data-labeling may seem complex at first glance. Nevertheless, our analysis shows that a methodical, structured approach can achieve the required compliance.
The success of a Compliant Labeling program rests on three essential pillars. Firstly, the expertise of specialized medical annotators guarantees data accuracy. Secondly, a robust validation and traceability system ensures the quality of the process. Finally, rigorous security protocols protect sensitive patient information.
The future of medical data-labeling will depend on the ability to maintain these high standards while adapting to technological developments. We are convinced that companies who invest in FDA-compliant processes today are positioning themselves favorably for tomorrow's medical innovations...
