Recommendations proposed for machine learning systems
February 2024—Members of the CAP Machine Learning Working Group, Informatics Committee, Digital and Computational Pathology Committee, and Council on Informatics and Pathology Innovation have proposed 15 recommendations for evaluating the performance of machine learning-based clinical decision support systems in pathology (Hanna MG, et al. Arch Pathol Lab Med. Published online Dec. 2, 2023. doi:10.5858/arpa.2023-0042-CP).
The evidence available to formulate guidelines is lacking, the authors write, but a review of the literature and existing documents made it possible to develop the proposed recommendations.
“The advancement in technology and computing resources coupled with a digital transformation of the pathology field has demonstrated a need to provide further guidance for systems that incorporate emerging technologies such as ML-based CDS,” they write.
They say their guidance document provides recommendations on performance evaluation of clinical tests that utilize ML in any part of the preanalytic, analytic, or postanalytic workflow.
Here’s a sampling of their proposed recommendations:
- The scope of the performance evaluation should be guided by a comprehensive risk assessment to evaluate sources of variability and error and the potential for patient harm.
- Variations of verification and validation terminologies by different domains should be understood.
- Predeployment of the machine learning model includes the model properties, case mix/data compatibility, bias and ethics, and laboratory influences.
- Data sources for the performance evaluation should include diverse real-world data including all clinically meaningful variations to which the model may be exposed.
- Inclusion of human factor evaluation during performance evaluation safeguards model explainability and trust.
- Personnel training and competency evaluation planning should be included for all personnel involved in the patient testing process in an end-to-end fashion.
The full article and list of proposed recommendations is at bit.ly/ML-12022023.
FDA clears Hologic’s Genius digital cytology platform
The Food and Drug Administration has cleared Hologic’s Genius Digital Diagnostics System with the Genius Cervical AI algorithm. It is a digital cytology system that combines deep-learning–based artificial intelligence with advanced volumetric imaging technology.
The system consists of the Genius Digital Imager for image acquisition, the Genius Cervical AI algorithm for image analysis, the Genius Image Management Server for image storage, and the Genius Review Station for local or remote case review.
The complete system is scalable and commercially available in Europe, Australia, and New Zealand. Commercial availability in the U.S. is expected early this year.