Listen to a discussion of this article through our AI-generated podcast.
CAP TODAY now offers the option to listen to a discussion of our articles with voices reminiscent of a Siri or Garmin speaker. In fact, the voices that you will hear are completely AI generated. Since this AI-generated podcast uses technology that is still in its early stages of development, there may be some errors in pronunciation and the tone may sound too upbeat for the material covered.
Molecular Oncology
Valerie Neff Newitt
October 2024—Machine learning applications in molecular oncology testing are largely in the research or early clinical implementation phase, though some ML methods have been part of bioinformatics tasks for years, such as variant effect prediction.
For the early adopters of other machine-learning-based methods—for example, for DNA methylation analysis of central nervous system tumors—existing CAP accreditation requirements can serve as a guide to implementing these tests.
“Our findings show that some components of the general CAP accreditation framework for traditional molecular oncology assay validation and maintenance can be considered when implementing a machine-learning-based test in a clinical laboratory,” says Larissa Furtado, MD, coauthor of an article that explains how the checklist requirements apply to ML-based molecular oncology assays (Furtado LV, et al. Arch Pathol Lab Med. Published online June 14, 2024. doi:10.5858/arpa.2024-0037-CP).
In molecular oncology, where genomic tests generate large volumes of complex data, ML models have the potential to improve molecular testing, says Dr. Furtado, molecular pathologist and an associate member of the Department of Pathology, St. Jude Children’s Research Hospital. They do so by increasing the analytical capabilities of bioinformatics pipelines, she says, and by providing sophisticated methods to extract relevant features, patterns, and correlations from genomic data, which “may matter for cancer management but have not yet been appreciated using the available analytical tools.”
“Unlike bioinformatics pipelines that process data based on preestablished analysis rules, machine learning can make predictions based on data learning and can derive meaning from data that may go beyond the known program concepts,” explains Dr. Furtado, who is a member of the CAP Artificial Intelligence Committee.
For some central nervous system tumors, ML methods are shaping and expanding the classification schemes, she says, which can improve the ability to elucidate their biology and better match tumor subtypes with therapies. She cites medulloblastoma as an example.
“If you think about the modern classification using supervised machine learning classifiers on DNA methylation data, medulloblastoma represents a heterogeneous tumor with multiple subtypes that present different risks of recurrence and have different long-term survival, and consequently require different intensities and types of therapies. Machine learning methods have the potential to enhance pathologists’ diagnostic, prognostic, and therapeutic prediction capabilities.”
DNA methylation profiling is often used to stratify tumors, she notes, and it’s being done using ML-based techniques in several clinical laboratories in the U.S., including at St. Jude’s.
“Machine learning models using gene expression and DNA methylation signatures can stratify tumors in clinically meaningful subgroups at a level of granularity that’s not achievable by any other molecular test,” Dr. Furtado says.
At St. Jude’s, the knowledge obtained in this way about the molecular classification of tumors is being used in a phase two clinical trial that stratifies treatment in infants with newly diagnosed medulloblastoma based on methylation classification and clinical characteristics (www.stjude.org/treatment/clinical-trials/sjimb21-study.html). “The integration of machine-learning-based DNA methylation profiling into the routine workup of CNS tumors has demonstrated improved diagnostic yield and precision,” she says. Some centers have reported performing ML-based sarcoma classification (Miettinen M, et al. Am J Surg Pathol. 2024;48[1]:112–122; Roohani S, et al. Clin Epigenetics. 2022;14[1]:149).
Dr. Furtado and other members of a CAP working group set out to write a review of the current and emerging applications of machine learning in molecular oncology testing, such as for DNA methylation profiling, variant calling, microsatellite instability status, mutational signature testing, and histology-molecular correlation.
To add to the practicality of the review, she says, they decided to do a “crosswalk between the checklist requirements for molecular oncology tests and how they could apply to the implementation of machine-learning-based tests,” using their own experiences with ML-based assays and in the belief that existing requirements could provide a framework for the validation, deployment, monitoring, and maintenance of ML-based assays. “We wanted to demonstrate to those planning to implement a machine-learning-based test in their clinical labs that it’s possible to use the existing accreditation requirements as a guide,” Dr. Furtado says, noting she used some of the same requirements to implement the ML-based assay in her laboratory.
She and her coauthors began by mapping out the components of an ML-based assay life cycle to identify the similarities and differences between it and the approaches used to design, develop, validate, and implement molecular oncology assays. They reviewed the latest editions of three CAP checklists: molecular pathology, all common, and laboratory general. That they found enough similarities to provide the guidance they aimed for was not unexpected, she says, because ML models are not used as autonomous devices in clinical laboratories. “Instead, they are implemented as a component of a test embedded in the core analytical pipeline of the test, such as in the methylation-based classifiers where the model is an integral part of the bioinformatics analytical pipeline.” Or the ML model might provide a separate analytical feature to the assay bioinformatics pipeline, such as the microsatellite instability analysis. “So the same general principles and requirements apply.”
She cites as an example that an ML-based assay should be validated or verified before it’s clinically implemented and revalidated when modifications are made, and that validation of ML models should be integrated with the wet bench validation, when appropriate, to ensure acceptable beginning-to-end test performance.
Additional example considerations for ML-based molecular oncology assays, drawn from existing checklist requirements (corresponding checklist requirement numbers shown in parentheses below), are as follows:
- During validation, laboratories should establish the minimal tumor content requirements for reliable, accurate, and reproducible performance of the ML assay. Limit of detection may be established by using mixtures of tumor and matched nonneoplastic control DNA samples at different neoplastic cell percentages or by in silico mixing using tumor and matched control files included in the cross validation and test sets (MOL.36153).
- For classification models, a threshold score for clinical reporting of a predicted class increases confidence in the classifier output and allows for unreliable calls to be discarded. Therefore, when appropriate, laboratories should validate or verify a threshold score for clinical reporting of ML-based assay classifications that provides acceptable sensitivity and specificity for classifier calls (MOL.31251).
Other such guidance and the checklist requirements the group drew from are provided in a table in their article, though the table isn’t comprehensive, Dr. Furtado says. “We chose areas of tests we consider very important such as validation and revalidation, quality management infrastructure, and analytical procedures. And of those areas, we went over examples of checklist requirements that are relevant. But we kept it more general to broaden the concept of applicability of the checklists for machine-learning-based assays” because different intended uses may make adaptations necessary.
The working group members came from three CAP committees: AI, Genomic Medicine, and Molecular Oncology. Their work demonstrates, she says, that although requirements that specifically address the use of machine learning in clinical laboratories are not available, “it is possible for laboratories looking to implement machine-learning-based assays for oncology testing to use CAP accreditation requirements as a guide.”
In addition to their use in DNA methylation profiling, machine learning models are being applied to variant calling in next-generation sequencing data.
The algorithm can learn from patterns in the data to distinguish between true variants and false positives “and serve as a quality check,” Dr. Furtado and coauthors write. Algorithmic methods also have been developed to determine microsatellite instability status, they say, and could be applied to mutational signatures and histology-molecular correlation.
Says Dr. Furtado, “There are many exciting ways in which machine learning could be used in molecular oncology to augment existing analytical methods, enable new analysis, and uncover new patterns in genomic and epigenomic data” that may help guide diagnosis and targeted therapeutic approaches. She describes machine learning techniques as “highly scalable, capable of processing vast amounts of data efficiently, and capable of handling and integrating complex, high-dimensional data more efficiently than standard bioinformatics methods.”
Outside of molecular oncology, the use cases for machine learning include, among others, quality control of the various phases of lab testing; image-based clinical decision support for tumor detection, grading, and classification; scoring of PD-L1 and HER2 biomarkers; assessment of tumor-infiltrating lymphocytes; screening of gynecologic liquid-based cytology specimens; analysis of kidney transplant biopsies; and parasite classification in stool samples.
With a machine learning model, Dr. Furtado says, “you’re always going to be using it as one component of your test, the same way we use bioinformatics pipelines or digital image analysis or an analyzer in the laboratory. We must treat it as a component of the test.” The performance of the model is tied to the quality and characteristics of the training data, she notes, stressing the importance of data collection and curation to ensure the robustness, accuracy, and generalizability of the model.
“So there are a lot of similarities but also some specific considerations it’s important to be aware of,” she says.
Valerie Neff Newitt is a writer in Audubon, Pa. Dr. Furtado refers readers to two articles for more information: doi.org/10.5858/arpa.2023-0042-CP; doi.org/10.5858/arpa.2020-0541-CP.