CAP TODAY Pathology/Laboratory Medicine/Laboratory Management
Editors: Liron Pantanowitz, MD, director of anatomical pathology, Department of Pathology, University of Michigan, Ann Arbor, and David McClintock, MD, associate chief medical information officer for pathology, Department of Pathology, University of Michigan.
An AI algorithm for prostate cancer diagnosis in whole slide images of CNBs
May 2021—As whole slide imaging has matured, pathologists have been focusing on the use of artificial intelligence algorithms. At the same time, the need to develop computer-assisted diagnostic tools to evaluate prostate core needle biopsies has intensified with the dramatic increase in the number of prostate cancer cases. Multiple studies have assessed the ability of AI-based tools to detect and grade prostate adenocarcinoma. However, few studies have performed clinical validation of such an AI-based algorithm or provided feedback about the clinical deployment of such tools. The authors conducted a blinded clinical validation study that involved deploying an AI-based algorithm in a pathology laboratory to aid prostate diagnosis. The algorithm was developed using H&E-stained slides of prostate core needle biopsies scanned with a Philips IntelliSite scanner in the Pathology Institute at Maccabi Healthcare Services’ centralized laboratory (Megalab) in Israel. The models were trained on 1,357,480 image patches from 549 H&E-stained slides and test datasets comprising 2,501 H&E-stained slides. The algorithm provided slide-level scores for probability of adenocarcinoma, Gleason score 7–10 versus Gleason score 6 or atypical small acinar proliferation (ASAP), Gleason pattern 5, and perineural invasion and calculated the percentage of carcinoma present in core needle biopsy material. It was subsequently validated on an external dataset of 100 consecutive cases (1,627 H&E-stained slides) scanned on an Aperio AT2 scanner at the University of Pittsburgh Medical Center. The algorithm was also implemented in the routine clinical workflow in the pathology laboratory at Maccabi as a second-read system for reviewing whole slide images of prostate core needle biopsies. Overall, the algorithm performed at high levels. It achieved an area under the receiver operating characteristic curve (AUC) of 0.997 for cancer detection in the internal test set and 0.991 in the external validation set. The AUC for distinguishing between a low-grade (Gleason score 6 or ASAP) and high-grade (Gleason score 7–10) cancer diagnosis was 0.941, and the AUC for detecting Gleason pattern 5 was 0.971 in the external validation set. The algorithm achieved an AUC of 0.957 for perineural invasion. The cancer percentage calculated by pathologists and the algorithm showed good agreement (r=0.882; P<.0001). In routine practice, the algorithm was used to assess 11,429 H&E-stained slides from 941 cases, leading to 90 Gleason score 7–10 alerts and 560 cancer alerts. Fifty-one (nine percent) cancer alerts led to additional cuts or stains being ordered, two (four percent) of which led to a third-opinion request. The authors concluded that this AI-based algorithm could be used as a tool to automate screening of prostate core needle biopsies for primary diagnosis, assess signed-out cases for quality control purposes, and standardize reporting to improve patient management.
Pantanowitz L, Quiroga-Garza GM, Bien L, et al. Clinical validation and deployment of an AI-based algorithm for prostate cancer diagnosis in whole slide images of core needle biopsies. Lancet Digital Health. 2020;2(8):e407–e416.
Correspondence: Dr. Liron Pantanowitz at lironp@med.umich.edu
Use of machine learning to enhance clinical decision support alerts
Clinical decision support comprises a set of powerful tools designed to assist provider decision-making and optimize clinical and operational workflows, such as appropriate ordering of laboratory tests. However, clinical decision support can be overused within an EHR, causing providers to overlook or ignore such alerts due to alert fatigue, minimizing its effectiveness. The authors reported on their development of a machine-learning model to predict whether a provider would review and accept an alert to cancel duplicate test orders (tests ordered on a patient who already had that test performed in a predefined time period) instead of overriding the alert. They hypothesized that a fully trained and validated logistic-regression model could be used to more accurately determine when to trigger an alert for specific providers so they would review the alert and use it to cancel a duplicate test order. After reviewing alert performance across 115 tests from eight hospitals, the authors zeroed in on five key tests (hemoglobin A1c, ferritin, vitamin B12, 25 hydroxy vitamin D, and thyroid-stimulating hormone) to develop their machine-learning models. They used logistic regression to analyze the impact of significant clinical and contextual predictor variables on alert compliance. The authors reviewed 60,399 triggered duplicate test alerts, ultimately focusing their efforts on outpatient alerts since those had much lower compliance rates (12.7 percent) than inpatient/emergency department alerts (66.9 percent). Factors that significantly reduced alert compliance across the five key tests were having a prior abnormal result for the test; entering orders based on outpatient encounters without a physical visit, such as via phone calls; and having attending providers place orders in outpatient settings in lieu of trainees, nurse practitioners, or physician assistants. Further, a provider-specific model created from the data pooled across the five key tests was the highest-performing model and had an area under the receiver operating characteristic curve of 0.82. Finally, the authors analyzed the potential effectiveness of using their provider-specific predictive model to reduce alert burden versus allowing a certain number of duplicate tests to be ordered. For example, they predicted that they could reduce physician alert burden by 1,900 alerts by allowing 200 duplicate tests to be ordered. Overall, the authors’ data and experience demonstrate that there are potential benefits to using clinical laboratory data to develop more intelligent clinical decision support tools.