Editors: Liron Pantanowitz, MD, director of anatomical pathology, Department of Pathology, University of Michigan, Ann Arbor, and David McClintock, MD, senior associate consultant, Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, Minn.
Use of artificial intelligence in clinical diagnosis: impact on pathologist decision-making
September 2022—Artificial intelligence in pathology has progressed recently, with at least four machine-learning algorithms classified for clinical use in the United States. While many challenges of implementing AI in pathology labs are well documented, one area that has not received much study is how an AI algorithm designed to augment pathologist performance will impact pathologists’ clinical decisions. The authors proposed three hypotheses regarding pathologists’ potential reliance on AI: AI decisions can influence pathologists’ decisions; pathologists may be more apt to rely on AI if information about the AI algorithm is disclosed; and the more pathologists trust an AI system, the more they will rely on it. To test these hypotheses, the authors performed a 1 × 3 within-subject online experiment, in which each subject tests three conditions and the results are compared across that subject’s performance. The authors conducted a 12-question online whole slide image-assessment and gathered demographic data and information about respondents’ AI beliefs. For the assessment, 116 pathologists (practicing and retired) and medical students analyzed the Gleason grade for a series of 12 prostate biopsies that composed three sets of four prostate whole slide images. The sets were labeled no AI aid (no expert advice), AI aid (expert advice respondents were told came from an algorithm but without any information about the algorithm), and AI aid+ (expert advice respondents were told came from an algorithm along with the algorithm’s accuracy rate and model information). The assessment used a mock AI algorithm, and participants were always presented with the correct (consensus) prostate Gleason grade for that slide as a recommendation. An analysis of the assessment data showed that participants’ clinical decisions for Gleason grading were significantly more accurate when AI-based recommendations were provided—92 percent accuracy with AI versus 87 percent without. However, there was no significant difference in accuracy when including explainability for the algorithm. Respondents’ reliance on AI correlated with their general beliefs about the overall clinical potential of AI but not their beliefs about the accuracy and usefulness of the actual AI advice provided. Furthermore, respondents made their clinical decisions faster when using the AI aid versus not using it. Of note, this study had limitations, including recruiting participants through an open Facebook group without verifying the credentials of the respondents and testing only one task—that is, assessing the Gleason grade. The authors concluded that additional study is needed to better discern how pathologists will be impacted by using AI algorithms in clinical practice.
Meyer J, Khademi A, Têtu B, et al. Impact of artificial intelligence on pathologists’ decisions: an experiment. J Am Med Inform Assoc. 2022. doi:10.1093/jamia/ocac103
Correspondence: Dr. Julien Meyer at julien.meyer@ryerson.ca
AI-driven whole slide image review of ThinPrep Pap tests
For many years the digitization of cytology slides lagged behind advances in digital pathology applications in such fields as surgical pathology. This is not surprising because it is extremely challenging to screen a digital cytology slide, such as a Pap smear, as it involves examining every cell. The ability to use artificial intelligence to analyze digital images has made it easier to screen Pap tests. There are several commercial digital cytology systems for this purpose, including the AI-driven BestCyte cell sorter imaging system (in development; CellSolutions, Greensboro, NC). This product classifies and sorts image tiles of scanned Pap test slides and displays cells of importance in galleries. It also allows the end user to pan the whole slide image of the scanned slide. The author sought to determine the time required and diagnostic accuracy when using BestCyte to screen 500 blinded, randomized ThinPrep Pap test digital slides according to Bethesda nomenclature. He used a Pannoramic P250 Flash III RX scanner with a pixel resolution of approximately 0.25 µm for the study. The author, who was trained on the BestCyte system, scanned ThinPrep slides at 20× magnification using a single focal plane. After the system analyzes the digitized slide, it creates a gallery containing variably sized images of clinically significant cells. Up to 100 image tiles can be displayed in each of eight galleries—overview, high N/C (nuclear:cytoplasmic ratio), halos, atypical, elongated, clusters, endocervical (T-zone), and InternalCtrl. The mean primary screening review times recorded were 1.38 minutes for the overall study; 1.23 minutes for negative for intraepithelial lesion malignancy (NILM); 1.18 minutes for atypical squamous cells of undetermined significance (ASCUS); 1.73 minutes for atypical squamous cells, cannot rule out HSIL (ASC-H); 1.84 minutes for atypical glandular cells of undetermined significance (AGUS); 1.49 minutes for low-grade squamous intraepithelial lesion (LSIL); 1.52 minutes for high-grade squamous intraepithelial lesion (HSIL); and 0.65 minutes for cancer. The diagnosis was downgraded in only two (0.4 percent) of these cases, and it was upgraded in 15 (three percent) following adjudicative whole slide image rescreening. The kappa coefficient for the study was 0.9305. The author concluded that by automatically classifying, ranking, sorting, and displaying clinically relevant cells, or clusters of cells, into galleries, BestCyte shortened ThinPrep review times. This study provides further evidence that AI-based technology has the potential to replace manual microscopy for primary screening of ThinPrep Pap tests.
Chantziantoniou N. BestCyte® cell sorter imaging system: Primary and adjudicative whole slide image rescreening review times of 500 ThinPrep Pap test thin-layers—an intra-observer, time-surrogate analysis of diagnostic confidence potentialities. J Path Inform. 2022. http://dx.doi.org/10.1016/j.jpi.2022.100095
Correspondence: CellPathology Plus at cellpathology.plus@gmail.com
Editor’s note: Dr. Chantziantoniou is a consultant to CellSolutions.