A look ahead at AI-based assistance in anatomic pathology

One of the key findings from the Paige study was a small but statistically significant reduction in false-positives when the pathologists were aided by AI, Dr. Retamero said. “This was a pattern that was reproducible across the generalists and the specialists and was present whether they were signing out remotely or on site, regardless of their age and level of experience.” He calls this, along with the “robustness to preanalytical variations,” another important aspect of generalizability, “which is a key benefit of using FDA-approved AI.” They found also that pathologists deferred (to immunostains, levels, second opinions) fewer cases that were clearly malignant and did not warrant deferral in reality, while deferring more cases that would have been wrongly diagnosed as benign.

Dr. Pantanowitz notes it’s unnecessary to wait for FDA approval to begin using AI-based tools, just as it was for digital pathology. For the latter, “Everyone was waiting for FDA-cleared products, and it made things murky and delayed adoption. If an AI product has been verified by the vendor and validated by the lab according to CAP requirements, then you can use it whether it’s FDA cleared or not,” he says.

But validation of AI, he admits, is “another murky area.” Laboratories need to perform clinical validation of AI-based tools on their own data or images. “The problem is there are no guidelines on exactly how to do this.” Dr. Pantanowitz, a member of the CAP’s AI Committee, says he and other committee members have discussed issuing guidelines to lessen confusion, standardize practice, ensure safety and good oversight, and promote adoption. But they decided not to do so at this point, he says, “because there’s no evidence out there to support our recommendations should we develop any, because very few labs are using AI and certainly not publishing about it.” The committee is working on a paper on principles based on good laboratory experience, “but it won’t be a formal guideline.” In the paper, the committee will point to CAP accreditation program checklists that are relevant to AI, for labs that are using AI for a particular task. “But that’s just the existing checklists,” Dr. Pantanowitz says. “There aren’t checklists yet for AI.”

Nor is there reimbursement for AI, though a precedent has been set with machine-learning algorithms to quantify biomarkers such as ER, PR, and HER2 for breast cancer, Dr. Pantanowitz says. The fear, he says, whether valid or not, is that the Centers for Medicare and Medicaid Services may pay less, not more. While he agrees the fear is realistic, he points to the lesson learned when Pap testing became automated. “People complained they didn’t want to move to computer-assisted screening. It was disruptive for them; they had to buy expensive technology. They initially didn’t think it was that great. Yes, it caught all the squamous lesions, but what about that rare endometrial cancer? It wasn’t trained to catch that.” Once there was a CPT code, he says, “almost everyone bought it.”

Hologic’s Quick points to Hologic’s track record. “What we did was work proactively with laboratory customers to develop both clinical and economic data” on outcomes, “not just from the perspective of a lab but from that of a payer.” Are downstream costs avoided? Is patient care better and worth a higher reimbursement? “You need to have that data to change the narrative around CPT coding and pricing.” The CMS is asking for guidance on AI, Quick says, “which is encouraging. But ultimately it needs to go beyond the efficiency of the laboratory. It needs to have a clinical benefit,” and the onus to provide the data, he says, is on industry, clinical laboratories, and hospitals.

A crossroad for many will be whether to believe the AI, Dr. Pantanowitz says. “If you look at a routine prostate case and there’s a heat map over a few glands, and the algorithm is saying, ‘These glands are adenocarcinoma,’ and you do not think it’s adenocarcinoma, you have a predicament.” He participated in a validation study of the Ibex Medical Analytics Galen platform for prostate core needle biopsies at the University of Pittsburgh Medical Center when he was on faculty there. In that study, he says, “we compared AI to MDs,” and there were 30 slides over which such discrepancies arose. They resolved the disagreements through consensus review with colleagues and experts. Ancillary studies also may be done if applicable, he notes. “And I suspect it would be the same in clinical practice.” In their study, he says, the AI was correct in all 30 cases. (Dr. Pantanowitz serves on Ibex’s medical advisory board. Galen received FDA breakthrough device designation in June 2021.)

In future practice, he says, “you won’t have to fight the computer machine—it’s not you versus the terminator. You can ask for help from your partners, and you can run other stains.” If manufacturers set it up so the AI makes a recommendation but the pathologist can weigh in and overrule it, “that seems reasonable,” he says.

Dr. Pantanowitz and his coauthors write in the review article that “AI-based algorithms may seem much more capable than they really are.” Humans are unable to fully comprehend, they write, how “millions of parameters contribute to a decision, leading to potential biases, misuse, and misdiagnoses.”

Dr. Retamero tells CAP TODAY, “We may be limited in understanding how the computer reaches certain conclusions, but part of the pathologist’s role in the diagnostic process when aided by AI is to question what the AI is telling you.” If what the AI says elicits a strong negative response, he says, “that probably means the AI is not accurate,” and that one’s own judgment may be more accurate. “All diagnostic tests produce false-negatives and false-positives, and AI is no exception here. The pathologist has the final say in the diagnosis. In that regard it’s no different than any immunohistochemistry assay or genetic assay,” he adds.

Hologic, Quick says, is beginning to use the term “digital assay”—which the company uses already to refer to its molecular testing offerings—to describe the content that will run on the Genius platform. They’re viewing the system “not just as a replacement for the ThinPrep imaging system, which is a natural progression, but as the creation of a platform for future technologies.” The company’s road map involves building out the menu of digital assays that will run on the Genius platform, including content looking at, among other things, endometrial and ovarian cancer.

“Exciting but incredibly complex” is how Quick describes AI in health care today, noting the regulatory environment is challenging and an opportunity for partnerships between industry and others. “We’ve built algorithms in the past and they’re locked down and we don’t touch them for years. That’s going to change in the future, but it will require a different regulatory strategy,” he says.

Dr. Pantanowitz, with his eye, too, on the future, says AI will change pathology practice, “but the way it will change practice will differ for pathologists in different settings.” GU subspecialists at a large academic medical center, for example, often are inundated with large volumes of prostate biopsies. AI-based tools may make their diagnostic work more efficient, allowing more time for research. Generalists working in a community setting, on the other hand, don’t need help with large volumes of biopsies. “What they need help with is diagnostic accuracy when they have a difficult case,” or assistance with second reads and quality control. “But I think pathologists in either setting would welcome AI because it’s beneficial from that point of view.”

And though some have speculated AI may deter new residents from entering pathology, he believes it’s an enticement. “I do think people coming into the field will see AI as more attractive than a 100-year-old microscope.”

Charna Albert is CAP TODAY associate contributing editor.