Closing the workflow loop: HistoQC for digital slides

It wasn’t Dr. Janowczyk’s first attempt at using a larger cohort. He and colleagues had built a series of in-house scripts and filters over the years to identify artifacts and regions that were out of focus in slides they wanted to use. “Building Histo­QC itself was a way to organize our experience into a single consolidated pipeline, as well as make it robust and easy to use for other people.” He says it can be used with little training.

“That’s how HistoQC began. I just sat down and started developing it,” he says.

Work began in 2017, and Histo­QC was officially released in June 2018 at the European Congress on Digital Pathology, where it won an innovation award. The source code is available in an open-source github repository (github.com/choosehappy/HistoQC). “Literally anyone can download it and start using it. And because it is part of the open-source paradigm, other people can now contribute to the effort. It has become a living, breathing project in the sense that people we don’t even know can add new components. They can integrate it, send us the code, and we can merge it into the central public repository.”

Dr. Janowczyk says it is hard to know how many HistoQC users there are. “One of the issues with giving something away free is that you have no real way to track how many people have used it. That said, I can say we receive comments from users all the time. Institutions, too, are starting to use it. In Switzerland we are already seeing uptake in a number of hospitals, and in Scandinavia too.” HistoQC was built in collaboration with Michael Feldman, MD, PhD, of the University of Pennsylvania Department of Pathology and Laboratory Medicine, “and he is interested in deploying it clinically there as well,” Dr. Janowczyk says.

“HistoQC is the first available tool of its kind and it happens to be free, so it is kind of growing like wildfire.”

Use of HistoQC will grow as users discover even more of its advantages, he says. There will be savings, for example, when staff knows precisely how long a stain is viable. “Stains are expensive. They are like oil in your car; you have to change them. But how often? Manufacturers will give you a conservative boundary to be sure the stain is working properly.” But with Histo­QC, the quality of the stain can be measured over a period. “You will have a quantifiable measure that shows you if a stain is still good. You may find that you don’t have to change the stain as often as you thought.

“On the other hand,” he continues, “if something goes wrong with your stain, you would know immediately, as soon as those first few slides come out of the scanner. You can hit that big red stop button before you unintentionally create a thousand bad slides.”

Alternatively, at the end of a month a laboratory could see that last month’s slides were of lower quality and know not to use them in the future for computational analysis or potentially clear them out of storage to make room for better quality data. It gives pathology a QC paradigm that’s everywhere, he says—building cars, steel beams, buildings—but was not previously possible in pathology because it wasn’t a digital science.

In research, HistoQC may act as a proving ground for algorithms in development and in use. “Researchers need to know they are testing new algorithms on slides that meet specifications. The question is always, ‘How can we trust this algorithm?’” Dr. Janowczyk says. “Previously we tried to show trustworthiness by curating large collections of slides from different hospitals and seeing if the algorithm works consistently across those cohorts. That’s a brute-force approach to validating an algorithm. But with HistoQC we can computationally look at a million slides and identify 5,000 of them that are the most diverse and then test an algorithm on those.” If it performs well, he says, they know they have covered the spectrum of what they might expect to see in the real world.

When an algorithm is introduced at a hospital, the metrics on that hospital’s slides can be compared against the metrics on the slides used to develop the algorithm. “We will be able to say, ‘Your slides are very dark or light compared with the slides we’ve built our algorithm on, so it may not work as expected. You should proceed with caution.’ It’s an opportunity to raise red flags to make sure an algorithm is not used in a way that was not intended.”

HistoQC’s collaborators, in addition to Dr. Feldman, are Ren Zuo of Case Western, Hannah Gilmore, MD, of University Hospitals Cleveland Medical Center, and Anant Madabhushi, PhD, of Case Western and Louis Stokes Cleveland VA Medical Center. They offered Histo­QC open source to spur a conversation, Dr. Janowczyk says.

“Digital pathology remains like a wild frontier,” he says. “Everyone is still developing their own tools and workflows. So it was important for us to take a step back and think how we can create a way that people can work together, to make this a little more efficient, a little more formalized, and a little less chaotic.”

It’s not that QC wasn’t performed but that it wasn’t centralized and uniform, he says. “It wasn’t organized and reproducible in a way that another lab would be able to do it in exactly the same way and get the exact same results.” He views that as “bad science.”

“One component of this project was the intent to formalize a quality control process and put it out there for others to evaluate and discuss. In its own way, it’s a bit revolutionary. But we won’t know how much of an impact this will have until we look back in a few years. Hopefully by then new standards and regulations will have been created as a result of realizations derived from using it.”

Perhaps scanner manufacturers will build these types of tools into their systems, he says, so slides can be identified as poor quality while still in the scanner. “And some are starting to do that.”

Dr. Janowczyk doesn’t necessarily see HistoQC still being needed decades from now. “To be honest, I would be surprised if it was still needed. I think HistoQC represents more of an awakening to a problem. We’re pointing to that problem and saying, ‘Here is our perceived solution. Here is a strong beta prototype that everyone can use. And now we can start from there to have useful discussions about what we think quality control should be, and why and how it should be implemented going forward.’”

Valerie Neff Newitt is a writer in Audubon, Pa.