Can machine learning algorithms predict lab values?

Charna Albert

February 2020—At Massachusetts General Hospital, machine learning is being used in the laboratories to build next-level clinical decision support, and in the latest phase, it’s undergoing trial for use in predicting laboratory results.

“I think this is the new paradigm for cost-effective laboratory medicine. This is an important way we’re going to change how we do business,” says Anand Dighe, MD, PhD, who spoke about machine learning techniques for labs during a CAP19 presentation last fall and in a recent interview with CAP TODAY.

Dr. Dighe, director of clinical informatics and director of the core laboratory at MGH, has been working with other scientists and pathologists to make this vision a reality. He and colleague Jason Baron, MD, a pathologist and clinical informatician within the MGH core laboratory and an assistant professor of pathology at Harvard Medical School, enlisted the help of two computer scientists at Massachusetts Institute of Technology. Together they studied ways to use machine learning to predict laboratory values using the results from other lab tests in the patient’s medical record (Luo Y, et al. Am J Clin Pathol. 2016;145[6]:778–788; Luo Y, et al. J Am Med Inform Assoc. 2018; 25[6]:645–653).

The collaboration with MIT was “particularly fruitful,” Dr. Baron tells CAP TODAY, in integrating MGH clinical laboratory and clinical data science expertise with computer science from MIT. “Although many mature machine learning methods developed outside of health care were available for us to use, some were not well suited to clinical data.” Existing prediction models required finesse to handle important nuances of clinical data, he says. “For example, no outpatient has a CBC every day. It’s not like a stock market ticker.” (Finance drove the development of some machine learning algorithms.)

“We had to figure out novel algorithms that could provide useful information, even in the face of the missing data that is so common with laboratory results.” The development of these algorithms was a key contribution of their MIT collaborators Peter Szolovits, PhD, professor of computer science and engineering and head of the clinical decision-making group within the MIT computer science and artificial intelligence laboratory, and Yuan Luo, PhD, who is now chief AI scientist and associate professor of preventive medicine at Northwestern University Feinberg School of Medicine.

One target of their work was predicting ferritin results from other laboratory tests. The MIT researchers worked with Dr. Dighe, Dr. Baron, and colleagues to develop imputation algorithms—methods that allowed them to infer the missing lab test values needed to train the model. In stage one of the two-step process, they imputed the results for lab tests that hadn’t been performed (other than ferritin). In stage two, they took the measured and imputed values for the predictor tests and used those, in addition to basic patient characteristics, to predict ferritin results.

“When looked at in isolation, ferritin values can lead to misdiagnosis. Ferritin often increases from inflammation, so non-iron-deficient patients undergoing inflammatory responses may have elevated ferritin levels. And normal ferritin values can obscure when a patient is in fact iron-deficient,” Dr. Dighe says. One application of the ferritin algorithm is to look for discrepant results. When predicted and measured ferritin don’t agree, “that’s almost always an important signal for us.”

“In those cases,” Dr. Baron says, “the obvious thing to do would be to append a comment to the test result warning the clinician, ‘Don’t rule out iron deficiency on the basis of a normal ferritin alone.’”

For now, implementation of the algorithm is on hold. “We didn’t have an obvious strategy for implementing it within our existing information systems,” Dr. Baron says.

Developing predictive models is only part of the solution, Dr. Dighe says. Many types of models will not be useful in improving patient care unless they are implemented as clinical decision support within existing workflows, processes, and health information systems, “and implementation can be challenging,” he says. Dr. Dighe and colleagues implemented a relatively straightforward, rule-based interpretive comment intended to flag substantially increasing creatinine values that may indicate acute kidney injury (Baron JM, et al. Am J Clin Pathol. 2015;143[1]:42–49).

This AKI flag “was much more difficult to implement than we would have guessed,” Dr. Dighe says. Developing the flag required calculating a “baseline” creatinine for each patient and then flagging subsequent creatinine values that were increased from that baseline according to certain rules. “However, there was no straightforward way to calculate the baseline creatinine within the version of the lab information system we were using at the time. We had to develop a complex workaround.”

The flagging rules provide a solution to the problem of overlooked AKI cases. While their current AKI flag identifies AKI only after the patient already has it, “the longer-term aim is to alert providers in advance that their patient is likely to develop AKI 24 hours or more into the future and perhaps even offer advice regarding actionable steps to take to reduce AKI risk,” Dr. Dighe says. One tack the team is taking involves extending their imputation work to forecast creatinine values into the future. “If future creatinine values are expected to increase, that could be a sign of AKI to come,” Dr. Dighe says.

The AKI algorithm was implemented at MGH more than five years ago and provider feedback has been positive, with changes in treatment and decision-making resulting from the AKI flagging. “What we found from subsequent surveys one of our hospitalist colleagues did,” Dr. Dighe says, “was that more than 50 percent of clinicians had made a change in patient care based on the AKI flag.”

“Luckily, our LIS team here is very creative and they were able to implement it,” he says of the difficulty. “When you’re doing analysis for a paper, you can do all kinds of wonderful things, but you sometimes find yourself limited by technology when you try to implement them.”

It helped that the creatinine flag could be reduced to simple if/then rules and that acute kidney injury is a common health problem. “We had a lot of high-level clinical requests to make this go through,” Dr. Baron says, noting that the AKI flag affects roughly 10 percent of MGH’s inpatients. “As a result, we were willing to put a lot of resources in and spend a lot of IT time, and we had a lot of clinicians helping.”

If the AKI algorithm had been based on an artificial neural network or a more complex model, Dr. Baron says, it would have been much more difficult to put into clinical practice at MGH.

Drs. Dighe and Baron collaborated recently with MGH colleagues Aliyah Sohani, MD, director of surgical pathology, and Lisa Zhang, MD, resident in anatomic and clinical pathology, to demonstrate the utility of machine learning models in predicting peripheral blood flow cytometry (PBFC) results, with the aim of optimizing the use of PBFC (Zhang ML, et al. Am J Clin Pathol. 2020;153[2]:235–242). Using decision tree and logistic regression models to analyze PBFC samples from MGH’s clinical flow cytometry laboratory, the study’s authors demonstrate that it’s possible to predict PBFC results by looking at the patient’s history of hematologic malignancy and CBC/differential parameters.