Can machine learning algorithms predict lab values?

“The results of multiple pathology and lab tests tend to be associated,” Dr. Baron says. “We’re finding this is one of many examples where we can predict what a test result will be with some degree of accuracy before we even perform the test.”

Dr. Dighe says information gleaned from machine learning can be more easily translated into clinical practice by transforming the machine learning model into a simplified rule-based approach. “It’s not like you need some super computer connected to the EHR to run the algorithm,” he says. “In many cases you can run the algorithm offline and then implement it using standard EHR tools.” The PBFC study is a good example, he says. “You can use machine learning to come up with rules”— in this case, whether the patient has a history of hematologic malignancy, the percentage of neutrophils, and presence or absence of blast cells—and then use those rules to implement standard EHR clinical decision support.

Their study, which included 784 PBFC samples (from 744 patients) with a concurrent or recent CBC/diff order, found that the triaging strategies “could potentially defer 35 to 40 percent of all PBFC (with concurrent or recent CBC/diff),” they and their coauthors write, noting that the deferred tests would be expected to produce no clinically significant findings.

Deciding what to do with the rules comes with a host of practical considerations, Dr. Dighe notes. Laboratory workflow as well as technical, administrative, and economic factors come into play. Put another way, validating an algorithm for a research paper is one thing; implementing the results of that research in a clinical setting is another.

“It requires a whole different standard of clinical evidence,” Dr. Baron says. “It requires working with clinicians and having reasonable evidence that this is something safe and good to do for patient care. We have to think not just about what we did for the research paper, but also a practical implementation strategy.”

For one thing, the bulk of their machine learning research analyzes certain lab test results to make predictions about the results of other lab tests. But in the real world of the clinic, the predictor tests aren’t always ordered first.

“With the flow cytometry project we’re trying to decide if a physician should move forward with flow cytometry,” Dr. Baron says. But the algorithm is predicated on knowing a patient’s CBC value. If the physician orders a CBC and flow cytometry in parallel, the prediction algorithm won’t work. This problem could potentially be solved with a reflex protocol, he says, where “we first perform a CBC, and then depending on the results of the CBC we would reflex to flow cytometry. Or we could say based on the CBC results that flow cytometry isn’t needed for this patient.”

Alert fatigue is another consideration when implementing decision support. It’s a well-known concern in health care. “It’s important if you’re going to stop a provider’s workflow,” Dr. Dighe says, “that you do it only when absolutely necessary and helpful.”

Making clear to clinicians that clinical decision support is based on carefully researched and validated rules is critical, he says. “It’s important for these machine learning algorithms not to be complete black boxes all of the time. We want clinicians to change their behavior, so we have to explain why we’re alerting them.”

The clinical version of the flow cytometry algorithm is now in what Dr. Dighe calls “silent mode,” a trial period during which the algorithm runs in the background while the system collects data about when an alert would have fired, without triggering alerts to clinicians. “You need a system to test these things out and look into those patients to make sure if it would have fired that it would have been appropriate,” Dr. Dighe says.

With the movement toward algorithms that are ever more complex, Dr. Baron says, “we need to think about how we’re going to leverage native LIS or EHR functionality, or how we’re going to build systems that can easily interface with existing health information systems.”

“If we had the full toolbox” for the AKI alert, Dr. Dighe says, “we could implement a very complex imputation method and a prediction algorithm. We would be able to look at not just the last or baseline creatinine but the whole picture of the patient.” Those approaches, however, would be almost impossible to implement within the current generations of LIS, he says, because none of the major companies permit external calculation engines.

“It isn’t hopeless, though,” Dr. Dighe adds. “You could potentially have a data repository and an external request to a clinical decision support engine, have all your computation occur somewhere else, and then bring the results back into your lab system or EHR.” Some EHRs now permit native machine learning implementation; algorithms that determine readmission risk and perform sepsis scoring in real-time are examples. “That same approach can work for lab tests too.”

“I think it’s very encouraging and a sign of recognition of the value of machine learning that EHRs have begun to create machine learning modules within the EHR build,” he says.

LIS and EHR functionality aren’t the only obstacles, Dr. Baron notes. Administrative and economic barriers also play a large role. One solution, he says, is to find a scalable model for shared clinical decision support. “It’s hard for an individual hospital to justify the resources to push these over the goal line by itself,” he says. “Let’s say it would take $2 million to build out a highly robust machine-learning–based solution for flagging of AKI. If you could build a solution that could be plugged into hospitals all over the country, then it could easily justify a few million dollar investment.”

Standards and the application of standards, like LOINC, SNOMED, and ICD-10, are holding things back too, Dr. Dighe says. In many organizations, “they’re typically not well applied, so even the basics like identifying a lab test can be a challenge. Now that we’re aggregating all our lab results from many EHRs in the New England area, we can build decision support inclusive of the entirety of the patient’s record, but we first have to manually and carefully map virtually all of those tests together for the decision support to work.”

“You can make this wonderful model that can look at all these parameters,” he continues, “but if you can’t identify and use a CBC result from an external organization that was deposited into your EHR, then it’s not as useful.”

Then, too, there is the tension around data sharing, Dr. Baron says. “In general, technology companies themselves don’t have direct access to patient data, so they try to partner with academic and nonacademic centers to collaborate on projects and get data.” Working with companies may be a solution, he says, to help future patients and build scalable models for decision support. “If we’re going to make this a reality, we’re going to need to develop these collaborations between health systems and industry.”

Charna Albert is CAP TODAY associate contributing editor.