Newsbytes

Create PDF

Machine learning algorithm shows promise for predicting need for massive transfusion

March 2020—At trauma hospitals, simplicity is considered a virtue. That’s why when Jansen Seheult, MD, and his colleagues decided to use machine learning to predict massive transfusion needs, they chose a decision tree algorithm. “It was easy to implement as if/then rules, and it didn’t require computational resources to deploy,” says Dr. Seheult, clinical assistant professor of pathology at the University of Pittsburgh Medical Center.

Dr. Seheult

Dr. Seheult and his colleagues reported on that algorithm in a 2019 proof-of-concept study (Seheult JN, et al. Transfusion. 2019;59[3]:953–964). For the study, they used data from the trauma patient registry at one of UPMC’s level one trauma centers to train a supervised machine learning model to predict when patients admitted to the trauma unit would require massive transfusion. The model used data outcomes to train an algorithm to classify new cases.

Most traditional scoring systems for predicting the need for massive transfusion, such as the Assessment of Blood Consumption score and Trauma Associated Severe Hemorrhage score, use univariate analysis to determine which predictor variables are most associated with massive transfusion and then combine those variables using logistic regression, says Dr. Seheult, who is also laboratory medical director at Vitalant Coagulation Laboratory, in Pittsburgh. But when scoring systems rely on this method, “you can have significant interactions between pairs or groups of variables that are not accounted for,” he explains. Furthermore, traditional scoring systems tend to be more successful at predicting which patients won’t need massive transfusion than which ones will need it.

This issue, often called the “accuracy paradox,” is a problem with traditional massive transfusion scoring systems, but it’s also a trap encountered more generally when building a predictive model using imbalanced data. Because only about five percent of all civilian trauma patients require massive transfusion, the data are highly imbalanced, and it’s especially critical to account for this imbalance, Dr. Seheult says.

Data imbalance was one factor that drove Dr. Seheult and his colleagues to choose a recursive partitioning decision tree algorithm to predict massive transfusion at UPMC. With a decision tree, data imbalances can be accounted for by weighting classes differently, says Michelle Stram, MD, clinical instructor in the Department of Forensic Medicine at New York University, who coauthored the Transfusion study during her residency training at UPMC. “You can tell the algorithm it’s more important for you to identify the group that needs massive transfusion,” she says. “You may end up with false-positives, but you won’t be missing the people that need [massive transfusion].”

Dr. Stram

The authors of the Transfusion study generated two decision trees. The first, called MTPitt, was trained using clinical and demographic parameters, such as vital signs, immediately available upon admission to the emergency department. The second, MTPitt+Labs, was trained using those variables and additional laboratory data.

A training data set comprises a set of inputs used to train the model, explained Dr. Stram during a presentation on machine learning applications in transfusion medicine at the 2019 AABB annual meeting. These inputs “should be representative of the types of cases you intend the algorithm to predict.” A validation data set, she added, “is a portion of the data set held back from training the model. This data is used to test the performance of the algorithm.”

In the Transfusion study, 15 patients from the validation data set received massive transfusion, and the MTPitt decision tree misclassified only two of these cases as false-negatives. The authors also compared the performance of the decision trees with those of the Assessment of Blood Consumption and Trauma Associated Severe Hemorrhage scores, which misclassified four and five cases as false-negatives, respectively. The machine learning algorithm predicted massive transfusion needs with higher sensitivity and balanced accuracy than either of the traditional scoring systems, the authors report. After adding laboratory data, the algorithm achieved significantly higher specificity and balanced accuracy, but the sensitivity did not improve because the initial version of the algorithm predicted too few false-negative cases for further stratification.

Another reason to use a decision tree is interpretability, Dr. Stram says. A decision tree is “human readable” and fits with trauma hospital workflow. And the output from the algorithm could be posted as a flowchart in the emergency department or trauma bay, Dr. Seheult adds. “You only need one individual to determine if it’s predicting [a patient will need] massive transfusion.” Furthermore, it’s easily deployed, he continues. Model implementation and interpretability “were our two primary concerns when we were trying to develop a new algorithm.”

Yet developing a predictive machine learning algorithm requires dialing in the correct level of model complexity, Dr. Seheult says. “As you increase the complexity of the decision tree, you reduce the bias [when the model is too simple to represent real-world data] so it starts to fit the training data very well. But you also increase the variance,” or how much the model differs when it’s trained using another data set. When this happens, he adds, you run the risk of overfitting the training data—that is, the model becomes too complex and begins to pick up irrelevant characteristics, or noise, in the data.

“You don’t want to train the tree [to the point that] it predicts every single case because that’s going to result in overfitting,” Dr. Seheult continues. He and his colleagues avoided overfitting by “pruning the tree,” or removing peripheral branches at four splits. In other words, they prevented the decision tree from splitting into additional subpopulations after four splits because they determined that at that complexity parameter, the model achieved the greatest balanced accuracy.

“Working with incomplete data is one of the fundamental challenges of machine learning,” Dr. Seheult says. Therefore, he and his coauthors chose predictor variables with a minimal amount of missing data to train the algorithm for the MTPitt prediction tool. “It’s important to stress to laboratories that they should . . . adopt storage practices that try to retain as much of the granular detail of the data as possible.” It’s also important, he adds, to think about the pattern of missing data. “If it’s missing at random, multiple imputation techniques can be used to come up with a predictive model of the missing data.”