Metagenomic NGS: More pros than cons?

Dr. Miller described the Texas Children’s Bacillus case as interesting, “because if we found that in a culture, even of an abscess fluid, we would probably write it off as being a contaminant. Probably 99 percent of the time that’s the case. Then there’s the tiny sliver of cases in which it’s real.” No test will be “the definitive end-all, be-all,” he said, and clinical context is vital. “Is this consistent with the patient? We know that in infants, Bacillus infections can happen” and there can be multiple abscesses. If Bacillus were detected in an adult patient with nonabscess-forming meningitis, it probably would not be a consideration.

Sample types do make a difference, Dr. Miller said. “Most of our experience has been with CSF where you’re assessing the actual intrathecal space for organism. That’s typically thought to be the site of infection.”

Plasma is trickier, he said, because from what he has seen in the literature and from UCSF’s own data thus far, “in cases where you do suspect a true bloodstream infection with a fastidious organism or one previously treated with antibiotics, those sites are more reasonable cases. We’ve certainly seen good utility in endocarditis cases.”

The majority of the plasma sample requests come when a patient has a suspected localized infection and clinicians want to do a liquid biopsy to see if an organism is present. That will not rule out infection, Dr. Miller said, because the negative predictive value is simply not high enough. A positive result can be helpful in those cases.

Dr. Miller said he advises his infectious disease and medicine colleagues that if they’re going to pursue mNGS testing, they need to assess the infectious site in some way. “Is there a plan for biopsies? If there’s a lung infection, is a bronchoscopy being performed? How do we look at that? Because the Hail Marys are usually pretty low yield.” The more disseminated presentations are more amenable to plasma metagenomics, he said.

“We’re all coming up with our own thresholds about what we think are good cases and not good cases to send.”

Dozens of case reports of successful applications of mNGS have been published, but less well published, moderator Dr. Erin Graf said, are the negative consequences of positive results that do not reflect infection. She noted that Drs. Muller and Miller reported data showing their laboratories having had similar numbers of clinically relevant versus clinically irrelevant positive test results, or similar numbers of false and real detections.

What, she asked Dr. Mitchell, do you think about the potential unintended consequences of clinically irrelevant positive results?

“This is where I fall so strongly on the con side,” Dr. Mitchell said. She agrees that “publication bias” exists in favor of mNGS successes but said more data are being reported on negative consequences, which she said will be helpful in developing criteria for which patients benefit.

Specificity becomes an issue with plasma-based samples, Dr. Mitchell said. “We’re finding there’s a lot of DNA background that we pick up using metagenomics, and it can be hard to interpret some of those results, especially if you see something that could be a true pathogen in a certain patient population.” Sending this test broadly without that key pretest probability can make it difficult to interpret the result. “And in the absence of having a medical director at the lab, or a dedicated ID physician who’s willing to look over this data, you can imagine there would be overtreatment or potentially misdiagnoses.”

Her biggest fear: “A false-positive will lead them down a rabbit hole that prevents them from looking at other reasons for the patient having these symptoms and making a true diagnosis.” It all comes down to developing criteria, she added, so the test is done on the right patients. Sensitivity “can be thrown in there as well,” she said, “especially in the highly cellular samples.” And contamination can be introduced at any point in the process.

The sensitivity issue makes her wonder if mNGS will replace conventional PCR testing, “or are they going to be sending everything plus this? And what do you do with discrepant or unexpected results?”

Texas Children’s Hospital has had sensitivity problems, too, “for the reasons described,” Dr. Palazzi said, noting missed cases of Mycobacterium tuberculosis, HSV, and a case of Staphylococcus aureus and Moraxella catarrhalis (Niles DT, et al. J Clin Microbiol. 2020;58[11]:e00794–20). And specificity is a limitation. In a retrospective study of cases, “the vast majority had organisms identified for which there was no clinical way that it could explain disease or the lab didn’t know what to do with the results,” and excessive antibiotic use results. “That’s a huge limitation,” she said.

Sensitivity may be limited for a variety of reasons, UCSF’s Dr. Miller said, and some can be overcome. “Some of our earlier cases were gaps in the database more than the sequencing data itself. Over time, I think those holes will get plugged.” But in cases in which the organism counts in a specimen are low, there are inherent sensitivity issues, he said, “and that speaks to how confident you can be that this is a good test and whether you are willing to accept that.”

For tuberculosis, his laboratory does a lot of PCR testing. Culture is the gold standard, and he explains often that PCR is faster, not more sensitive. Metagenomics is “comparable to PCR in terms of detection sensitivity,” he said, “but a low organism load is going to be an issue.”

“The specificity issue is real,” he said, but it can be addressed by considering and acting on the results in the clinical context rather than treating every organism as a pathogen and increasing antibiotic use.

Contamination as a cause of false-positives can come from the source itself during collection, most commonly a flora, whether it’s skin, oral, or GI. “Sometimes teasing out what might be significant from that is difficult,” though in some ways not much different from assessing organism growth in culture, he said.

“We’ve learned there are ways you can assess it in terms of the metagenomic sequencing data,” Dr. Miller said. When flora is seen in a sample, there tends to be a variety of species and a spectrum of organism detection, whereas in true infections there are fewer organisms that can be collapsed into one or two categories.

“In that way, I think of it almost like a urine culture,” he said, “because if you get a urine sample that’s growing a lot of different Gram-negative rods, that’s probably just a contaminated urine.”

In metagenomics, there could be flora, laboratory contamination, or environmental species, “and that is a challenge we try to address with good technique and controls.” But in some cases the samples are contaminated with organisms that may be clinically significant, such as Ralstonia or Pantoea species. “Those can be difficult to assess,” he said.

“Looking at this from the whole patient perspective is important,” Dr. Miller said, “so we try to make it clear in the interpretive portion of the report what the assessment is from the laboratory side.” UCSF’s clinical microbial sequencing boards offer opportunities for discussion with providers.

And that’s an important distinction, in Dr. Mitchell’s view. “We have to remember,” she said, “that we’re in an academic bubble and have opportunities to collaborate with infectious disease and to make these multidisciplinary teams.” She worries about commercial laboratories that perform mNGS on urine samples sent from clinics with potentially complicated patients, “where they think it’s an advantage. They are sequencing and working up everything. Even if it’s seven different organisms, they consider that significant and are treating as if the patient has a UTI with all of those organisms.”

“The answer to that is education and developing guidance documents,” replied Dr. Miller, who said the practice she described is not supported by the literature, especially for a nonsterile site like urine.

“This isn’t just a lab medicine problem,” Lurie Children’s Dr. Muller pointed out. “This is a societal problem. Just look at the Internet for examples of misuse of technology or at least use in a way that wasn’t intended.”

Of a finding that is not clinically relevant, Dr. Muller asked, “Is that a limitation of a test if it tells you something about the patient? I don’t know if I would say that.” He does admit it requires an understanding of how to interpret the information, “which not everybody is equipped to do.”

For Texas Children’s, the cost question is an open one. “We haven’t done a deep dive because it’s so multifactorial,” Dr. Palazzi said. “We know these tests just experientially are not decreasing costs in our hospital because it’s not avoiding other testing. It’s almost like it results in even more testing sometimes to confirm or refute what is found.”

Most mNGS testing is performed on inpatients, and with UPMC having its own health insurance plan, UPMC pays for the metagenomic testing when it’s done. “That takes away from the ability to acquire new equipment, new laboratory staff,” at a time when the lab is facing staff shortages just for routine testing, Dr. Mitchell said. “When you pull that money away, we start to lose the ability to do other things elsewhere in the lab.” Thus the need to use mNGS on the right patients.

The problem, Dr. Muller said, is not the cost of the test per se but the way health care is paid for in the U.S. And cost does enter into the calculation at Lurie Children’s for when and whom to test. “We are not sending it on patients who were previously healthy and will likely get better with empiric treatment,” Dr. Muller said. “That will be incrementally not as cost-effective as sending it on the patient who’s already cost the health care system hundreds of thousands if not a million dollars and for whom the incremental cost to that patient’s care in the big picture is low.”

Dr. Miller agrees the cost benefit is likely to be seen in the patients for whom the most health care resources are used. “If you want to predict who is a potential beneficiary based on cost,” he said, “a patient who is in the ICU is a high predictor.”

At UCSF, he said, patients scheduled for brain biopsies are also a high predictor. “If you can manage them better with the results, whether positive or negative, you can potentially decrease their overall cost of care,” whether by avoiding the biopsy or reducing length of stay or therapy, for example.

Metagenomic NGS was piloted at Texas Children’s Hospital for a period, at no cost. The hospital’s laboratory stewardship committee initially wasn’t involved in mNGS testing, but the committee has been collecting data with an eye to using the data to develop an algorithm and negotiate for resources, Dr. Palazzi said. Someone has to look at the tests, she said, “and we are in the process of determining what that’s going to look like.” In addition to the financial impact on the lab, there’s an antimicrobial stewardship impact.

“When I think of costs from a stewardship standpoint, I think of all the unintended consequences of the adverse effects of the antibiotics for our institution and the individual patient,” Dr. Palazzi said. “We are in the process of presenting the data and trying to come up with a rational plan to start stewarding this resource in a better way.”

At Lurie Children’s Hospital, “the lab medicine folks were initially very skeptical” about the impact of mNGS, which explains the laboratory’s oversight for all testing that’s done, Dr. Muller said. “It does matter that people are paying attention.”

Said Dr. Mitchell, “Healthy skepticism is good because it causes people to think and pause and put a policy in place.”

Amy Carpenter Aquino is CAP TODAY senior editor.