Leaving behind outdated AST breakpoints

Karen Titus

May 2022—Among the countless interruptions COVID-19 has inflicted on the medical community, one of the most obvious has been conversational. In the face of a global pandemic, other topics can seem unworthy of discussion.

But as some post-pandemic normalcy creeps back in, so does the focus on topics of equal, if less dramatic, importance.

Two years ago, experts were just digging into data collected through a voluntary supplemental questionnaire, distributed with a 2019 CAP Bacteriology Survey, about antimicrobial susceptibility testing breakpoints. The information, shared with Surveys participants, “was a bit of a teaser,” recalls Trish Simner, PhD, D(ABMM), associate professor of pathology and medicine, Johns Hopkins University School of Medicine, and director of the medical bacteriology and infectious disease sequencing laboratories, Johns Hopkins Hospital. She and others involved in the work called the results striking but couldn’t say more, at least not publicly, at the time.

Now comes the big reveal, in the form of a February publication in Open Forum Infectious Diseases (Simner PJ, et al. Open Forum Infect Dis. 2022;9[3]:ofac007). That article, together with a new requirement in the 2021 edition of the CAP accreditation microbiology checklist, should be the revving engine that drives laboratories to adopt up-to-date breakpoints.

The new requirement, MIC.11385, says laboratories subject to U.S. regulations must, by Jan. 1, 2024, use current breakpoints for antimicrobial susceptibility test interpretation. By that date, at minimum, laboratories will need to employ breakpoints that have been updated within three years of official publication by the FDA, though the laboratory may use currently accepted breakpoints from other standards development organizations (for example, CLSI) with validation to support use.

The February publication points out what’s at stake—not only how labs are falling short, but the cost to patients.

Recipients of the 2019 questionnaire (sent to 2,296 labs, with a 60 percent response rate) were asked if they were applying current breakpoints for seven so-called drug-bug combinations—combinations with breakpoints that the FDA or CLSI had updated relatively recently (since 2010). Those who weren’t using them, or were unsure which breakpoints they used, were asked why they weren’t up to date.

Depending on the organism/antimicrobial agent, between 40 to 70 percent of U.S. labs were still using breakpoints now considered obsolete. (For international labs, the range was 20 to 40 percent, “which really drives home the point that this is a larger issue within the United States,” Dr. Simner says.) The results reflected when the breakpoint changes occurred, she says—essentially, whether automated AST manufacturers and clinical labs have had sufficient time to devise strategies for updating the breakpoints. The number of labs using current breakpoints for the cephalosporins and carbapenems (updated around 2010) was greater than the number of labs using current breakpoints for the fluoroquinolones (updated in 2019).

Of the labs that were still applying outdated breakpoints, roughly 50 percent reported they recognized the problem and planned to update them. The other half had diminished ambitions, saying they either didn’t intend to update their breakpoints or had no current plans to do so.

“When we further asked why they’re using the outdated breakpoints,” Dr. Simner says, “the largest response we received, about half, was manufacturer related.” More than 90 percent of labs in the United States use automated antimicrobial susceptibility testing for their methodology to determine AST results; if a manufacturer’s panel and software aren’t updated, labs are left in the lurch. (In the current regulatory environment, when a manufacturer submits a susceptibility testing panel to the FDA to obtain clearance, it must apply the FDA breakpoints in place at that time. However, the manufacturer is not required to update those breakpoints if and when they change after FDA clearance.)

Another 20 percent of labs cited lack of internal resources to perform the validation of new AST breakpoints, Dr. Simner continues, including being short of staff, money, and perhaps expertise. In addition to lack of time and other resources, “it comes down to the cost of doing a validation,” Dr. Simner says. Unless an institution’s upper management sees the need to make this a lab priority, outdated breakpoints will remain in force.

“There are so many competing demands,” Dr. Simner observes. “Hospitals are finding it a challenge to prioritize this.” Or they may not understand the need to update breakpoints, she says, or the impact of using outdated breakpoints.

Interestingly (or perhaps unnervingly), another slice—13 percent—of respondents reported being unaware of breakpoint changes or the need to update them. Or, as Dr. Simner puts it: We had no clue this was an issue.

That clarifies the mission for Dr. Simner and colleagues: to increase awareness among labs of all sizes, given the patient safety and public health repercussions of using outdated breakpoints. If labs can’t detect critical and urgent threats, she says, “then we’re not going to be good at containing or preventing the further spread of these resistant bugs.” Standard AST methods are the primary methods used globally to detect antimicrobial resistance, she adds. When detection fails, “we’re actually perpetuating the further spread of these bugs.”

The February publication is crucial, says coauthor Carol Rauch, MD, PhD, adjunct associate professor of pathology, microbiology, and immunology, Vanderbilt University School of Medicine, and formerly with the CDC’s Antibiotic Resistance Laboratory Network. “We can have an opinion, and we can say there’s a problem, but it’s very different once something’s in the published literature. These data provide a focal point to have the discussion.”

She agrees with Dr. Simner and other colleagues about the urgency. In her role with the AR Laboratory Network, she says she had concerns that the “front line of this program, where resistance is initially detected in clinical laboratories, isn’t functioning as well as we all need.” Not only are there patient care consequences, Dr. Rauch says, but “these initial results from clinical labs are what feeds into the public health infrastructure. From any viewpoint,” she adds, “we cannot afford to miss resistant pathogens.”

AST breakpoints for each bug-drug combination are set initially based on three categories of data, says Isabella Martin, MD, D(ABMM), medical director of microbiology, Department of Pathology and Laboratory Medicine, Dartmouth-Hitchcock Medical Center. Standards development organizations such as the FDA and CLSI look at the following: the distribution of minimum inhibitory concentration values in hundreds of wild type bacterial isolates versus those with known acquired resistance mechanisms; pharmacokinetic/pharmacodynamic data for the antimicrobial agent in question; and clinical outcomes data, whenever possible. Breakpoints subsequently change when new data come to light in one of those three categories.

“Specific studies detailing the overall effect on patient outcomes from widespread use of outdated breakpoints do not currently exist. However,” Dr. Martin says, “we know that those breakpoints were updated for a reason and that continued use of outdated breakpoints puts patients at risk for being treated with ineffective therapy.”

“At the end of the day,” says Dr. Martin, a coauthor on the Open Forum publication, “most of us will need an antimicrobial at some point in our lives to treat an infection. It’s important we are able to accurately assess the effectiveness of common antimicrobial agents in our laboratories.”

The labs that aren’t aware that breakpoints have changed may be misclassifying a particular organism as susceptible, whereas a current breakpoint might identify that same organism as resistant. Says Dr. Rauch, “If we’re going to be tracking rates of resistance, we had better make that determination accurately.”

She says she found it fascinating to read through the hundreds of notes from respondents explaining why they weren’t using current breakpoints. The reasons are predictable: lack of resources, including expert personnel. Others noted that they awaited the green light from corporate, as in, Corporate has to tell us. “Who’s corporate?” Dr. Rauch asks. “Corporate probably doesn’t include a microbiologist.”

It’s clear that laboratories need to act—but so do others. How can regulatory agencies, accrediting organizations, clinical labs, and manufacturers come together to solve the problem?

All share a common goal—patient safety—but each group faces a different barrier, Dr. Simner says. “So how can we make this better without pointing fingers?”

It’s a subtle dance. “The CAP has taken a difficult but necessary step,” Dr. Rauch says of the new checklist requirement. In some ways, the logistics are the least of the problems.

As she and others worked to bring about changes, Dr. Rauch recalls, they spent untold hours trying to strike the right tone. “We don’t want anyone to come to this in anger or frustration.”

“This is big enough that we should all want to pay attention.”

She explains further: “You want to say that it’s a patient safety issue, and it is. That’s totally legitimate. What we don’t want is for people to overreact.” And given that every party has different constraints, “we want to communicate this in a positive and responsible way.”

Choosing a deadline showed how this challenge played out, says Dr. Rauch. “We wanted to pick a time off in the future, because we recognize labs have work to do to prepare.”

“We also had to lay out a path for how to get from here to there,” she continues. “We want to be deliberate and rational and calm and serious. We can’t ignore this, and we don’t want labs to be panicky. We don’t want people worried that they’re giving out a bad result without any clue what to do about it. So we need to give them support in the form of guidance and resources.”

There’s another delicate issue at play: the comfort level between a manufacturer and the lab. A lab might have been using a particular commercial system for years and, as with any relationship, change is hard. So even if the manufacturer hasn’t updated its breakpoints, no one’s necessarily agitating to break up.

In those cases, says Dr. Rauch, “we are asking them [labs] to either switch to a newer panel that applies current breakpoints, or to do a little bit of what is technically off-label work. And CAP does not enter into a recommendation like that lightly.” She calls the CAP conservative, careful to protect laboratories, and bent on “staying within every regulatory guideline we can think of.” But with AST, “we unfortunately don’t have a regulatory environment that has kept up with the resistant bugs. We have carbapenem-resistant organisms out there. Sometimes they’re called ‘the nightmare bacteria.’ We don’t like to use inflammatory language like that, especially with patients. But if we miss one,” the impact can be huge—for individuals, hospitals, and communities.

As she identifies the gaps between the key players, Dr. Rauch could be describing a Jane Austen plot. “There are insufficient relationships between and among all these partners.”

And even if a manufacturer’s breakpoints were sufficient at one point, they may not be now. Lead pipes were deemed sufficient for municipal water supplies at one point, too. The organisms are evolving in real time; the regulatory environment supporting revised breakpoints is not.

That leaves it to labs, at least for now, to “do some extra work that allows them to apply different interpretive criteria,” beyond what may have been established years ago when their commercial devices were cleared by the FDA.

Manufacturers, for their part, will cite the huge financial burden of updating breakpoints, as well as regulatory speed bumps along the way. Newer agents have a more streamlined process for this, but for older agents, it’s not simply a matter of providing data to the FDA for recognizing updated breakpoints. If the submission process for older agents could be similarly streamlined, Dr. Simner says, it would be much easier for manufacturers to update their panels and systems with current, FDA-cleared breakpoints.

Another complication: FDA and CLSI breakpoints aren’t always in agreement. The FDA’s breakpoints may be based on its initial approval for indicated organisms and dosing, which may differ from what happens in clinical practice. “When we revisit the breakpoints,” says Dr. Simner, “we’re generally applying the dosing that is used currently.”

If, for example, manufacturers want to update their carbapenem breakpoints—for, say, Enterobacterales—they might then lose the ability to address carbapenem breakpoints in other organisms.

Work has already been done to decouple breakpoints from prescribing information on the FDA STIC (susceptibility test interpretive criteria) website, although more work remains. “There’s got to be some sort of further lobbying, essentially, to get this changed,” Dr. Simner says. The FDA already recognizes CLSI as a standards development organization and can recognize its breakpoints. So why not apply it more broadly, she asks, and allow manufacturers to apply either group’s breakpoints for clearing their devices?

A similar give-and-take can occur when manufacturers puzzle through how to allocate their own resources, Dr. Simner says. Is it better to develop a new generation of more rapid, more accurate phenotypic or genotypic tests to detect antimicrobial resistance, or to update clinical breakpoints for past devices?

For its part, the CAP, with the February publication and its accreditation checklist requirement, has taken steps to frame and quantify the problem and improve lab practices.

Says Dr. Simner: “It’s nice to see the CAP draw that first line in the sand to prioritize this. We know it won’t be easy for clinical labs, but at the same time we know it needs to be done.” The CAP’s move should nudge other groups to act, she says.

Discussions at CLSI meetings, for example, often have a timeworn feel to them, she says. “In our M100 documents, we still refer to, ‘If you use the old 2010 breakpoints, you do this.’” But why the tacit encouragement to use outdated breakpoints? Now that the CAP has decided to discourage use of obsolete breakpoints, she says, the CLSI should do the same. She’s hoping for a domino effect.

The move should also encourage a larger gathering among all the groups. While manufacturers are meeting with their clinical labs, as well as with the FDA, “we all need to sit at the table together. We need to use our entire efforts and influence as these large groups to make the change,” Dr. Simner says.

How confident is she that everyone will do so? “I tend to be very optimistic. If we can describe our barriers and come together, I think we can do it. It’s not going to be easy. But we are a collegial group, and we are used to working together.” The next step is to address the key question: How should they do this? Everyone already agrees it should be done. “We need to brainstorm and provide a unified path forward.”

With the pandemic consuming most of the headlines and energy in the past two years, Dr. Simner says it makes sense to talk about a second pandemic—not the influenza/COVID-19 “twindemic” that many feared heading into fall 2020, but antimicrobial resistance and COVID-19. AMR is simmering in the background, she says. Indeed, the topic had been garnering plenty of attention and resources in recent years, right up through 2019. “And then COVID hit, and a lot of resources were taken away from addressing antimicrobial resistance.” But from a clinical and public health perspective, COVID and AMR are the two main global issues.

“We do get excited to talk about non-COVID topics,” adds Dr. Martin with a laugh.

Dr. Martin defines her institution as a midsize, rural academic site. “We’re approaching this with excitement, recognizing the value to patient care that it represents, but also with a little bit of trepidation, because it is potentially a heavy lift to figure it all out and get it down on paper.”

Her lab uses two different automated susceptibility platforms, from different commercial vendors. “We also do a handful of gradient diffusion tests as well as disk diffusion tests. So some breakpoint interpretations are going to be applied at the LIS level while others are applied at the instrument level.”

Even though they went live with some new panels in 2019 (“I have confidence we’re up to date, though it never hurts to verify that”), Dr. Martin has no doubt that she and her colleagues will uncover surprises, especially with the disk and gradient diffusion tests. “Just because we’ve been doing them so long. It’s not clearly documented anywhere what breakpoints we’re applying.”

Ultimately, having updated breakpoints, with the information all in one place, will ease the way for future changes. It will be nothing less than “a source of truth,” she says.

Truth, as any seeker knows, doesn’t come easily. “It is a challenge, in the midst of being chronically short staffed, and having many other projects and ongoing quality improvement initiatives,” Dr. Martin concedes.

She’s not alone in hoping that vendors will anticipate the lab’s questions and needs, and will be able to interrogate the instruments to provide a list of breakpoints being applied for each drug-bug combination. But even if labs have helpful vendors, they could still find themselves manually exporting large documents into an Excel file, for example. It may not be as easy and as seamless a process as some labs hope, she says diplomatically.

Those in the thick of working on AST breakpoints have a passion for the topic, Dr. Simner acknowledges. “But we realize there are labs of all sizes, with varying levels of directorship and guidance available to them, that now need to face this reality of understanding what breakpoints are applied to their systems.”

That’s why revised accreditation checklist requirement MIC.11380 focuses on making sure labs understand the breakpoints applied by their automated system, she says, or are used in the lab to interpret susceptibility testing results. (In checklist editions prior to 2021, MIC.11380 was MIC.21930.)

Before labs can tackle a problem, they need to know its scope. Asks Dr. Simner: “How big is your problem? How far behind are you?”

While it might seem straightforward to figure out what breakpoints are in use, Dr. Simner is quick to disabuse labs of that notion. “To be honest, I found this exercise [outlined in MIC.11380] extremely helpful.” Indeed, she and her colleagues uncovered issues with breakpoints they were using. “It affects us all.”

Not only is it a matter of understanding the breakpoints applied by the automated system, but also by the electronic health record, she says. “It’s complex.” Some labs will have all their breakpoint interpretations put into the middleware associated with their automated instruments; others will do interpretations of their own, transmitting changes from the LIS into the EHR.

The first thing labs should do, Dr. Simner advises, is to ask their manufacturers, Can you pull all of the interpretations that my automated instrument is using to interpret the AST results right now? “Get a list,” she says.

From there, labs will need to compare those with current FDA and CLSI breakpoints. It is, she acknowledges, a tedious task. “I’ve done it myself. It took a lot of time.”

Once a lab has identified the gaps between its own breakpoints and those of the FDA, CLSI, and even EUCAST (from the European Committee on AST), it can prioritize those that need updating sooner rather than later. For some labs, that could be an extensive list; others might need to address only a few.

Working with antimicrobial stewardship committees will also help prioritize the work. Which breakpoints are most important clinically, based on the formulary, patient population, etc.?

Dr. Simner wants to be clear: “Labs don’t need to do all of this review and documentation at once. They just need to show that they’re addressing it, that they’re working closely with their antimicrobial stewardship committee or with a clinical champion to become up to date.” Otherwise, she says, it may seem overwhelming to those who haven’t updated any breakpoints since 2010. “As long as you’re making progress, that’s what we want to see.” Once this work is underway, labs will need to check annually to ensure they stay up to date.

Currently, CAP-accredited labs are not required to use updated breakpoints—MIC.11380 in the 2021 edition of the microbiology checklist permits use of obsolete breakpoints. The new checklist requirement, MIC.11385, will soon close that gap.

A CAP FAQ document, to be posted on the CAP website soon (in the Accreditation Resources section of the e-Lab Solutions Suite, microbiology checklist requirement QA subsection), will point labs to the FDA STIC website (https://bit.ly/FDA-breakpoints). The tables are fairly straightforward, Dr. Simner says, and the site includes an area that helps labs identify when the FDA recognized the breakpoints. (A CAP TODAY “Q&A” published in 2020 addresses validation studies for breakpoint updates, https://bit.ly/0320QA.)

When Dr. Simner peered into her own lab’s breakpoints, she found they hadn’t updated the breakpoint for daptomycin and Enterococcus faecium. “But I found we were using the current M100 document when we were manually interpreting the results.”

The disconnect surprised her. “You should be applying the same breakpoints whether you’re using a manual disk diffusion or a gradient diffusion method, compared to what’s in your automated system,” she says.

She also found that some breakpoints they thought had been updated didn’t get switched to expert rules. “It wasn’t put live in the system, so despite our efforts to validate the breakpoint, it wasn’t turned on and wasn’t being reflected in our results.” The checklist exercise is “great,” she says. “You catch those little things that might not be caught otherwise.”

It became clear that their LIS wasn’t good at susceptible dose-dependent results (SDD), she says. “So we had to make those transitions occur from our LIS, which listed SDDs as an intermediate result, and cross into our electronic health record as an SDD result.

“That’s tricky,” Dr. Simner continues. “If you just go to your breakpoints within your LIS, you might not realize this. So in addition to looking at what’s in your LIS, you should also be looking at your patient reports,” especially for labs that report out the MIC and the interpretation. At every juncture—the middleware from the AST system, the LIS, and the EHR—there’s a possibility that one of the systems will switch the interpretation or the results, based on how each is programmed.

“I don’t want to make this too complex,” she says, “but laboratories, as they go through and try to identify what breakpoints are applied to their final results in their patient report, they need to take all of those into consideration.”

If the pandemic brought the laboratory to light, so to speak, can that same attention be harnessed to make AST breakpoints a priority? After all, momentum had been building. “We need to make this a bigger deal,” Dr. Simner suggests. “We need to make this newsworthy. Because that’s what finally got leadership—and the world—to recognize what we do for patient care.”

“Everybody knows that antimicrobial resistance is a big deal,” she adds. But they may not understand that the lab is the gatekeeper. It’s a little more complex, she says, than the paradigm COVID-19 presented, which was: Can we test for it or not? And without concrete examples of what can go wrong, it’s hard to muster concrete support. “We need to make a ruckus. We need to make a noise.”

Labs would do well to take a page from the playbook of their infectious disease colleagues, Dr. Simner says, noting they’ve been the advocates for antimicrobial stewardship teams and addressing antimicrobial resistance from a prescribing standpoint. There’s been no similar focus on how clinicians’ actions hinge on how AST is performed and interpreted.

Dr. Rauch, for her part, calls for a gentle ruckus, at least within the health care community. In extreme cases, involving a patient’s death, the topic can almost feel off-limits. “It’s radioactive. People are feeling too awful to talk about it.”

Near-misses, in her experience, can be the ideal chance to make changes: Something awful almost happened—but we caught it. Then the conversation becomes, How are we going to make sure we catch it again? And make sure we prevent it from happening in the first place? “That’s the space that I have found most productive for catching everyone’s attention,” Dr. Rauch says. “Those are the cases where people can engage and use their problem-solving skills in a creative way, and they’re not just trying to hide and apologize.”

With clinical and public health consequences for AST and resistance detection, she says, “we need everyone to help solve this.” 

Karen Titus is CAP TODAY contributing editor and co-managing editor.