Risk management steps up labs’ QC game under IQCP

Anne Paxton

September 2014—Industrial risk management. It may not seem all that sexy as a concept, but in the field of laboratory quality control, risk management has become about as buzzworthy as is possible. One of the key reasons: The Centers for Medicare and Medicaid Services has embraced risk management as the foundation of a new option for meeting CLIA quality control standards called IQCP, or Individualized Quality Control Plan.

A voluntary QC option under CLIA, IQCP aims to give labs greater flexibility in achieving QC compliance by customizing lab QC plans to each unique testing environment via the use of risk-management strategies. Employing electronic/integrated controls, IQCP intends to adapt to future technological advances and strengthen manufacturer-laboratory partnerships.

“Many assays appear to be okay when we run our QC,” says Alison Woodworth, PhD, director of esoteric chemistry, associate director of clinical chemistry, and assistant professor in the Department of Pathology, Microbiology, and Immunology at Vanderbilt University Medical Center. “But when we do more in-depth risk analyses, they may not be okay.”

With IQCP, “labs will have a choice now. They can use a one-size-fits-all approach of running two levels of external QC material once a day, the bare minimum under CLIA requirements, or they can do a risk-assessment evaluation to better determine how their assays are performing and how much QC they should run.”

IQCP is available for all CLIA-regulated nonwaived tests except pathology and its subspecialties, which the CMS says will be considered later. The CMS announced IQCP last fall as an exercise of its enforcement discretion under CLIA; CLIA regulations of QC remain unchanged from the last update in 2003.

Following IQCP’s official launch on Jan. 1, labs were given two years to make the transition to IQCP and to test laboratory operations under that option. After Jan. 1, 2016, the old EQC, or Equivalent Quality Control, procedures will no longer meet CLIA standards.

What does IQCP mean for laboratories in specific terms? No process maps, fishbone diagrams, or formal risk-assessment charts and protocols will be required for laboratories to develop an IQCP, the CMS promises. But laboratories will need to have sufficient data to support their decisions, and all IQCP activities must meet documentation requirements.

IQCP and its related method, the EP23 standard offered by the Clinical and Laboratory Standards Institute since 2011, may save laboratories some work, but do not necessarily represent a lowering of QC requirements, says quality control expert James Westgard, PhD. A cofounder of Westgard QC in Madison, Wis., Dr. Westgard was the first chairman of the Evaluation Protocols Area Committee in CLSI (then known as NCCLS).

Although he would prefer to see more rigorous regulation of QC, “Technically, if you look at the CMS regulations, the requirements haven’t been reduced with IQCP,” Dr. Westgard says.

For laboratories tasked with performing quality control, he says, “You have option A, you have option B, and now you have option C. Option A is guidance on doing the right QC for precision and accuracy and detecting medically important errors. Option B, if you do not want to do that [option A], is to run at least two levels of QC once a day. And now option C is this risk-based QC, and if you do it, you can probably get by with less.”

Risk management—using analytical tools to assess the risk of reporting unreliable patient results and risks that attend such results—is a major new direction within the field of laboratory quality management, Dr. Woodworth says. “Risk management didn’t start in the clinical lab. It started with manufacturing and the folks using six sigma, but now with the publication of CLSI’s EP23 and other similar documents, it is a hot topic in laboratory medicine.”

Although the CLSI EP23 guideline is a tool for implementing IQCP, the fact that it has been in place for a few years already has helped people become oriented to the concept of risk management, she adds.

Using risk assessment, Dr. Woodworth’s research focusing on HbA1c has vividly demonstrated some of the inadequacies of analytical QC programs currently in place in clinical laboratories. Looking at eight different HbA1c platforms at four academic medical centers, her team used QC materials and analytical tools to assess risk.

“What we found is that most assays out there are not that good. The bare minimum approach of two levels of QC once a day is not sufficient to evaluate the risk of reporting unreliable results for HbA1c.”

All but one platform showed that the laboratory should be running the maximum number of QC (three levels three times a day). “It caused us to take a step back and think about the performance of this particular assay,” Dr. Woodworth says (Woodworth A, et al. Clin Chem. 2014;60: 1073–1079).

That doesn’t mean every test should have the maximum rather than the minimum QC, she notes. “We calculated sigma metrics, which are an estimate of risk. They take into account the allowable total error for the assay along with the analytical performance, the precision, and the bias of the assays. The sigma is equivalent to the six sigma goal in manufacturing. In the laboratory the sigma metric helps to estimate the risk of reporting an unreliable patient result—one outside the allowable total error limits. Then we can use recommendations from national thought leaders to determine an appropriate analytical QC program based on sigma metrics—i.e. how many levels and how often QC should be run each day.”

“It’s really just a simple formula to calculate a sigma metric. Anybody who uses Excel can gather the analytical data they are already generating, imprecision can be gathered from the QC they run, and bias can be garnered from their proficiency testing results.”

Software tools by such companies as Bio-Rad, CRI, and Carepoint Solutions have been developed to help. (Randox’s Acusera 24.7 wasn’t designed to be used to develop an IQCP but can be a useful tool in carrying out an IQCP, according to a company spokesperson.) Dr. Woodworth’s laboratory now uses Bio-Rad’s Unity, which receives data from the automated instrumentation in the clinical chemistry lab and allows frequent evaluation of QC. “With Unity, I can look at shifts and trends in my analytical QC as well as generate sigma metrics and receive recommendations on amount of QC to run.”

A current limitation is that not all labs connect all instruments to the Unity program. “If you are using devices that are not interfaced, then you have to manually enter the data,” she says. For example, with Vanderbilt’s point-of-care testing and some of the endocrine lab assays, QC results are entered manually. “It can be done, but it’s more challenging.”

But Dr. Woodworth is enthusiastic about the potential contribution of IQCP to laboratory quality. “We all should be looking more closely at how each assay performs prior to determining how many QCs to run. But the only way IQCP is going to work is if people do a good job of evaluating the risk. Where it could falter is if the risk-assessment piece is not complete. All labs must understand that they need to start working on more complex statistical evaluations of the analytical piece as well as incorporate the total testing process, the preanalytical and postanalytical.”

It’s important to keep in mind, Dr. Woodworth stresses, that EQC is going away and IQCP is already available to start trying out. “Very soon, labs will not be able to use EQC. Most labs do assess risk and work constantly to refine processes, but IQCP is providing a little more structure to what we do already.”

With EP23 and IQCP, “CMS is getting out of micromanaging and dictating that there is only one way to run QC,” says James Nichols, PhD, medical director of clinical chemistry and professor in the Department of Pathology, Microbiology, and Immunology at Vanderbilt. IQCP and EP23 really evolved over the last 10 years out of concerns about the limitations under CLIA of one-size-fits-all QC, he adds.

“The EQC protocol was developed to interpret CLIA regulations for levels of QC, but it obviously wasn’t scientifically based, and things like molecular testing and single-use cartridges with a lot of built-in QC were coming out. People wanted a standard that could be enduring for future devices on the market, so they wouldn’t have to go back to the drawing board when a new method came out.”

As chair of the EP23 group, Dr. Nichols helped shepherd that standard to approval. He sees EP23 as representing a return to the historical principles of total quality management—looking at the entire test process, finding the weaknesses in that process, and addressing those weaknesses systematically. IQCP, he says, is basically an interpretation of the principles that are in the EP23 document for regulation and inspection of laboratories.

Is the introduction of industrial risk-management principles into the clinical laboratory a reasonable evolution of QC? “Yes, absolutely,” he says. “We know that running two levels of QC on our analyzers is no insurance against getting bad results. Some of our technologists have a false sense of security that if they run QC, their results are good. But every day we get complaints from physicians that a result doesn’t match the clinical situation with the patient.”

“QC will not tell you about bubbles, clots, hemolysis, lipemia, and so on, and there are other random things that can go wrong with individual samples that QC isn’t going to tell you about. To address those particular sources of error, we need to take a different QC perspective.” For example, he says, at Vanderbilt the laboratory is running indices on its chemistry assays. “Basically we’re looking spectrophotometrically at every sample as it comes down the line, and when we get an amount of hemolysis above a particular amount, we know we may have a problem with a specific test.”

An added benefit of IQCP is that it will help support the laboratories’ need for resources, he believes. “We do a lot of things in the lab that we don’t get a lot of credit for, such as running proficiency surveys, training, competency—things we can’t bill for. But every one of those goes into managing the quality of that test result. Now you have a plan, a total quality management plan, where you’re integrating all of that and linking it directly to weaknesses in your testing process and showing how you address quality in the lab.”

IQCP will be particularly helpful for point-of-care testing, he says. “Ask anybody in the lab and they’ll confirm that point-of-care testing, using these waived devices, really needs better quality management. As it is now, under the regulations, all you have to do is follow manufacturer instructions and pay your fees, even though we know if you get wrong results it can lead to wrong decisions and bad patient care. For a long time, the waived category has slipped under the radar screen, but I hope in the future there will be more attention paid to it.”

Where IQCP will really make a difference in his laboratory is with POC tests for which the lab is not running QC every day but maybe once a week or once a month, per manufacturer instructions. “By looking at all the aspects of your testing process, figuring out where the weaknesses are, and going through the processes outlined by CMS to develop an IQCP, this will really benefit labs because they will identify weaknesses before an inspector does, or before a bad patient outcome.”

Developing an IQCP will bring more errors to light that weren’t discovered before, he says. Even though IQCP isn’t going to be enforced for glucose meters, “simply mapping your processes, getting out of the laboratory, watching the fingersticks, watching how the test is run, you can discover and see steps where errors can occur. Because of our expertise as laboratorians, we can see and predict preanalytic problems that we may not understand when we wrote the original procedures. And that will help us improve the process.”

He urges laboratories not to see IQCP as “Oh my God, another thing I have to do in my already too busy day.” “This is going to help improve the quality of testing out there and will help improve your testing processes. It doesn’t take a lot of time for most devices. You can write a good QC program in under an hour.”

“It simply means going out on the floors or to the stat lab or ED lab and watching samples being collected and transported to the device, watching how everything is being conducted up to transmission of results back to the clinician. Then you close that loop and look for problems, run through your mental checklist and think about how to address each of those points. That’s all a QC plan is.”

It’s something of an anomaly that EP23 emerged first, since it gives guidance on how to do a risk-based QC plan, says Andy Quintenz, scientific and professional affairs manager of Bio-Rad’s Quality Systems Division in Irvine, Calif. “The IQCP is really a sort of checklist of what you have to have in your document; EP23 tells you one way to create the document with the steps to go through to get the information needed to make a decision.”

Having been developed by CLSI, the EP23 guideline must be purchased, while IQCP interpretive guidelines are available free online from the CMS. But both EP23 and IQCP were created to answer the question of how often QC should be run, Quintenz says. “It’s a question that’s never really been answered before.”

“QC rules are all about acceptance or rejection, and how much variance is allowed in an instrument’s performance. But how often to run QC became a big question with the explosion of popularity of POC testing devices, where a lot were cartridge-based and manufacturers promoted that, due to the sophistication or smart technology of the instruments, you didn’t really need to run QC as often—maybe only once every 30 days or when a new shipment comes in, to make sure the cartridges are stable.”

Many factors come into play in determining how much QC should be done, Quintenz explains. “The two levels are the minimum mandated by CLIA—typically a low and a high, or a positive and a negative with a qualitative test—and some manufacturers recommended three levels because of the range of their analytes. But the choice of levels is more a function of the level of quality you’re going after at your laboratory.”

The frequency of QC is set at a minimum of once a day by CLIA, with some exceptions. For example, blood gases have to have QC conducted once per shift. But Quintenz knows of high-volume laboratories that regularly run QC every two hours. “The reason is that where they’re doing thousands upon thousands of tests per day, if they go four or eight hours without running QC, then if they have a QC failure, they have to go back and retest all those samples. They would have to waste a whole other day doing the repeats.”

But if a hospital runs only two levels of QC once a day, each morning, for example, and one morning the QC fails, it opens the question about the tests performed the day before: Do they need to be repeated? “That’s why we talk about risk-based QC,” Quintenz says, because for some tests the risks would be much higher than others.

For example, if a troponin is used to determine whether a patient had a heart attack, a positive result would quickly send the patient off to some kind of interventional therapy. “So that is a very, very high-risk test, as opposed to an HbA1c, where the result may be relating to a trend and not as critical. You may be more willing to have a lower frequency of QC for a test like that, as opposed to a cardiac test.”

Bio-Rad is completing development of an IQCP-oriented software tool to complement its Unity program. “This tool will allow labs to look at the performance of the test along with their QC rules and their risk comfort level and give a recommendation for QC frequency. If a lab says it is more comfortable with more patients being tested between QC events, for example, that’s taken into account. We do this on an analyte-by-analyte basis, because different analytes have different risk levels associated with them.”

Called Risk Calculator, the Bio-Rad software is slated for release in early 2015. Other programs designed to help laboratories with IQCP include Carepoint Solutions’ EZ-QCP and a software package available from CRI, the educational arm of COLA, called IQCP E-Optimizer.

Laboratories seem eager for guidance on QC plans, Quintenz says. “We thought most of the bigger labs really understand sigma metrics and wouldn’t be into this sort of tool, but we had a preview of the Risk Calculator software at the recent annual meeting of the American Association for Clinical Chemistry and we were happily overwhelmed by the interest from labs of all sizes, saying this is really something that helps them answer a question they’ve had for a long time.”

In talking to labs, he tries to emphasize that IQCP and risk management are processes to make decisions about QC frequency, but they are processes with limits. “There’s nothing in there that leads you to assume that if you do this, it’s going to be right. It really varies from instrument to instrument and lab to lab, as to what your risk comfort level is and what your instrument performance is. So we tell labs to take time to understand it, talk to people, get some education, and then start breaking it down into reasonable-sized projects and start chipping away at creating their own IQCP.”

The CAP Laboratory Accreditation Program plans to introduce IQCP in its July 2015 checklist, subject to CMS approval. The CAP Checklists Committee, together with the Point-of-Care Testing Committee, is working on changes and will submit a plan for concept approval in early fall.

But even as IQCP is being phased in, the Laboratory Accreditation Program will be adapting to the new QC option, says William J. Castellani, MD, medical director of clinical chemistry at Penn State Hershey Medical Center, interregional commissioner for the CAP Laboratory Accreditation Program, chair of the CAP ISO 15189 Committee, and member of the CAP Council on Accreditation.

The CMS has already issued revisions to the state operations manual (which goes to state departments of health) to instruct inspectors about IQCP because it will replace the EQC program. Although both IQCP and EQC are still permissible right now, the checklist will need to change.

“CAP is very actively looking at issues associated with how to inspect for an IQCP-compliant program,” Dr. Castellani says. “The revisions CMS published in the state operations manual, in my opinion, make interpretation very subjective, and we need to try to provide something consistent with IQCP and CMS’ expectations that can also be inspected appropriately by a volunteer team.”

For example, Dr. Castellani says, storage condition requirements are a common QC situation. “It’s not unusual for refrigerator temperature monitors to transiently detect temperatures above the target point, if the door is open too long. Does this transient exceeding of guidelines really affect the quality of reagents to the point where there is an increased risk?” What exactly is adequate is an area still open to interpretation and possible differences of opinion by the inspector and the laboratory.

Dr. Castellani thinks IQCP offers laboratories a significantly improved means of QC. “To implement IQCP under CMS, there must be an ongoing quality monitoring protocol for the effectiveness of the QC. Right now under standard CLIA QC, not IQCP, the basic corrective actions are fairly limited.”

While IQCP is likely to be more expensive for laboratories because it will be their own creation, he says, “this will be a change in the way labs deal with QC. Because not only do they need to deal with outliers on an immediate basis, they must also monitor the effect of their QC programs on an ongoing basis. It’s more than just ‘we did the controls, we did the standard deviations, so we should be okay.’”
Risk assessment itself is fairly straightforward, but the rest will require additional laboratory staff time and “paperwork, paperwork, paperwork,” Dr. Castellani says. “You’re not only justifying the initial implementation but also justifying how you are continually putting patients at risk using this plan. It’s very possible for labs to have multiple plans for each specific platform.” For example, “Different IQCP plans will be required for glucose meters than for iStats because they are not interchangeable devices and their plans cannot be interchangeable.”

Until now, EQC has been used for a lot of POC devices, and implementing IQCP for point-of-care devices will be a challenge, Dr. Castellani adds. “When you look at the instrumentation and the huge numbers of devices out there, scattered throughout the hospital and routinely run by non-laboratory individuals, this is where a major impact of IQCP will be.” Another likely use, he says, is in developing a quality control system for tests assessing rare diseases, “especially genetic tests where patient material demonstrating the target abnormality may be hard to come by. A protocol that evaluates the testing system, rather than the specific analytical target, may satisfy IQCP requirements if properly documented.”

One other element of IQCP that laboratories should be aware of is required under CLIA rules (42 CFR 493 1282 (b)(2)): “standard corrective actions.” Specifically, “If I run QC on a device today and it fails on my IQCP, not only do I have to remediate the QC issue, but I have to call into question all of the results on that instrument, and that will be really challenging under IQCP, since each device is a separate test system. There has to be some process for evaluating whether or not patient results were compromised and how far back they were compromised,” Dr. Castellani says.

That procedure could require a lot of effort for tests where QC is intermittent. “When I’m running daily QC, I only have to go back to the last 24 hours. But if my instrument has not been checked in the last month”—which could happen under the EQC protocols for electronic controls put in place previously by CMS and under IQCP protocols in the future—“I’ve got a month’s worth of data I have to look at and evaluate and have a strategy to assess the impact on patients.”
Moreover, there is no standard protocol for notifying clinicians about QC problems. “In my chemistry lab, it’s in our quality management plan for daily QC to address patient results that are compromised. But that will not transfer necessarily to another section like microbiology or blood bank or point of care.”

Dr. Castellani believes that IQCP will increase laboratories’ awareness of the broader context of QC. “To implement an IQCP program takes an understanding of the entire process from sample collection to results reporting, and realizing that past results may also need to be re-investigated and having a plan to cover all those contingencies, and when the plan fails, responding to that proactively. When you develop an IQCP, you need to realize that now you’re not just looking at QC results; you’re looking at the entire system.”
[hr]

Anne Paxton is a writer in Seattle. Dr. Castellani will give a CAP webinar on Oct. 15, with Guang Fan, MD, PhD, on “Quality Control: What You Need to Do and How You Can Show That You Do It.”