| Looking for an equivalent
quality |
 |
|
| Looking for an equivalent
quality |
|
|
|
May 2005 Current QC protocols When the Centers for Medicare and Medicaid Services outlined three new options for performing what it calls "equivalent quality control," or EQC, under CLIA ’88 in January 2004, the agency expected that the options would better accommodate new technologies, reduce the frequency of QC for stable test systems, and provide flexibility. What it didn’t expect was that EQC would be so confusing for so many labs to implement. "After we released the three EQC options in our CLIA surveyor guidelines, it wasn’t long before we started hearing concerns. We had calls and e-mails from laboratories, manufacturers, professional groups, and accrediting organizations," said Judith Yost, MA, MT(ASCP), director of the CMS Division of Laboratory Services, speaking at the March 18 "QC for the Future" workshop in Baltimore. "There is innovative and new technology that we did not anticipate, and we have created some policies that are perhaps inconsistent with what the accrediting organizations have," she explained. Recognizing that it may be time to re-evaluate the direction laboratory QC is taking, Yost, her fellow CLIA ’88 regulators, and professional organizations—including the CAP—partnered with the Clinical Laboratory Standards Institute to convene the "QC for the Future" workshop. "Through this workshop, we want to generate the framework for the development of future quality control consensus protocols," said Thomas L. Hearn, PhD, president of CLSI and associate director of the Division of Laboratory Systems, Centers for Disease Control and Prevention. The CMS could then potentially use the consensus protocols to turn the one-size-fits-all CLIA regulation into one that provides scientifically based QC options for labs that are embracing innovative technologies. As laboratorians know, CLIA ’88 now requires that two levels of external QC be performed each day for nonwaived tests, and more frequently for certain specialty and subspecialty tests. CLIA regulators have long recognized, though, that this may not be an optimal way to perform QC. "When we first began to discuss QC under CLIA ’88," Yost said, "both the CDC and CMS had meetings with nationally recognized experts regarding quality control. Because of the variation in laboratories, and in the types of technologies and test methodologies, we were unable to come to any real conclusions, so we defaulted to the two external controls per day." Every test system may not need this level of external QC at this frequency, she concedes, and that’s one of the reasons she and others created the equivalent quality control options. The EQC options are detailed in CLIA’s interpretive guidelines. Basically, the first option applies to test systems with internal QC that monitors all analytical components of the test process, the second option addresses systems with internal QC that monitors a portion of the analytical test components, and the third option can be used for stable test systems without built-in controls. But the use of these QC alternatives is contingent on a performance evaluation by the laboratory that institutes the EQC option, and a test is not eligible if the manufacturer requires more frequent controls in its package insert. A number of specialty and subspecialty tests are excluded from EQC as well. Speaking at the workshop, James O. Westgard, PhD, president of Westgard QC Inc., pointed out the drawbacks of using the current EQC options. "If you do any of the EQC protocols and you don’t have any rejection signals, that is not evidence that things are okay. You have not tested whether the system would even respond if you had problems," he said, adding that before regulators think about reducing QC, they should be sure the analytical performance of current assays is truly acceptable. He provided data showing that for most of today’s common tests, the quality of the result is three to four Sigma on the Six Sigma scale, with six Sigma being the goal for world-class quality and three Sigma being the minimum acceptable quality for a routine testing process. He showed that the CLIA minimum QC requirement of two levels per day is appropriate only for methods having five Sigma quality or better and recommended that methods demonstrate six Sigma performance for adoption of any EQC option. In addition, if there is a QC failure when using one of the EQC options, the lab must repeat the test to determine whether the problem is isolated. If the problem is a fluke, the lab can continue testing. If not, the lab must return to the last successful run and repeat tests on all of the subsequent samples to be sure the results are accurate. With the first EQC option, labs are allowed to go 30 days without doing QC, which means if there is an inexplicable failure, they may have to retest samples as far back as 30 days. This can be a problem for labs that keep samples for only three to seven days. "At the moment, no one is pleased with the EQC options," Gerald Hoeltge, MD, treasurer of the Clinical Laboratory Standards Institute and the CAP’s liaison to the CLSI, told CAP TODAY. "If you look at the surveyor guidelines, I believe the approaches detailed there are scientifically difficult to explain and may be risky to implement, depending on the instrumentation." What he heard "loud and clear" at the workshop from many in the laboratory community, he says, "was that you can’t do quality control any less frequently than the stability of the sample. In fact, he adds, any QC intervals that are longer than the stability of the sample are out of the question from the user’s point-of-view. Thomas L. Williams, MD, director of the Department of Pathology at Nebraska Methodist Hospital, Omaha, and a member of the Clinical Laboratory Improvement Advisory Committee, agrees with Dr. Hoeltge’s assessment. "Those of us on the laboratory side were quite baffled about how to approach EQC. If you look at the evaluation processes for the three different options and how many times you do controls, and what you must do if QC fails, none of that seemed to make sense to any of us," he told CAP TODAY. Dr. Williams, who was at the workshop, says there’s a disconnect between the way laboratorians view the EQC options and the way in vitro diagnostics manufacturers approach EQC. "Those in industry seemed to immediately look at EQC as a risk management question, while those of us in the laboratory looked at it as a quality control question," he says. "I’ve reached the conclusion that for us to proceed with EQC, we really need to learn more about risk management in the laboratory, and look at EQC as a risk management challenge." Why does risk management make sense for clinical laboratories? "It provides greater insight into the patient safety risks that we’re all trying to deal with, and it supports systematic, well-informed decision making regarding QC and other risk controls," said Donald Powers, PhD, a quality systems/regulator compliance consultant and founder of Powers Consulting in Pittsford, NY, who spoke at the workshop. "It also fosters quality by design, continuous improvement, and leads to enhanced predictability of results," he said. "QC should be thought of as a form of risk control." In addition, risk management dovetails with other quality system initiatives that laboratories may be undertaking already. "Risk management has been incorporated into the regulatory schemes of all the major industrial countries, and the new European IVD directive places a very strong emphasis on risk management as an essential requirement," said Dr. Powers, adding that the quality systems regulation in the U.S. has similar requirements. Dr. Powers explained that the internal built-in monitoring systems used in a number of today’s IVD devices exist because of IVD manufacturers’ risk management process. If laboratories would use risk management in those parts of the process under their control, the partnership could be powerful. "One of the things that concerns me is that the manufacturers are doing a lot of risk management, but the laboratories haven’t really started yet," he said. "However, they are an integral part of making sure that the risks are minimized to ensure the safety of patients. So we have to find a way for manufacturers and labs to work together." If labs want to adopt a risk management approach to QC, guidance is already available to them through the International Organization for Standardization, or ISO. ISO standard 14971 is a general life-cycle risk management approach, and new guidelines for applying the standard to IVD products were developed recently by ISO TC212 and added to ISO 14971. "Applying risk management to in vitro diagnostics, where failure has an indirect impact on the patient, is a little more complicated than applying it to pacemakers or dialysis machines, where failure causes direct harm. That’s why we needed additional guidance," Dr. Powers said. The new IVD risk management guidelines describe one risk model that begins with a manufacturer experiencing a failure in a quality system, which leads to a defective IVD medical device. This, in turn, causes the laboratory to have a failure in its testing process, which then produces an incorrect result. Since the incorrect result in the physician’s hands can lead to a wrong diagnosis or an inappropriate treatment, it is considered a "hazardous situation" in the terminology of the standard. As defined in the standard, a "hazard" may lead to a hazardous situation, but not all hazards lead to hazardous situations, and not all hazardous situations lead to harm. For example, though an incorrect test result may create a hazardous situation, it won’t cause harm unless the physician uses the incorrect result to change the course of therapy or treatment in some way. Risk management sounds like it might be good in theory, but could it work? Says Dr. Williams, "Delineating the boundary between where the manufacturer’s responsibility ends and where the lab’s responsibility begins is going to be difficult, and I think as I understand the risk management concept, we in the laboratory are going to have to understand enough of our own risk management environment to know where failures are likely to happen as their device is used with our operators in our environment." Communication will be key in the future because labs will need information from manufacturers to control risks and monitor hazards over the life of the device. "We want the manufacturers to be able to communicate with us about what the potential points of risk are in an instrument," workshop attendee Stephen Sarewitz, MD, CAP Laboratory Accreditation Program checklist commissioner, told CAP TODAY. Manufacturers, he says, tend to want to put out the instrument as a black box. "That may or may not be true," he concedes, "but since we have responsibility for the result, we believe we have to be given some degree of insight into how the test works to make sure it’s working right." Powers agrees that more open communication is necessary. "The ISO standard requires manufacturers to disclose any residual risks to users," he says. IVD manufacturers proposed another possible solution to the EQC conundrum, and its working title is EQC Option 4. "We wanted to propose something that made scientific sense," said Luann Ochs, director of regulatory and reimbursement affairs at Roche Diagnostics and chair of the AdvaMed CLIA Working Group. "So we called CMS, CDC, and FDA into a meeting at AdvaMed in May 2004, and we proposed what we called Option 4, which we believe is a scientifically valid process for establishing equivalent quality control while still maintaining government oversight." At the moment, Option 4 consists of three steps. First, the manufacturer performs a scientifically rigorous risk assessment and validation of a proposed alternative QC protocol that would provide substantial equivalence to traditional QC. Second, the manufacturer submits the validation data to the FDA using processes already in place today. The FDA then reviews, approves, or clears that proposed QC protocol if the validation data are sufficient. Third, labs use the alternative QC protocol in place of the CLIA-mandated QC requirements for that assay. "The objective of Option 4 is to mitigate as many risks as possible through a combination of fail-safe mechanisms and quality control—internal and external, if needed—and reduce the risk to minimal or negligible levels or at least reduce them as well as current traditional QC practices," Ochs said. At the workshop, participants were asked to provide their input into designing a QC program for the future. Most agreed the initial three EQC options needed to be changed or eliminated, and that future QC should be based on scientifically sound data. Many supported the exploration of Option 4 and listed a number of new tools labs might be able to use for QC. The CLSI is producing a monograph from the conference that lists all of the participants’ suggestions. Laboratorians will have a chance to be heard in the future as a CLSI subcommittee begins work to determine what constitutes a "scientifically rigorous" validation to support the Option 4 proposal. This definition will be part of a consensus document describing data requirements for alternative QC validation. The proposed title for the document is "Principles of Manufacturers’ Validation of Risk Mitigation Using Quality Controls." Another suggestion workshop attendees made was that unit-use devices become their own separate category for QC purposes. Some felt it was wrong to require labs to do "old school" QC on today’s innovative devices. "For unit-use devices," says Dr. Sarewitz, "traditional external quality control really isn’t applicable, and that was one of the major problems with the equivalent quality control procedures as initially proposed. CMS didn’t make a special provision for unit-use devices." Dr. Williams adds, "I believe unit-use devices are safer to impose limited quality control on than are the typical flow-through systems we use all of the time in the laboratory." While CLIA regulators sort out the ideas expressed at the "QC for the Future" workshop, labs can expect QC to remain as it is for now. Also, because QC policy is unclear at the moment, Yost announced at the workshop that surveys for the QC arena of CLIA ’88 will remain educational. Don’t expect CAP checklists to change anytime soon either, Dr. Sarewitz says. "We don’t need to make any changes to the checklist at this time," he says. "Because the situation is unsettled and we don’t know what exactly is going to come out of CLSI and CMS and other discussions in the next year or so, we don’t want to make changes because of something we think might happen and then make other changes in six months if there’s something new. So we’re going to leave the checklist as is and await development." In the meantime, the CAP expects to have a voice in the development of the Option 4 document and future CLSI conferences on this topic. (Dr. Sarewitz was appointed advisor to the Subcommittee on Validation of Risk Mitigation that will write the Option 4 document.) Dr. Hoeltge says the next CLSI Leadership Conference is expected to address risk management, and he believes the Option 4 project will be the consensus document upon which the CMS will build much of its future foundation for QC. "In fact, I think the future of EQC is riding on this CLSI document," he says. Sue Parham is a writer in Edgewater, Md. |
|||
|
|
