Idea to ease PT standard for HbA1c stirs alarm

Anne Paxton

July 2019—Excellence is considered the hallmark of progress throughout health care. We don’t expect to hear that, for a particular diagnostic test, standards may already be too high. Yet a proposed rule by the Centers for Medicare and Medicaid Services, announced Feb. 4, does convey that message in its recommendation to add HbA1c to the list of analytes regulated under CLIA’s proficiency testing regulations related to analytes and acceptable performance.

If the rule is adopted, the CMS would CLIA-regulate HbA1c proficiency testing for the first time. But in an atypical turn to lower standards, for HbA1c—for which the CAP sets an acceptance limit of ±six percent around the target value—the CMS is proposing a looser CLIA PT accuracy standard of ±10 percent.

The almost universal reaction of the diabetes and laboratory medicine community to this proposal to weaken the HbA1c accuracy standard has been alarm—and puzzlement. “I was shocked” when the proposed rule was issued, says David B. Sacks, MB, ChB, FRCPath, senior investigator and chief of the clinical chemistry service in the Department of Laboratory Medicine at the National Institutes of Health. He had no inkling that a proposal to weaken the standard was being prepared. David C. Klonoff, MD, medical director of the Diabetes Research Institute at Mills-Peninsula Medical Center, part of Sutter Health, in San Mateo, Calif., also was taken aback. “It’s one of the strangest situations I have seen in medicine.”

The CMS’ invitation to submit public comment about the proposed rule produced 108 responses between February and the extended deadline of June 4, when comments closed. Along with others, the CAP voiced support for the CMS’ effort to update the list of regulated analytes by proposing the addition of 29 and the deletion of six current analytes, but objected to loosening the HbA1c accuracy standard.

Comments on the rule submitted to the CMS by individuals and organizations expressed more than strong reservations: “Would totally erase the progress in quality improvement in measuring HbA1c that has served the diabetes community so well in the past few years” (International Society for Pediatric and Adolescent Diabetes). “The diagnostic utility of the test will be lost” (Paul Yip, Institute for Quality Management in Healthcare, University of Toronto). “Flies in the face of successful efforts to tighten the assays and harmonize them across the country” (David Nathan, Massachusetts General Hospital). “An unexplainably retrograde step” (Simon O’Neill, Diabetes UK).

In fact, it appears that no commenters support the ±10 percent standard for HbA1c. “I have heard no one publicly support this—not one doctor, not one organization, not one company,” Dr. Klonoff says. So how did the idea garner enough consensus to appear in a proposed rule? On that question, he says, “I am just baffled. Why they came up with ±10 percent, I don’t know.”(Dr. Klonoff notes that the opinions he expresses are his own and he is not speaking on behalf of Sutter Health.)

Dr. Klonoff and Dr. Sacks are among eight leading endocrinologists and clinical chemists—all caught by surprise by the proposed rule—who penned a joint editorial for Journal of Diabetes Science and Technology (2019;13[3]:424–427), explaining why they believe the CMS proposal is a step backward. The authors’ core message: “We believe that CLIA’s proposed rule loosening the acceptance limits for proficiency testing from the current level of ±6% to a proposed level of ±10% would significantly harm the effectiveness of HbA1c testing and threaten patient safety.”

The recent history of HbA1c is one of progressive improvement. Criteria for passing the CAP Survey tightened from ±15 percent in 2007 to ±six percent by 2013, and in 2018 the CAP began to offer educational grading at ±five percent. There may be further improvement in the future. By contrast, the authors believe, the accuracy standard offered in the proposed CLIA rule is regressive.

“It is very unusual for a regulatory agency to say, ‘We want to have less stringent standards,’” Dr. Klonoff says. “There would have to be some really unusual problem—maybe a shortage of a product, or somebody did something wrong. But in normal practice, as technology improves, the capability of making an accurate measurement improves. That’s why the CAP standards have been mandating a lower and lower variability. For any regulatory body to permit higher variability for any analyte is strange. There has been no shortage of HbA1c instruments and no unusual events that would support changing standards, except for the gradual tightening of standards.”

Some companies with devices on the market are rightly upset about the proposed rule, he adds. “They say, ‘We go to a lot of trouble to build an accurate product; now you—CMS and CLIA—want to allow lower quality products to compete with us.’ So this is an instance where industry and academia are in agreement.”

The CMS had not proposed a standard for HbA1c before now because HbA1c was not a high-profile test the last time the CMS established CLIA rules, Dr. Klonoff says. “They’ve specified proficiency testing limits since the mid-1990s for a group of analytes they thought were important, but they previously decided not to include HbA1c.”

In the absence of professional testing rules, “the CAP has had proficiency testing policies that all labs have been following, and the NGSP has similar requirements.” The NGSP (originally known as the National Glycohemoglobin Standardization Program) sets strict international standards that manufacturers must adhere to, and sometimes laboratories don’t do as well as the method does in the hands of the manufacturer, he notes.

As Dr. Klonoff explains, the CAP is the de facto organization that sets the performance of clinical chemistry devices to make sure they are sufficiently accurate. “The NGSP also wants to make devices that measure HbA1c accurate, but for an additional purpose.” The NGSP seeks to ensure all the instruments in the country are standardized to the Diabetes Control and Complications Trial by calibrating a network of reference labs to the DCCT reference values; those reference labs work with manufacturers to standardize their methods.

“This year, NGSP has reset the standard for manufacturers at ±five percent for HbA1c. Manufacturers have been preparing for the shift to ±five percent,” Dr. Klonoff says. Since their methods will still have to be NGSP-certified, he believes manufacturers won’t be able to ignore the tighter NGSP accuracy standard, even if the CMS adopts its proposed rule.

The leadership of NGSP is also opposed to a lower standard, Dr. Klonoff says. “This proposed rule would apply only in the U.S., so it would probably not affect the international body. But you never know. If the U.S. were to mandate a looser requirement, then possibly the international body would follow. Although they probably would want to know why these looser standards are acceptable, and I haven’t read anything showing that.”

A ±10 percent accuracy margin is difficult to compare across analytes, Dr. Klonoff says. “I don’t think it’s meaningful to say that because a test for one analyte has a proficiency test margin of ±10 percent, then another test should also be specified to have the same range. The permitted range depends on biological variability among people and how easily the analyte can be measured. If you have an analyte that is difficult to measure, then you’re going to allow greater differences, and if people’s results can change from one day to the next, then as a general rule you’d also allow greater differences.”

However, Dr. Klonoff adds, there has been clear progress toward less variability in the HbA1c standard. “It has taken a long time to get these instruments down to a proficiency target of ±six percent. Ironically, almost every device on the market right now is ±six percent. So why go back to ±10 percent?”

The CAP does support part of the proposed rule. As it said in its comments to the CMS, it agrees with the agency’s effort to update the list of regulated analytes by proposing the addition of the 29 new analytes and the deletion of six current analytes. It also agrees with the effort to allow PT programs more autonomy to establish peer groups and parameters for PT offerings, and to protect the integrity of the program by ensuring nonprofit status for administrative activities.

But the CAP does not support a change to acceptable performance criteria of target value ±10 percent for HbA1c. The CAP offered for consideration two sets of criteria for HbA1c: “one for commutable (matrix effect free) whole blood material, where the criterion should be ±6%, and the second criterion for other matrix, where the acceptable performance could be a target value of ±10%.” The CAP said it “can accept use of ±10% for non-commutable material so long as the accuracy-based criterion of 6% is simultaneously implemented.”

“Initially, for CAP PT they didn’t use whole blood,” Dr. Sacks explains. “So the grading was peer group grading—comparing performance on PT only to that of labs that use exactly the same method, not to everybody. Subsequently, the CAP introduced whole blood PT, which eliminated the problem with artificial materials. Whole blood eliminates the matrix effect and the CAP changed to accuracy-based grading in 2007. So now the performance of all labs, regardless of method, is evaluated by comparison to the true target.”

It was a combined effort of the NGSP, the CAP, and manufacturers that produced the dramatic improvement in HbA1c measurement, Dr. Sacks says.

HbA1c was not considered by the American Diabetes Association or other influential clinical diabetes organizations to be suitable for diagnosis until 2010, when an international committee representing all the major worldwide clinical diabetes organizations came up with the recommendation that HbA1c may be a better means of diagnosing diabetes than glucose. “If anyone reads the ADA position statement, their rationale for that recommendation is that the improvements in the assay made by the community resulted in HbA1c being accepted for diagnosis,” he says.

In 2011, the same recommendation for use of HbA1c appeared in World Health Organization guidelines. “So the PT standard was ±10 percent in 2009, ±eight percent in 2010, ±seven percent in 2011, and now it is ±six percent. If one looks at the CAP Surveys, the coefficients of variation are progressively tightening, so the assay continues to improve. Now there is great concern about going backward,” Dr. Sacks says.

He gives an example of the difference in clinical impact between a ±five percent standard and a ±10 percent standard. “Say a patient has a true HbA1c of 6.5 percent, the value at which there is a diagnosis of diabetes. If the CAP sample has a true value of 6.5 percent, with a ±five percent standard the laboratory would pass PT if its results were anywhere from 6.2 to 6.8 percent, which is fairly tight, and doesn’t have a substantial clinical impact. If you change the same standard to ±10 percent, the acceptable range of values now becomes 5.8 percent to 7.2 percent, which is a huge range. This is clearly unacceptable.”

The clinical impact of less consistent HbA1c results could be serious, he and the other authors warn in their editorial. Patients will be at greater risk for unrecognized high HbA1c results and increased risk for microvascular complications over time, they say. In addition, for every 10 percent relative difference in HbA1c, the risk for retinopathy progression increases by 43 percent, while falsely high HbA1c results will place patients at greater risk for hypoglycemia, they point out.

Another one of the risks of a lower accuracy standard cited in the editorial is the potential effect on Food and Drug Administration standards for approval of new therapies. Apart from their role in diagnosing and treating diabetes, HbA1c results help determine whether the FDA will approve a new drug application. Comparative trials can indicate whether a new medication can lower glucose levels based on a 0.3 to 0.4 absolute difference in HbA1c concentration.

For a new diabetes drug application to the FDA, “a drug’s outcome has to be within no more than an absolute 0.4 percent HbA1c for noninferiority,” which is a measure of effectiveness, Dr. Klonoff says. “So if you have developed a new drug and the HbA1c is a little better or worse, all else being equal the FDA will approve the drug. But if you allow a greater amount of HbA1c measurement inaccuracy, then there will be some drugs that will be incorrectly classified as effective when they are not effective or ineffective when they are effective, all because of low-quality assays.”

Mitchell Scott, PhD, medical director of chemistry and point-of-care testing at Barnes-Jewish Hospital, Washington University School of Medicine in St. Louis, is as mystified as Dr. Sacks and Dr. Klonoff about why the CMS has moved to loosen HbA1c standards. “I don’t know the origin or the logic behind this. What is the motivation, who initiated this, who made the final decision? It’s a mystery to me. It doesn’t make any sense.”

He has not heard from anyone, including manufacturers, that the proposed weakened standard is a good thing. “It would be like taking lipids back to the late 70s, early 1980s,” he says. In those years, a widespread public relations campaign urged people to “know your number”—the number being your cholesterol value. “But the different cholesterol methods produced results that could differ by 50 or 60 mg/dL. The CDC jumped on that and standardized lipid methods in the 1980s.”

Dr. Scott likens the NGSP to that standardization effort. At the time of the DCCT study, which ended in 1993, many HbA1c methods were not very accurate, he says. “There is a target value for HbA1c (seven percent HbA1c) that came out of the DCCT and yet the methods were all over the map in 1993. A 2001 study compared CAP data from 1993 and CAP data from 2000 and found there were samples in 1993 that could vary by 100 percent at a target value of 10.5 percent HbA1c. Different methods gave values of nine percent HbA1c and 18 percent HbA1c for the same sample. It was clear something had to be done.”

By 2000, many improvements had been made, he says. Different methods were showing results differing by 1.0–1.5 percent HbA1c. “But we were at ±10 percent bias at that point, which is where CMS wants to go back to. Between 2000 and today, accuracy has increased such that NGSP-certified methods are within ±six or ±five percent.”

Dr. Scott notes that the CAP accuracy-based PT challenges originated with the NGSP and that the NGSP/ CAP collaboration is “one of the exemplary cases of standardization and harmonization in laboratory medicine.” He would compare it not only with the CDC standardization of lipids in the 1980s but also the recent standardization of creatinine by the National Kidney Disease Education Program of the National Institutes of Health and several lab organizations, allowing labs to do calculations and predict risk regardless of which creatinine method is used. “NGSP is an incredible success story and it even extends to point-of-care HbA1c methods.”

High standards of accuracy for HbA1c have a direct link with reduced patient risk of complications and death. “There is a J-shaped relationship between HbA1c and mortality in people with type 2 diabetes, which was shown in the ADVANCE and other trials about 10 years ago,” he says. “If HbA1c is too low, and the nadir of the J shape is in the low six percent HbA1c range, then risk and mortality increase. If we allow ±10 percent accuracy to enter the picture, and you are targeting a patient at seven percent HbA1c and your method reads seven percent HbA1c but the actual HbA1c is 6.3 percent HbA1c, your method still fulfills the proposed CMS criteria, but the patient could actually be in an HbA1c range that is potentially dangerous. That would set the field of glycemic control and HbA1c harmonization back 20 years.”

As the authors of the editorial point out, the fact that HbA1c was not included as a regulated analyte in CLIA 1988 enabled the CAP to contribute to improved assay performance by gradually narrowing the acceptance limits for PT.

However, Dr. Scott says, if the CMS adopts its proposed rule, manufacturers are unlikely to intentionally decrease the accuracy of their methods. “They aren’t going to use shoddy manufacturing methods. But what happens over time is that methods can drift.”

“So right now, a manufacturer has a lot release criterion that they are within ±six percent of the reference method assigned by NGSP. NGSP provides reference samples and reference methods for manufacturers to compare themselves to, and they certify the manufacturer’s methods as NGSP-certified. But if that accuracy goes from ±six percent to ±10 percent, then maybe the manufacturer releases lot numbers that would not have passed previously, but would now. Then an absolute value of seven percent HbA1c, instead of being ±0.4 HbA1c, is now ±0.7 HbA1c, which will allow a patient to be in a potentially dangerous region of glycemic control versus whatever the physician is targeting.”

Drs. Klonoff, Sacks, and Scott agree it would be a mistake for the CMS to lower standards by adopting the proposed CLIA rule for HbA1c. “This proposal will be harmful to the practice of medicine by making it more difficult to accurately diagnose and treat diabetes and will be harmful to patients if it leads the FDA to make decisions based on incorrect information,” Dr. Klonoff says. “We have appropriate standards in place now and there’s no benefit to patients or physicians in diluting those standards.”

Anne Paxton is a writer and attorney in Seattle.