Valerie Neff Newitt
November 2020—The CAP released in September its proficiency testing program for SARS-CoV-2 antigen testing, with the first shipment to laboratories set for Nov. 30.
It also introduced recently a Quality Cross Check program that makes it possible for labs performing nucleic acid amplification testing for SARS-CoV-2 to monitor performance across multiple instruments, in compliance with the CMS directive prohibiting proficiency testing on multiple instruments.
“Even with its limitations, antigen testing could have a role in a well-designed testing strategy, but nucleic acid testing remains the standard of care for a definitive diagnosis,” says Daniel D. Rhoads, MD, member of the CAP Microbiology Committee, which helped to develop the proficiency test. With laboratory medicine having “pushed to the forefront of news” during the pandemic, “it’s critically important that we are performing well in all testing formats and meeting the need in this time of crisis,” says Dr. Rhoads, assistant professor and section head of microbiology at the Cleveland Clinic.
The latest proficiency test release follows the release last spring of PT for NAAT and for serologic tests that detect the presence of antibodies against SARS-CoV-2. Laboratories that enroll in the antigen PT will receive several samples with inactivated virus “containing all of the antigens that could ever be tested for in SARS-CoV-2,” says Dr. Rhoads, adding it will work with all antigen assays, including those yet to come.
Dr. Rhoads calls the antigen PT and the other CAP SARS-CoV-2 proficiency tests “an opportunity for labs to challenge the system—both the performance of the assay and the technologists doing the tests—and compare themselves to their peers.”
He offers a few cautions about antigen testing for SARS-CoV-2. The tests on the market (as of September) are designed to be performed with a dry swab. “It needs to be collected and tested oftentimes within an hour of that collection, depending on the manufacturer. Different manufacturers have different instructions that have been authorized by the FDA, so not all of them are only an hour. You need to know that before you order the test and put all your eggs in that basket. You want to make sure you know the limitations.”
If these tests are performed in the laboratory and not at the point of care, Dr. Rhoads says, there is often no good way to transport them. “When we test the PCR nasopharyngeal swab and put it in transport media, we screw the lid on tightly and then everything is contained.” The dry swabs can be slipped back into the sleeves they came in and placed in a bag, “but it’s a little less tidy. It’s a concern,” he says.
Most antigen tests are designed for use at the point of care. “When that dry swab is tested, the specimen is used up. You cannot then reflexively test for flu or RSV or other respiratory viruses or even retest for SARS-CoV-2 off the same specimen. You would need a new specimen.”
Antigen tests are not used at the Cleveland Clinic, where the focus has been to increase capacity for NAAT. Fast turnaround time is the allure of the antigen tests, but knowing what to do with the result can be a problem, Dr. Rhoads says.
Dr. Rhoads
“If a provider orders a test but cannot be confident of the results, it often leads to retesting until there is greater confidence. So we are thinking hard about the problem we’re trying to solve and then trying to build the best solution to meet the need.” With more data and a better understanding of how the antigen tests perform, he says, the decision about whether to use them may change. “But as it is right now, I do not see a compelling use case.”
When it comes to testing for SARS-CoV-2, “everything has pros and cons,” Dr. Rhoads says, noting that the FDA authorized antigen testing for testing symptomatic people, making it inappropriate for screening, despite the market for such. “Different assays have different criteria as to how long after symptom onset you’re able to test someone and report results as presumptively negative,” he says. Insufficient data has been collected to date in independent studies about antigen testing for SARS-CoV-2. “But with experience with antigen testing for other respiratory viruses, we know it is generally less sensitive than a nucleic acid amplification test.” Thus, some false-negatives are expected.
“But we don’t expect to see false-positives, and yet there have been anecdotal reports of false-positives.”
“We need clarity,” through clinical studies, Dr. Rhoads says, on “whether there is a problem with human error, instrumentation, or the assay itself. We must consider how confident we are in results being accurate.”
For now, he says, antigen tests are believed to be best used in symptomatic people shortly after symptom onset when the antigen load would be its highest and it would be easiest to detect.
“The more tools we have the better,” Dr. Rhoads says, “and certainly there are advantages to an antigen test.” But it is matching the best tool or solution with each need—screening, diagnosis, surveillance—that is the challenge.
An adequate supply is another. “Abbott made news because they said they’re going to make 50 million tests a month. But even at that rate it would take six months before they could make enough tests for everyone in the country to be tested just once,” he says.
Orders for proficiency testing will be taken through Jan. 15, 2021. Next year’s shipments for the antigen PT are scheduled for April 12 and Oct. 4.
Some of the CAP’s proficiency testing data collected to date for molecular SARS-CoV-2 testing were reported in a letter published online Aug. 12 in Clinical Infectious Diseases (doi:10.1093/cid/ciaa1199) and written by Dr. Rhoads and other members of the CAP Microbiology Committee. In response to an article on the impact of SARS-CoV-2 viral load on risk of intubation and mortality, they provided caveats to consider when applying published findings regarding cycle threshold values to patient results. One of those caveats: “Ct values can vary significantly between and within methods.”
CAP data from more than 700 laboratories using PT material produced from the same batch revealed that the median Ct values reported by the instruments for different FDA EUA methods varied by as much as 14 cycles, Dr. Rhoads and colleagues wrote. “Within a single test performed on the same instrument, the difference in the median Ct-values for different targets was as high as 3.0 cycles,” they continued. And “within a single gene target for a single method, up to 12.0 cycle differences were seen across all laboratories.” The assay and gene target used by the authors of the article on viral load and intubation and mortality risk, ORF1a detected by the Roche Cobas, “differed by approximately 6.0 cycles across all laboratories responding to the survey,” they said.
They pointed out, too, that many clinical labs are using multiple tests that assess different gene targets for SARS-CoV-2 and are testing on different platforms, both of which add to the potential variability of Ct values produced by a single lab. “The ongoing shortage of commercial testing reagents presents a major obstacle to conducting large research studies comparing testing platforms,” Dr. Rhoads and colleagues wrote.
Dr. Rhoads tells CAP TODAY: “There has been growing interest and research around Ct values and the relevance for PCR testing. So, using the data from laboratories that participated in the nucleic acid proficiency testing, we were able to share that information with the medical community and help to caution infectious disease doctors and laboratories not to over-interpret a Ct value.”
Laboratories that enroll in the Quality Cross Check—SARS-CoV-2 Molecular program, known as COV2Q, receive three noninfectious samples that contain the whole SARS-CoV-2 genome, says Lauren N. Pearson, DO, MPH, chair of the CAP Instrumentation Committee and assistant professor in the Department of Pathology, University of Utah School of Medicine, and director of laboratories, University of Utah Health Sciences Center. They analyze the samples on up to three instruments and report the results, indicating on which instrument a given result was obtained. If a lab has more than three instruments, it can purchase more than one kit. The report labs receive from the CAP tells the lab if its results were correct and provides a peer group result comparison.
Dr. Pearson
The report also lets laboratories know how their own instruments performed in comparison with one another. “It will tell you if your results were identical among the instruments that you reported,” she says. If not, the laboratory has to determine where there is a clinically meaningful difference. “Let’s say I got a negative on my ID Now and a positive on my Cepheid instrument. I would want to know about that. It’s helpful information gained from the use of this kit because it’s a discrepancy I would investigate.”
The kit can be used to help laboratories meet the laboratory accreditation twice yearly requirement to ensure instruments are analyzing the same samples equitably, Dr. Pearson says.
A “nice bonus,” she adds, is that labs can use the QCC in meeting the accreditation program training and competency assessment requirements. “Integrating the samples from this kit into that process is one way to do that.”
Shipments of the QCC kits are spaced between PT events so labs can monitor performance across multiple instruments and still maintain compliance with the 2015 CMS directive that prohibits PT on multiple instruments unless that is how the lab tests patient specimens, Dr. Pearson says. “There’s a defined line between what QCC offers compared with proficiency testing. The results from the QCC challenges do not go to any accrediting agency. They don’t go to CMS.”
With labs having implemented testing on or expanded testing to multiple platforms to meet demand during the pandemic, they “want to be sure, at least qualitatively, that the interpretation of the result for a patient’s specimen is the same, no matter which instrument it’s analyzed on,” Dr. Pearson says. For her, as a lab director, it boils down to three things, she says.
“One, am I putting out comparable patient results regardless of which instrument I put the specimen on? Two, how am I going to most easily meet the accreditation program requirements for inter-instrument comparison? And, three, how do I do this while keeping my staff safe? From my perspective, this kit offers the ability to meet and satisfy all of those concerns.”
Valerie Neff Newitt is a writer in Audubon, Pa. To order SARS-CoV-2 antigen PT (COVAG), Quality Cross Check–SARS-CoV-2 Molecular (COV2Q), or any of the CAP’s other COVID-19-related programs, visit cap.org and select Shop at the top of the page.