Steps to verifying SARS-CoV-2 antibody assays and what’s known about protective immunity

Sensitivity and specificity thresholds are determined by the lab and there are two questions to consider, Dr. Anderson said. Are your providers going to want to test earlier than day 14 post-symptom onset? Guidance advises against this, he said, but “a lot of our providers may want to use it in this way and it may be hard to control, so you may consider a high sensitivity threshold early in the disease course.”

Second, what patient population will be tested? Will it be used for symptomatic patients for diagnostic purposes, or for asymptomatic screening and surveillance? “You’re going to approach those two tests very differently,” he said. In Fig. 3 are three different theoretical tests, with specificities in the lower left, all relatively high. They’re used to test the three example populations (the estimated prevalences of 20 percent, 1.69 percent, and 0.10 percent were based on molecular testing about mid-April).

“We see that our positive predictive value and our prevalence of 20 percent is in the 90 percent range. However, it begins to drop as that prevalence drops, and when we get to the point where the prevalence is below one percent, the positive predictive value becomes quite abysmal,” meaning most of the positives will be false-positives. Thus, screening of asymptomatic populations must be performed using a high-specificity approach.

So a lab may wonder: How specific is our test? Dr. Anderson shared sample data (Fig. 4) on a test that has 100 percent specificity. “However,” he said, “we need to keep in mind the confidence intervals.” For the test in Fig. 4, “the 95 percent confidence interval goes as low as 83 percent. Your assay could be 100 percent specific or it could be 83 percent specific—you don’t really know.” Testing more specimens—in this case up to 200—will tighten the CI, in this case to a 98 percent specificity “you can be more sure of.”

“The bottom line here is if you are going to use an assay for population screening, you need to do a more rigorous verification to provide acceptable specificity, to be comfortable you’re doing the right thing.”

The CDC recommendations recognize this need for verified high-specificity assays for population-based screening. If a lab cannot achieve this, the CDC says, “it can avoid testing low pretest probability populations altogether,” Dr. Anderson said, or use a combination of assays in an algorithmic fashion. A PPV calculator is available for that purpose (www.fda.gov).

The Barnes Jewish Hospital clinical laboratory uses a frequently asked questions document to get SARS-CoV-2 test-related information to providers. “We found it useful to have a form of communication that’s centralized and can be updated frequently,” he said. Also used is a clinical decision support tool, in which providers are told at the point of ordering what sensitivity to expect at different days post-symptom onset and what can cause a false-positive.

There are secondary benefits to such tools, Dr. Anderson said. “Depending on how they’re built, you can use them to monitor appropriateness of testing. What we’ve done at our hospital is have our providers answer a question about days post-symptom onset. We are then able to mine that data and figure out exactly what type of testing practices we have. This can be very important because it can give you insight into the effectiveness of education and where you need more education.”

Included in the educational material should be information on the interpretation of positive results. “That’s because there are so many misconceptions about what a positive means,” he said.

What a positive result doesn’t mean or doesn’t reveal is when the person was infected, whether he or she is shedding virus (live or remnant DNA), or whether the person is protected against reinfection, Dr. Theel said.

Binding antibodies are often produced at high levels but unable to independently prevent infection. Neutralizing antibodies are able to bind the virus and lead to loss of infectivity by blocking the virus from entering host cells, and this is largely accomplished independent of other immune system components.

The current commercially available assays do not distinguish neutralizing from non-neutralizing antibodies, Dr. Theel said, and testing for neutralizing antibodies is challenging because classically it requires plaque reduction neutralization testing using live virus, which for SARS-CoV-2 requires BSL-3 facilities for viral culture. “Increasingly, though, BSL-2-level methods are being developed for this purpose,” she said, using a variety of different viral constructs, including, for example, pseudotyped Vesicular Stomatitis Virus expressing the SARS-CoV-2 spike protein.

For common coronaviruses, studies performed decades ago showed that in volunteers infected with 229E, IgG levels peaked at about two weeks post-infection but then returned to baseline at about one year, Dr. Theel said. Re-challenge of these volunteers did not lead to symptomatic infection, although two-thirds of them still shed virus for a period of time. These studies also suggested that protective antibodies likely drop off to insignificant levels, leading to loss of protective immunity after about 18 to 24 months.

For SARS-CoV, Dr. Theel said, neutralizing antibodies peak at about four to five months post-infection and then decline over the next two to three years and become undetectable by six to seven years. For MERS-CoV, neutralizing antibodies remain detectable for at least three years. The studies didn’t go beyond that, she said.

“The one question these studies didn’t address or didn’t have the opportunity to address is what levels of neutralizing antibodies are clinically significant and correlate to protective immunity.”

For SARS-CoV-2, a few studies performed in rhesus macaques found that initial infection led to the development of binding and neutralizing antibodies against the virus, Dr. Theel said, and post-initial infection and recovery, re-challenge with SARS-CoV-2 at 30 to 35 days post-initial infection led to very low levels of detectable viral mRNA and no recoverable virus post day two (Chandrashekar A, et al. Science. 2020. doi:10.1126/science.abc4776). “These animals did not develop any clinically significant illness, suggesting that the presence of antibodies, and likely other components of the immune system, does lead to at least short-term immunity,” she said.

In another study, researchers looked at neutralizing antibodies in 175 recovered patients and found that while titers peaked at about 10 to 15 days post-symptom onset, those neutralizing antibody levels were variable across all individuals (Wu F, et al. medRxiv preprint. www.medrxiv.org/content/10.1101/2020.03.30.20047365v2). About six percent did not develop any neutralizing antibodies (< 1:40), and about 30 percent developed low-level neutralizing antibodies (< 1:500). “The unknowns that remain are what neutralizing antibody titer is clinically significant and potentially associated with protective immunity,” Dr. Theel said, “and then how long they persist.”

While knowing the neutralizing antibody level may play an important role in the future, tests to detect neutralizing antibodies will not be routinely performed in clinical laboratories, she said. “So one of the questions is, do the currently available commercial tests in any way correlate with neutralizing antibody titers?” It’s a tricky question to answer, Dr. Theel said, because the EUA assays now are qualitative and few studies published to date explored this correlation. Among the few studies published, the methods are highly variable, making it hard to reach comparative conclusions. “But in at least three studies, the correlation between spike and nucleocapsid-based ELISAs with neutralizing titers does seem to occur” (To KKW, et al. Lancet. 2020;20[5]:565–574; Okba NMA, et al. Emerg Infect Dis. 2020;26[7]:1478–1488; Amanat F, et al. Nat Med. 2020. Epub ahead of print: doi.org/10.1038/s41591-020-0913-5). The findings are preliminary and the subset of samples was fairly small, “so we should interpret these results with caution,” she said.

A positive result suggests only recent or prior infection, though the positive predictive value is affected by the assay’s specificity and the anticipated prevalence in the community. Here is what Mayo Clinic reports with its positives: “SARS-CoV-2 IgG antibodies detected. Results suggest recent or prior infection with SARS-CoV-2. Correlation with epidemiologic risk factors and other clinical and laboratory findings is recommended. Serologic results should not be used to diagnose recent SARS-CoV-2 infection. Protective immunity cannot be inferred based on these results alone. False positive results for IgG antibodies may occur due to cross-reactivity from preexisting antibodies or other possible causes.”

“For your own comments,” Dr. Theel said, “review the manufacturer’s instructions for use to make sure there’s nothing specific they recommend including.”

Sherrie Rice is CAP TODAY editor. The American Society for Microbiology EUA SARS-CoV-2 antibody tests verification protocols are at www.asm.org/Protocols/Verify-Emergency-Use-Authorization-EUA-SARS-CoV-2.