NGS checklist takes in infectious disease testing

William Check, PhD

October 2017—The CAP issued its first accreditation checklist for next-generation sequencing in 2014, as NGS was becoming a tool used in a growing number of clinical laboratories. The list of requirements, which was a new section in the molecular pathology checklist, focused on constitutive (germline) testing and oncology testing.

Now, with the growing penetration of NGS into the infectious disease field, the CAP has issued a revised edition of the NGS requirements, one that includes, among other things, greater coverage of NGS in the microbiology laboratory.

“The main point is that we’ve added information to almost all of the notes in many existing requirements that clarify the requirements for infectious disease testing,” says Benjamin Pinsky, MD, PhD, a member of the CAP’s Microbiology Resource Committee and of its NGS Project Team. “We did not yet break out the applications of next-generation sequencing to infectious disease from the rest of the checklist.” As NGS becomes more commonly used in infectious disease, those applications are likely to be set apart, adds Dr. Pinsky, associate professor of pathology and of medicine (infectious diseases), Stanford University School of Medicine, and medical director of the clinical virology laboratory, Stanford Health Care and Stanford Children’s Health.

Sheldon Campbell, MD, PhD, a member of the CAP Checklists Committee and associate professor of laboratory medicine, Yale School of Medicine, notes there are no FDA-approved applications for NGS in infectious diseases. “However, it has become an important tool in outbreak investigations and epidemiology, and it has become clear that NGS-based techniques for strain typing and following strains in outbreaks give you more information than do older methods of bacteria, parasite, fungus, and virus identification,” he says.

In the 2014 checklist, cancer and constitutive testing of human DNA were strongly represented, says Rakesh Nagarajan, MD, PhD, chief biomedical informatics officer at PierianDx and adjunct associate professor of pathology and immunology, Washington University School of Medicine. “Over the next few years there was increasing recognition that NGS could be used in infectious diseases and pharmacogenomics. There was some information pertaining to those areas at the time of the initial checklist, but they were not a major focus.”

For the revised 2017 checklist, released in August, he and others reviewed the recently published literature and the checklist as a whole and made either structural or higher-level recommendations that can be taken on in future years. The reviewers were in four subgroups, each with its own focus: molecular oncology, inherited disease, infectious disease, and pharmacogenomics. Their members came not only from the Checklists Committee and NGS Project Team but also from the CAP Molecular Oncology, Microbiology Resource, Personalized Health Care, and Histocompatibility/Identity Testing committees and the CAP/ACMG Biochemical and Molecular Genetics Resource Committee. The CAP Council on Accreditation leads the work to reexamine and revise checklists.

Amajor change in the 2017 checklist addresses the validation of NGS testing methods and how many and what variety of samples should be used for validating NGS. “We added a note about input specimen types,” Dr. Nagarajan explains. “The specimen types you use for validation should be the same types you expect to encounter during testing. You don’t need to test all possible specimen types. However, you shouldn’t evaluate those specimen types in cell lines alone.” What’s important, Dr. Nagarajan emphasizes, is the context. “Cell lines alone are not adequate to satisfy this requirement. You also have to have primary specimen types from patients and a representation of those types you expect to get in clinical work.”

MOL.36015 on “NGS Analytical Wet Bench Process Validation” says, in part, the following: “Due to extensive microbial genetic variation and diversity, it is not possible to perform an NGS test validation that would assess the ability of the test to accurately and reliably detect every possible organism or variant that may be present in a specimen. To address this limitation, a methods-based approach can be used for validation wherein the specimens used for validation contain a representative spectrum of the types of organisms, resistance variants, pathogenic factors, and host-response markers that the test is designed to detect. For tests that are designed for organism detection, common pathogens found in a particular specimen type should be included, when feasible, in the validation to ensure their accurate detection.”

Dr. Nagarajan describes as a “huge” addition the inclusion of language illustrating that the checklist no longer addresses only sequencing the human genome but detection of sequences of microorganisms as well. “Whether you are using specific probes or a metagenomic approach, you must validate specimen types inclusive of the microbiology process,” he says.

In the bioinformatics section, the checklist acknowledges the unique requirements for NGS testing in microbiology. Microbiology needs pipelines and databases for testing and identifying microorganisms, Dr. Nagarajan notes. “We were never prescriptive. In these emerging fields it is sufficient to present general concepts,” he says. The checklist describes, for example, the features of a dependable database: It must identify pathogenic organisms and host response markers and give the sequence and identification of the microorganism. But no specific databases or pipelines are recommended.

In oncology two new checklist requirements focus on the lower limit of detection (LLD). One (MOL.36108) says that neoplastic cellularity should be part of the laboratory report and that reporting of variants and their allele frequency should be done in the context of neoplastic cellularity. “If you have 50 percent tumor in your sample, your lower limit of detection will be quite different than if the sample has 20 percent cellularity,” says Dr. Nagarajan, a member of the CAP Molecular Oncology Committee.

A second requirement (MOL.36118) addresses LLD assay validation and how to do it. In what situations do you perform LLD validation in somatic variant detection and in germline variant detection? “This section also addresses mosaicism LLD as it applies to different variant types, so LLD goes with what variant you are doing. And the checklist tells how to validate LLD by spiking in plasmids or nucleotides or using reference standards,” Dr. Nagarajan says. Cell line mixtures can be used also. However, “You need more than all these molecular tools—you have to have a certain number of patient samples,” he cautions.

For exome and genome sequencing, a validation study should show that your method is able to identify genetic or germline variants. “We listed approaches you can use to augment but not supplant the use of patient samples,” he says.

The final area of the revised checklist is on interpreting and reporting NGS results. “One major component was identification of causal germline variants by doing NGS,” Dr. Nagarajan says. “This section calls out validation and being able to identify causal variants through the variant strategy you adopt. It talks about strategies and describes the process you go through to identify and interpret pathogenic variants.” In an analogous fashion, this section includes a reporting component for microbiology, saying that a laboratory report should describe the algorithm used to classify and interpret these tests.

Only a few changes were made for pharmacogenomics, and they address calling out race and ethnicity in reference to variants (MOL.36155). “How common or rare a variant is depends in some cases on a person’s race or ethnicity,” Dr. Nagarajan notes. The checklist recommends appropriate nomenclature to report pharmacogenomic results.

Dr. Campbell, director of laboratories for the VA Connecticut Healthcare System, New Haven, cites the likely growth areas for NGS in infectious disease that justify the greater focus on this area. “Next-generation sequencing has been used to characterize bacteria and other pathogens that cause unusual infections, such as B. cereus strains that contain anthrax toxin and so produce anthrax-like disease.” At this point such strains have been found primarily in Africa, but there has been a case in Texas.

The other major important emerging application of NGS in infectious disease is in patients with mysterious illnesses, and in particular central nervous system infections. “The use of very broad-range sequence approaches has allowed us to diagnose patients with CNS infections like leptospirosis.”

Detection of resistance genes is primarily a research tool now, but Dr. Campbell says that could change. “Right now it’s complicated and not routine. At some point it will become so.”

Validation of specimens is especially critical for microbiology, he says. “For genetic testing you almost always have whole blood or some other relatively straightforward sample type. In oncology it is more complex because of tumor heterogeneity. In infectious disease we do more specimen types than anybody else”—blood, CNS, pleural fluid, sputum, others. “Not all of them are easily obtained for validation. And many can be challenging from the inhibition point of view.” Many specimen types have normal microbiota that can be difficult to distinguish from potential pathogens. “So it really is quite a challenge to figure out what’s important in validating an NGS test for infectious disease,” he says. “That’s still evolving.”

Also evolving is the number of specimen types a laboratory needs to validate. “If you demand that a lab do 30 samples of 20 specimen types and each costs $5,000, the cost of validation alone could amount to a half million dollars.” The new edition of the checklist doesn’t address this. “It’s early days yet,” Dr. Campbell says. “We have lots of work to do to sort it out.”

As language is added to the checklist reflecting the growing importance of NGS testing for infectious disease, language appropriate to microbiology reporting is also making its appearance. “The earlier edition of this checklist talked predominantly about sequence variants in infectious disease,” Dr. Campbell says. “That is not the right language. One can talk about sequence variants in relation to a canonical known genome, for example, oncogenes in cancer. That’s not exactly what we are mostly trying to do in infectious disease.” Here, the question is whether an organism is present or absent, or whether a specific organism in a particular site is normal or abnormal. Or one might ask whether an organism carries a resistance marker. “To a first approximation,” Dr. Campbell says, “we included appropriate language in the infectious disease sections.”

Also important is what you compare your results to. For instance, in oncology you might ask, What is your database of meaningful oncogene markers? “That drives targeted therapy,” he says. “In infectious disease the analogous question is, What database of microbial sequences are you using and how well is it curated? How capable is your information pipeline at excluding normal microbiota? Can it distinguish between closely related species that may not differ at the locus you are using? We tried to make sure those fundamental issues were addressed for infectious disease in this edition of the checklist.”
[hr]

William Check is a writer in Ft. Lauderdale, Fla.