Home >> ALL ISSUES >> 2015 Issues >> Paths to validating next-gen sequencing assays

Paths to validating next-gen sequencing assays

image_pdfCreate PDF

Kim Scott

July 2015—As more clinical laboratories tread the unfamiliar ground of next-generation sequencing, they are faced with the age-old challenges of establishing validation and quality control processes. Two experts tackled the topic of molecular QC during a recent CAP TODAY webinar presented in cooperation with Horizon Diagnostics and available for viewing on demand at www.captodayonline.com/cap-today-hosted-webinars/#horizon.

Because the Food and Drug Administration classifies most sequencers as research instruments, the majority of clinical NGS tests require full validation, not just verification. When validating a test, laboratories must compare a number of parameters, including accuracy, trueness, precision, reproducibility, and robustness, according to the CAP’s recommended principles and practices for validating clinical molecular pathology tests (Jennings L, et al. Arch Pathol Lab Med. 2009;
133[5]:743–755). Of these, accuracy is perhaps the most important, said Josh Deignan, PhD, associate director of the UCLA molecular diagnostics laboratories.

“How a lab approaches accuracy for a next-generation sequencing test really sets the stage for how to address all of the other relevant parameters,” Dr. Deignan said.

One approach to validating the accuracy of an NGS test is to compare test results with those of another clinical laboratory performing the same test and whose results are presumed to represent the gold standard.

When performing validations for single gene and small variant panels, a lab typically would require multiple (at least 20) positive and negative samples, with each positive sample containing at least one clinically relevant variant. While it is possible to compare many variants from a single sample, multiple samples are still recommended due to the inherent matrix variability in different extractions and the complexity of other genomic alterations that may exist in specific examples, Dr. Deignan said.

The problem with comparing variants from a single sample, he explained, is other laboratories may not be performing the exact same test that your lab is, which makes it difficult to compare NGS data from the same type of test between two clinical laboratories.

A second approach to validating the accuracy of an NGS test is to compare NGS results with the results obtained from a gold standard method. While Sanger sequencing once was considered the gold standard, Dr. Deignan noted there are cases where a Sanger test is negative while clinical exome sequencing is positive for a particular variant.

Sanger sequencing is no longer the gold standard, he said, because there is potential for allele dropout due to polymorphic positions under primers or unknown heterozygous deletions. When this happens, the sequencing may either miss variants or may erroneously assign homozygosity to a heterozygous/hemizygous variant. What’s more, Sanger sequencing can only detect a minimum allele frequency of 15 percent to 20 percent.

“The conclusion is there may not be a different gold standard method to compare next-generation sequencing to anymore,” Dr. Deignan said. “I would argue that next-generation sequencing is the new gold standard.”

A third approach to validating the accuracy of a next-gen test is to compare the NGS results with the results from well-characterized reference material. A new consortium called Genome in a Bottle is developing the technical infrastructure, including reference standards, methods, and data, to enable translation of whole human genome sequencing to clinical practices. The National Institute of Standards and Technology, in April, made available for purchase its first reference material—8398, human DNA for whole-genome variant assessment. Additional reference materials for sequencing are under development.

One of the QC challenges of NGS is whether to run positive control and no-template control samples with every exome run. This is expensive; it also can be difficult to define a “positive” exome or genome since there are so many variants. Dr. Deignan noted that Sanger sequencing tests don’t require separate positive controls if all of the peaks in the sequence match the sequence of the gene.

Dr. Deignan

Dr. Deignan

“If you think about treating genomic DNA as its own next-gen sample type, which is something that a lot of clinical laboratories are now subscribing to, to me it makes a lot more sense,” he said.

Dr. Deignan does recommend that laboratories validate an NGS assay using their own routine extraction method. However, there may be instances in which other institutions wish to send your laboratory existing extracted DNA for testing instead of fresh blood or cells. Because it’s not possible to validate all DNA extraction methods, one approach is for the laboratory to develop defined QC criteria that confirm sequencing was acceptable. Another approach is to use Sanger sequencing for confirmation, but Dr. Deignan believes this is not the best approach.

“What we did [at UCLA] for the first year and a half of offering clinical exome sequencing was to confirm all reported variants,” he explained. “After that time, we essentially validated our ability to do away with Sanger confirmation for high-quality single nucleotide variants.”

The UCLA molecular diagnostics laboratories still use Sanger confirmation for all low-quality single nucleotide variants and all insertions and deletions. Reads for any variants not confirmed are manually evaluated. The quality score cutoff that UCLA uses is specific to its laboratory; each laboratory will need to develop its own cutoff.

The UCLA laboratory does not confirm somatic NGS panels, mainly because of the challenges in finding a confirmatory method with comparable sensitivity. The CAP leaves it up to individual laboratory directors to decide whether or not to confirm testing.

“I encourage everyone to decide for themselves, using a well-thought-out, data-driven process, how they want to approach this,” he advised. “Next-generation sequencing is the new gold standard for clinical molecular diagnostic testing, and while NGS validation and quality control is challenging and may require slightly different approaches, the principles are the same.”

Dr. Corless

Dr. Corless

Knight Diagnostic Laboratories in Oregon is a good example of a laboratory that has successfully developed an NGS testing program and is performing comprehensive validations. Knight first got involved with next-generation sequencing in 2012, primarily to better target therapeutics, said Christopher Corless, MD, PhD, Knight’s director and chief medical officer and a professor of pathology at Oregon Health and Science University.

Knight has developed a number of custom amplicon-based panels validated for DNA or RNA from FFPE tissue (see “GeneTrails next-gen sequencing tests”).

“These are relatively small panels ranging from 20 to 76 genes because we have chosen to focus on those genes that we think are truly actionable with drugs today,” Dr. Corless said.

To validate the panels, Knight Diagnostic Laboratories follows a four-step process:

CAP TODAY
X