Anne Ford
September 2016—Whether it was “This is your brain on drugs,” “Take a bite out of crime,” or “Friends don’t let friends drive drunk” popping up onscreen, few of us watching TV in the 1970s and ’80s enjoyed having our programs interrupted by those public service announcements. Yet those important messages stuck in viewers’ brains—and stuck hard, if homages such as the Washington Post’s “10 Best PSAs of All Time” are anything to go by.
John D. Pfeifer, MD, PhD, had similar hopes for his thoughts on next-generation sequencing, which he presented during a webinar produced by CAP TODAY in collaboration with Horizon’s Diagnostics Division, “Assessing the Clinical Genome: Thoughts and Lessons for Improving Diagnostic Tests” (http://bit.ly/062116_capwebinar).
Dr. Pfeifer
The two other webinar speakers focused on the wet-bench parts of NGS and on bioinformatics, and Dr. Pfeifer said his own talk was positioned between the two. “I’m going to raise some issues that may not be as reassuring, and that may actually create some unease. . . . And so my talk today is going to be, if you will, a public service announcement.
“It’s like one of those commercials that interrupts your favorite TV show,” he said, “and it raises your awareness about something that maybe is not entirely comfortable . . . But hopefully you’re a better person at the end of it.” Dr. Pfeifer is vice chair for clinical affairs in pathology and immunology and a professor of pathology and immunology and of obstetrics and gynecology at Washington University School of Medicine.
What was this potentially uncomfortable topic? The variability of next-generation sequencing among expert laboratories, largely owing to the use of different platform and test designs, and the need to communicate more often and more openly with the clinicians who order the tests.
The pathologists and laboratory professionals who perform next-generation sequencing, he said, tend to focus on the analytic components of the sequencing, but often fail to recognize a question that creates a lot of ambiguity in the broader context of patient care: Is NGS a test or a method?
“The reason I ask that question,” he told listeners, “is to help us all remember that in the laboratory, we have a specific view about what we’re doing, but those of us who are laboratory professionals sometimes lose sight of what our clinical colleagues who are ordering the testing may or may not know.”
Pathologists and laboratory professionals tend to think of NGS as a methodology, Dr. Pfeifer said, whereas their colleagues in clinical practice often view it as a test. “In their mind, it often centers on the intended use: What is the information they hope to get out of the test?” he said. “We in the laboratory tend to think of NGS as a method where we go through all these steps . . . and we end up with a set of sequence variants through an assay that is very highly validated. . . .”
Analytically, he said, pathologists and laboratory professionals know there are “different platforms, different assay designs, different targeted genes within a panel, different mutation classes, the single nucleotide polymorphisms or single nucleotide variants, and indels, copy variants, and structural variants such as translocations. And that there are different bioinformatics pipelines that need to be optimized to find each one of those.”
In contrast, “our clinical colleagues may not recognize that NGS is not one thing,” he said.
This ambiguity and lack of communication about it can create problems, in Dr. Pfeifer’s view.
First, different NGS instruments use different chemical methods to produce the raw sequence. Some of the best examples, he said, are the Illumina HiSeq, NextSeq, and MiSeq series of instruments, which perform DNA sequence analysis via synthesis and which employ chemistry that utilizes fluorescence from incorporated bases during strand elongation.
The Thermo Fisher Ion Torrent, Ion Proton, Ion PGM, and Ion Chef series of instruments use semiconductor technology and utilize pH changes to measure incorporation of bases during strand elongation—also sequencing by synthesis.
Qiagen recently released its GeneReader NGS platform, and other platforms, such as from Oxford Nanopore, are in development.
There are a number of different platforms and pathologists and other laboratory professionals know they have different strengths and weaknesses. “But our clinical colleagues may not be aware,” Dr. Pfeifer said. He referenced a slide from a 2013 paper showing significant differences when the same DNA preparation was sequenced by different technologies (Boland JF, et al. Hum Genet. 2013;132:1153–1163). “When you start looking at indels, and these would be small indels, nothing that’s in the 50 or 60 base pair length but more in the range of a few nucleotides long, you can see that the level of agreement between those different platforms decreases,” he said.
All of this, of course, is made more complicated by there being two different general assay designs: amplification-based assays generally limited to target regions of about 50 kb, which are well suited to the detection of single nucleotide variants and small indels and which require lower DNA inputs; and hybrid capture–based assays, which are well suited to target regions of all sizes up to the whole exome, as well as to detection of SNVs, indels, copy number variants, and structural variants and which require higher DNA inputs.
He raises this point, he said, because clinicians may order what they think is a comprehensive cancer test and be unaware that the test has intrinsic limitations on the types of variants it can detect and the range of genes present in the panel. “And consequently, they may assume that certain mutations that are not being reported represent an absence of the mutations without recognizing that those particular mutations were not actually queried by the test,” he said.
Even if the same NGS platform and assay designs are used, there’s still the issue of different annotation interpretation schemes. Many laboratories query different databases, Dr. Pfeifer noted, and reports are interpreted by pathologists and other laboratory professionals who may or may not emphasize a particular result or paper—“and they may have access to some proprietary or internal database that other laboratories do not have access to.”
There’s another question to consider: What is a comprehensive cancer gene set? When he and his colleagues started doing next-generation sequencing about four and a half years ago, their “comprehensive cancer gene set,” which was assayed through a hybrid capture–type approach, had 25 genes. Their version two panel, launched about 18 months later, had 50 genes. Their version three panels are all disease-specific and generally have from 15 to 50 genes.
Other vendors and other laboratories perform the same type of sequencing, with different numbers of genes, such as Illumina TruSight Cancer with its 94 genes by hybrid capture and FoundationOne’s 315 genes by hybrid capture, he said. “And so here we use this term, even internally, as a ‘comprehensive cancer gene set,’ and our clinical colleagues who are ordering this testing are not aware of the platform, not aware of assay design. They see the phrase comprehensive cancer test, and they may not . . . recognize that the data they’re getting back is markedly different between laboratories, all of which are running a very highly validated, very highly characterized assay with a very expert bioinformatic pipeline.” The concern is that the results they’re receiving can suggest significantly different therapeutic options for the care of their patient.
Simply put, in Dr. Pfeifer’s view, there is a need for methodologic/analytic quality-based standards for traditional metrics—sensitivity, specificity, positive and negative predictive values, and so on—for different classes of mutations so physicians who order tests based on NGS methods know what they’re getting. In addition, standards are needed that account for the wide variety of “tests” (quotation marks his) subsumed by the term “NGS,” standards that account for the databases used for annotation and interpretation as well as the analytic test components, and agreement on what constitutes the gold standard.
“It’s not that I’m casting disparaging remarks on the quality of labs that are doing this testing,” Dr. Pfeifer said. “We’re doing testing with a very high degree of expertise. It’s just that what doesn’t come through to our clinical colleagues is that there is so much variability between the testing that is being done by different labs in terms of the genes, the type of mutations, and the reports that are being generated, and that two different labs may give fundamentally different answers because of a difference in test design.”
Of course, pathologists and laboratory professionals are used to talking with their colleagues about the strengths, weaknesses, and limitations of the other types of testing they do. “It just strikes me that the NGS community has not been as organized in communicating that message to our clinical colleagues,” he said.
Also important is the need to help clinicians understand whether the ordered test is clinically appropriate and whether the result will be clinically useful. “That in some settings, an amplification-based test focused on those mutations that are the targets of specific chemotherapeutic agents is the test that should be run. Other times it’s a more extensive, larger panel of genes that looks for a broader range of mutations in a larger set of genes all the way up perhaps to the whole exome. We need to do a better job communicating that,” Dr. Pfeifer said.
Then, too, there is the question of who will pay the bill, and a host of specimen requirements.
And there are interpretive issues. “We need to ask our clinical colleagues what the information is they are looking for, because someone who interprets a test from a center that has a lot of expertise and their own database may come up with a more sophisticated interpretation,” he cautioned.
Wrapping up, he reiterated his own public service announcement: “We as a community need to do a better job of helping our clinical colleagues understand that NGS as a test is not one thing. It’s a very broad range of different tests that is unified essentially by the fact that everybody is using a similar massively parallel sequencing technology. And by being more communicative about that, I think we can enhance patient care.”
Anne Ford is a writer in Evanston, Ill.