Method or test? Providing clarity to clinicians on NGS

There are a number of different platforms and pathologists and other laboratory professionals know they have different strengths and weaknesses. “But our clinical colleagues may not be aware,” Dr. Pfeifer said. He referenced a slide from a 2013 paper showing significant differences when the same DNA preparation was sequenced by different technologies (Boland JF, et al. Hum Genet. 2013;132:1153–1163). “When you start looking at indels, and these would be small indels, nothing that’s in the 50 or 60 base pair length but more in the range of a few nucleotides long, you can see that the level of agreement between those different platforms decreases,” he said.

All of this, of course, is made more complicated by there being two different general assay designs: amplification-based assays generally limited to target regions of about 50 kb, which are well suited to the detection of single nucleotide variants and small indels and which require lower DNA inputs; and hybrid capture–based assays, which are well suited to target regions of all sizes up to the whole exome, as well as to detection of SNVs, indels, copy number variants, and structural variants and which require higher DNA inputs.

He raises this point, he said, because clinicians may order what they think is a comprehensive cancer test and be unaware that the test has intrinsic limitations on the types of variants it can detect and the range of genes present in the panel. “And consequently, they may assume that certain mutations that are not being reported represent an absence of the mutations without recognizing that those particular mutations were not actually queried by the test,” he said.

Even if the same NGS platform and assay designs are used, there’s still the issue of different annotation interpretation schemes. Many laboratories query different databases, Dr. Pfeifer noted, and reports are interpreted by pathologists and other laboratory professionals who may or may not emphasize a particular result or paper—“and they may have access to some proprietary or internal database that other laboratories do not have access to.”

There’s another question to consider: What is a comprehensive cancer gene set? When he and his colleagues started doing next-generation sequencing about four and a half years ago, their “comprehensive cancer gene set,” which was assayed through a hybrid capture–type approach, had 25 genes. Their version two panel, launched about 18 months later, had 50 genes. Their version three panels are all disease-specific and generally have from 15 to 50 genes.

Other vendors and other laboratories perform the same type of sequencing, with different numbers of genes, such as Illumina TruSight Cancer with its 94 genes by hybrid capture and FoundationOne’s 315 genes by hybrid capture, he said. “And so here we use this term, even internally, as a ‘comprehensive cancer gene set,’ and our clinical colleagues who are ordering this testing are not aware of the platform, not aware of assay design. They see the phrase comprehensive cancer test, and they may not . . . recognize that the data they’re getting back is markedly different between laboratories, all of which are running a very highly validated, very highly characterized assay with a very expert bioinformatic pipeline.” The concern is that the results they’re receiving can suggest significantly different therapeutic options for the care of their patient.

Simply put, in Dr. Pfeifer’s view, there is a need for methodologic/analytic quality-based standards for traditional metrics—sensitivity, specificity, positive and negative predictive values, and so on—for different classes of mutations so physicians who order tests based on NGS methods know what they’re getting. In addition, standards are needed that account for the wide variety of “tests” (quotation marks his) subsumed by the term “NGS,” standards that account for the databases used for annotation and interpretation as well as the analytic test components, and agreement on what constitutes the gold standard.

“It’s not that I’m casting disparaging remarks on the quality of labs that are doing this testing,” Dr. Pfeifer said. “We’re doing testing with a very high degree of expertise. It’s just that what doesn’t come through to our clinical colleagues is that there is so much variability between the testing that is being done by different labs in terms of the genes, the type of mutations, and the reports that are being generated, and that two different labs may give fundamentally different answers because of a difference in test design.”

Of course, pathologists and laboratory professionals are used to talking with their colleagues about the strengths, weaknesses, and limitations of the other types of testing they do. “It just strikes me that the NGS community has not been as organized in communicating that message to our clinical colleagues,” he said.

Also important is the need to help clinicians understand whether the ordered test is clinically appropriate and whether the result will be clinically useful. “That in some settings, an amplification-based test focused on those mutations that are the targets of specific chemotherapeutic agents is the test that should be run. Other times it’s a more extensive, larger panel of genes that looks for a broader range of mutations in a larger set of genes all the way up perhaps to the whole exome. We need to do a better job communicating that,” Dr. Pfeifer said.

Then, too, there is the question of who will pay the bill, and a host of specimen requirements.

And there are interpretive issues. “We need to ask our clinical colleagues what the information is they are looking for, because someone who interprets a test from a center that has a lot of expertise and their own database may come up with a more sophisticated interpretation,” he cautioned.

Wrapping up, he reiterated his own public service announcement: “We as a community need to do a better job of helping our clinical colleagues understand that NGS as a test is not one thing. It’s a very broad range of different tests that is unified essentially by the fact that everybody is using a similar massively parallel sequencing technology. And by being more communicative about that, I think we can enhance patient care.”

Anne Ford is a writer in Evanston, Ill.