Next-gen sequencing now: a restless wave

“Definitely there is a learning curve,” Dr. Jones says, “both from the technical bench part of the assay and the informatics part. CLIA provides the framework for validating these assays, regardless of lab size, but it may be more difficult for smaller labs to bring up and sustain NGS. Large reference labs are likely to be major players because of the volume and expertise it takes to develop and keep the tests up and running.”

Probably not many laboratories will be able to put full NGS testing in place, Dr. Mardis agrees. “Like many things in pathology, commercial entities, the Quests and LabCorps of the world, are in the position to accommodate the necessary throughput to incorporate these more advanced assays,” making them available to many more laboratories.

Dr. Kulkarni’s view is more encouraging: “Laboratorians shouldn’t think this type of testing is reserved for academic centers. That was true three years ago.” The situation changed, in his view, with the publication last year of the first guidelines on quality control of NGS (Gargis AS, et al. Nat Biotechnol. 2012;30:1033–1036). Guidelines on bioinformatics for NGS are due out soon.

To hasten the spread of expertise in NGS, Washington University offers a fellowship in clinical genomics, which Dr. Kulkarni directs and is accredited by the American Board of Medical Genetics. “Our goal is to produce the next generation of clinical genomicists, who will be trained in molecular pathology and bioinformatics,” Dr. Kulkarni says.

Dr. Mardis offers an overview of NGS. “I would like to say it’s revolutionary, but I’m a bit hesitant. There are so many moving parts that can shut this down quickly, regulatory and reimbursement especially.” On the positive side, Dr. Mardis calls cancer “a fantastic paradigm” for NGS, with its ability to look at normal and tumor tissue for each patient at the level of DNA and RNA. However, she cautions, “This look is highly dependent on the bioinformatics algorithm used to tease apart the data. Unless you know what you’re doing, you can miss things that are clinically important.”

For most laboratories, Dr. Guettouche says, targeted gene panels—which come in two flavors—will be the vehicle of choice for their first exposure to NGS. Hotspot panels look at single nucleotide variants or small deletions that don’t cover the whole gene. “These tend to cover well-known mutations,” Dr. Guettouche says. Many are theranostic mutations; others are suspected of having actionable results but not yet validated.

Larger whole-gene panels sequence every exon of a cancer gene or the whole gene. “Once you have the results, you can choose to look at only hotspots or only novel findings,” Dr. Guettouche says. “You should be able to determine if this mutation has any influence on the protein. It could be a lot of work, depending on how well that mutation has been described. For some, you may have to determine yourself whether it has any influence on treatment.” At a conference Dr. Guettouche attended earlier this year, “A presenter claimed that if you are only running hotspot panels, you are likely missing some actionable mutations.”

Many NGS vendors now have panels. AmpliSeq from Ion Torrent (owned by Life Technologies) was one of the first. Illumina and some third-party vendors, such as RainDance Technologies, also have panels that can run on existing instruments.

“Right now all sequencing tests are RUO,” Dr. Guettouche says. “None are validated to the point required to get an in vitro diagnostic clearance as a clinical assay. Every sequencing company is working toward that goal.” He is confident that if an NGS assay is cleared, it will be a panel, not a whole exome sequence method, “since it is so much easier to validate a limited panel.”

In the meantime, users must validate any panel they want to use for clinical reporting. Sensitivity and specificity should be above 90 percent, Dr. Guettouche says, and the assay must be robust, which means not much affected by the amount or quality of DNA.

Dr. Kulkarni and his colleagues are using a panel to do targeted, ultra-deep (the sequencing program reads each nucleotide between 1,000 and 3,000 times) NGS-based diagnostic testing of cancer genes. The work is performed at Genomics and Pathology Services, a CAP-accredited, CLIA-licensed lab in the Washington University School of Medicine Department of Pathology and Immunology. It’s here that all clinical NGS testing in support of patient care at the university’s medical center is done. (In partnership with the pathology department, WashU’s Genome Institute is developing a CLIA-licensed lab that the institute can use to support NGS unique to its research.) To do the clinical testing, Dr. Kulkarni uses an NGS laboratory-developed panel consisting of nearly 50 genes. A few examples: ALK for lymphoma and lung; CEPBA, DNMT3A, FLT3, and RUNX1 for AML; MAP1 (ERK) and MAP2 (MEK) for lung and melanoma; and TP53 for colon, lung, and pancreas.

“We made this panel from scratch by sequence capture,” Dr. Kulkarni says. “Some clinical labs have information for 100 or 200 genes. Our philosophy is that less is more. We focus on those genes and variants where clinical utility is unequivocal.”

He and his colleagues have validated this panel based on the guidelines of the CDC, AMP, CAP, and other professional societies, which is more difficult in cancer because genomic alterations are found at low frequency. Two facts account for this. First, not all cells within a sample are tumor cells. Second, not all tumor cells have a particular aberration. “Cancer is a polyclonal disease,” Dr. Kulkarni says. “Even in a pure tumor cell population, maybe 10 percent of the tumor cells have a KRAS mutation. So our NGS approach should be able to find low-level sequence variants.” Sensitivity can be validated by dilution experiments.

Reproducibility of the panel (inter-tech, intra-tech, inter-lane, and combinations) is between 97 percent and 99 percent, Dr. Kulkarni reports. “We worked with several cell lines with a known profile of bases as well as DNA from blinded positive controls obtained from peer labs.”

Diagnostic sensitivity and specificity were derived in two ways. First, during validation, they used many positive and negative controls. A second way was to take advantage of existing knowledge to work on annotations. (Annotation is the process of identifying a small subset of functionally or clinically important variants from large amounts of sequencing data. For an example of an early annotation program, ANNOVAR, see: Wang K, et al. Nucl Acids Res. 2010;38:e164.) On the laboratory team are several doctoral-level scientists who are notation specialists. “Each of us is responsible for a subset of genes,” Dr. Kulkarni says. “We go through each annotation scoring system and either grade a variant as pathogenic or downgrade it to unknown significance. We do this for each gene during the initial validation.”

All of this work is aimed at obtaining reimbursement. “We have been very successful in getting good reimbursement because of our focus on clinical utility,” he says. “All the clinical testing in our CAP/CLIA lab is completely funded by insurance because we focus on the direct impact on patient management.”

The team has now analyzed more than 1,000 cases. A clinically actionable variant was found in 40 to 50 percent of all cases.

A major barrier to achieving clinical utility from NGS in cancer is the challenge of setting up an effective bioinformatic pipeline. “Cancer genomes are extremely complex and have diverse genomic aberrations,” Dr. Kulkarni says. “Gross genomic instability leads to gains or losses of chromosomes, segmental gains or losses, amplifications or deletions of chromosomes and structural variants such as translocations. All of these can significantly affect the bioinformatics analysis of tumors, starting with alignment.” To handle this complexity, his team wrote its own informatics software called Clinical Genomicist Workstation, or CGW. “When we started two years ago you couldn’t buy anything. And going to PubMed for each variant was a nightmare.” Now variant-calling software based on published guidelines is beginning to come on the market. “However, none of it is clinical grade,” Dr. Kulkarni cautions, adding, “What we developed in-house is fantastic.”

One caveat is that some genes or parts of genes cannot be captured very well, such as GC-rich regions. For many genes this includes exon 1. “Our clinical report distinguishes those parts and contains a disclaimer that they are not validated up to our standard,” Dr. Kulkarni says.

Quest’s first NGS-based test consists of a panel of seven commonly mutated myeloid-associated genes to be used in the diagnostic workup of patients with leukocytosis and suspected myeloproliferative neoplasm or to diagnose myelodysplastic syndromes in patients with cytopenia. The assay can also be used to identify mutations as clonal markers. Other NGS-based tests are for HIV tropism and genetic testing to assess hereditary risk of cancers. Quest began to offer BRCA testing in October, “using two NGS platforms for enhanced reliability,” Dr. Jones says. “These assays exemplify the use of NGS in large genes that are expensive or difficult to sequence by other methods,” he adds.

Quest’s results on fixed tissues are “somewhat dependent on the referring lab,” Dr. Jones says. “We find some labs have fixatives that are more troublesome.” The chief problem in a small subset of samples is extraction of high-quality DNA. “Even with those issues, suboptimal results that prevent a complete study should be seen in no more than five percent of samples,” he says. On the flip side, NGS can solve another problem. “With Sanger sequencing we used to worry about small samples. Now there is a real possibility for this technology to overcome the problems we see with limited samples such as fine needle aspirates.”

Dr. Jones’ colleague, David Ross, PhD, handles the bioinformatics component of Quest’s NGS work. Systematic errors occur with advanced sequencing machines, says Dr. Ross, who is director of computational biology at Celera, a Quest subsidiary. For this reason, bioinformatics pipelines need controlled data to evaluate assay performance. In addition to cell lines with defined engineered mutations, such as those from Horizon Discovery, and clinical samples with mutations validated by another procedure, Dr. Ross recommends in silico analysis as one way to do this. For in silico analysis, anonymous whole exome or whole genome sequencing samples containing known variants are combined to make complex datasets that are run through the software repeatedly. In addition, they can be reassembled to present the interpretive pipeline with varied looks.

Dr. Jones says storage is “one of the major housekeeping tasks” in doing this type of sequencing and one of the hurdles for smaller laboratories. “You really need a data storage plan,” he says. “It is a major part of our effort at Quest. Right now there are no definitive guidelines, so we are erring on the side of caution. We are keeping analyzed sequenced data until it’s a bit clearer if the regulatory environment may change.”

To confirm or not to confirm suspected clinically relevant variants is another important question. Dr. Jones says Quest has generally  not yet moved away from the confirmatory approach. How to confirm? “Pyrosequencing, for example, with an average five percent sensitivity for mutation detection is a pretty good match for most next-gen platforms,” he says. Confirmation provides confidence about clinical calls and a good sense of the accuracy of the system. “Over time you may become fairly confident that a particular mutation or gene is associated with a particular condition so that you can skip confirmation on those variants.”

Dr. Mardis of The Genome Institute at Washington University co-directs an ambitious effort that combines whole genome and whole exome sequencing of tumor and matched normal DNA, and sequencing of the entire tumor transcriptome for selected patients to provide therapeutic options for the patient and the treating oncologist to consider. Transcriptome analysis evaluates whether mutated genes are expressed. All possibly actionable variants are verified in a CLIA-certified laboratory with pathology sign-off.

“What we do happens currently in a research setting,” Dr. Mardis says. “To qualify specific results for return of information to the clinician, we work through our university pathology group to identify and help specify the assay used to confirm the NGS results.” Custom-designed PCR primers can be used in the CLIA setting to amplify a mutated region from the patient’s tumor identified by NGS analysis.

To illustrate the sensitivity and resolving power of NGS, Dr. Mardis cites a case, reported in the July 7, 2012 New York Times, in which an actionable variant—overexpression of the FLT3 gene—led to therapy that put the patient’s cancer, adult acute lymphoblastic leukemia, into remission (http://tinyurl.com/FLT3gene).
Of two pediatric brain tumor cases done subsequently with the NGS protocol, in one no driver mutations were found, and in the other the patient was enrolled in a trial of a MEK inhibitor as a result of the analysis. Outcomes are not yet known. The program is not being scaled to accommodate increased numbers of patients, since the work is entirely paid for with discretionary funds.

Turnaround time is a major consideration. In a case of acute promyelocytic leukemia reported in 2011, sequencing and validation took seven weeks (Welch JS, et al. JAMA. 2011;305:1577–1584). “That is one of the things we worry about most,” she says. Dr. Mardis is confident the time frame can be shortened greatly. Newer commercial sequencing instruments provide a sequence virtually overnight. “The next wave of clinical feasibility will be centered around software acceleration,” Dr. Mardis says. She would like to see an overnight sequence and two weeks to interpret the sequence and provide information to the patient’s oncologist.

Further shortening could come from having a pathologist sign off based on the NGS result itself. “I’m convinced we can do that,” Dr. Mardis says. “I’m hopeful that CAP will recognize that, with the appropriate metrics outlined in CLIA documents, a pathologist with correct training who is used to interpreting NGS data will be able to sign off directly without taking additional time for further verification in a CLIA laboratory.”

At the University of California, San Francisco, Trever G. Bivona, MD, PhD, assistant professor of medicine/hematology-oncology, leads a group that is sequencing genomes and transcriptomes from lung cancers of patients who present to the clinic. “Genetic changes occur specifically in tumor cells that drive the growth of these cancers. We are looking to classify and diagnose the genetic roots of each patient’s tumor and use that knowledge to direct care,” Dr. Bivona says.

“We know from large sequencing studies and clinical trials that many of the genetic alterations we observe in lung cancer cells can be targeted with therapy,” Dr. Bivona continues. “We believe, based on that observation, that we can use genomic technology to identify changes that can result in more effective therapies.” A prototypical example of this idea is the efficacy of the tyrosine kinase inhibitor erlotinib (Tarceva) to treat non-small cell lung cancers that have an EGFR mutation.

Encouraged by this example, Dr. Bivona and his colleagues have embarked on more comprehensive studies across patients’ tumors: “We think other molecular changes in the tumors, even in the presence of EGFR or other driver mutations, may affect treatment response.”

They have been doing this for about six months. Their first publication, based on results in two patients, is now under review. “Our findings were surprising and quite illuminating from both biological and clinical standpoints,” Dr. Bivona says. Tumors are known to be genetically complex. “What we have uncovered is that tumors are genetically complex perhaps beyond our expectations. In addition, we found that tumors evolve rapidly on treatment. If we understood the genetic basis for evolution of treatment resistance, we could use that information as a basis for improved therapy.”
Dr. Bivona stresses that this is “very much a research protocol,” not a clinical test. “We have parallel efforts to do this in a CLIA- and CAP-certified setting. In the research project, we can’t act on our findings clinically unless we validate a particular alteration in the CLIA setting,” he says.

For now, Dr. Bivona agrees that findings from research projects are increasingly used to develop multigene panels of the most clinically actionable genetic alterations. “We see our research as fueling and sharpening the clinical implementation of NGS,” he says. On the other hand, he does not necessarily agree that full-fledged NGS will remain a research tool. “The cost and process for doing this is becoming democratized. Time will tell whether whole genome sequencing will be used routinely in the future. Cost, time to results, and interpretability will evolve and hopefully improve rapidly, as in many other technology sectors.”

Dr. Jones is of the school that, at least for the foreseeable future, even multigene panels will be out of reach for most clinical laboratories. Many people have begun this journey, he says, but not many have completed it. “I think that is what we are going to see. Laboratories that are highly motivated in a particular specialty in some area of lab medicine may get to an assay. But the broad roster of assays will probably be fairly restricted in their impact on routine diagnostics.”

However results from NGS programs are introduced into the clinical laboratory, they are already changing the testing landscape. “We are at the dawn of an incredible era,” Dr. Bivona says. “There has been no more exciting time to be in science and medicine than now, given the empowering ability of these technologies to help us understand and treat human disease.” 
[hr]

William Check is a writer in Ft. Lauderdale, Fla.