Steep climb to suitable reference standards

While expensive certified reference materials won’t be used in daily routine, assays can be calibrated to a high level of accuracy by using the certified materials to calibrate EQA materials, which can then be used to calibrate laboratory controls. In a 2012 PT challenge for Huntington disease diagnosis, EMQN gave laboratories an opportunity to adjust their Huntington disease PT material sizing to the NIST standard. Genotype (triplet repeat numbers) of the PT material was verified with a NIST standard reference material. Participants could then calibrate their assays using the PT materials.

Counting GAA repeats in Friedreich ataxia presents a similar conundrum. In a PT challenge, laboratory results showed substantial scatter. “The range wouldn’t change the category,” Dr. Barton says. “All alleles reported as disease-causing were disease-causing, and those reported as in normal range were in the normal range. However, the scatter raises a concern that, if you did have an allelle close to the borderline value, some labs would get it wrong.”

“All labs think they are reporting correct allelle size,” he adds. “Only through EQA or PT can you show people they are not conforming with the consensus.” For Huntington disease this is easier, since a NIST standard value can be provided, which is powerful in the argument. “Otherwise,” Dr. Barton says, “strong-minded lab directors will say, ‘I’m right, they’re wrong.’” Well-characterized standards for Friedreich ataxia, which are lacking, would serve the same function.

Genomic reference materials are also available from the National Institute for Biological Standards and Control (NIBSC), a government-funded body much like NIST, which, Dr. Barton says, is “the only laboratory in the world making biological reference materials for WHO certification, not just in genetics but across the scope of clinical laboratory testing.” NIBSC offers 11 genomic reference materials at this time, and three more are in development (www.nibsc.ac.uk).

CRMGEN, a preview project that Dr. Barton chaired, determined what type of reference materials would be best for genetic testing. “What people want most,” he says, “the most versatile format in everyday life, is genomic DNA. Anything else restricts the usefulness of reference material to specific assays.”

Highlighting the necessity to use well-characterized reference materials, Dr. Barton cites detection of the R117H mutation in cystic fibrosis. “In most populations this mutation is moderately rare. So homozygous mutant individuals are not seen.” In Ireland, on the other hand, there is a high frequency of R117H. “So we have several samples from patients homozygous for this mutation. When we shared that material with test developers, on two occasions they found their normal signal was not discriminating between normal and mutant alleles. It was not specific for the normal allele. So they got a heterozygous signal on our homozygous mutant sample.”

Turning to the U.S. experience, Dr. Barton says he is an admirer of Dr. Kalman’s work at the CDC. “That program has taken a very pragmatic approach to characterizing existing materials. Making certified materials is an extremely expensive business. If they had set out to do that, they might have two or three certified reference materials on the market. Instead, they have many dozens of verified control DNAs that may not meet the standard of CRMs but are still useful to labs.”

Dr. Kalman coordinates the Genetic Testing Reference Materials Program (GeT-RM), which evolved from a pilot program completed in 2003. After that effort successfully created and characterized 27 new cell lines, the GeT-RM was initiated in 2004. Since that time, the GeT-RM has characterized more than 300 cell lines. It is essentially an ad hoc program. “People become involved as we do different projects,” Dr. Kalman, health scientist in the CDC Division of Laboratory Science and Standards, said in an interview. Volunteer laboratories characterize the reference materials. “When you think about this, it can cost thousands of dollars for them to run these samples,” she says. “And we do not reimburse them. They do it because they feel these materials are necessary.”

Reference materials can be of varying quality, Dr. Kalman says, with the degree of characterization being the chief variable. “It is well defined as to what you have to do to meet the requirement for a certified or standard reference material,” she says. QC materials, which include genomic DNA, are homogeneous and stable but not necessarily well characterized. What certified and standard reference materials have in addition is a certified value, along with its uncertainty, and stated traceability to a known standard, original standard, or reference method. Dr. Kalman estimates that about 20 standard reference materials, certified reference materials, or WHO standards are available, along with about 300 characterized genomic DNAs resulting from the work of GeT-RM. GeT-RM has created characterized genomic DNA reference materials for a number of disorders, including fragile X, Huntington disease, BRCA1/2, and Duchenne muscular dystrophy. In development are reference materials for Rett syndrome, pharmacogenetics, and cytogenetics.

“We do exclusively genomic DNA,” Dr. Kalman says. All of the genomic DNAs characterized by GeT-RM come from Coriell repository cell lines, which originated from patients with a particular disease. To emphasize the meaning of genomic DNA, Dr. Kalman says, “NIST’s fragile X standard is a PCR amplicon, not genomic DNA. It is a ‘synthetic DNA molecule’ amplified from a human cell line.” She points out, as does Dr. Barton, that genomic DNA from cell lines doesn’t have the same function as certified or NIST reference materials. GeT-RM cell line DNA is logistically and economically feasible to use for daily controls, she says.

Developing reference materials for Duchenne muscular dystrophy is a situation in which GeT-RM had to develop new cell lines for some mutations, since existing cell lines covered only deletions, not point mutations or duplications in the DMD gene. Working from a patient registry with known mutations sponsored by a patient advocacy group, and going through institutional review board approval and patient consent, researchers collected blood from patients with mutations of interest, and Coriell made 10 new cell lines. Both male probands and female carriers were represented. “We’ve facilitated the creation of new cell lines for a number of projects,” Dr. Kalman says, including the ongoing Rett syndrome work.

New projects for GeT-RM include pharmacogenomics, cytogenetics, molecular oncology (with Dr. Jennings), and next-generation sequencing. “We are now working on reference material for the whole genome sequence,” she says.

For the NGS project, existing and new sequence data are being collected for two human cell lines from more than 36 clinical gene panels, exome and whole genome tests. As usual, volunteer laboratories are doing the work. The resulting format will be not one sequence but what each laboratory produced with its method. Information collected includes an assessment of data quality, such as coverage and quality scores. Data are sent to the National Center for Biotechnology Information, which is building a whole genome browser to allow access to the reference material sequence data in collaboration with clinical laboratories.

Dr. Kalman distinguishes between these reference genomes, which will represent two specific individuals with determined sequences, and the sequence from the Human Genome Project, which averaged data at each base position from many individuals. The purpose of the NGS project is not for diagnosis, since these are two supposedly healthy individuals. “We are trying to characterize the sequence of these samples,” Dr. Kalman says, “so that when someone wants to validate a new next-generation sequencing assay, they can buy samples from Coriell and compare their sequence to our results. That should enable them to troubleshoot their assay.”

New challenges will arise from the NGS project. “The quantity of data will be enormous,” Dr. Kalman says. Also, it is difficult to characterize a genomic sequence with respect to SI units, as one can do with clinical chemistry analytes, such as sodium or glucose. These chemicals can be precisely quantified, but not so for genomic DNA. “NIST is currently creating highly characterized reference materials for the whole human genome, three billion base pairs with four possible nucleotides in each position,” Dr. Kalman says. “The result is not quantitative, not how much of each nucleotide, just a sequence.” Each sample will contain three billion analytes. “Even one gene is orders of magnitude more difficult than anything we have done previously,” she says. “We will have to develop a different model of how you make this new reference material, of how dependable it has to be to make a medical genetic diagnosis.” Even with these differences, the NGS project will adhere to the same underlying principle that guided previous projects: Truth in laboratory testing requires well-characterized reference materials.

William Check is a writer in Ft. Lauderdale, Fla.