October 2009
Feature Story

Anne Paxton

Following many months of consideration and discussion, the CAP announced in September a new policy position on laboratory-developed tests, or LDTs, informally known as homebrew or in-house tests. For the first time, the College is recommending that the Food and Drug Administration play a role in the oversight of LDTs classified as high risk. The CAP is also calling for stronger CLIA accreditation standards for laboratories that have developed low- or moderate-risk tests.

“Our position for a number of years was that CLIA [Clinical Laboratory Improvement Amendments of 1988], under which the College has deemed status for accrediting labs, was sufficient to regulate LDTs,” says Jay F. Schamberg, MD, a CAP governor and chair of the CAP Council on Accreditation. “But as more and more LDTs came along, we decided it would be appropriate to restudy the issue. We believe the resulting position will preserve safety while allowing innovation.”

The three-tier system proposed by the CAP would classify as low, moderate, or high risk all tests developed within CLIA-certified laboratories and used in patient management, including genetic, molecular, conventional, and other tests. High-risk tests would be reviewed by the FDA before they could be used clinically, while moderate-risk tests would be reviewed by the laboratory’s accreditor before being offered clinically. For low-risk tests, the laboratory accreditor would verify validation procedures and compliance with accreditation standards during normally scheduled inspections.

It’s the College’s view that its regulatory proposal would require modification of CLIA regulations but not new legislation.

The CAP has long been reluctant to foster an over-regulated environment for LDTs, Dr. Schamberg says. “The fear was that if the FDA had to approve all LDTs in a manner similar to what it does for drugs and complex devices, it could stifle innovation in lab testing and delay patient access to valuable tests,” he explains. “That’s not our purpose; our purpose is to assure safety for patients.”

A number of forces have caused the College to rethink its position on LDTs, says Gail H. Vance, MD, a CAP governor and principal author of the proposed three-tier LDT regulation model. “A couple of years ago, the Secretary’s Advisory Committee on Genetics, Health, and Society of the Department of Health and Human Services was instructed to review and develop a strategy for the oversight and regulation of genetic tests, and the bulk of those are LDTs. Around the same time, two Senate bills were introduced—one by the late senator Ted Kennedy and one by then-senator Barack Obama—and both concerned regulation of genetic tests and LDTs by the FDA and CLIA.”

These have stirred the pot, Dr. Vance says, and test manufacturers have been weighing in with concerns about perceived abuse of the LDT rules. “Most recently,” she notes, “Genentech [now part of Roche] submitted a citizen’s petition to the FDA seeking parity of LDTs and test kits made by manufacturers. And AdvaMed, the trade association for medical device manufacturers, then sent its own proposal for how to provide oversight of LDTs by the FDA. So there’s been a lot of controversy and discussion about LDTs.”

Hovering over commercial interests and validation issues is the growing need to revise CLIA. “CLIA ’88 is now 21 years old, and although it is sufficient for most lab tests, the complexity of test technology has changed,” Dr. Vance says.

All of these factors led the CAP to form a working group, about a year ago, to develop a position on LDTs. “The discussion was really across the College,” Dr. Schamberg says. “It included the Laboratory Accreditation Program and Council on Scientific Affairs, including people from the molecular side. And they worked pretty hard putting this together, with conference calls week­ly for a number of months.”

CAP governor Paul N. Valenstein, MD, vice chair of the College’s Council on Scientific Affairs, believes the CAP is taking a nuanced approach to regulation of LDTs.

“Laboratory-developed tests vary widely and pose different levels of risk to patients,” he says. “The results of some LDTs—such as cytokeratinare used only by pathologists and only in conjunction with other information before a diagnosis is rendered. The results of other LDTs—such as for HER2 or EGFR—are reported directly to oncologists and pivotally impact decisions about regulated therapies.”

Furthermore, the scale of LDT testing varies widely across laboratories, Dr. Valenstein points out. “Some LDTs are used only for patients already being treated by a particular health system,” he explains. “Other LDTs are promoted nationally to tens of thousands of individuals, sometimes by specialized laboratories specifically set up to perform a handful of esoteric LDTs that are not available from any other source.”

A laboratory’s incentive to overstate an LDT’s capabilities varies from one setting to another, as does an ordering physician’s ability to independently verify a laboratory’s accuracy, Dr. Valenstein says. “A one-size-fits-all approach to LDT regulation spells trouble,” he cautions.

Until now, the FDA has asserted the right to regulate all laboratory-developed tests but has exercised discretion about which tests to review. “The evolving CAP position is designed to spell out which types of LDTs require which types of oversight,” says Dr. Valenstein. “This approach should provide clarity to the laboratory community and IVD manufacturers.”

Through the years, he adds, some in vitro diagnostics manufacturers have objected to claims being made by some LDT laboratories and have asserted that the LDT companies are using the CLIA framework to “get around” the FDA.

That was the gist of Genentech’s complaint in its petition to the FDA in December 2008. Genentech, which makes Herceptin, urged that LDTs meet the same premarket review requirements that apply to commercial test kits. The company also requested that the FDA concurrently initiate an enforcement action, under its existing authority, against any laboratory selling and making claims about an LDT without sufficient analytical and clinical evidence to support its claims.

The CAP Laboratory Accreditation Program, as a deemed accreditor for CLIA, has always required analytic validity in all testing, Dr. Schamberg says. “That’s a demonstration that we’re measuring what we say we are measuring. But more recently, we’ve also been asking for clinical validity in molecular testing. Clinical validation requirements have been included in the CAP proposal. They ensure that tests are classified accurately and that claims about the accuracy or usefulness of a test are validated and correlated to a clinical condition.

“For example,” he continues, “you may have something nonspecific like hyponatremia, or low sodium, where you have a number of symptoms, like light-headedness, nausea, and vomiting, going all the way to seizures and coma. But low sodium results can be associated with many diseases and physical states. So measuring sodium tells you a person has a condition but doesn’t tell you what the condition is. It’s a very complex area figuring out exactly what we in the lab can delineate and validate and what needs to be validated by a larger system.”

Standards for clinical usefulness or clinical utility of a test are a step beyond validation. “Clinical validation guidelines are something we’ll be working on in our Center for Pathology Laboratory Quality, which is just getting off the ground,” Dr. Schamberg says. “We haven’t yet decided what role the College should play in determining clinical usefulness. To determine the usefulness of a test in practice, you have to consider factors like cost and access. And I don’t know where we’ll end up on that.”

Several factors enter into the calculation of the level of risk an LDT presents.

Cytokeratin, for example, would be considered low risk, Dr. Schamberg says. “It’s an immunohistochemistry test that we use when looking at malignant tumors. We don’t use it to determine whether a tumor is malignant or not—which is the largest distinction we have to make. But once we’ve ascertained a tumor is malignant, we use cytokeratin stain to determine whether it’s of epithelial origin, or fibrous origin, or hematologic. If it’s positive, that’s evidence it came from the epithelium in the lung, or GI tract, or prostate. Cytokeratin is a low-risk test because while it may influence how the tumor is treated, it usually isn’t going to affect treatment independently or directly.”

Also considered low risk would be a test the FDA has already approved but for which the laboratory perhaps changed a reagent, as well as tests for rare diseases. “One interesting and I think new twist,” says Dr. Schamberg, “is that we recognized there are many rare diseases that affect a very small population of individuals. Maple-syrup urine disease is one example. Nobody has the motivation to spend the money and resources necessary for the FDA to approve or clear such a test, and we think there is value in allowing LDTs to be developed for very rare tests. That will help innovation occur, where it would not occur if the regulations were too restrictive.” As long as there is no risk to the patient inherent in the test, he adds, the working group has tried to err on the side of not raising the bar unreasonably.

For the moderate-risk category, the CAP is proposing as a requirement that the test be validated by desk review of the accrediting agency, whether it be the CAP, Commission on Office Laboratory Accreditation, Joint Commission, or others, before it’s offered for clinical testing, Dr. Schamberg explains.

HER2, which presents fewer problems than Genomic Health’s Oncotype DX breast cancer assay, is an example of a moderate-risk test, he says. “With HER2, you can look at the literature and see the performance of the test in predicting responsiveness to treatment with Herceptin. If a patient’s tumor does not express HER2, then using Herceptin poses a risk with little likelihood that it will be effective. The risk/reward ratio is much improved if treatment is limited to patients whose tumors express HER2. With Oncotype DX, it is much harder for an oncologist to really validate the usefulness of the laboratory report.”

The distinction between moderate and high risk in this case is about the transparency of the way in which the test result is obtained, Dr. Schamberg continues. Tests that use a proprietary algorithm or computation end users can’t access and that are being used to determine treatment “are tests we think the FDA should look at because the FDA is best equipped to ensure necessary controls are applied to protect public health and safety,” he says.

The high-risk category of tests would include proprietary tests for which there isn’t much public review of the testing methodology, Dr. Vance says. It would also include IVDMIAs (in vitro diagnostic multivariate index assays), which combine findings from multiple individual analyses into a single composite test result using a formula that isn’t transparent.

“Oncotype DX is really an IVDMIA,” she says. “Accreditation inspectors have been encountering this type of test more frequently in recent years, and it’s very difficult for an inspector, during an on-site inspection, to understand all the statistical algorithms and data weighting for the test and to say that test has been properly validated.” If there were an FDA guideline that would have jurisdiction over those types of tests, Dr. Vance adds, “we might not have had to develop the LDT policy that we have.”

The evolution of labs that were developed around a single test “is a new phenomenon for us,” says Dr. Vance. “The whole milieu has changed. Whereas before, CAP has accredited labs, it’s now being tasked with accrediting tests—and that can be technically challenging.”

Nevertheless, aware that FDA review of a test involves a lot of time and money, the CAP has tried to take a conservative approach to regulation. “With three tiers of risk,” Dr. Vance says, “there would be more validation than currently exists. But the laboratory could do the validation in-house and there would be either an internal review on site at the lab or accreditor review for what’s designated as moderate or level two risk.”

“The reason we’re comfortable with this approach is that most of the LDTs out there have a long history of safety and providing health care benefits. It’s not the bulk of LDTs that are causing the concern; it’s a minority of LDTs,” Dr. Vance emphasizes. “I direct a cytogenetics lab that has been in operation since the 1970s. We’ve been performing LDTs for years and have demonstrated that these tests can be performed safely and benefit patient care.”

As for pathologists’ role under the proposal, “We’re looking for pathologists to demonstrate clinical validity, but that is not new because we already have the requirement for clinical validation in the molecular checklist,” Dr. Vance says. “We’re just extending that requirement to LDTs beyond molecular tests.” Pathologists, she continues, are in charge of the labs that are producing the LDTs, “so we are the ones in a position to offer enlightened statements and proposals about the oversight of LDTs.”

“What we’re looking for is a balance,” Dr. Vance adds. “It’s very critical that we provide oversight without becoming overly burdensome. The beauty of our proposal is that it includes both CMS and FDA—it’s a bridging between the regulatory agencies. It’s also a public-private partnership between regulatory agencies and accrediting agencies. So there are a lot of plusses to the proposal.”

The proposal is a good compromise between the notion that all LDTs should be cleared by the FDA one way or another and the idea that virtually none should be, Dr. Schamberg agrees. “It’s based on the risk to the patient. We think it’s a good middle ground that allows innovation but recognizes that where there’s risk, we have to take more care.”

Related Links

• CAP laboratory-developed test oversight model (PDF, 20 KB)