Valerie Neff Newitt
July 2020—If a manufacturer assists a laboratory in setting up a new FDA-approved or -cleared test, the lab must make sure that the personnel who will perform the test participate in the verification or validation study. If the personnel don’t participate, there must be some way to confirm that performance is consistent with in-house studies performed by lab personnel.
That is a new condition of an existing requirement that is in the 2020 edition of the all common accreditation program checklist, released in June. It can be found in the introduction to the test method validation and verification section and in COM.40300 “Verification of Test Performance Specifications—FDA-Cleared/Approved Tests.”
In CMS validation inspections of CAP-accredited labs, the CMS “discovered that often only the manufacturers’ representatives were setting up machines, getting tests up and running, and doing validation studies necessary to start performing the test,” says Harris S. Goodman, MD, of Alameda Health System, Oakland Calif., and chair of the CAP Checklists Committee.
The CAP, too, became aware that in many cases when a new FDA-approved or -cleared test or instrument was introduced to labs, it was the vendor representatives, not the lab personnel, who were performing the verification studies, which are the lab’s responsibility, says Stephen J. Sarewitz, MD, vice chair of the Checklists Committee.
Dr. Sarewitz
“In the laboratory’s defense, it happened because such studies are labor-intensive and many labs are strapped for labor. Everyone is running a tight ship. So allowing manufacturer reps to do the verification was convenient.” But inspectors raised a concern, Dr. Sarewitz says: “If the lab personnel aren’t doing the verification, how do we know they can achieve the same level of performance that was documented by the verification studies performed by someone else?”
This concern has now been addressed in the form of the following statement in the checklist: “If an FDA-cleared or approved method was verified by someone other than the laboratory’s personnel (eg, manufacturer’s representative), the laboratory must ensure that the verification correlates with its in-house test performance by showing confirmation of performance specifications by laboratory personnel testing known specimens.”
“We’re primarily looking at accuracy, precision, and reportable range,” Dr. Sarewitz says, “which are the three major requirements for verification of FDA-approved instruments and methods. Lab personnel could take known patient samples of known concentration and run those and see if they can obtain the results that the manufacturer representative obtained, and then document those results.” Those records would be required to show compliance, he says. “In addition to the full verification records, there must be written evidence that lab personnel also did some of those studies.”
For labs not subject to U.S. regulations, the 2020 all common checklist now contains COM.40325 “Verification of Test Performance Specifications—Tests Approved by an Internationally Recognized Regulatory Authority—Laboratories Not Subject to US Regulations.”
In the past, Dr. Goodman explains, if a test was not FDA approved but had a European Union CE mark, for example, labs still had to do the complete validation studies as if it were a laboratory-developed test. “They were not allowed to do the more limited verification studies that are now allowed” by COM.40325.
Validation applies to testing instruments that are not FDA approved or cleared. “For those,” Dr. Sarewitz says, “the laboratory must establish its own performance specifications, which is a more extensive study and requires looking at additional parameters. An important question came up: How about labs not subject to U.S. regulations, which may be using instruments that are not approved by the FDA and yet are approved by another internationally recognized organization? How do we make the process equivalent and fair for them?”
“Previously we were not taking into account the stress that accompanied international validation,” says Bharati Suketu Jhaveri, MD, immediate past chair of the CAP Council on Accreditation and a member of the CAP International Accreditation Committee and Inspection Process Committee.
Dr. Jhaveri
Dr. Jhaveri and other Council on Accreditation members reviewed the CE regulations. “We wanted to make sure we were comparing apples to apples and that the CE is indeed a highly monitored group like the FDA,” she says. “We decided that the CE requirements do fit appropriately with our accreditation requirements.” Thus international laboratories will no longer have to perform redundant validation studies to meet CAP requirements.
“This will save labor, time, and money,” Dr. Jhaveri says. “Labs will be under much less stress and pressure and will be able to bring on these tests faster.”
This addition to the checklist documents for international laboratories the same thing that is required of U.S. laboratories, Dr. Sarewitz says. “In other words, non-U.S. labs implementing tests or instruments that are approved by a body such as CE need only verify the manufacturer’s performance characteristics and show they can meet the level of performance in the manufacturer’s instructions.”
As always, Dr. Jhaveri adds, “Laboratories must practice to the maximum requirements applicable to their geographic regions.”
If a laboratory performs tests that are not FDA approved or cleared or not approved by an internationally recognized regulatory authority, those tests are considered LDTs and subject to the requirements for test method validation found in COM.40350 “Validation of Test Performance Specifications—Modified FDA-Cleared/Approved Tests and LDTs.”
“This simply says that if a test is not approved, the lab must do a full validation. If the lab modifies an approved test, the modification must be validated,” Dr. Sarewitz says.
COM.40350 now says that for labs not subject to U.S. regulations, this requirement also applies to tests not approved by an internationally recognized regulatory authority and to approved tests that the lab has modified.
Dr. Goodman
Test modification is sometimes so subtle, Dr. Goodman says, that lab personnel may not be aware a modification has been made. “For example, let’s say I have an FDA- or CE-approved method of running a gonorrhea test on a cervical swab, and now I want to run it on a throat swab but my platform hasn’t been approved for that specimen type. That is now a modification of an FDA- or CE-approved test and it becomes an LDT.
“Or let’s say I have an FDA- or CE-approved test to look at alpha mutations in lung cancer using a specific probe but I decide to use a different probe. I’ve now modified the test,” he explains, “and it is no longer FDA or CE approved. In both of those cases I would have to do studies to validate them.”
Even when labs are aware of these subtleties, he says, often clinicians are not. “They’ll bring down a specimen and say, ‘We just want you to run this fluid and see if it has creatinine in it,’ but the fluid is not urine. That would mean changing an FDA- or CE-approved serum test for creatinine into a body fluid test for creatinine, and that would not be FDA or CE approved. That modification would require validation.” With a simple change, he says, it’s easy to turn a cleared or approved test into a modified test or an LDT.
COM.40830 says the lab must maintain a list of LDTs and modified approved tests, and the same is true of labs not subject to U.S. regulations.
The all common checklist says, in COM.40640 “Clinical Claims Validation,” that all claims the laboratory makes must be validated for LDTs and for FDA-cleared or -approved tests for which the claim is not in manufacturer instructions. It now says this requirement applies to labs not subject to U.S. regulations that make clinical claims not included in manufacturer instructions about tests approved by an internationally recognized regulatory authority.
“Many laboratories perform testing without making any clinical claims about their testing. They perform a test, then deliver their results, period,” Dr. Goodman says.
If a lab suggests an FDA-approved or -cleared test may be used for something not in the manufacturer instructions, or if a non-U.S. lab does so for a test approved by an internationally recognized regulatory authority, then the lab has to validate the clinical claim.
“Let’s say a lab has an FDA- or CE-approved blood test that detects some kind of cancer that is detailed in the manufacturer instructions,” Dr. Sarewitz says, “but the lab sends out a memo on its catalog saying the test can be used to detect another condition not in the manufacturer instructions. The lab must show it has done its due diligence and completed studies to support that contention.”
Valerie Neff Newitt is a writer in Audubon, Pa.