Ask SAM: How well do you do on HER2 testing?

December 2007
Feature Story

Karen Lusky

Meet SAM, evaluator and instructor. It's the College's latest endeavor to ensure HER2 testing is as accurate as it can be in identifying patients whose breast cancer may respond to trastuzumab therapy.

Piloted at CAP '07, the SAM, short for self-assessment module, made it possible for pathologists to test their mettle in applying the American Society of Clinical Oncology/ CAP HER2 testing guidelines published one year ago.

Pathologists M. Elizabeth H. Hammond, MD, Kenneth Bloom, MD, and David Hicks, MD, offered a HER2 testing symposium at CAP '07 to complement the SAM, a 35-item test they created in conjunction with the CAP Education Committee.

Dr. Hammond, a coauthor of the ASCO/CAP HER2 testing guidelines, says the symposium and SAM focused on the pathologist's role in HER2 testing rather than on the lab's role.

"We pitched it to pathologists who would be doing the testing and needed to either qualify the test in their lab or do the test themselves," says Dr. Hammond, professor of pathology at the University of Utah and former chair of pathology at LDS Hospital of Intermountain Healthcare.

The SAM didn't keep test-takers in suspense about how well they did. The computerized exam provided immediate feedback on whether selected answers were right or wrong and why.

By providing the feedback, the SAM, in fact, earned high marks from participants as a source of instruction in its own right. Seventy-four percent of participants ranked the SAM as a "five" on a five-point scale in terms of whether it helped them learn more about HER2 testing, says Arlene Thompson, a Chicago-based instructional design consultant who provided technical assistance in developing the module. The average rating was 4.71.

Forty-one pathologists at CAP '07 took the self-assessment test, and 32 passed it with a score of 80 percent or higher. Those who took the test said it was challenging, says Dr. Hicks, director of surgical pathology at the University of Rochester (NY) Medical Center. "It's our feeling that anyone doing HER2 testing should be able to pass the test," he adds. "If they couldn't, they had some knowledge gaps that need to be addressed."

Current plans include offering the HER2 testing SAM online instead of during the annual meeting. "Many people who came out of the test said they wanted to make sure others in their practice took [it]," Dr. Hammond says.

Pathologists attending CAP '07 had the option of using the SAM as a stand-alone test—that is, independent of the symposium. But two-thirds of the SAM users reported that they attended the symposium, Thompson says. And 59 percent of those said the symposium helped them prepare.

E. Randy Eckert, MD, a pathologist at North Austin (Tex.) Medical Center, who attended the symposium and took the self-assessment test, believes that to pass the test, a pathologist would have had to at least download and review the pre-reading materials available via symposium or SAM.

In the symposium, Dr. Hammond reviewed the HER2 testing guidelines, explaining what they require and why. Dr. Bloom talked about immunohistochemistry HER2 testing; Dr. Hicks presented on FISH.

"One of the key messages" from the symposium, Dr. Hicks says, was that IHC and FISH each pose technical and performance challenges, "and the two methods are actually complementary."

Dr. Eckert says the symposium and materials explained why it's important to patient care for a pathologist to know how to manage an indeterminate category, such as a 2+ result using IHC. "If you get a 2+ IHC result," Dr. Eckert says, "you need to attempt to determine the patient's HER2 status by doing FISH. Or if the values for FISH fall within the borderline range, it's important to reflex back to IHC."

The HER2 symposium and SAM also addressed the ASCO/CAP guideline requirements that breast tissue be fixed in formalin for a minimum of six hours to a maximum of 48 hours before undergoing HER2 testing.

Dr. Hammond shares a typical situation that might involve a fixation issue: A 50-year-old woman undergoes a lumpectomy after having had a mammogram, the result of which was equivocal. The lumpectomy specimen is rushed through same-day processing because the clinician wants the results at 9 AM the next day. So the tissue is fixed in formalin for only two hours instead of the minimum six hours required by the ASCO/CAP guidelines for HER2 testing. The lumpectomy sample shows high-grade infiltrating intraductal breast carcinoma that's HER2 negative and ER/PR negative.

Because of the short fixation time, what would be the pathologist's appropriate response to the negative result?

"The best response," Dr. Hammond says, "would be to call the test equivocal due to the short fixation time and state that the test will be sent out for FISH." She points out that FISH is much less sensitive than IHC to under-fixation.

The scenario offers a couple of clues that under-fixation may have led to the negative result: The estrogen and progesterone receptors were also negative, and the tumor was high grade. Thus, one might expect the HER2 result to be positive, Dr. Hammond says.

When Dr. Hammond writes SAM questions on HER2 in the future, she's going to try to include questions for the pathologist about how to deal with those in an institution who insist that breast tissue be fixed less than the required six hours.

The SAM also had a visual component. "The test included pictures of slides where the pathologist was asked to interpret the images as part of case presentations," Dr. Hicks says. "Some questions showed a series of images and you had to make a choice about which one best represented a positive or negative result." Or the test might give a HER2 result and ask the pathologist if it's adequate for interpretation and, if so, what the correct interpretation is.

John W. Turner, MD, staff pathologist at Johnston-Willis Hospital, Richmond, Va., took the self-assessment test at CAP '07 and describes it as a "real-world practical type of test." He says, "It was an 'if you were troubleshooting a problem in your lab, what would you do?' kind of quiz." How to do QC in the lab and how to know "you have all of your QC ducks in a row before you read out a result" was the central message, Dr. Turner says. "If you couldn't answer a lot of the QC-related questions, you might think hard about the quality of procedures in your lab."

Dennis Frisman, MD, founder of ImmunoQuery (www.immunoquery.com), an informatics system for selecting and ordering immunostains, thought the SAM was a "difficult but important test" of how well HER2 laboratory testing is understood. "You not only need much understanding about the meaning of the HER2 immunohistochemical results but how you arrived at them," he says. And Gail H. Vance, MD, a coauthor of the guidelines and professor of medical and molecular genetics at Indiana University, Indianapolis, liked the way SAM "pulled it all together in a practical way."

Dr. Eckert thought the test was "very well done" and noted only one drawback: The images were sometimes "difficult to discern" on the small laptop screens used to display the test. Projecting a clear image on small screens consistently is difficult because the screens are set at various resolutions and color saturations, he notes. "Even though you could click on and enlarge the images, it wasn't the same quality as I'd get in examining specimens under my own microscope in my office. And since sometimes HER2 is an interpretive test...being able to see all of the details in high resolution is important," he says. Providing larger monitors with higher resolutions and using photos taken at the highest resolution possible would be the only way, in his view, to address his concern about image quality.

And that's where pilot testing proves useful. "It enables us to identify necessary content and delivery improvements," Thompson says, "including the impact of the learning technology itself." She says image clarity will be addressed.

The CAP Education Committee is set this month to review the HER2 SAM pilot results and feedback and to discuss how to proceed with the SAM, says Ann Neumann, PhD, CAP director of educational program evaluation. Based on the positive evaluation results from the pilot, only minor modifications are expected before the HER2 testing SAM is released to the broader pathologist audience.

The CAP intends to develop SAMs for the maintenance of certification, or MOC, program required by the American Board of Medical Specialties for all physicians. Pathologists who were certified in June 2006 and thereafter will have to be recertified every 10 years, and included in that process is the requirement to complete SAMs successfully. "If the HER2 SAM is what MOC is about, I'm all for it," Dr. Vance says.

Even on HER2 testing, more SAMs are being considered, says Dr. Hammond, who is going to recommend developing separate tests for pathologists who do FISH and IHC. She foresees the HER2 self-assessment modules playing a role in the future in terms of giving pathologists a way to demonstrate their HER2 testing competency to their colleagues and potentially also to patients.

"There are noises from medical oncologists about wanting pathologists doing HER2 testing to pass a competency exam," she says. That idea could "percolate into the patient population where patients may feel more comfortable that the pathologist doing their testing has passed the competency exam."

She has advocated that labs notify people receiving their pathology report that the lab is accredited by the CAP and performs proficiency testing. And maybe in the future, pathologists can also say they have passed the SAM. "Those things would help assure the public that a qualified lab was doing their testing. That could give a lab a competitive edge."

Like the SAM, the HER2 testing guidelines remain a work in progress. Dr. Vance says the guidelines are a "living document" and there are plans to expand them.

The CAP and ASCO are also talking about developing guidelines for estrogen receptor testing, so there could eventually be a SAM for that also.

"But HER2 is much more critical than ER because the HER2 test is very expensive and has a high rate of false-positives"—18 percent for IHC testing and 13 percent for FISH, Dr. Hammond says. The highest false-negative rate reported is about seven percent, though Dr. Hammond believes the actual false-negative rate is quite a bit lower than that. "False-positives are the biggest problem," she notes. And patients who receive them can end up exposed to an unnecessary treatment that poses an eight percent risk of their developing congestive heart failure, she says. The cost of one year of the therapy is $100,000.

What the CAP is trying to do with the ASCO/CAP HER2 testing guidelines, ongoing educational symposia, and now the SAM is to reduce the variation in testing for HER2 that leads to inaccurate results. The test "must be done in a standardized way every time," Dr. Hammond says. "When it isn't, people are harmed."