Anne Paxton
July 2015—Secondary review of surgical pathology cases is a common, if not universal, practice in U.S. anatomic pathology departments. The evidence has shown that case reviews detect errors. But until now, one important thing has been missing: consensus on the actual standard of practice for such reviews. Anatomic pathology departments have lacked evidence-based guidelines on how and when to conduct reviews.
Dr. Nakhleh
A new guideline, “Interpretive Diagnostic Error Reduction in Surgical Pathology and Cytology,” jointly sponsored by the CAP and the Association of Directors of Anatomic and Surgical Pathology, aims to fill the gap.
Developed by a 10-member expert panel after an extensive literature review, the guideline’s main recommendations clarify that a systematic process of secondary review should be adopted and that such reviews should be timely, while other consensus statements in the guideline address monitoring and documentation, relevant procedures, and responding to poor agreement, if present. The guideline was published online in May (Nakhleh RE, et al. Arch Pathol Lab Med. Epub ahead of print May 12, 2015. doi:10.5858/arpa.2014-0511-SA), following an open comment period that indicated strong agreement—87 percent to 93 percent—from those who weighed in.
“An informed second review of cases is an important step to decrease errors. There’s no question about it,” says study coauthor Vania Nosé, MD, who, as co-chair of the guideline committee, represented ADASP. “We have a lot of information on different practices in error reduction throughout the country, and the majority of U.S. hospitals, the majority of pathology departments, do have a way of reviewing cases. But it’s not standardized. There are few rules or recommendations to help directors of surgical pathology develop a system of case reviews.”
In 2012, the CAP and ADASP jointly developed and published criteria for “critical” diagnoses in anatomic and surgical pathology, says Dr. Nosé, professor of pathology at Harvard Medical School and director of anatomic pathology and molecular pathology at Massachusetts General Hospital. Later, the CAP and ADASP agreed that recommendations for the review of pathology cases to detect or prevent interpretive diagnostic errors were needed, and the organizations decided to jointly develop the interpretive diagnostic error reduction guideline.
The idea for this guideline was approved in 2010 by the CAP and ADASP, says Raouf Nakhleh, MD, guideline co-chair representing the CAP and a professor of pathology at Mayo Clinic in Jacksonville, Fla. “It took a couple of years to get everything in place and get the expert panel together. Then we worked to evaluate the literature, which took a little over two years.” The expert panel reviewed 823 published articles.
Helping to push the guideline forward are the more conservative surgical approaches that are changing surgical pathology, Dr. Nakhleh notes. “Because the trend is to conduct less-invasive procedures, there are more endoscopic biopsies, more fine-needle aspiration, resulting in less tissue. So as we get smaller and smaller tissues, we’re challenged to be as precise as possible.” Within this context, the need for secondary review has become even greater, he believes.
Dr. Nakhleh says a correct diagnosis boils down to five factors: the pathologist’s knowledge and experience, clinical correlation, use of standardized terminology, use of confirmatory ancillary studies when available, and some type of review process to make sure diagnoses are correct.
The last component, case review, has been discussed often. “But there hasn’t been any systematic, methodical examination of this topic. That was our intent in this project,” Dr. Nakhleh says.
Pathologists have traditionally had an informal process of case review, he adds. “Instinctively, most pathologists—when they have a case with an unfamiliar lesion—will share that case with someone who is more familiar with that lesion. But it’s done in a haphazard way; the process is not defined. So we wanted to examine the literature to find out if this is effective and, second, whether there is a method of review that is superior in identifying discrepancies.”
After a comprehensive literature review, the expert panel crafted five high-level recommendations and expert consensus statements to formalize the recommended process for review of surgical pathology and cytology cases.
The panel’s recommendations are that anatomic pathologists should: develop procedures for the review of selected pathology cases to detect disagreements and potential interpretive errors; perform case reviews in a timely manner to avoid having a negative impact on patient care; have documented case review procedures that are relevant to their practice setting; and continually monitor and document the results of case reviews. The fifth of the recommendations is that if pathology case reviews show poor agreement within a defined case type, anatomic pathologists should take steps to improve agreement.
To develop these recommendations, the panel asked two key questions during its systematic literature search: Does targeted review at either the analytic or postanalytic phase of surgical pathology or cytology cases (slides and/or reports) reduce the error rate or increase the rate of interpretive error detection compared with no review, random review, or usual review procedures? And: What methods of selecting cases for review have been shown to increase or decrease the rate of interpretive error detection compared with no review, random review, or usual procedures?
The panel evaluated more than 800 studies and extracted data from 137 articles, suggesting a broad review, but Dr. Nakhleh insists that the focus was actually very limited. “We really focused on papers that discussed a review of cases that compared original diagnoses with subsequent diagnoses. We didn’t try to look at other factors. And it was pretty clear that if you reviewed cases, diagnostic disagreements were found, and some of those were real errors.”
The quality of the literature was graded as to rigorousness, though most of it was not up to the gold standard of a double-blind prospective randomized trial. “In pathology, we don’t really have very many studies at that level,” Dr. Nakhleh says. “It’s a bit of a problem, because if you find an error in a diagnosis, you’re always going to correct it, just as you’re not going to test whether parachutes save lives by randomizing and having some people jump out of planes without parachutes. So it may not be possible to get to a high-level quality of evidence.”
Of the more than 800 studies, “we actually found only five studies that compared different types of review. Prospective review was compared with retrospective review of surgical pathology and cytology cases,” Dr. Nakhleh says.
The analytic phase of surgical pathology and cytology, unlike that of clinical pathology, involves the inherent judgment of the pathologist at the time of slide interpretation, the expert panel notes, saying, “It is therefore more subjective than clinical laboratory tests.” But this label does not mean the judgments in surgical pathology and cytology are less scientific.
A key finding of the study, in addition to the recommendations, is that the pathologist’s experience and knowledge are paramount in any review process. “Obviously, the more experienced a pathologist is in a certain type of lesion, the easier it is to make that diagnosis,” Dr. Nakhleh says. “A pathologist’s expertise comes not only in the form of being able to make a diagnosis but also in sharing information about addressing clinical correlation, or the type of ancillary testing needed to confirm a diagnosis. The interaction with a more knowledgeable pathologist helps complete the balance involved in managing the five factors leading to an accurate diagnosis.”
Some lesions require a high level of judgment because they lack objective criteria, Dr. Nakhleh notes. “It depends on what type of diagnosis you’re looking at. For some lesions where we have objective criteria—for example, the presence of an organism, seen morphologically or by special stain—a second opinion may not be necessary. But there are many lesions, particularly those in a gray zone between benign and malignant, where subjective criteria are used. Are the nuclei big enough to be dysplastic? Or too big to be benign? This requires some judgment. With experience, a pathologist becomes better and develops a level of comfort at making the diagnosis.”
The pathologist’s knowledge and experience remain the most important factors in interpretive diagnosis, Dr. Nosé says. But pathology is not one of the most subjective areas in medicine. “I would say that pathology is a science you learn by putting things together. But the research shows that pathology is not like a mathematical science,” she says. “In my experience, training is very important for error reduction. With knowledge and experience, a pathologist becomes expert at recognizing different lesions, but no one knows all the lesions. That’s why we share cases and have case reviews.”
Dr. Nosé acknowledges that people don’t like guidelines in general. But the case review process has proven value, and some standardization in how the process is applied will, in her view, significantly improve its role in reducing error. She hopes, as a result of this guideline, more institutions will develop procedures for review of cases.
The five studies brought out the critical importance of timeliness. “If a case is reviewed before it is signed out and a discrepancy is detected, that’s the best-case scenario,” Dr. Nakhleh says. “But we want to make sure people understand a review after sign-out can also be timely, so long as the patient has not been treated. Sometimes cases are reviewed for a clinical-pathologic correlation conference that’s intended as a working conference, but the review is after the report is signed out. That’s still timely because the patient is not going to be treated until after that conference.” Speaking generally, he says, “As long as the review occurs in a timely fashion—before the patient is treated—if errors are detected, a lot of headaches can be avoided.”
Despite the extensive professional training and experience pathologists may have, Dr. Nosé believes discretion should give way to standardized and systematic practices when it comes to case review. “I’m a scientist, but when you talk about surgical pathology sign-out cases, we feel there should be standard procedures for review and reviews should be timely. That means they should not be reviewed in a year; it doesn’t help patients if you take that long and review it later. We should review as soon as we have the case.”
Some components of the five-part guideline did not rise to the level of a recommendation but are presented as “expert consensus opinion.” “The College has designated that there needs to be a certain amount of evidence for something to be a recommendation,” Dr. Nakhleh says.
The expert panel arrived at the consensus that anatomic pathologists “should have documented case review procedures that are relevant to their practice setting.” Possible choices include a review of the following: selected types of diagnoses or a selected percentage of cases, selected organ system or specimen type, random cases, cases for multidisciplinary conferences, in-house cases sent outside for review, cases during cytology-histology correlation, and cases in a consensus conference. The laboratory medical director, the guideline says, is responsible for choosing which methods are best suited to the particular practice setting. One study did demonstrate that review focused on an organ system was more efficient at detecting discrepancies than a random review of cases.
An important measure to ensure error reduction is to include negative cases in the secondary reviews. “This is definitely valuable,” Dr. Nosé says.
“A common recommendation that is made is to review ‘all cancers’—in other words, all positive cases,” says Dr. Nakhleh. “But the literature shows us that up to 70 percent of interpretive errors are diagnoses in which a lesion was missed on initial review. So in a secondary review procedure, negative cases should be included that are high risk for being missed, or else you’re never going to catch them. If you review only the positive cases, you’re never going to catch a false-negative case.”
In his own department at Mayo Clinic Florida, “We like to review all initial breast biopsies by two pathologists, because we view that as a high-risk area, regardless of whether they are positive or negative.”
Mayo Clinic Florida implemented this policy about 10 years ago, Dr. Nakhleh says. An additional personal preference that is not a department policy: “Sometimes I have multiple prostate biopsies totaling several trays of slides. And if they’re all negative, I personally like for someone else to double-check that work, because it’s very easy to miss a small lesion.”
Collaborative discussions with clinicians help improve the diagnostic process in general, Dr. Nakhleh notes. Depending on the size of the institution, “at a certain point, each pathologist tends to serve as a point person to their clinical counterparts.” In the case of breast cancer, at his hospital, a pathologist regularly goes to the radiographic correlation conference and other meetings with oncologists and surgeons. “Those discussions help clinicians understand what we’re doing in pathology and vice versa. So clinical correlation is very important,” says Dr. Nakhleh, who meets regularly with his hepatology colleagues and surgeons. “This is part of the overall spectrum of what we do. Case reviews are only a tiny part of it.”
Which lesions are tricky or difficult to diagnose can change over time and with experience, he notes. “That’s why, in this guideline, we’re not specifying the type of cases that a department should review. What could be difficult today may be easier in the future, while something else will come up later. That’s why we feel we have to continuously re-assess a review policy within the context of a quality assurance plan, and continuously focus on what is difficult at the moment.”
Many institutions already have review processes in place that are similar to what is recommended in the guideline. “They may not be doing it in a very systematic way, but a lot of institutions do have prescribed lists of lesions they like to show to other pathologists.”
The expert consensus statement on how to address poor agreement on diagnoses within anatomic pathology groups notes that the causes of poor agreement are variable. Studies do show that some diagnoses have inherently higher interobserver variation than others; assessment of thyroid lesions by fine-needle aspiration cytology and assessment of esophageal dysplasia in the setting of Barrett esophagus are examples where low diagnostic agreement is found. But the expert panel notes that such methods as intradepartmental consensus conferences, acceptance and use of uniform diagnostic criteria, and use of calibration slide sets can reduce interobserver disagreement.
Smaller institutions often have to take a different approach to handling pathology reviews. “They have to be a little more creative, and many pathologists understand this very well,” Dr. Nakhleh says. “For example, many small institutions don’t try to tackle bone marrow biopsies or flow cytometry, or renal biopsies. They will send those complex specimens out to a laboratory that has a higher volume. Down the road, there may be a role for digital pathology where a pathologist could be practicing anywhere and can share images of a particular lesion with willing partners anywhere else in real time and be able to discuss the case.”
“We’re so much better when we are able to talk to a colleague about a case, and think about it and go through the workup to arrive at a confident diagnosis,” he says. “It’s much more difficult for pathologists practicing on their own.” In many cases a diagnosis made at a small hospital will get referred to another larger institution where the patient will receive definitive treatment, and those cases usually get reviewed at that second hospital. “So there are mechanisms in place to deal with such cases.”
To Dr. Nakhleh, the bottom line is that making a diagnosis is a multifactorial process, and review of cases is one important part of that. “Case review is actually a very nice mechanism when done in an effective way. With prospective reviews a pathologist can discuss the case with a knowledgeable colleague, plan out what needs to be done with the case, and arrive at an accurate diagnosis. It helps build a team and it’s also better for patients.”
[hr]
Anne Paxton is a writer in Seattle.