May 2009

Jared N. Schwartz, MD, PhD

On why we write guidelines

“All we want are the facts, ma’am.” —Joe Friday

When my brother and I were growing up, our parents had certain rules. We were expected to do our chores. We were expected to keep up with our homework. And we were expected to watch “Dragnet” with them. We were pretty good about the third one.

Every week, my family spent 30 minutes with detective Joe Friday, a terse, low-key cop with a mind for details and unfailing intuition. Friday had a good heart effectively disguised by a maniacal devotion to procedure and a low tolerance for dra­ma. He was the no-frills, no-nonsense guy that no criminal could outwit—a made-to-order role model, the prototypical Evidence-Based Man.

I hadn’t thought about Joe Friday in years but then he popped up as a role model again, this time in an engaging little book about how we think—as opposed to how we think we think.

To write Sway: The Irresistible Pull of Irrational Behavior, Ori and Ron Brafman examined the work of researchers who look at what happens when we confuse instinct and cultural bias with rational decisionmaking. For example, they describe work by Bradley University professor Allen Huffcutt, an expert on hiring techniques, whose meta-analysis determined that candidates screened via a “Joe Friday” approach were six times more likely to succeed on the job than were those selected after a traditional unstructured job interview. If you want to find the best person for the job, Huffcutt said, stick to the facts. Stay away from speculative questions about five-year plans. Hire on evidence of specific experience and skill. Save the chatter for lunch.

Evidence-based thinking is at the heart of the new CAP Pathology and Laboratory Quality Center charged with writing comprehensive pathology-focused best practices. M. Elizabeth Hammond, MD, chairs the CAP ad hoc committee that is structuring plans for the Center; she is also coordinating collaboration with the American Society of Clinical Oncology (ASCO) to write guidelines for estrogen receptor/progesterone receptor (ER/PR) testing.

We learned a lot from the HER2 guidelines, in no small part because we had the best of partners in ASCO, physicians who were equally committed to clarity and completeness. As pathologists, we knew a lot about the nuances of analytical testing but not so much about what attending physicians really needed to manage the clinical implications of test results. As oncologists, they knew a lot about patient management but not so much about the limitations and potential variables of the tests they order and the subtle judgment calls that keep pathologists late in the lab. Neither of us really understood the nature of what the other did for our patients and now we do, which is empowering on both sides and will be immensely valuable moving forward.

While Dr. Hammond’s group is still ironing out a master plan for the Center, the fundamentals are known. We know that guidelines must be carefully written, stipulating criteria for inclusion and exclusion of outliers. Best practices should be evidence-based or, if the topic is too new, carefully drafted by experts who sign their work. Membership on consensus committees should be inclusive, and terms should be limited. The guidelines themselves should be viewed as living documents that can be amended quickly in response to new scientific evidence.

Antonio Wolff, MD, an oncologist at Johns Hopkins who worked closely with the CAP on the HER2 guidelines, is collaborating with Dr. Hammond’s group on ER/PR testing. Dr. Wolff has worked on clinical practice guidelines for ASCO and is sensitive to the differences between writing standards for clinical validity (oncology) and writing standards for analytic validity (pathology). He has helped us to understand the thinking that underlies clinical practice guidelines, which are more informative and less prescriptive. Patients have variable needs and limitations, he says; in the patient care setting, inflexible mandates would only lead to nonadherence.

Guidelines for the laboratory are a little different because analytic validity rests upon precise measurements in controlled conditions. Laboratory professionals are dealing with hard numbers and concrete measurements, which is why we adhere to best practices until new scientific evidence is sufficiently persuasive to prompt an update.

Another member of Dr. Hammond’s committee, Steven Gutman, MD, came to appreciate the challenges of interpreting and acting on test results in the clinical setting during his tenure as director of the FDA Office of In Vitro Diagnostic Device Safety and Evaluation. Dr. Gutman, now a professor of pathology at the University of Central Florida, underscores the value of validating research-based judgments in the clinical context. “Nobody has greater reverence for tests than I do,” Dr. Gutman says. “I spent 17 years regulating them. But a test is a test and the patient is a patient and you have to connect the two. If the test doesn’t agree with the patient, the patient always wins.”

CAP Council on Scientific Affairs vice chair Paul Valenstein, MD, always has provocative things to say about quality; he thinks about it a lot. He reminds us that many physicians, patients, and regulators share an exaggerated belief in our ability to effectively treat human disease. Experts who write guidelines and individual caregivers can sometimes devise effective treatments, he says, but our ability to do so is limited. It is important to remember that humans are extremely complex biological systems thrust into extremely complex social and physical environments. To think that we can control for every eventuality is hubristic at best.

Dr. Valenstein is right, of course, and maybe it’s perverse, but I take encouragement from his warnings. We cannot expect to control for everything, but we can get closer, and we’re doing that with the Center. We’ll never know all the answers, but we are discovering new questions and more of our patients are surviving to pose them. As Dr. Hammond likes to say, we write guidelines to free up the necessary intellectual bandwidth to contemplate the complexities of the outliers. Or, as detective Friday might have put it, first we work with the facts.