Q & A, 7/13

[pulledquote]Q. Can you clarify the difference between the terms “optimization,” “validation,” and “verification” as used in immunohistochemistry?[/pulledquote]

A. These three terms relate to the processes that the laboratory must undertake before new diagnostic, prognostic, or predictive immunohistochemistry markers are used for clinical and/or pathologic decisionmaking.

Optimization is the process by which the laboratory director determines provisional assay conditions, which most often involves staining a single case or small number of cases at varying assay conditions. The conditions that may be altered include primary antibody dilution, duration of primary antibody incubation, type of antigen retrieval buffer, antigen retrieval time, and detection chemistry, with the goal of having the strongest positive reaction with appropriate subcellular localization and minimizing, or eliminating, any reaction in cells that do not contain the protein in question. Once the laboratory director is satisfied that the quality of staining is optimal, as assessed in this small number of cases, one can proceed to the validation or verification step.

The terms validation and verification are often used interchangeably. Strictly speaking, however, they apply to different types of immunohistochemistry assays.

Verification is the process by which a laboratory determines that an assay performs according to the recommendations set forth by the manufacturer as documented in the product insert at the assay conditions determined during the optimization step. This process typically involves staining a number of cases that span the range of expected protein expression of the chosen protein, including a number of anticipated negative cases. The laboratory director should determine the number of cases that should be stained. Generally speaking, the number of cases to be tested during verification is larger if the results are to be used solely as a prognostic or predictive marker (for example, HER2). Also, if the number of result categories is higher than simply positive or negative, the number of verification cases should be high enough to test each of the result categories.

Validation is a more rigorous process than verification and applies only to laboratory-developed tests (LDTs). Since the test performance characteristics of an LDT have not, by definition, been determined by a manufacturer, a greater number of cases must be stained to confirm that the LDT performs according to the specifications determined by the laboratory director.

Both verification and validation require that the results of the assay be compared to a known standard.  These standards include cases stained in the same laboratory using a previously validated/verified assay, cases stained in another laboratory with a validated/verified assay, comparison with another technique, or comparison of results with findings reported in the peer-reviewed literature.

1. Clinical and Laboratory Standards Institute. Quality assurance for design control and implementation of immunohistochemistry assays; approved guideline—2nd edition. CLSI document I/LA-A2. Wayne, Pa: CLSI;2011.
2. Taylor CR, Cote RJ. Immunomicroscopy: A Diagnostic Tool for the Surgical Pathologist. 3rd edition. Philadelphia, Pa.: Saunders Elsevier;2006.
3. Fitzgibbons PL, Murphy DA, Hammond ME, et al. Recommendations for validating estrogen and progesterone receptor immunohistochemistry assays. Arch Pathol Lab Med. 2010;134(6):930–935.

Jeffrey Goldsmith, MD

Director, Surgical Pathology Laboratory
Beth Israel Deaconess Medical Center, Boston
Advisor, CAP Immunohistochemistry Committee

[pulledquote]Q. Is there a published set of minimum standards that a laboratory information system or anatomic pathology LIS must meet before it is put on the market? Vendors of hospitalwide electronic medical record systems are offering inadequate laboratory software (in terms of patient safety standards) for free, or at deep discounts, as part of their overall EMR software to entice hospital administrators to purchase their products, often over the lab’s protests.[/pulledquote]

A. Assessing an LIS in terms of metrics like “minimum standards” and “threats to patient safety” can be extremely difficult, if not impossible. An axiom in the field is that highly competent pathology personnel can make an inadequate LIS perform at a high level and, conversely, less competent personnel can cause a best-of-breed LIS to perform poorly. This comment highlights the important interactions of lab professionals with LIS software and the futility of a “regulatory” approach to most health care software. Another reason for the lack of “minimum standards” for LISs is that no two systems are identical. Each may be running different versions of the same software and will vary based on local differences, such as test names, reference ranges, and interfaces to other hospital systems.

However, it is incumbent on all pathologists to assess the functionality of any LIS under consideration for purchase or even after a system has gone live. This can be accomplished through the use of what can be called “functionality statements,” which decompose a complex lab process such as test order entry or result reporting into a number of declarative statements. These “functionality statements” can be incorporated into a request for proposal (RFP) that a vendor must respond to as part of an LIS purchase process. These vendor responses can then be incorporated into the contract with the chosen vendor and become legally binding after contract signing. These same functionality statements can be woven into scripted scenarios that are presented to vendors to guide their live demos of LISs under consideration. These scripted scenarios serve a dual purpose: 1) they provide proof of the veracity of the vendor responses to the RFP; and 2) they provide a glimpse of the efficiency of the LIS under consideration through a number of workflow vignettes.

Although a “regulatory” process is not generally available to assess the safety of LIS and most other health care software, a detailed assessment of the functionality and workflow requirements of an LIS under assessment can provide an effective means to ensure patient safety and effective and efficient lab operations with such a system. The Association for Pathology Informatics (API) is in the final stages of developing a Functionality Assessment Tool that includes a set of about 1,000 functionality statements, sample scripted scenarios, and a sample total cost of ownerships (TCO) spreadsheet comparing a hypothetical set of LISs. It will be available for download on the API Web site at no cost.

Bruce A. Friedman, MD

Active Emeritus Professor of Pathology
University of Michigan Medical School
Ann Arbor, Mich.