Cytopathology and More | Automated screening workload limits are too high

Tarik M. Elsheikh, MD

January 2015—A task force of the American Society of Cytopathology in 2009 began the work involved in developing workload recommendations for cytotechnologists who screen image-guided Pap tests. The available data strongly suggested that Pap test screening workloads, as currently approved by the FDA and practiced in some laboratories, are too high and may represent a patient safety risk for the women whose Pap tests are reviewed under those conditions.

Of 65 million Pap tests examined annually in the United States, it is estimated that 85 to 90 percent are processed by liquid-based cytology and that 50 to 60 percent of the latter are image guided. The two most commonly used imaging systems in cervical screening today are the ThinPrep Imaging System and the FocalPoint GS. The image processor in these systems locates 22 or 10 fields of view (FOV) for every slide imaged, by ThinPrep Imaging or FocalPoint GS, respectively. The cytotechnologist then evaluates all FOVs and if no abnormalities are found, the case is finalized as “negative.” The finding of any abnormalities on any of the FOVs requires full manual review (FMR) of the entire slide.

Many studies have shown increased sensitivity associated with the imaging systems, including higher rates of detection of ASCUS, LSIL, and HSIL. The most striking outcome of using these imaging devices, however, has been not enhanced sensitivity but rather increased productivity. The current FDA-approved workload limits are doubled for image-assisted Paps compared with manually reviewed slides. (Imaged—FOV only—slides are counted as 0.5. Daily limit is 200 slides.) As a result, some laboratories are encouraging cytotechnologists to meet desired productivity expectations, not “quota” or “performance targets.” Expectations are not usually considered illegal because they are determined on an individual basis and do not require a minimum number of slides to be screened daily.

In September 2011, the American Society of Cytopathology published six recommendations (see page 46) that were endorsed unanimously by most major pathology/cytology professional societies, including the American Society for Clinical Pathology, American Society of Cytotechnologists, and Papanicolaou Society of Cytology. For the purpose of this summary, I will highlight three—Nos. 1, 3, and 4—of those recommendations. But first let us examine the evidence behind the recommendations—the literature review—which included the FDA clinical trials studies, retrospective studies, and prospective studies.

FDA clinical trials for the ThinPrep Imaging System and FocalPoint GS have suffered from major limitations, including small sample sizes (9,000 to 12,000 cases) as well as nonroutine laboratory (clinical trial) settings. That is, screening time calculations did not take into consideration computer time, including detailed check of clinical information/history or entry of results into the laboratory information system. In addition, cytotechnologist productivity, including the reported “high day rates,” were extrapolated from hourly rates, and “high eight-hour” daily screening rates were never actually achieved by any cytotechnologist (also extrapolated numbers). Extrapolated screening rates are not realistic because they don’t take into account necessary breaks or fatigue.

Several retrospective studies correlated productivity and sensitivity using image-guided systems (all slides were counted as 1.0, manual or imaged only). There were extremes in results ranging from no appreciable change in screening speed up to a more than 200 percent increase in productivity (about 200–228 slides per day). No significant gain in sensitivity or specificity was achieved at higher speeds (140–160 slides per day), and those studies that reported significant increases in sensitivity showed only modest gains in productivity. Overall, workloads of more than 100 slides per day appeared to lead to decreased rates of detection of HSIL and lower screening performance by the cytotechnologists. Renshaw, et al.,1 reviewed all studies that compared manual screening versus image-guided screening with the ThinPrep Imaging System, and identified three distinct workload ranges (all slides counted as 1.0): low (<60 slides per day), where workload did not influence screening accuracy; intermediate (60–103 slides per day), where the imager consistently increased cytotechnologist detection of HSIL; and high (>103 slides per day), where the imager did not increase HSIL detection. Interestingly, when detection of ASCUS increased, HSIL decreased, suggesting that cytotechnologists tended to diagnose more abnormal cases as ASCUS rather than classify them precisely as HSIL.

Also evident from those retrospective studies was that increased speed was achieved mostly by reducing the time spent examining fields of view and reducing the percentage of cases that underwent full manual review (as low as three percent FMR has been reported in the literature). As workload increased, the time devoted to screen fields of view decreased. Cytotechnologists also struggled to identify ASCUS and HSIL at higher speeds, which led to more misses. Most false-negative cases were due to failure to identify abnormal changes that were present in at least one of the fields of view.

Two prospective longitudinal studies, performed by Elsheikh, et al.,2 and Levi, et al.,3 correlated image-guided screening with workload, using the ThinPrep Imaging System and FocalPoint GS, respectively. Each study evaluated the performance of three cytotechnologists with variable levels of experience and screening speeds. They were asked to progressively increase their productivity over three phases (six to eight weeks), and they were told they did not have to screen a mandatory minimum number of slides. The findings from both studies were similar: Overall, as cytotechnologist workload increased to more than 100 slides per day, screening time spent per field of view decreased, %FMR decreased, abnormal rate decreased, and false-negative fraction increased (all rates were statistically significant). In addition, the Elsheikh, et al., study demonstrated that the decreased detection rates of abnormalities were associated with decreased ASCUS and elevated ASCUS-HPV+ rates (all values statistically significant), suggesting that the cytotechnologists established a higher threshold for calling atypia, which led to undercalling of abnormals.

I turn now to the American Society of Cytopathology recommendations for productivity and QA in the automated screening era. Again, for the purpose of this writing I am highlighting only three of the six recommendations. The complete list of recommendations, including explanatory notes and references, can be viewed at http://j.mp/ASCworkloadrecs. These recommendations apply only to gynecologic cytology specimens with image-assisted screening.