Tarik M. Elsheikh, MD
January 2015—A task force of the American Society of Cytopathology in 2009 began the work involved in developing workload recommendations for cytotechnologists who screen image-guided Pap tests. The available data strongly suggested that Pap test screening workloads, as currently approved by the FDA and practiced in some laboratories, are too high and may represent a patient safety risk for the women whose Pap tests are reviewed under those conditions.
Of 65 million Pap tests examined annually in the United States, it is estimated that 85 to 90 percent are processed by liquid-based cytology and that 50 to 60 percent of the latter are image guided. The two most commonly used imaging systems in cervical screening today are the ThinPrep Imaging System and the FocalPoint GS. The image processor in these systems locates 22 or 10 fields of view (FOV) for every slide imaged, by ThinPrep Imaging or FocalPoint GS, respectively. The cytotechnologist then evaluates all FOVs and if no abnormalities are found, the case is finalized as “negative.” The finding of any abnormalities on any of the FOVs requires full manual review (FMR) of the entire slide.
Many studies have shown increased sensitivity associated with the imaging systems, including higher rates of detection of ASCUS, LSIL, and HSIL. The most striking outcome of using these imaging devices, however, has been not enhanced sensitivity but rather increased productivity. The current FDA-approved workload limits are doubled for image-assisted Paps compared with manually reviewed slides. (Imaged—FOV only—slides are counted as 0.5. Daily limit is 200 slides.) As a result, some laboratories are encouraging cytotechnologists to meet desired productivity expectations, not “quota” or “performance targets.” Expectations are not usually considered illegal because they are determined on an individual basis and do not require a minimum number of slides to be screened daily.
In September 2011, the American Society of Cytopathology published six recommendations (see page 46) that were endorsed unanimously by most major pathology/cytology professional societies, including the American Society for Clinical Pathology, American Society of Cytotechnologists, and Papanicolaou Society of Cytology. For the purpose of this summary, I will highlight three—Nos. 1, 3, and 4—of those recommendations. But first let us examine the evidence behind the recommendations—the literature review—which included the FDA clinical trials studies, retrospective studies, and prospective studies.
FDA clinical trials for the ThinPrep Imaging System and FocalPoint GS have suffered from major limitations, including small sample sizes (9,000 to 12,000 cases) as well as nonroutine laboratory (clinical trial) settings. That is, screening time calculations did not take into consideration computer time, including detailed check of clinical information/history or entry of results into the laboratory information system. In addition, cytotechnologist productivity, including the reported “high day rates,” were extrapolated from hourly rates, and “high eight-hour” daily screening rates were never actually achieved by any cytotechnologist (also extrapolated numbers). Extrapolated screening rates are not realistic because they don’t take into account necessary breaks or fatigue.
Several retrospective studies correlated productivity and sensitivity using image-guided systems (all slides were counted as 1.0, manual or imaged only). There were extremes in results ranging from no appreciable change in screening speed up to a more than 200 percent increase in productivity (about 200–228 slides per day). No significant gain in sensitivity or specificity was achieved at higher speeds (140–160 slides per day), and those studies that reported significant increases in sensitivity showed only modest gains in productivity. Overall, workloads of more than 100 slides per day appeared to lead to decreased rates of detection of HSIL and lower screening performance by the cytotechnologists. Renshaw, et al.,1 reviewed all studies that compared manual screening versus image-guided screening with the ThinPrep Imaging System, and identified three distinct workload ranges (all slides counted as 1.0): low (<60 slides per day), where workload did not influence screening accuracy; intermediate (60–103 slides per day), where the imager consistently increased cytotechnologist detection of HSIL; and high (>103 slides per day), where the imager did not increase HSIL detection. Interestingly, when detection of ASCUS increased, HSIL decreased, suggesting that cytotechnologists tended to diagnose more abnormal cases as ASCUS rather than classify them precisely as HSIL.
Also evident from those retrospective studies was that increased speed was achieved mostly by reducing the time spent examining fields of view and reducing the percentage of cases that underwent full manual review (as low as three percent FMR has been reported in the literature). As workload increased, the time devoted to screen fields of view decreased. Cytotechnologists also struggled to identify ASCUS and HSIL at higher speeds, which led to more misses. Most false-negative cases were due to failure to identify abnormal changes that were present in at least one of the fields of view.
Two prospective longitudinal studies, performed by Elsheikh, et al.,2 and Levi, et al.,3 correlated image-guided screening with workload, using the ThinPrep Imaging System and FocalPoint GS, respectively. Each study evaluated the performance of three cytotechnologists with variable levels of experience and screening speeds. They were asked to progressively increase their productivity over three phases (six to eight weeks), and they were told they did not have to screen a mandatory minimum number of slides. The findings from both studies were similar: Overall, as cytotechnologist workload increased to more than 100 slides per day, screening time spent per field of view decreased, %FMR decreased, abnormal rate decreased, and false-negative fraction increased (all rates were statistically significant). In addition, the Elsheikh, et al., study demonstrated that the decreased detection rates of abnormalities were associated with decreased ASCUS and elevated ASCUS-HPV+ rates (all values statistically significant), suggesting that the cytotechnologists established a higher threshold for calling atypia, which led to undercalling of abnormals.
I turn now to the American Society of Cytopathology recommendations for productivity and QA in the automated screening era. Again, for the purpose of this writing I am highlighting only three of the six recommendations. The complete list of recommendations, including explanatory notes and references, can be viewed at http://j.mp/ASCworkloadrecs. These recommendations apply only to gynecologic cytology specimens with image-assisted screening.
1. Cytotechnologists’ workday should not include more than seven hours of gynecologic (Pap test) screening in a 24-hour period, provided there are no additional duties or distractions.
Based on available evidence that fatigue and discomfort increase over time, No. 1 above is considered good practice. These recommendations apply only to gynecologic screening and do not necessarily apply to nongynecologic cytology, including immediate evaluation of adequacy of fine-needle aspirates. Non-screening time of gynecologic specimens must include at least two paid mini-breaks of 15 minutes each and a 30-minute lunch break in an eight-hour day. Breaks constitute a complete break from microscopy work and cannot include other activities such as data entry, quality assurance, and nongynecologic specimen immediate evaluation and screening. Time allotted for breaks is intended for mental and muscular rest, so it cannot be “worked through.” Employment for fewer than eight hours must also assume non-screening time of gynecologic specimens, including breaks, prorated to the total number of hours worked. For example, a person scheduled to screen Pap tests for a four-hour shift should have at least one 15-minute paid break and one 15-minute lunch break, which adds up to 3.5 hours of actual gynecologic screening and 30 minutes of non-screening.
3. Cytotechnologist average laboratory productivity should not exceed 70 slides per day using the Centers for Medicare and Medicaid Services recommendations for calculating workload (fields of view [FOV] only = 0.5 slide, full manual review [MR] = 1.0 slide, FOV + MR = 1.5 slide).
The current FDA workload limits for automated image-assisted screening methods, including the ThinPrep Imaging System and FocalPoint GS, are 100 slides per day, where slides are counted per the 2010 FDA bulletin. These rates are extremely high and may be associated with significant reduction in sensitivity. This American Society of Cytopathology recommendation is assuming a full manual review rate of imaged slides to be at least 15 to 20 percent (see recommendation No. 4). For example, with a 20 percent FMR, maximum number of slides examined per day will equal 80 “field of view only” slides (calculated as 80 × 0.5 = 40) plus 20 FOV + FMR slides (calculated as 20 × 1.5 = 30). So the total number of actual slides screened in this example is 100 slides (FOV and 20 percent FMR). We understand that screening rates vary from hour to hour, screener to screener, and slide to slide. This variation is expected as the complexity of the slides examined varies and performance of the cytotechnologist changes over time. These screening rates, therefore, are recommended as a maximum laboratory average, not as a maximum individual cytotechnologist performance.
4. The percentage of imaged slides that undergo full manual review should be at least either 15 percent, or twice (2×) the epithelial cell abnormality (ECA) rate, whichever is greater.
Studies have demonstrated that as workload and productivity increase, there is a tendency for FMR to decrease, which leads to decreased detection rates of abnormalities in the Pap test.
In summary, higher screening rates proportionally cancel out the increased sensitivity gained by imaging because there is a direct relationship between the amount of time spent screening slides and the accuracy of the reading. Increased cytotechnologist workload leads to reduced accuracy and higher false-negative rates. The current maximum FDA workload limits for automated image-assisted screening, including the ThinPrep Imaging System and FocalPoint GS (100 slides per day), are certainly too high for most cytotechnologists to achieve. Workload limits should not be based on extrapolated numbers and should take into account microscopic screening time, LIS time, and necessary breaks.
Cytotechnologists are not machines.
- Renshaw AA, Elsheikh TM. Predicting screening sensitivity from workload in gynecologic cytology: a review. Diagn Cytopathol. 2011;39:832–836.
- Elsheikh T, Kirkpatrick JL, Cooper MK, Johnson ML, Hawkins AP, Renshaw AA. Increasing cytotech workload above 100 slides per day using the ThinPrep imaging system leads to significant reductions in screening accuracy. Cancer Cytopathol. 2010;118:75-82.
- Levi A, Schofield K, Elsheikh TM, Harigopal M, Chhieng D. Effects of increasing cytotechnologist workload using the location guided imaging system FocalPoint GS on SurePath Pap tests (abstract). Cancer Cytopathol. 2010;118(suppl 5):307.
Dr. Elsheikh is medical director of AP services, Cleveland Clinic Laboratories, Cleveland, Ohio.