Cytopathology in focus: Statistical reporting—benefits beyond the numbers

Janie Roberson, SCT(ASCP)
Sana Tabbara, MD

January 2022—The CAP has a robust Laboratory Accreditation Program with a commitment to continually improving the programs and providing appropriate resources needed for compliance. As a deemed status organization, validation surveys are performed annually through the Centers for Medicare and Medicaid Services and the feedback obtained provides direction for education. Along with the validation survey findings (2019 American Society for Cytotechnology Services), CAP inspection findings are monitored for frequency of citations. CYP.07650 was one of the most common phase one accreditation deficiencies in 2016–2019, and CYP.07600 was one of the most common phase two deficiencies in 2017–2019. CYP.07400 was also a common phase two citation (2017).

In response to these findings and to add clarity to the statistical reporting requirements, the CAP cytopathology checklist was recently reviewed and revised.^1-3 The revisions are not new requirements but clarifications to those in the existing checklist. These changes, related to statistical reporting, are effective in the 2021 checklist edition (Table).

For CYP.07600 (Statistical Records—Gynecologic Cytopathology), the revision clarifies the need to collect statistical data for the number of gynecologic cases reported by diagnosis for each specimen type. Additionally, the data must be evaluated by the laboratory and included in the annual cytopathology statistical report. Inclusion of atypical glandular cells (AGC) data is optional. Separate statistics for conventional and each type of liquid-based preparation (SurePath and ThinPrep) are required. Benchmarking data for conventional (including data for annual test volumes greater than 60 cases) and liquid-based preparations (including data for annual test volumes greater than 300), against which laboratory data should be evaluated, are provided in the checklist.

For CYP.07692 (Statistical Records—Non-gynecologic Cytopathology), the revision clarifies that records of the total number of nongynecologic cases examined need to be maintained. Subcategorization of nongynecologic specimen types (e.g. urine, pleural fluid, peritoneal fluid, FNA) and diagnostic categories (e.g. benign, atypical, malignant) is at the discretion of the laboratory. A record of unsatisfactory specimens is required. Written criteria for identification and reporting of unsatisfactory specimens must be defined in written laboratory policy and should be based on established guidelines (e.g. Bethesda System) where available.

Statistical reporting policy. The first step in achieving evidentiary compliance is developing a written policy that defines the components of the laboratory statistical report and parallels a sustainable laboratory practice. Careful alignment to the language in the requirements can ensure compliance. The policy should specify standardized methodology for retrieving data, data sources, who performs the review, and the frequency of retrieval and review.

Data acquisition. Cytology laboratories have evolved toward more sophisticated acquisition and use of data. While cytologic interpretation is largely subjective, the 30-plus years of CLIA regulatory requirements have fortuitously caused the laboratory to become rich in data to support objective assessments. One of the most beneficial developments toward data acquisition is the standardization of diagnostic terminology such as the Bethesda Systems for gynecologic and thyroid reporting.^4,5

Most laboratory information systems are now able to retrieve data for these basic reports, albeit not always with a straightforward approach. Laboratory staff should seek to understand the LIS data structure in order to build or request retrievals in a meaningful way. Input in building the LIS can pay dividends long term for standardized data acquisition. While gynecologic statistical requirements are well defined, latitude is given for laboratories to define categories of nongynecologic reporting. Rates of diagnostically unsatisfactory specimens are required for both categories.

As well as the mandated statistical reports, other periodically reviewed records are required, such as daily workload recording and workload assessment every six months, which yield valuable statistical data. Additionally, most institutions establish internal metrics that are appropriate and meaningful to their scope of practice. Turnaround time expectations and customer satisfaction are typical examples.

Data compilation, review, and analysis. Data in and of itself is not useful, however, and if not summarized in a meaningful way, much of the benefit (and regulatory compliance) is lost. The data gathered must be compiled in a format that allows for periodic review and facilitates comparison of data over time, an essential component of effective data utilization.

Laboratory staff may lack data management experience, which can handicap establishing a data review structure that is relevant and beneficial to the laboratory. A comprehensive dashboard of required statistics is a best practice for laboratories. Additionally, a few simple tools, such as Andon Signals, Control Charts, or similar functions can facilitate the visual assessment of data and trends that can inform even the novice reviewer.^6-8

The schedule of review is mandated for some data sets but may be laboratory defined for others. While formal review of the annual statistical report provides a static snapshot in time—of laboratory test volume, diagnostic distribution, and even turnaround times, for example—monthly or quarterly reviews can better reveal the dynamics of the laboratory including trends that may be seasonal or cyclical. Maintaining monthly or quarterly statistics that roll into the required annual report is efficient and educational. Timely reviews can also identify trends and guide potential change before problems escalate.

At a minimum, the cytology general and technical supervisors should be involved in a documented review process. Outliers (i.e. ASC/SIL ratios) must be addressed with corrective actions when appropriate (CYP.07650, CYP.07653).

Accurate analysis of this data can focus resources on areas in which improvements are needed, not just those of recency, personal bias, or high profile. CAP benchmarking information and laboratory-established thresholds based on the voice of the customer are important elements by which assessments are made.

In addition to showing evidence of compliance, documented analysis is necessary to maintain continuity and sustainability of processes where trending over time is important for reliable conclusions. The retention period for these records is typically two years, although state and institutional retention times may vary. Monitoring the data over longer periods has added benefit because the data tell a story of the laboratory technical evolution, reveal longitudinal trends, and simply preserve institutional history. Stable data management will transcend staffing changes and other laboratory transitions.

Data utilization. Laboratories expend valuable resources managing data and records for regulatory compliance. Beyond basic compliance, a comprehensive plan for gathering and evaluating annual statistics and other data-rich records offers quality and operational benefits to many aspects of laboratory practice. A final step in data management is using these data to drive decisions for future process improvements or efficiencies.

Diagnostic distributions, ASC/SIL ratios, human papillomavirus results, diagnostic reclassifications from random rescreens, cytology/histology correlations, and retrospective reviews can be indicators of diagnostic accuracy and overall laboratory quality. These data also provide the basis for calculating other accepted statistics such as false negative fractions (FNF) and comparison of individuals with overall lab statistics. Outliers that emerge can be addressed in a timely, systematic manner.

Monitoring turnaround times, unsatisfactory rates, and corrected reports provides opportunities to work with clients to target education and improve customer service as well as to assess internal staffing and reporting accuracy.

Understanding the laboratory’s entire “book of business” can go far when advocating with administrative staff for equipment, additional labor, or other resources. Test volumes and specimen types help in establishing supply inventory levels and directing capital purchases such as automated screening and telecytology systems. These data are essential in generating returns on investment and budgetary planning.

Staffing needs can be assessed with the help of cytotechnologists’ workload records by reviewing the total number of slides screened and rescreened; the diagnostic distributions indicate how many cases cytopathologists review. Time spent screening is captured through the prorated process for workload recording. While quotas are not advocated for, reasonable productivity expectations based on the laboratory setting can be determined. Quantifying the volume and labor requirements for rapid on-site evaluation and fine-needle aspiration procedures may justify staffing needs as well as reveal areas in which efficiencies can be gained.

Maintaining statistical records is a regulatory requirement and compliance brings a wide range of opportunities. The laboratory benefit is fully realized by taking advantage of the wealth of data collected through systematic retrieval, analysis, and utilization. A well-developed laboratory data analysis plan can have an impact on and inform every phase of the testing process and touch on virtually every element of the quality management system.^9-11

The requirement is clear. Compliance is mandated. The benefits to quality, operations, and future planning are worth the additional effort—to go beyond the numbers.

We acknowledge the assistance of Lyn Wielgos for data and document review.

1. Electronic Code of Federal Regulations. Accessed April 2021. https://www.ecfr.gov/cgi-bin/ECFR?page=browse.
2. Clinical Laboratory Improvement Amendments of 1988, final rule. Fed Register. 2003 (Jan. 24):7169[42CFR493.1274(e)(4)].
3. College of American Pathologists. Cytopathology checklist and Laboratory general checklist. Sept. 21, 2021.
4. Nayar R, Wilbur DC. The Bethesda System for Reporting Cervical Cytology: Definitions, Criteria, and Explanatory Notes. Springer; 2015.
5. Cibas ES, Ali SZ. The 2017 Bethesda System for reporting thyroid cytopathology. Thyroid. 2017;27(11):1341–1346.
6. Renshaw AA, Birdsong GG. Freeing the data from cytology databases in order to improve the quality of cytology. Diagn Cytopathol. 2019;47(1):48–52.
7. Chen Y, VanderLaan PA, Heher YK. Using the Model for Improvement and Plan-Do-Study-Act to effect SMART change and advance quality. Cancer Cytopathol. 2021;129(1):9–14.
8. Heher YK, Chen Y, VanderLaan PA. Measuring and assuring quality performance in cytology: a toolkit. Cancer Cytopathol. 2017;125(S6):502–507.
9. Clinical and Laboratory Standards Institute. QMSO1: A Quality Management System Model for Laboratory Services; Approved Guideline, 5th ed.; 2019.
10. Chandra S, Chandra H, Kusum A, Singh Gaur D. Study of the pre-analytical phase of an ISO 15189: 2012-certified cytopathology laboratory: a 5-year institutional experience. Acta Cytol. 2019;63(1):56–62.
11. Tresserra F, Temprana J, Vasquez C, et al. Developing indicators for quality assurance in cytopathology. Catalan Society of Cytopathology. Diagn Cytopathol. 2021;49(2):273–286.
12. Renshaw AA. Quality improvement in cytology: where do we go from here? Arch Pathol Lab Med. 2011;135(11):1387–1390.

Janie Roberson, a member of the CAP Cytopathology Committee, is senior director for laboratory quality, anatomic pathology, and cellular therapy, University of Alabama at Birmingham. Dr. Tabbara, chair of the CAP Cytopathology Committee, is director of cytopathology, Department of Pathology, Moffitt Cancer Center, Tampa, Fla.