Sigma analysis, role and limitations: development of a QC program for the Beckman Coulter AU5812

Create PDF

Fig. 2. Improvements in cholesterol assay (AU5812 Apollo) with implementation of enhanced monitoring and maintenance (cholesterol TEa: 10%)

An additional result of this increased monitoring and maintenance was an increase in the metrics of other analytes. See Fig. 3 for an example of accompanying improvements in the calcium assay.

Fig. 3. Accompanying improvements in calcium assay (AU5812 Apollo) with implementation of enhanced monitoring and maintenance (calcium TEa: 1.0 mg/dL)

Discussion

Observation of the sigma metric variation between identical analyzers and assay chemistries—and its apparent dependence on routine maintenance procedures—highlights the directional utility of the sigma metric as a periodic indicator of overall system performance and a guide for the appropriate QC protocols in the clinical laboratory. Assays with sigma levels of three and four should be examined for ways to improve, perhaps in terms of more maintenance, calibrations, or personnel training, or all three. Some assays, owing to the strict CLIA-mandated TEa, may be difficult to raise above four sigma.

For example, at an Xc of 150 mmol/L, a sodium level must have no bias and a CV of 0.67 percent (TEa 4 mmol/L) or better to achieve four sigma or higher. Simplistic comparison of absolute values of sigma metrics for a specific analyte between different analyzers and assay chemistries without careful consideration of the confounding factors—such as the selection of TEa, study protocol, comparative method, reagent lot-to-lot variability, calibrator lot-to-lot variability, and maintenance schedule—may lead to erroneous conclusions about assay performance.

The periodic nature of common QC protocols creates the constant risk of within-run system issues not being detected until the next scheduled QC event. For a high-volume laboratory like RRMC, this means potential reruns performed on many hours of test results, as well as incremental risk of the erroneous clinical test results propagating to the point of affecting patient care. Moving averages (or, better yet perhaps, moving medians) might be best used to supplement traditional QC in critical assays where they can detect shifts and trends in real time. One limitation of moving averages is that they are specific to the patient population undergoing testing. For example, electrolytes in outpatients analyzed in the afternoon might be quite different than electrolytes measured in ICU or critical care inpatients in the morning, though today’s algorithms can be configured to account for these variables. Examples that demonstrate the effect of using moving averages or medians have been published in multiple articles.^10,11

Given the potential negative impact on patient care, the quality of laboratory test data must remain of paramount concern at all times. Experience suggests the need for holistic approaches to laboratory quality rather than relying on a single parameter such as sigma metrics. Assays and instrument systems should be assessed and monitored on multiple measures, including but not limited to precision, accuracy, standardization/traceability, reagent and calibrator stability, ease of use (and resistance to operating errors), and analytical measuring range. Other important metrics are lack of interferences, commonality of reagents and results across platforms, optimized workflow, daily setup time/FTE needs, preventive maintenance time/FTE needs, system uptime metrics, and scalability.

Conclusions

The real-world analysis of Beckman Coulter AU5812 analyzers in a high-volume central laboratory revealed excellent sigma metric performance, consistent with the data provided by the manufacturer, through the method validation studies. Detected dependence of the sigma metrics on routine maintenance procedures highlights the dynamic nature of the metric with both bias and precision changing over time, necessitating periodic monitoring and incremental QC techniques such as moving averages. While sigma metrics can help guide analyte-specific QC rules, as well as the number and frequency of QC observations used each day, a holistic approach to QC protocols is needed. This includes the skill and training of operators, instrument performance directly related to maintenance schedules and procedures as described in the preceding results section, reagent and calibrator quality and stability, and preanalytic sample handling. A total quality program is a multifaceted approach requiring the training, participation, and cooperation of the entire laboratory staff.

1. Clinical and Laboratory Standards Institute. QMS20-R: Understanding the Cost of Quality in the Laboratory; A Report. May 2014.
2. Clinical and Laboratory Standards Institute. C24: Statistical Quality Control for Quantitative Measurement Procedures: Principles and Definitions, 4th ed.; 2016.
3. Westgard JO. Six Sigma Quality Design and Control, 2nd ed. Madison, Wis.: Westgard Quality Corp.; 2006.
4. Clinical and Laboratory Standards Institute. EP21: Evaluation of Total Analytical Error for Quantitative Medical Laboratory Measurement Procedures, 2nd ed.; 2016.
5. U.S. Code of Federal Regulations 42 CFR §493.909–942 CFR §493.959.
6. Ricós C, Alvarez V, Cava F, et al. Current databases on biological variation: pros, cons and progress. Scand J Clin Lab Invest. 1999;59(7):491–500.
7. Richtlinie der Bundesärztekammer zur Qualitätssicherung laboratoriumsmedizini­scher Untersuchungen. Gemäss dem Beschluss des Vorstands der Bundesärztekammer vom 1.04.2014 und 20.06.2014. Deutsche Ärzteblatt. 2014;111(38):A1583–A1618.
8. Allowable limits of performance. Royal College of Pathologists of Australasia. www.rcpaqap.com.au/docs/2014/chempath/ALP.pdf. Accessed Dec. 13, 2016.
9. Shih J. Using six sigma metrics to monitor the quality of results. Publication BR-51862. Beckman Coulter Inc.; 2016.
10. Wilson A, Roberts WL, Pavlov I, Fontenot J, Jackson B. Patient result median monitoring for clinical laboratory quality control. Clin Chem Acta. 2011;412(15–16):1441–1446.
11. Van Rossum HH, Kemperman H. Optimization and validation of moving average quality control procedures using bias detection curves and moving average validation charts. Clin Chem Lab Med. 2017;55(2):218–224.

Jack Montgomery is the chemistry technical specialist, Asante Rogue Regional Medical Center, Medford, Ore.