Introduction
In this era of evidence-based medicine, clinical laboratory test results play a crucial role in providing valuable information about a patient’s health condition based on the analysis of various biological specimens, such as blood, urine and other body fluids. The goal of clinical laboratories is to provide accurate and reliable results within the stipulated time, as medical practitioners rely on these reports for diagnosis, treatment, progress monitoring and to make informed decisions about patient care management.1 Regular monitoring of quality in a clinical laboratory is essential to ensure the accuracy and precision of multiple analytes testing performed in the laboratory. Quality assurance and quality control measures are implemented to maintain high standards and reliability in laboratory testing. Under a quality management system, clinical laboratories process both internal and external quality control samples for quality assurance of all biochemical analytes.2 Currently, most clinical laboratories are implementing sigma metrics to monitor test performance quality, focus on continuous quality improvement based on the sigma score for all the analytes, and minimize operational defects in the process.3, 4
A higher sigma score indicates fewer errors in the pre-analytical, analytical, and post-analytical processes, as well as improved accuracy and precision of test results reported. Sigma score analysis helps identify analytes that have robust performance. Such analytes do not need frequent quality checks. Inversely, analytes exhibiting imprecision or inaccuracy require a more stringent quality control protocol.5, 6 Overall, monitoring the performance of all the analytes processed in the clinical laboratory, based on sigma metrics, offers the advantage of identifying the quality baseline. It aids in the quality improvement of laboratory practices to ensure accurate and reliable test results are reported for all patients.
The incorporation of sigma metrics scale in analytical performance in our lab revealed the poor performing assays in clinical chemistry and immunoassay tests. A detailed root cause analysis done identified operational defects like non-compliance with SOPs, frequent reagent lot changes, infrequent test requests, and staff competency issues. Solutions included staff education, training, regular competency assessments, preparation and implementation of detailed SOPs, and reagent lot reservation from manufacturers.
The goal of this study was to analyze and monitor the performance of 48 analytes based on both Chemistry and Immunological assays, which were processed at a tertiary care center using VITROS XT 7600 Integrated clinical laboratory systems.
Materials and Methods
This study was conducted at the Department of Biochemistry, Medanta Hospital, Lucknow, UP, India. The NABL-accredited Department of Biochemistry provides clinical laboratory services to all the patients undergoing health care management in the NABH-accredited tertiary care Hospital, Medanta Hospital, Lucknow, UP, India. It is a retrospective study spanning twelve months from Jan 2023-Dec 2023 for 48 clinical chemistry and immunoassays.
Data collection: Data for IQC was collected via UNITY REAL TIME software from Bio-Rad. Data for EQAS were collected via QCnet.com by BIO-RAD.
Inclusion criteria: All acceptable QC run data points were included for the duration of the study for the selected parameters.
Exclusion criteria: For the IQC, no data points were excluded for the selected parameters. In the case of folate, in the EQAS performance, the EQAS samples in the month of April, July, and October 2023 showed high values above the linearity limit of 20 ng/mL. Those results were not reported and excluded in the data analysis. Further, the parameters excluded from this study consisted of less frequently asked tests like Progesterone, for which IQC was processed only on demand and hence a limited data was available for analysis.
Analyzer
Department of Biochemistry at Medanta Hospital, Lucknow, is equipped with VITROS Total Lab Automation System (QuidelOrtho, USA) consisting pre-analytical systems viz., sample sorter with bar-code identification, decapper, sample router through tracker; analytical systems - fully automated VITROS XT 7600 Integrated systems for processing both Chemistry and immunological assays; and post-analytical system viz., recapper and buffer modules. All these modules are controlled by the middleware - VITROS Instrument Manager. All the biochemical analytes are processed using VITROS MicroSlide Technology and VITROS MicroTip Technology, and immunological assays are processed using VITROS enhanced chemiluminescence technology in VITROS XT 7600 Integrated systems.
VITROS XT 7600 system utilizes microsensor technology to monitor the quality of all the serum, plasma, and CSF samples in real time with respect to hemolysis, icteric and lipemic and report the sample quality in the form of HIT Index (Hemolysis, Icteric and Turbidity index). Affected results are flagged automatically alerting the user who then initiates requisite corrective action.
The Vitros XT 7600 integrates microslide, microtip, and enhanced chemiluminescence technologies, offering a wide range of tests. Dry chemistry systems, used by microslide technology, provide more stable results than wet chemistry and minimizes interferences from hemolysis, icteric, Lipemic and paraprotein issues. Use of disposable tips eliminate sample and reagent carryover, enhancing accuracy. The system boasts better reagent stability, long calibration stability, onboard refrigeration, and faster results, leading to excellent precision and cost efficiency. However, the system also has certain disadvantages like high setup costs, which deters laboratories with less workload from opting dry chemistry, the need to maintain ambient temperature and humidity is crucial for the system and reagent stability on board.
Figure 1
Number of clinical chemistry assays (1a) and immunoassays (1b) performing at various sigma levels
Table 1
The list of assays analyzed for its performance in sigma scale
Table 2
The internal quality control performance of clinical chemistry assays in terms of CV%
Table 3
The internal quality control performance of immunoassays in terms of CV%
Table 4
The EQAS performance of clinical chemistry assays in terms of bias%
Table 5
The EQAS performance of immunoassays in terms of bias%
Table 6
Comparison of sigma score of clinical chemistry assays based on CLIA '88 and 2024
Table 0
Table 7
Sigma score of immunoassays based on selected TEa
Assays
Twenty-six clinical chemistry assays based on microslide technology, two clinical chemistry assays based on microtip technology (direct LDL Cholesterol and direct TIBC), and 20 immunological assays based on microwell technology were included in the study. These analyte’s performance were monitored using sigma metrics and analyzed on a monthly basis for a period of twelve months. All the assay reagents were obtained from QuidelOrtho, USA, stored, and used as per the manufacturer’s instructions.
Controls
Bio-Rad (California) assayed controls Level 1, Level 2, and Level 3 as third-party control were used as internal quality control (IQC) for regular monitoring of the assay performance on a daily basis. The frequency of the control run was in accordance with the guidelines formulated by the National Accreditation Board for Testing and Calibration Laboratories (NABL). For clinical chemistry assays, Bio-Rad Lyphocheck assayed chemistry controls (Level 1 and 2), and for immunoassays, Bio-Rad Lyphocheck Immunoassay Controls (Level 1, 2 and 3) were used as per the manufacturer’s instructions. Control storage, reconstitution, and subsequent use were in accordance with the manufacturer’s specifications. Unity Real Time software was used for Quality Control (QC) management. QC outliers were determined based on Westgard rule violations as per laboratory policy. Root cause analysis was carried out for outliers with appropriate corrective and preventive actions and documentation of the same. Further, the clinical laboratory participates in the Bio-Rad External Quality Assessment Services (EQAS) program, which includes monthly analysis and peer group comparison of data received in the form of a detailed report from Bio-Rad. Both IQC and EQAS results were included in this study.
QC run frequency
Chemistry: Every 8 hours, 2-level controls are processed in the morning, followed by alternating single levels every 8 hours. The frequency of QC runs in a day is 3.
Immunoassay: Every 8 hours, 2-level controls are processed in the morning for all immunoassay parameters, followed by a single level of control processed for Thyroid profile hormones and Beta hCG in the evening. For remaining immunoassay parameters, a single level of QC is processed in the evening time, only on an ad-hoc basis, as and when a test request is received.
Statistical Analysis
Accuracy and precision in the performance of each assay were calculated as both % of Bias and % of coefficient of variation (CV). Accuracy in terms of Bias% was calculated based on the EQAS report by using the following formula:
Bias% = (EQAS result reported by laboratory – Peer group mean value)/(Peer group mean value) * 100
The Bias% was calculated for 12 months from January to December 2023 based on the performance of the analytes in the EQAS cycle, and the mean Bias% was calculated. Precision in terms of CV% was calculated based on the IQC results obtained for 2 (or 3) levels of controls. Monthly data in terms of mean (X), standard deviation (SD), and CV% for all analytes across each level were recorded, and the average CV% for all the levels of controls was calculated. The global specifications for the performance of various analytes in clinical laboratories are defined in terms of total allowable error limits (TEa). For the application of sigma metrics, the total allowable error goal was sourced from Clinical Laboratories Improvement Act (CLIA) Acceptance limits for Proficiency testing 2024 along with old AP (CLIA 1988).7 In CLIA 1988, TEa was available only for 22 out of 28 clinical chemistry assays and 6 out of 20 immunoassays evaluated in this study. In CLIA 2024, when TEa was not available for 3 assays (Lipase, CA19.9 and Free T3), it was sourced from RCPA allowable limits of performance for Biochemistry for sigma score calculation.8 When compared to CLIA 1988, the total allowable error was made stringent in CLIA 2024. Hence, the sigma score analysis was made and compared based on the total allowable error limit given in both CLIA 1988 and CLIA 2024. For the immunoassays, since the TEa is available only for 6 out of 20 parameters in CLIA 1988 guidelines, the sigma score analysis was done only using TEa sourced from CLIA 2024.
With the total allowable error (TEa), mean Bias% and CV%, the sigma score was measured for each analyte using the formula:
Sigma = TEa – Bias%/CV%
The performance of analytes in terms of sigma score for the period of 12 months was calculated and tabulated.
The assays performing at ≥ 6 sigma level were considered excellent. The analytes performing at <5.9 to ≥3 sigma level were considered as good. The analytes performing at < 3 sigma were considered poor, which needs stringent monitoring and improvement. The assays performing at a sigma score below 6 provide the scope for continuous quality improvement. The root cause for the assay performance below 6 sigma may be either imprecision or inaccuracy, or both. The other reason for an assay performing below 6 sigma may be due to the stringent TEa limit. All the assays performing at < 6 sigma were analyzed using a quality tool called the Quality Goal Index (QGI). The QGI was calculated using the formula.
QGI = Bias/1.5 * %CV
If the QGI is below 0.8 for an assay, the root cause may be imprecision. If the QGI is above 1.2, the root cause may be inaccuracy. If the QGI is between 0.8–1.2, the root cause may be both inaccuracy and imprecision. Accordingly, the required corrective and/or preventive action was implemented.
Results
A total of 48 assays (26 assays based on VITROS MicroSlide technology, 2 assays based on VITROS MicroTip Technology, and 20 assays based on VITROS MicroWell Technology) were analyzed for their performance in VITROS XT 7600 integrated system based on sigma metrics (Table 1). The performance of assays in terms of precision (CV%) and accuracy (Bias%) in the observation period of 12 months are tabulated (Table 2, Table 3, Table 4, Table 5). The obtained sigma scores at both level 1 and level 2 controls and, where available, level 3 controls based on the TEa sourced are tabulated (Table 6, Table 8).
Based on the sigma score obtained, the assay performance was classified into 4 categories viz., world-class performance (with > 6 sigma), good performance (sigma score 5–6), optimum performance (sigma score 3–5), and performance which needs more focus and improvement (Sigma score < 3.0). The number of analytes with performance as per the above-mentioned classification based on CLIA 1988 and CLIA 2024 is represented in the form of a pie chart (Figure 1 a, b).
With the TEa source from CLIA 1988, out of 22 clinical chemistry assays, 14 assays (64%) showed world-class performance with a sigma score of > 6.0; 6 assays (27%) showed good to moderate performance with a sigma score between 3.0 and 5.9; 2 parameters (9%) showed performance with sigma score of < 3.0 which needs to be focused for quality improvement. With the TEa source from CLIA 2024, out of 28 assays, 13 assays (46%) showed world-class performance with a sigma score of > 6.0; 12 assays (43%) showed good to moderate performance, and 3 assays (11%) showed performance with the sigma score of <3.0. Sodium assay showed performance below 3.0 because of the very narrow TEa of around 3%.
With the TEa source from CLIA 2024 and RCPA, out of 20 immunoassays, 11 assays (55%) showed world-class performance with a sigma score of >6.0; 7 assays (35%) showed good to moderate performance with a sigma score between 3.0 to 5.9 and 2 assays (10%) showed performance with the sigma score of <3.0. CA 19.9 assay showed performance below 3.0 because of the narrow TEa sourced from RCPA. Overall, most of the assays showed excellent performance on the sigma scale.
The performance of assays with < 6.0 sigma score was analyzed using the QGI tool to identify the possible causes for the sigma score of < 6.0. Based on the data, it was observed that marginal improvement is required to improve the accuracy and/or precision of the assays to move most of the assays to the performance level of > 6.0 sigma.
Discussion
Sigma metrics is a quality management tool used in clinical laboratories to monitor the performance of the assays and to work towards continuous quality improvement in the process. It provides a quantitative measurement of the performance of each assay based on the allowable error limit specified and the scope for quality improvement of each assay as per the requirement.3
In our study, the majority of the parameters (about 68% of the Clinical Chemistry assays and 75% of Immunoassays) are performing well with a sigma score of above 5.0 with the TEa source from CLIA 2024. About 21% of the Clinical Chemistry assays and 15% of Immunoassays are performing satisfactorily, with a sigma score between 3.0 to 4.9. Only 3 assays, viz., Sodium, Iron, and Bicarbonate, showed performance with a sigma score of < 3 either with any one level of control or both. With the TEa source from CLIA 1988, about 86% of Clinical Chemistry assays are performing at a sigma score of > 5.0, and about 5% of the assays are performing satisfactorily with a sigma score between 3.0 to 4.9. The reason for this variation observed between CLIA 1988 and CLIA 2024 is due to the narrow Total allowable error limit for 16 out of 22 assays like Amylase (TEa is revised from 30% to 20%); Alkaline phosphatase (30% to 20%); Creatinine (15% to 10%); Glucose (10% to 8%), Potassium (+/- 0.5 to 0.3 mmol/L), etc.
The sodium assay showed a performance below 3.0 sigma because of the very narrow TEa of around 3% in both CLIA 1988 and CLIA 2024. Joshi and Patel (2022) reported that Sodium is likely to remain at low levels of sigma across methods/analyzers unless the guidelines revise the TEa to a higher level.9 Poor sigma performance of electrolytes especially Sodium is not unique to our study, even though we achieved the CV% around 0.8% and bias% around 1.6. Heerden et al., 2022, reported that tight quality specifications are expected to give low sigma results.10
Another assay which showed performance below 3.0 sigma score for level 2 control was bicarbonate. The root cause for this performance is due to imprecision observed with level 2 control. Bicarbonate is considered as less stable as it is present in equilibrium with carbon dioxide. If there is exposure to air or if the control sample is not properly sealed and stored, carbon dioxide can escape, leading to imprecision.11 So, it is crucial to follow proper sample handling and processing procedures. Chakravarty et al., 2017 reported the low sigma score of Bicarbonate assay in both level 1 and 2 controls because of imprecision observed due to volatile nature of analyte in the sample.12
In this study, Iron showed performance below 3.0 sigma score for both level 1 and level 2 controls. One of the possible reasons for this performance is the narrow TEa of 15% of CLIA 2024 and the higher CV% observed with level 2 control having a low value (< 70 ug/dL) of Iron. TEa of Desirable Biological Variation Database Specification for Iron is 31%. With this TEa of 31%, the level of performance is increased to Sigma score of 6.6 and 3.7 for level 1 and level 2 control. As per the QGI, the lower level of sigma score is due to imprecision observed in IQC performance. Iron can undergo changes in its chemical form and reactivity over time, especially in the presence of oxygen and other reactive substances in control fluids. This instability can lead to imprecision while measuring iron levels in control samples. It is recommended that precautions be followed while handling, processing, and storing control fluids to maintain the stability of iron in control samples.
In the immunoassay, all parameters showed excellent performance except for folate, which showed performance below 3.0 sigma score for both level 2 and level 3 control. The obtained sigma scores were 2.5 and 2.4. This is majorly due to the slightly higher imprecision observed with the controls. The obtained CV% was around 8%, which is comparable with the manufacturer’s performance characteristics of folate assay. Owen and Roberts, 2003, while doing a comparative study between 5 different folate assays, reported that the imprecision of all methods was acceptable with coefficients of variation of less than 10%, even at low folate concentrations, with the exception of the Elecsys 2010 method which had an overall imprecision of 16% at the lowest concentration of folate examined.13 VITROS 19.9 assay showed the performance with a sigma score between 2.4 to 2.7 for all 3 level controls with the TEa of 15% sourced from RCPA. With TEa from Desirable Biological Variation Database Specification (23%), the level of performance was increased above 4.0 sigma. Marginal improvement in terms of imprecision in IQC, as well as Bias in EQAS performance, helps to enhance the sigma score of folate.
Our study showed that both clinical chemistry and immunoassays performed in the VITROS XT 7600 Integrated system could produce results with both accuracy and precision for the majority of the assays. Miller and Gammie, 2024 developed an algorithm to extract the QC data from more than 100 VITROS analyzers and derived the sigma metric for 115 analytes.14 In this analysis, 79 out of 115 (68.7%) of the assays assessed achieved >6 sigma, and 98 out of 115 (85.2%) achieved > 5 sigma. Sigma metrics is a great quality tool for accessing the analytical performance of all assays in the clinical laboratory, but there are some limitations for a few assays like sodium. For those assays, it is also important to monitor the performance of assays in terms of precision in IQC and accuracy in EQAS, in comparison with the manufacturer’s claim. Sigma score assessment may also help the laboratories to select the IQC rules for those assays that are performing with high sigma values to reduce false rejections, and stringent rules may be followed for those assays that are performing with low sigma values based on the clinical needs.15
Conclusion
This study showed that about 90% of clinical chemistry and immunoassays produced results with excellent performance in sigma scale. The analysis helped to identify the root causes for the low performance of few assays with sigma score below 3.0 and performance improvement steps undertaken.
