Introduction
The modern clinical laboratory is an extremely demanding field that requires to produce faster results without compromising on the quality. Laboratory systems are complex involving multiple procedural steps and many technical personnel with variable training and handling abilities. More than 70% of clinical decisions rely on test results and recommendations thereby increasing the need for high precision and throughput.1, 2
Quality control (QC) is the core aspect of good laboratories that assures that the quality of results critical for clinical diagnosis and patient care are performed with utmost precision with no window of error. A strong quality management model is mandatory so that the lab work is performed efficiently and each stage in its workflow operates without mistakes.1, 3 Various practices are adopted by clinical laboratories to maintain quality; these are monitoring Levey Jenning (LJ) graphs, following Westgard rules, and recording coefficient of variation (CV%) for internal quality control purposes. Medical laboratory technicians are often trained to test and rerun controls till it reaches acceptable limits, only after which patient samples can run. To further assure quality, external quality assurance (EQA) programs are established and Z score or Standard deviation index (SDI) is calculated. These tools allow an estimation of precision by minimizing random errors and ensures accuracy by reducing bias respectively. The backbone of a good laboratory thus rests on the quality control program adopted by the laboratories.4
Although these tools play an important role, an exact quantitation of errors is often difficult. Counting errors becomes a subjective phenomenon which leads to difficulties in providing a correct and objective assessment of the analytical performance. As an answer to this dilemma, an assessment method based on quantitative Sigma metrics is often used. The Six sigma model is like a bull's eye graph, which graphically displays the degree to which any result deviates from its target. Sigma (σ) is the mathematical symbol of standard deviation (SD). Motorola Company introduced it as part of their quality improvement; on seeing remarkable success, many companies adopted six sigma principles as their operational motto. This system's main advantages are that it helps reduce cost, prevent errors, and detects variability in the system.5, 6
The six sigma model advocates five steps in contrast to the traditional total quality management (TQM) model. The five steps are define, measure, analyze, improve and control (DMAIC). In contrast to the TQM model, an extra step of 'Control' guards against errors returning to the total process and this is a crucial step. "In sigma metrics, errors identified are quantified as percentage errors or DPM (defects per million). 1 sigma (σ) represents 6,90,000 errors/million reports, 2 sigma represents 3,08,000 errors/million reports, 3 sigma represents 66,800 errors/million reports, 4 sigma represents 6,210 errors/million reports, 5 sigma corresponds to 230 errors/million reports and 6 sigma represents 3.4 errors/million reports."7, 8, 9 Therefore any process with greater than six sigma, indicates very low variability and defect rate. Based on the sigma obtained the process can be divided into the following categories:10
>6: world class performance
5-6: Excellent
4-5: Good
3-4: Acceptable
2-3: Poor
<2: Unacceptable
A Six Sigma classification in the clinical laboratory can result in fewer controls and fewer rates of false rejections for methods with a sigma metric of 5 or better. When a larger number of analytes or methods perform at higher than 5 sigma, the cost for controls, reagents and other supplies required to monitor these methods correspondingly decrease. The first study utilizing sigma metrics in the clinical lab was published by Nevalainen et al., in the year 2000 and since then many similar studies have been done throughout the world.8 The studies utilizing Sigma metrics to gauge laboratory performance is of limited number in the Middle East region and hence this study was undertaken.
The aim of this study is to calculate sigma metrics of various biochemistry analytes and compare analyzer performance with other similar studies in the Middle East.
Materials and Methods
This is a retrospective study for a period of 12 months from January to December 2020 conducted at Aster Medical Centre clinical laboratory, Doha, Qatar using biochemistry analyzer ERBA-XL 200. Internal quality data of 16 analytes including, albumin (ALB), alanine amino transferase (SGPT), aspartate amino transferase (SGOT), alkaline phosphatase (ALKP), bilirubin total (BIL T), bilirubin direct (BIL D), calcium (CAL), cholesterol (CHOL), creatinine (CREAT), gamma glutamyl transferase (GGT), glucose (GLUC), high density lipoprotein (HDL), triglyceride (TG), total protein (PROT), uric acid (UA) and urea was extracted.
The control materials used were Erba Norm and Erba Path (normal and high levels respectively) supplied by the manufacturer (Erba Lachema s.r.o, Brno, CZ). These controls are routinely assessed once a day before processing patient samples. The same lot of quality control materials was used for this study and instrument calibration done according to manufacturer guidelines.
Coefficient of variation (CV%)
For each level of control, the monthly CV% was calculated from the mean and standard deviation (SD) of internal quality control data which is automated by the analyzer and then average was taken. The CV% was calculated as follows:
CV% = SD/ mean x 100
Bias%
The laboratory uses the Randox international quality assessment scheme (RIQAS) where, one sample of varying concentration is analyzed every month and the subsequent bias% was used. The Bias% was calculated as follows:
Bias% = mean of all laboratories using same instrument and method-lab mean/mean of all laboratories using same instrument and method×100.
Results
Table 1 and Table 2 shows average CV% for all parameters in Level I and Level II while Table 3 and 4 shows bias and sigma calculation. In this study, the CV% was found to be the lowest for albumin (Level 1- 1.35%) and SGOT (Level II-1.2%) and the highest for urea (Level 1 and 2). Furthermore, the minimum average bias was observed for albumin (1.2%) while maximum bias was observed for bilirubin total (10%).
Table 1
CV% of all 16 analytes during the study period for level 1
Out of the 16 analytes studied at two levels of concentration, it was found that five analytes in level 1 and eight analytes in level 2 had greater than 6 sigma performance indicating world class quality. Moreover, many of the parameters showed >3 sigma performance which is considered acceptable performance. The problem analytes having <3 sigma was identified as Urea (both levels) and GGT (level 1). (See Figure 1 and Figure 2).
Table 2
CV% of all 16 analytes during the study period for level 2
Table 3
Bias % and sigma metrics for level 1 QC
Table 4
Bias % and sigma metrics for level 2 QC
Discussion
Sigma metrics is an improvement method which concentrates on reducing variability in process outputs. Furthermore, it is an excellent tool to predict and compare assay and instrument quality and is a pointer to the tests that require minimal quality control rules to monitor the performance of the method. Based on the sigma values obtained the QC can be tailored as follows: 10, 8
>6σ (Excellent performance): IQC can be run once per day with one level (alternating levels) and follow 13.5s rule.
4σ–6σ (Suited to purpose): IQC can be run once per day with two levels per day and follow single IQC rule.
3σ–4σ (Poor performers): IQC can be run twice per day with two levels of IQC per day and use multi-rule system.
<3σ (Problematic): IQC should be run three times per day with three levels; consider testing in duplicate and use maximum IQC rules.
As is evident from this classification, analytes that display >6 sigma require very minimal QC rules to monitor the method performance. If the sigma is <3 or shows a wide variation between two levels, a close monitoring of the method with use of multiple QC rules or even a change in method is mandated.11 In our study, Urea (both levels) and GGT (level 1) was identified as problem analytes with <3 sigma metrics. Our study correlates well with similar studies from Asia and Middle East. In these studies, urea was also found to be <3 sigma while ALKP was found to have >6 sigma12, 13, 14, 15, 16, 17, 18, 19, 9 (Table 5)
Table 5
Comparision of present study with similar studies in the Middle East and Asia
A recent study in India by Vijatha et al., shows excellent correlation with the present study.9 Our study also showed good correlation with a similar study by Nanda et al where >6 sigma was observed for uric acid, total bilirubin, SGOT, SGPT, TG and ALP. In contrast, albumin and cholesterol had <3 sigma. In our study, these parameters showed better performance.20 Similar studies in India by Adiga US et al., using Erba XL-640 showed >6 sigma for HDL, ALKP, UA, TG and Alb, whereas <3 sigma was obtained for SGPT and SGOT. This contrasts with our study as these parameters showed better sigma metric in our study.21 The discrepancies observed in sigma metrics in various studies can be attributed to various factors such as difference in methods, reagents, IQC material, bias calculations and varying EQAS providers. Creatinine showed a wide variation between L1 and L2 with L1 performing at 4 sigma, whereas L2 showed >6 sigma.
The parameters showing wide differences in sigma results at different levels of QC should be carefully and meticulously evaluated. A thorough root cause analysis and troubleshooting should be conducted for those analytes. A change in QC techniques, including run number and strictly abiding by Westgard multirules should be performed, and results are to be re-analysed.21
Another study analyzed the reason for the lower sigma in analytes using another quality parameter known as Quality goal index ratio (QGI). A QGI value less than 0.8 (QGI < 0.8) is an indicator that the precision is affected, whereas if the result is greater than 1.2 (QGI > 1.2), it points towards improving accuracy. A QGI value falling between 0.8 and 1.2 (0.8 ≤ QGI ≤ 1.2) means that both the precision and accuracy of the corresponding analyte is affected and has to be simultaneously corrected after thorough evaluation.11
El Sharkawy et al., in 2018, proposed a harmonised protocol for sigma calculation and highlighted the importance of selecting TEa goals.14 Sigma metric calculation changes according to the chosen TEa goal and each lab should have a selection criterion for choosing the same. The lab world is yet to reach a consensus regarding the most ideal quality goal to be used, and this is the biggest challenge of using sigma metrics. A false estimation of sigma metric leads to overwork for the laboratory personnel and error in patient results.22 However, in the evolution of lab processes, sigma metric analysis is considered a revolutionary quality assessment tool. The old 'one size fits all model of quality management is considered outdated and incapable of meeting the ever-changing cost and efficiency demands of the modern lab.
Few limitations of this study include the use of manufacturer supplied controls for calculation of precision instead of third-party controls. This was due to financial constraints. Another limitation is the lack of a pilot study using the new proposed IQC frequency demonstrating process improvement in comparison to the existing one.
Conclusion
To the best of our knowledge, this is possibly one of the first studies in Qatar to gauge analytical clinical laboratory performance using six sigma metrics. Before the sigma study, our laboratory was using two level IQC and Westgard multirule blanket approach for all analytes. Based on the study findings, we conclude that the six sigma metrics can be used to customize IQC frequency for effective and improved quality control.
