Background: Manual validation of laboratory test results is time-consuming, creating a demand for expert systems to automate this process. We have started to set up the program "LabRespond", which covers five validation levels: administrative, technical, sample, patient, and clinical validation. We present the evaluation of a prototype of an automated patient validation system based on statistical methods, in contrast to the commercially available program "VALAB", a rule-based automated validation system.
Methods: In the present study, 163 willfully altered, erroneous test results out of 5421 were submitted for validation to LabRespond, VALAB, and to a group of clinical chemists (n = 9) who validated these test results manually. The test results rejected by three or more clinical chemists (n = 281) served as a secondary reference standard.
Results: The error recovery rates of clinical chemists ranged from 23.9% to 71.2%. The recovery rates of LabRespond and VALAB were 77.9% and 71.8%, respectively (difference not significant). The false-positive rates were 82.7% for LabRespond, 83.6% for VALAB, and 27.8-86.7% for clinical chemists. Using the consensus of three or more clinical chemists as the secondary reference standard, we found error recovery rates of 64.8% for LabRespond and 72.2% for VALAB (P = 0.06). Compared with VALAB, LabRespond detected more (P = 0.003) erroneous test results of the type that were changed from abnormal to normal.
Conclusions: The statistical plausibility check used by LabRespond offers a promising automated validation method with a higher error recovery rate than the clinical chemists participating in this study, and a performance comparable to VALAB.