Failure of Current Laboratory Protocols to Detect Lot-to-Lot Reagent Differences: Findings and Possible Solutions

BACKGROUND: Maintaining consistency of results over time is a challenge in laboratory medicine. Lot-to-lot reagent changes are a major threat to consistency of results. METHODS: For the period October 2007 through July 2012, we reviewed lot validation data for each new lot of insulin-like growth factor 1 (IGF-1) reagents (Siemens Healthcare Diagnostics) at Mayo Clinic, Rochester, MN, and the University of Virginia, Charlottesville, VA. Analyses of discarded patient samples were used for comparison of lots. For the same period, we determined the distributions of reported patient results for each lot of reagents at the 2 institutions. RESULTS: Lot-to-lot validation studies identified no reagent lot as significantly different from the preceding lot. By contrast, significant lot-to-lot changes were seen in the means and medians of 105 668 reported patient IGF-I results during the period. The frequency of increased results increased nearly 2-fold to a high of 17%, without detectable changes in the underlying patient demographics. Retrospective statistical analysis indicated that lot-to-lot comparison protocols were underpowered and that validation studies for this assay required testing >100 samples to achieve 90% power to detect reagent lots that would significantly alter the distributions of patient results. CONCLUSIONS: The number of test samples required for adequate lot-to-lot validation protocols is high and may be prohibitively large, especially for low-volume or complex assays. Monitoring of the distributions of patient results has the potential to detect lot-to-lot inconsistencies relatively quickly. We recommend that manufacturers implement remote monitoring of patient results from analyzers in multiple institutions to allow rapid identification of between-lot result inconsistency. (c) 2013 American Association for Clinical Chemistry