Reliability of the assessment of preventable adverse drug events in daily clinical practice

Purpose To determine the reliability of the assessment of preventable adverse drug events (ADES) in daily practice and to explore the impact of the assessors' professional background and the case characteristics on reliability. Methods We used a combination of the simplified Yale algorithm and the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) scheme to assess on the one hand the causal relationship between medication errors (MEs) and adverse events in hospitalised patients and on the other hand the severity of the clinical consequence of MEs. Five pharmacists and five physicians applied this algorithm to 30 potential MEs. After individual assessment, the pharmacists reached consensus and so did the physicians. Outcome was both MEs' severity (ordinal scale, NCC MERP categories A-I) and the occurrence of preventable harm (binary outcome, NCC MERP categories A-D vs. E-I). Kappa statistics was used to assess agreement. Results The overall agreement on MEs' severity was fair for the pharmacists (kappa=0.34) as well as for the physicians (kappa=0.25). Overall agreement for the 10 raters was fair (kappa=0.25) as well as the agreement between both consensus outcomes (kappa=0.30). Agreement on the occurrence of preventable harm was higher, ranging from kappa=0.36 for the physicians through kappa=0.49 for the pharmacists. Overall agreement for the 10 raters was fair (kappa=0.36). The agreement between both consensus outcomes was moderate (kappa=0.47). None of the included case characteristics had a significant impact on agreement. Conclusions Individual assessment of preventable ADEs in real patients is difficult, possibly because of the difficult assessment of contextual information. Best approach seems to be a consensus method including both pharmacists and physicians. Copyright (c) 2008 John Wiley & Sons, Ltd.