The Reliability of Determining the Presence of Surgical Site Infection Based on Retrospective Chart Review

Purpose Surgical site infection (SSI) can be a challenging complication after hand surgery. Retrospective studies often rely on chart review to determine presence of an SSI. The purpose of this study was to assess reliability of Centers for Disease Control and Prevention (CDC) criteria for determining an SSI as applied to a chart review. We hypothesized that interobserver and intraobserver reliability for determining an SSI using these criteria while reviewing medical record documentation would be none to minimal (K < 0.39) based on an interpretation of Cohen's K statistics. Methods We created and used a database of 782 patients, 48 of whom received antibiotics within 3 months of a surgical procedure of the hand. Three fellowship-trained orthopedic hand surgeons then evaluated the charts of those 48 patients, in which each reviewer determined whether an SSI was present or absent based on CDC criteria provided to the reviewers. Patients' charts were then reassessed 1 month later by the same reviewers. Kappa statistics were calculated for each round of assessment and averaged to determine intraobserver and interobserver reliability. Results Overall K values were 0.22 (standard error, 0.13), indicating fair reliability. Average K value between reviewers was 0.26 (standard error, 0.13. On average, intrarater reliability was 68.7%. Conclusions We found poor interobserver and intraobserver reliability when using CDC criteria to determine whether a patient had an SSI, based on chart review. Clinical relevance Better criteria or documentation may be needed in patients with an infection after hand surgery. Retrospective chart reviews to assess infection may be unreliable. Copyright (c) 2020 by the American Society for Surgery of the Hand. All rights reserved.