Measuring the Intensity of Resident Supervision in the Department of Veterans Affairs: The Resident Supervision Index

Purpose To develop a survey instrument designed to quantify supervision by attending physicians in nonprocedural care and to assess the instrument's feasibility and reliability. Method In 2008, the Department of Veterans Affairs (VA) Office of Academic Affiliations convened an expert panel to adopt a working definition of attending supervision in nonprocedural patient care and to construct a survey to quantify it. Feasibility was field-tested on residents and their supervising attending physicians at primary care internal medicine clinics at the VA Loma Linda Healthcare System in their encounters with randomly selected outpatients diagnosed with either major depressive disorder or diabetes. The authors assessed both interrater concurrent reliability and test-retest reliability. Results The expert panel adopted the VA's definition of resident supervision and developed the Resident Supervision Index (RSI) to measure supervision in terms of residents' case understanding, attending physicians' contributions to patient care through feedback to the resident, and attending physicians' time (minutes). The RSI was field-tested on 60 residents and 37 attending physicians for 148 supervision episodes from 143 patient encounters. Consent rates were 94% for residents and 97% for attending physicians; test-retest reliability intraclass correlations (ICCs) were 0.93 and 0.88, respectively. Concurrent reliability between residents' and attending physicians' reported time was an ICC of 0.69. Conclusions The RSI is a feasible and reliable measure of resident supervision that is intended for research studies in graduate medical education focusing on education outcomes, as well as studies assessing quality of care, patient health outcomes, care costs, and clinical workload.