期刊
AMERICAN JOURNAL OF SURGERY
卷 221, 期 2, 页码 369-375出版社
EXCERPTA MEDICA INC-ELSEVIER SCIENCE INC
DOI: 10.1016/j.amjsurg.2020.11.044
关键词
Surgery education; Natural language processing; Entrustable professional activities; Assessment; Feedback
类别
资金
- National Cancer Institute of the National Institutes of Health [T32CA090217]
Analyzing actual EPA assessment narrative comments using natural language processing can enhance understanding of resident entrustment in practice. Research indicates that machine learning algorithm, specifically LDA topic analysis, can identify topics relevant to EPA entrustment levels.
Background: Entrustable Professional Activities (EPAs) contain narrative 'entrustment roadmaps' designed to describe specific behaviors associated with different entrustment levels. However, these roadmaps were created using expert committee consensus, with little data available for guidance. Analysis of actual EPA assessment narrative comments using natural language processing may enhance our understanding of resident entrustment in actual practice. Methods: All text comments associated with EPA microassessments at a single institution were combined. EPA-entrustment level pairs (e.g. Gallbladder Disease-Level 1) were identified as documents. Latent Dirichlet Allocation (LDA), a common machine learning algorithm, was used to identify latent topics in the documents associated with a single EPA. These topics were then reviewed for interpretability by human raters. Results: Over 18 months, 1015 faculty EPA microassessments were collected from 64 faculty for 80 residents. LDA analysis identified topics that mapped 1:1 to EPA entrustment levels (Gammas >0.99). These LDA topics appeared to trend coherently with entrustment levels (words demonstrating high entrustment were consistently found in high entrustment topics, word demonstrating low entrustment were found in low entrustment topics). Conclusions: LDA is capable of identifying topics relevant to progressive surgical entrustment and autonomy in EPA comments. These topics provide insight into key behaviors that drive different level of resident autonomy and may allow for data-driven revision of EPA entrustment maps. (C) 2020 Elsevier Inc. All rights reserved.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据