Review
Anesthesiology
Jennifer M. Weller, Ties Coomber, Yan Chen, Damian J. Castanelli
Summary: Specialist training bodies have been developing innovative methods of workplace assessment tools to meet the requirements of competency-based medical education. A recent review identified 30 innovative WBA tools, categorised across seven dimensions, which highlighted the interdependencies and trade-offs in their design. This study emphasizes the importance of deliberate consideration of tool characteristics to suit the local context.
BRITISH JOURNAL OF ANAESTHESIA
(2021)
Article
Anesthesiology
Jennifer M. Weller, Ties Coomber, Yan Chen, Damian Castanelli
Summary: This study aims to understand the current use and desired information about trainees by decision-makers and the methods of obtaining this information. Through interviews with 21 training supervisors in Australia and New Zealand, four interrelated themes were identified, including the local context of training sites, the impact of the information gathering and documentation process, when to pay attention to possible deviations from performance norms, the effects of transparent and covert information gathering on assessment outcomes, and the distribution of assessment control between trainees and supervisors. Based on these themes, a set of design principles for future workplace assessment is proposed.
BRITISH JOURNAL OF ANAESTHESIA
(2022)
Article
Education, Scientific Disciplines
Lauren B. Phinney, Angelina Fluet, Bridget C. O'Brien, Lee Seligman, Karen E. Hauer
Summary: This study explores the changes and tensions in a clerkship feedback activity system with the implementation of a workplace-based assessment (WBA) tool. Using cultural historical activity theory (CHAT), the study identifies the elements and tensions within the feedback activity system. The results show that the design of the tool and student autonomy in documenting feedback are important factors for successful WBA implementation.
Article
Education, Scientific Disciplines
Damian J. Castanelli, Jennifer M. Weller, Elizabeth Molloy, Margaret Bearman
Summary: In workplace-based assessment, the behavior of trainees is influenced by the invitation and trustworthiness of supervisors. Supervisors' trust and investment in trainee development can encourage trainees to authentically engage and take risks, creating assessment for learning.
Article
Obstetrics & Gynecology
Evan Tannenbaum, Melissa Walker, Heather Sullivan, Ella Huszti, Michele Farrugia, Mara Sobel
Summary: The study found that the assessment of residents in gynecologic surgery is influenced by the assessor's knowledge of the resident's past performance, with assessors tending to give higher scores to residents with favorable reputations.
OBSTETRICS AND GYNECOLOGY
(2021)
Article
Surgery
Shady Elmaraghi, Keon Min Park, Nikdokht Rashidian, Ava Yap, Kara Faktor, Doruk Ozgediz, Eric Borgstein, Abebe Bekele, Adnan Alseidi, Girma Tefera
Summary: This study examined the needs and limitations of surgical educators in Eastern, Central, and Southern Africa, and identified priorities for educational development. The survey revealed that faculty development and skills and simulation training were the areas with the greatest perceived need for improvement. Barriers to development were commonly cited as clinical responsibilities, cost, and technical support.
JOURNAL OF THE AMERICAN COLLEGE OF SURGEONS
(2023)
Review
Education & Educational Research
Lara Teheux, Ester H. A. J. Coolen, Jos M. T. Draaisma, Marieke de Visser, Nynke D. Scherpbier-de Haan, Wietske Kuijer-Siebelink, Janielle A. E. M. van der Velden
Summary: Intraprofessional learning in postgraduate medical education shows great potential for learning, influenced by factors such as sociocultural environment, learners and learning context. Awareness of learning opportunities and explicit reflection are crucial, while challenges include complexity of care, professional stereotyping, and negative perceptions. Emphasized learning outcomes include collaborative decision making, communication, and teamwork.
BMC MEDICAL EDUCATION
(2021)
Review
Education, Scientific Disciplines
Hannah L. Anderson, Joshua Kurtz, Daniel C. West
Summary: This study aims to fill the knowledge gap on how to effectively implement Workplace-based assessment (WBA) to support competency-based medical education (CBME). Analysis reveals that many WBA tools and programs have been implemented, with common barriers across fields and specialties; theoretical perspectives emphasize the necessity for data-driven implementation strategies; user perceptions of WBA vary depending on implementation factors; technology solutions could offer useful tools to support WBA; and many areas for future research and innovation remain.
Article
Education, Scientific Disciplines
Jessica V. Rich, Ulemu Luhanga, Sue Fostaty Young, Natalie Wagner, J. Damon Dagnone, Sue Chamberlain, Laura A. McEwen
Summary: Developing comprehensive practice guidelines for programmatic assessment in competency-based medical education is crucial for guiding assessment theory into practice. The CBME Programmatic Assessment Practice Guidelines (PA Guidelines) were developed using a multiphase, multimethod approach and were found to be valuable for guiding the implementation of programmatic assessment by various stakeholders.
Review
Education, Scientific Disciplines
Roy Khalife, Manika Gupta, Carol Gonsalves, Yoon Soo Park, Janet Riddle, Ara Tekian, Tanya Horsley
Summary: This study examines the involvement of patients in the assessment of postgraduate medical learners through a scoping review. The findings indicate that patients are willing to participate in assessments, but barriers such as language and reading comprehension challenges, as well as a lack of resources in training programs, exist. Patients may participate as a standalone group or as part of multi-source feedback, and they generally provide high ratings but comment on professional behaviors and communication skills.
Article
Multidisciplinary Sciences
Anil Kamat, Condell Eastmond, Yuanyuan Gao, Arun Nemani, Erim Yanik, Lora Cavuoto, Matthew Hackett, Jack Norfleet, Steven Schwaitzberg, Suvranu De, Xavier Intes
Summary: This study presents a fNIRS dataset acquired on mobile subjects performing FLS tasks in a laboratory environment. Relevant information and FLS scores are provided to facilitate the use of this open-access dataset.
Article
Green & Sustainable Science & Technology
Sara Siddique, Ali Ahsan, Neda Azizi, Omid Haass
Summary: This study introduces a new approach for developing the competence of socio-technical aspect by focusing on personal competence in the Project Management (PM) curriculum. The research identifies communication skills as a key competency for PM students and provides a comprehensive training plan and evaluation tools to address this need. The findings of this research have significant implications for building PM competence and improving the curriculum.
Article
Education, Scientific Disciplines
Trevor J. G. Robinson, Natalie Wagner, Adam Szulewski, Nancy Dudek, Warren J. Cheung, Andrew K. Hall
Summary: This study aimed to explore factors influencing faculty's rating choices immediately following WBA assessments and revealed faculty's experiences using WBAs with entrustment anchors. The study found that faculty most commonly identified the amount of guidance the trainee required as the influencing factor in their ratings.
Article
Education, Scientific Disciplines
Damian J. Castanelli, Jennifer M. Weller, Elizabeth Molloy, Margaret Bearman
Summary: The study highlights the importance of trust between trainees and their assessors in workplace-based assessments, as well as the significant influence of the assessor's power on the trainee's learning opportunities and progress. Trust was granted based on how trainees anticipated supervisors would use their power, and trainees were more open to learning when supervisors used their power for their benefit.
Article
Education, Scientific Disciplines
Thurarshen Jeyalingam, Catharine M. Walsh, Walter Tavares, Maria Mylopoulos, Kathryn Hodwitz, Louis W. C. Liu, Steven J. Heitman, Ryan Brydges
Summary: This study explored rater entrustment decision making in workplace and simulation settings for entrustable professional activities (EPAs) in medical education. The results showed that there were variations in how raters understood and defined entrustment, which led to differences in their entrustment decisions. The study suggests that these variations should be considered when integrating EPA assessments from different settings and emphasizes the importance of clear definitions and purposes in workplace and simulation-based assessments.
Article
Education, Scientific Disciplines
Olle ten Cate, Carol Carraccio, Arvin Damodaran, Wade Gofton, Stanley J. Hamstra, Danielle E. Hart, Denyse Richardson, Shelley Ross, Karen Schultz, Eric J. Warm, Alison J. Whelan, Daniel J. Schumacher
Summary: The Miller's pyramid divides assessment in medical education into different levels, emphasizing innovation and importance in workplace assessment. With the rise of competency-based medical education, assessment of learners' competence in clinical workplaces has become a focus.
Article
Obstetrics & Gynecology
Jessica L. Bienstock, Pooja Shivraj, Kenji Yamazaki, AnnaMarie Connolly, George Wendel, Stanley J. Hamstra, Eric S. Holmboe
Summary: The study evaluated the correlation between the Accreditation Council for Graduate Medical Education Obstetrics and Gynecology Milestones and residency program graduates' performance on the American Board of Obstetrics and Gynecology qualifying examination. The results showed a substantial association between milestone ratings and candidates' examination scores, particularly in Medical Knowledge subcompetencies, indicating the importance of competency-based assessment in graduate medical education.
AMERICAN JOURNAL OF OBSTETRICS AND GYNECOLOGY
(2021)
Article
Surgery
Brigitte K. Smith, Abigail Luman, Kenji Yamazaki, Ara Tekian, Stanley J. Hamstra, Eric Holmboe, Erica L. Mitchell, Yoon Soo Park
Summary: The study found that a significant percentage of VSFs do not achieve the ACGME Vascular Surgery Milestone targets for graduation in the competencies of SBP and PBLI, indicating a need for improvement in curricula and assessment strategies across vascular surgery fellowship programs. Reported milestones levels at different time points are predictive of ultimate achievement upon graduation and should be used to provide targeted feedback and individualized learning plans for graduates to engage in personal and health care system improvement in unsupervised practice.
ANNALS OF VASCULAR SURGERY
(2021)
Article
Education, Scientific Disciplines
Kenji Yamazaki, Eric S. Holmboe, Stanley J. Hamstra
Summary: This study investigates the milestone data of obstetrics and gynecology residents and finds that the clinical competency committee ratings reflect the 6 general competency domains and further distinguish the patient care competency domain into three factors. The study provides validity evidence for milestone assessment in a single specialty and sheds light on the understanding of milestones by the clinical competency committees.
Article
Surgery
Brigitte K. Smith, Kenji Yamazaki, Ara Tekian, Eric Holmboe, Stanley J. Hamstra, Erica L. Mitchell, Yoon Soo Park
Summary: This study investigates the at-risk scores of semiannual Accreditation Council for Graduate Medical Education (ACGME) Milestone ratings for final achievement of competency targets in vascular surgical trainees. The study found that lower milestone ratings throughout training are associated with a higher risk of not achieving recommended performance targets upon graduation.
Article
Education, Scientific Disciplines
Warren J. Cheung, Andrew K. Hall, Alexandra Skutovich, Stacey Brzezina, Timothy R. Dalseg, Anna Oswald, Lara J. Cooke, Elaine Van Melle, Stanley J. Hamstra, Jason R. Frank
Summary: This study evaluated the readiness of Canadian postgraduate training programs to implement Competence by Design (CBD), a model of Competency-Based Medical Education (CBME). The majority of respondents were supportive of successful CBD implementation, but there were also challenges that need to be addressed for future improvement.
Article
Education, Scientific Disciplines
Martin V. Pusic, Robert J. Birnbaum, Brent Thoma, Stanley J. Hamstra, Rodrigo B. Cavalcanti, Eric J. Warm, Anna Janssen, Tim Shaw
Summary: The information systems supporting clinical care and health professions education have evolved separately, leading to a digital divide that hinders practitioners and organizations. To address this, we propose enhancing existing health information systems to intentionally facilitate learning. We present three frameworks for learning that can guide the evolution of healthcare information systems to better support education.
JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS
(2023)
Article
Surgery
Brigitte K. Smith, Stanley J. Hamstra, Kenji Yamazaki, Ara Tekian, Benjamin S. Brooke, Eric Holmboe, Erica L. Mitchell, Yoon Soo Park
Summary: This study aimed to investigate the association between educational competencies in vascular surgery and patient care outcomes. Through a series of surveys and discussions, the conceptual alignment between educational competencies and patient care outcomes across different clinical registries was determined. The study found that systems-based practice competencies had a high level of relevance to patient outcomes, while practice-based learning and improvement and professionalism competencies had lower relevance to patient outcomes.
JOURNAL OF VASCULAR SURGERY
(2022)
Article
Education, Scientific Disciplines
Brigitte K. Smith, Kenji Yamazaki, Abigail Luman, Ara Tekian, Eric Holmboe, Erica L. Mitchell, Yoon Soo Park, Stanley J. Hamstra
Summary: This study investigated the potential for generating early predictors of final Milestone ratings in professionalism and interpersonal and communication skills. It found that as many as 1 in 4 vascular surgery trainees did not achieve the ACGME Milestone targets in these competencies. Therefore, early assessment ratings can be used to predict achievement of competency targets at the time of graduation, allowing programs to address potential deficits early in training.
JOURNAL OF SURGICAL EDUCATION
(2023)
Article
Education, Scientific Disciplines
Daniel E. E. Kendrick, Angela E. E. Thelen, Xilin Chen, Tanvi Gupta, Kenji Yamazaki, Andrew E. E. Krumm, Hoda Bandeh-Ahmadi, Michael Clark, John Luckoscki, Zhaohui Fan, Greg M. M. Wnuk, Andrew M. M. Ryan, Bhramar Mukherjee, Stanley J. J. Hamstra, Justin B. B. Dimick, Eric S. S. Holmboe, Brian C. C. George
Summary: Accurate assessment of clinical performance is crucial for graduating residents. The study investigates the association between milestone competency ratings of U.S. general surgical residents and their patient outcomes in early career practice. The results showed no significant associations between milestone ratings and patient outcomes.
Article
Medicine, General & Internal
Misop Han, Stanley J. Hamstra, Sean O. Hogan, Eric Holmboe, Kelly Harris, Eric Wallen, Gerald Hickson, Kyla P. Terhune, Donald W. Brady, Bruce Trock, Kenji Yamazaki, Jessica L. Bienstock, Henry J. Domenico, William O. Cooper
Summary: This study investigated the association between resident Milestone ratings and posttraining patient complaints. The results showed that trainees with low Milestone ratings near the end of residency were at increased risk for patient complaints in their early posttraining independent physician practice. Therefore, evaluating trainee performance in graduate medical education training programs and providing support for those with lower ratings is crucial.
Article
Emergency Medicine
Simranjeet Singh, Warren J. J. Cheung, Sebastian Dewhirst, Timothy J. J. Wood, Jeffrey M. M. Landreville
Summary: Longitudinal coaching relationships did not influence the quality of EPA assessments.
AEM EDUCATION AND TRAINING
(2023)
Review
Education, Scientific Disciplines
Olle ten Cate, Natasha Khursigara-Slattery, Richard L. Cruess, Stanley J. Hamstra, Yvonne Steinert, Robert Sternszus
Summary: This article proposes a multilayered conceptualisation of medical competence, including a core layer of canonical knowledge and skill, context-dependent knowledge, skill, and attitude, and personalised competence. The assessment methods for each layer are different, with standardized tests for canonical knowledge and skills, merged multiple sources of information for context-dependent competence, and recognition and reward for personal excellence.
Article
Education, Scientific Disciplines
Sebastian Dewhirst, Timothy J. Wood, Warren J. Cheung, Jason R. Frank
Summary: Work-based assessments (WBAs) are commonly used to determine trainee progression. However, they often fail to differentiate between trainees of different abilities and have poor reliability. Entrustment-supervision scales may improve WBA performance, but there is limited research comparing them to traditional WBA tools.
Meeting Abstract
Surgery
Brigitte K. Smith, Kenji Yamazaki, Ara Tekian, Benjamin S. Brooke, Eric Holmboe, Stanley J. Hamstra, Erica L. Mitchell, Yoon Soo Park
JOURNAL OF VASCULAR SURGERY
(2022)