4.6 Review

Systematic review automation tools improve efficiency but lack of knowledge impedes their adoption: a survey

Journal

JOURNAL OF CLINICAL EPIDEMIOLOGY
Volume 138, Issue -, Pages 80-94

Publisher

ELSEVIER SCIENCE INC
DOI: 10.1016/j.jclinepi.2021.06.030

Keywords

Automation; Automation tools; Systematic review; Systematic review automation; Health technology assessment; Clinical practice guideline

Funding

  1. NHMRC [APP1195676]

Ask authors/readers for more resources

The survey found that the majority of respondents have used systematic review automation tools, primarily during the screening stage. They believed these tools saved time and increased accuracy, but lack of knowledge was identified as the main barrier to tool adoption. Respondents suggested the development of new tools for the searching and data extraction stages.
Objective: We investigated systematic review automation tool use by systematic reviewers, health technology assessors and clinical guideline developerst. Study design and setting: An online, 16-question survey was distributed across several evidence synthesis, health technology assessment and guideline development organizations. We asked the respondents what tools they use and abandon, how often and when do they use the tools, their perceived time savings and accuracy, and desired new tools. Descriptive statistics were used to report the results. Results: A total of 253 respondents completed the survey; 89% have used systematic review automation tools - most frequently whilst screening (79%). Respondents' top 3 tools included: Covidence (45%), RevMan (35%), Rayyan and GRADEPro (both 22%); most commonly abandoned were Rayyan (19%), Covidence (15%), DistillerSR (14%) and RevMan (13%). Tools saved time (80%) and increased accuracy (54%). Respondents taught themselves to how to use the tools (72%); lack of knowledge was the most frequent barrier to tool adoption (51%). New tool development was suggested for the searching and data extraction stages. Conclusion: Automation tools will likely have an increasingly important role in high-quality and timely reviews. Further work is required in training and dissemination of automation tools and ensuring they meet the desirable features of those conducting systematic reviews. (c) 2021 Elsevier Inc. All rights reserved.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.6
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Editorial Material Medicine, General & Internal

Likelihood ratio interpretation of the relative risk

Suhail A. R. Doi, Polychronis Kostoulas, Paul Glasziou

BMJ EVIDENCE-BASED MEDICINE (2023)

Letter Medicine, General & Internal

Effect of an Individualized Audit and Feedback Intervention on Rates of Musculoskeletal Diagnostic Imaging Requests Reply

Denise A. O'Connor, Paul Glasziou, Rachelle Buchbinder

JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION (2023)

Article Mathematical & Computational Biology

How should we handle predatory journals in evidence synthesis? A descriptive survey-based cross-sectional study of evidence synthesis experts

Timothy H. Barker, Danielle Pollock, Jennifer C. Stone, Miloslav Klugar, Anna M. Scott, Cindy Stern, Rick Wiechula, Larissa Shamseer, Edoardo Aromataris, Amanda Ross-White, Zachary Munn

Summary: Synthesizers of evidence may encounter studies from predatory journals during the evidence synthesis process. This study explores the attitudes and opinions of evidence synthesis experts towards predatory journals. A survey found that most experts agreed with the definition of predatory journals but had hesitations about including studies from these journals in evidence synthesis projects.

RESEARCH SYNTHESIS METHODS (2023)

Article Health Care Sciences & Services

Nondrug interventions for reducing SARS-CoV-2 transmission are frequently incompletely reported

Sharon Sanders, Elizabeth Gibson, Paul Glasziou, Tammy Hoffmann

Summary: The study aims to investigate the completeness of reporting of behavioral, environmental, social, and system interventions (BESSI) for reducing SARS-CoV-2 transmission in randomized trials. The researchers assessed the completeness of reporting using the TIDieR checklist and contacted investigators for missing intervention details. The results show that incomplete reporting of BESSI is a significant problem, leading to wasted research resources and a lack of essential information for implementing interventions.

JOURNAL OF CLINICAL EPIDEMIOLOGY (2023)

Review Health Care Sciences & Services

Misdiagnosis of Uncomplicated Cellulitis: a Systematic Review and Meta-analysis

Rachael Nightingale, Krishan Yadav, Laura Hamill, Paul Glasziou, Anna Mae Scott, Justin Clark, Gerben Keijzers

Summary: Background Cellulitis is often misdiagnosed due to the lack of gold standard diagnostic criteria. This review analyzed the proportion of misdiagnosis and identified the common alternative diagnoses in primary or unscheduled care settings.

JOURNAL OF GENERAL INTERNAL MEDICINE (2023)

Review Multidisciplinary Sciences

Amitriptyline's anticholinergic adverse drug reactions-A systematic multiple-indication review and meta-analysis

Maria-Sophie Brueckle, Elizabeth T. Thomas, Svenja Elisabeth Seide, Maximilian Pilz, Ana I. Gonzalez-Gonzalez, Truc Sophia Dinh, Ferdinand M. Gerlach, Sebastian Harder, Paul P. Glasziou, Christiane Muth

Summary: This study aimed to explore and quantify the (anticholinergic) adverse drug reactions (ADR) in patients taking amitriptyline vs. placebo in randomized controlled trials (RCTs). The results showed that amitriptyline had a higher frequency of anticholinergic ADRs compared to placebo, with dry mouth, drowsiness, and fatigue being the most frequently reported ADRs. However, non-anticholinergic ADRs were similar between amitriptyline and placebo. Future studies should focus on older patients who are more susceptible to anticholinergic ADRs.

PLOS ONE (2023)

Article Multidisciplinary Sciences

Which clinical research questions are the most important? Development and preliminary validation of the Australia & New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network Research Question Importance Tool (ANZMUSC-RQIT)

William Taylor, Robin Willink, Denise A. O'Connor, Vinay L. Patel, Allison Bourne, Ian Harris, Samuel Whittle, Bethan S. Richards, Ornella G. Clavisi, Sally Green, Rana Hinman, Chris Maher, Ainslie E. Cahill, Annie McPherson, Charlotte C. Hewson, Suzie May, Bruce Walker, Philip Robinson, Davina Ghersi, Jane Fitzpatrick, Tania Winzenberg, Kieran Fallon, Paul Glasziou, Laurent Billot, Rachelle Buchbinder

Summary: The Australia and New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network developed a Research Question Importance Tool (ANZMUSC-RQIT) to rank research projects based on their importance. The tool consists of 5 dimensions and is designed for scoring by committee consensus.

PLOS ONE (2023)

Review Endocrinology & Metabolism

Low-risk prostate lesions: An evidence review to inform discussion on losing the cancer label

Caitlin R. Semsarian, Tara Ma, Brooke Nickel, Alexandra Barratt, Murali Varma, Brett Delahunt, Jeremy Millar, Lisa Parker, Paul Glasziou, Katy J. L. Bell

Summary: Active surveillance (AS) reduces overtreatment of low-risk prostate lesions. Recalibrating diagnostic thresholds and adopting alternative diagnostic labels could increase AS uptake. Evidence suggests that AS has low rates of metastasis and prostate cancer-specific mortality, but termination of AS and conversion to treatment can occur in a significant number of men. Prevalence of subclinical prostate cancer increases with age, and there is variability in reproducibility of histopathological diagnosis. Diagnostic drift is evident, with cases being upgraded or downgraded based on contemporary diagnostic criteria.

PROSTATE (2023)

Review Rehabilitation

Telerehabilitation versus face-to-face rehabilitation in the management of musculoskeletal conditions: a systematic review and meta-analysis

Natalia Krzyzaniak, Magnolia Cardona, Ruwani Peiris, Zoe A. Michaleff, Hannah Greenwood, Justin Clark, Anna Mae Scott, Paul Glasziou

Summary: A systematic review and meta-analysis of five randomized controlled trials found no significant differences in pain, function, quality of life, and satisfaction between telehealth and face-to-face consultations for musculoskeletal conditions. However, the evidence is limited due to the small number of included studies and sample size.

PHYSICAL THERAPY REVIEWS (2023)

Review Health Care Sciences & Services

Comparison of Telephone and Video Telehealth Consultations: Systematic Review

Oyungerel Byambasuren, Hannah Greenwood, Mina Bakhit, Tiffany Atkins, Justin Clark, Anna Mae Scott, Paul Glasziou

Summary: This study compared telephone and video telehealth consultations and found no notable differences in the management of patients with an established diagnosis. However, the study also highlighted a significant lack of telehealth research in primary care settings.

JOURNAL OF MEDICAL INTERNET RESEARCH (2023)

Review Clinical Neurology

Anomalies in the review process and interpretation of the evidence in the NICE guideline for chronic fatigue syndrome and myalgic encephalomyelitis

Peter White, Susan Abbey, Brian Angus, Harriet A. Ball, Dedra S. Buchwald, Christine Burness, Alan J. Carson, Trudie Chalder, Daniel J. Clauw, Jan Coebergh, Anthony S. David, Barbara A. Dworetzky, Mark J. Edwards, Alberto J. Espay, John Etherington, Per Fink, Signe Flottorp, Beatrice Garcin, Paul Garner, Paul Glasziou, Willie Hamilton, Peter Henningsen, Ingrid Hoeritzauer, Mujtaba Husain, Anne-Catherine M. L. Huys, Hans Knoop, Kurt Kroenke, Alexander Lehn, James L. Levenson, Paul Little, Andrew Lloyd, Ira Madan, Jos W. M. van der Meer, Alastair Miller, Maurice Murphy, Irwin Nazareth, David L. Perez, Wendy Phillips, Markus Reuber, Winfried Rief, Alastair Santhouse, Tereza Serranova, Michael Sharpe, Biba Stanton, Donna E. Stewart, Jon Stone, Michele Tinazzi, Derick T. Wade, Simon C. Wessely, Vegard Wyller, Adam Zeman

Summary: Chronic fatigue syndrome/myalgic encephalomyelitis (CFS/ME) is a disabling long-term condition of unknown cause. The recent guideline from the National Institute for Health and Care Excellence (NICE) sparked controversy by recommending against graded exercise therapy (GET) and downgrading the importance of cognitive-behavioural therapy for recovery. The anomalies in evidence processing and interpretation by the NICE committee, such as creating a new definition, omitting data, and deviating from accepted practices, contributed to this controversy.

JOURNAL OF NEUROLOGY NEUROSURGERY AND PSYCHIATRY (2023)

Review Health Care Sciences & Services

Telehealth versus face-to-face delivery of cognitive behavioural therapy for insomnia: A systematic review and meta-analysis of randomised controlled trials

Anna Mae Scott, Ruwani Peiris, Tiffany Atkins, Magnolia Cardona, Hannah Greenwood, Justin Clark, Paul Glasziou

Summary: This systematic review and meta-analysis compared the effectiveness of telehealth and face-to-face delivery of care to patients with insomnia. The results showed no significant differences in insomnia severity, quality of life, and satisfaction between the two delivery methods. Telehealth may be a viable alternative to face-to-face cognitive behavioural therapy for insomnia.

JOURNAL OF TELEMEDICINE AND TELECARE (2023)

Article Medicine, General & Internal

The differences and overlaps between 'explanatory' and 'pragmatic' controlled trials: a historical perspective

Paul Glasziou, Robert Matthews, Isabelle Boutron, Iain Chalmers, Peter Armitage

JOURNAL OF THE ROYAL SOCIETY OF MEDICINE (2023)

Article Medicine, General & Internal

Trial analysis by treatment allocated or by treatment received? Origins of 'the intention-to-treat principle' to reduce allocation bias: part 2

I Chalmers, R. Matthews, P. Glasziou, I Boutron, P. Armitage

JOURNAL OF THE ROYAL SOCIETY OF MEDICINE (2023)

Article Emergency Medicine

Evaluating the use of clinical decision aids in an Australian emergency department: A cross-sectional survey

Zoe A. Michaleff, Laetitia Hattingh, Hannah Greenwood, Sharon Mickan, Mark Jones, Madeleen van der Merwe, Rae Thomas, Joan Carlini, David Henry, Paulina Stehlik, Paul Glasziou, Gerben Keijzers

Summary: This study investigated the knowledge, use, and documentation of clinical decision aids (CDAs) among healthcare professionals in a large Australian emergency department (ED). The results showed that healthcare professionals had low knowledge and self-reported use of CDAs, but the usage varied across different CDAs. Most respondents expressed a desire to increase their use of valid CDAs and supported the integration of CDAs into the electronic medical record (EMR) system. Potential barriers to CDA use included knowledge, social/professional role and identity, and social influences.

EMERGENCY MEDICINE AUSTRALASIA (2023)

No Data Available