4.4 Article

The semi-automation of title and abstract screening: a retrospective exploration of ways to leverage Abstrackr's relevance predictions in systematic and rapid reviews

Journal

BMC MEDICAL RESEARCH METHODOLOGY
Volume 20, Issue 1, Pages -

Publisher

BMC
DOI: 10.1186/s12874-020-01031-w

Keywords

Systematic reviews; Rapid reviews; Machine learning; Automation; Efficiency

Funding

  1. Alberta Strategy for Patient-Oriented Research (SPOR) SUPPORT Unit Knowledge Translation Platform - Alberta Innovates
  2. Canadian Institutes of Health Research
  3. Canada Research Chair in Knowledge Synthesis and Translation

Ask authors/readers for more resources

Background We investigated the feasibility of using a machine learning tool's relevance predictions to expedite title and abstract screening. Methods We subjected 11 systematic reviews and six rapid reviews to four retrospective screening simulations (automated and semi-automated approaches to single-reviewer and dual independent screening) in Abstrackr, a freely-available machine learning software. We calculated the proportion missed, workload savings, and time savings compared to single-reviewer and dual independent screening by human reviewers. We performed cited reference searches to determine if missed studies would be identified via reference list scanning. Results For systematic reviews, the semi-automated, dual independent screening approach provided the best balance of time savings (median (range) 20 (3-82) hours) and reliability (median (range) proportion missed records, 1 (0-14)%). The cited references search identified 59% (n = 10/17) of the records missed. For the rapid reviews, the fully and semi-automated approaches saved time (median (range) 9 (2-18) hours and 3 (1-10) hours, respectively), but less so than for the systematic reviews. The median (range) proportion missed records for both approaches was 6 (0-22)%. Conclusion Using Abstrackr to assist one of two reviewers in systematic reviews saves time with little risk of missing relevant records. Many missed records would be identified via other means.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.4
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available