4.4 Review

The automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrations

Journal

BMC MEDICAL RESEARCH METHODOLOGY
Volume 21, Issue 1, Pages -

Publisher

BMC
DOI: 10.1186/s12874-021-01485-6

Keywords

Systematic reviews; Trial registrations; Document similarity; Hierarchical clustering

Funding

  1. National Library of Medicine, National Institutes of Health [R01LM012976]

Ask authors/readers for more resources

This study evaluated methods for ranking clinical trial registrations for inclusion in systematic review updates. Document similarity outperformed hierarchical clustering in identifying relevant trials, indicating its potential for automatic identification of new trials for updates.
Background Clinical trial registries can be used as sources of clinical evidence for systematic review synthesis and updating. Our aim was to evaluate methods for identifying clinical trial registrations that should be screened for inclusion in updates of published systematic reviews. Methods A set of 4644 clinical trial registrations (ClinicalTrials.gov) included in 1089 systematic reviews (PubMed) were used to evaluate two methods (document similarity and hierarchical clustering) and representations (L2-normalised TF-IDF, Latent Dirichlet Allocation, and Doc2Vec) for ranking 163,501 completed clinical trials by relevance. Clinical trial registrations were ranked for each systematic review using seeding clinical trials, simulating how new relevant clinical trials could be automatically identified for an update. Performance was measured by the number of clinical trials that need to be screened to identify all relevant clinical trials. Results Using the document similarity method with TF-IDF feature representation and Euclidean distance metric, all relevant clinical trials for half of the systematic reviews were identified after screening 99 trials (IQR 19 to 491). The best-performing hierarchical clustering was using Ward agglomerative clustering (with TF-IDF representation and Euclidean distance) and needed to screen 501 clinical trials (IQR 43 to 4363) to achieve the same result. Conclusion An evaluation using a large set of mined links between published systematic reviews and clinical trial registrations showed that document similarity outperformed hierarchical clustering for identifying relevant clinical trials to include in systematic review updates.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.4
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Public, Environmental & Occupational Health

Ensuring Prevention Science Research is Synthesis-Ready for Immediate and Lasting Scientific Impact

Emily A. Hennessy, Rebecca L. Acabchuk, Pieter A. Arnold, Adam G. Dunn, Yong Zhi Foo, Blair T. Johnson, Sonya R. Geange, Neal R. Haddaway, Shinichi Nakagawa, Witness Mapanga, Kerrie Mengersen, Matthew J. Page, Alfredo Sanchez-Tojar, Vivian Welch, Luke A. McGuinness

Summary: It is crucial to robustly synthesize available evidence to inform and improve prevention efforts and policy, yet barriers such as inaccurate terminology and unclear reporting hinder comprehensive evidence synthesis. Practical guidelines and tools are provided to assist prevention scientists in preparing synthesis-ready research, with step-by-step guidance and software suggestions for standardizing data design and public archiving to facilitate synthesis-ready research. Using a recent mindfulness trial as an example, ways to ensure discoverability of primary studies and the presence of necessary data are demonstrated.

PREVENTION SCIENCE (2022)

Article Substance Abuse

Social connections influencing e-cigarette use and intentions in Australia: a survey

Samia Amin, Liliana Laranjo, Adam G. Dunn

Summary: The study findings suggest that higher social acceptability of e-cigarettes in workplaces or educational settings is associated with increased likelihood of use or intention to use among individuals.

JOURNAL OF ADDICTIVE DISEASES (2022)

Article Medicine, Research & Experimental

Characteristics of clinical trials associated with early results reporting at ClinicalTrials.gov

Rabia Bashir, Adam G. Dunn

Summary: This study aims to investigate the characteristics of clinical trials associated with earlier results reporting on ClinicalTrials.gov. The results show that non-industry, non-drug, and earlier phase trials report results on ClinicalTrials.gov more slowly. The findings suggest that incentives and tools targeting these types of trials are also needed to improve trial reporting.

CONTEMPORARY CLINICAL TRIALS (2022)

Article Multidisciplinary Sciences

Social interactions and quality of life of residents in aged care facilities: A multi-methods study

Joyce Siette, Laura Dodds, Didi Surian, Mirela Prgomet, Adam Dunn, Johanna Westbrook

Summary: This study examined the frequency and duration of social interactions among residents in aged care facilities and their association with quality of life. The results showed a positive correlation between time spent with other residents and quality of life, while time spent with facility staff was negatively correlated with quality of life.

PLOS ONE (2022)

Review Computer Science, Interdisciplinary Applications

Natural language processing for clinical notes in dentistry: A systematic review

Farhana Pethani, Adam G. Dunn

Summary: The objective of this study was to review the applications of natural language processing (NLP) for information extraction and retrieval from clinical notes in dentistry. A search strategy was implemented in EMBASE, CINAHL, and Medline to identify relevant studies. A total of 17 studies were included, demonstrating heterogeneity in study design and reporting quality. The findings suggest a need for standardization in reporting and improved connections between NLP methods and applied NLP in dentistry.

JOURNAL OF BIOMEDICAL INFORMATICS (2023)

Review Computer Science, Information Systems

Using automated methods to detect safety problems with health information technology: a scoping review

Didi Surian, Ying Wang, Enrico Coiera, Farah Magrabi

Summary: This research summarizes the literature on evaluating automated methods for early detection of safety problems with health information technology (HIT). The study found that various rule-based, statistical, and machine learning methods have been applied to automate the detection of HIT safety problems, but there are still many opportunities to study their application and effectiveness in real-world settings.

JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION (2023)

Editorial Material Biochemistry & Molecular Biology

Disruptions to the hearing health sector

Isabelle Boisvert, Adam G. Dunn, Erik Lundmark, Jennifer Smith-Merry, Wendy Lipworth, Amber Willink, Sarah E. Hughes, Michele Nealon, Melanie Calvert

NATURE MEDICINE (2023)

Article Health Care Sciences & Services

Evolving Face Mask Guidance During a Pandemic and Potential Harm to Public Perception: Infodemiology Study of Sentiment and Emotion on Twitter

Divya Ramjee, Catherine C. Pollack, Marie-Laure Charpignon, Shagun Gupta, Jessica Malaty Rivera, Ghinwa El Hayek, Adam G. Dunn, Angel N. Desai, Maimuna S. Majumder

Summary: During the COVID-19 pandemic, there were fluctuating policies on face mask use by the US Centers for Disease Control and Prevention. Analyzing sentiment on masks surrounding guideline changes can inform future communication strategies and improve widespread and sustained adoption.

JOURNAL OF MEDICAL INTERNET RESEARCH (2023)

Article Health Care Sciences & Services

Multiple Automated Health Literacy Assessments of Written Health Information: Development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor v1

Julie Ayre, Carissa Bonner, Danielle M. Muscat, Adam G. Dunn, Eliza Harrison, Jason Dalmazzo, Dana Mouwad, Parisa Aslani, Heather L. Shepherd, Kirsten J. McCaffery

Summary: The SHeLL Health Literacy Editor is an automated tool that provides objective and specific guidance for producing easy-to-read health information. It improves the quality and safety of written health information by offering immediate feedback on various aspects, such as readability and person-centered language. It can be used as a scalable intervention to promote the adoption of health literacy guidelines by health services and information providers.

JMIR FORMATIVE RESEARCH (2023)

Article Mathematical & Computational Biology

A comparison of machine learning methods to find clinical trials for inclusion in new systematic reviews from their PROSPERO registrations prior to searching and screening

Shifeng Liu, Florence T. Bourgeois, Claire Narang, Adam G. Dunn

Summary: This study compared different methods for finding relevant trial registrations using a PROSPERO entry, without prior screening. The results showed that term-based representations and MetaMap outperformed PICO extraction in ranking trial registrations. When ranking trial articles, term-based representations were the best performing approach. These findings suggest that automated methods can reduce workload in systematic reviews, but additional processes are still needed to efficiently identify relevant trial registrations or trial articles.

RESEARCH SYNTHESIS METHODS (2023)

Article Computer Science, Cybernetics

RHMD: A Real-World Dataset for Health Mention Classification on Reddit

Usman Naseem, Matloob Khushi, Jinman Kim, Adam G. Dunn

Summary: People on social media using disease and symptom words to discuss their health can introduce biases in data-driven public health applications. This study presents a new dataset called RHMD, which consists of 10,015 manually annotated Reddit posts. The dataset is labeled with four categories and provides a comprehensive performance analysis of baseline methods. The release of this dataset is expected to facilitate the development of new methods for detecting health mentions in user-generated text.

IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS (2023)

Article Primary Health Care

Experiences and influences of online health information-seeking about statin use in patients with high cardiovascular risk: a qualitative study

Hooi Min Lim, Chirk Jenn Ng, Adam G. Dunn, Adina Abdullah

Summary: This study explores the views and experiences of patients with high cardiovascular risk on online health information-seeking about statins, and how this information influences their decisions. The results suggest that patients need different types of online health information throughout their disease trajectory, and unintentional passive exposure to online health information has an influence on patients' adherence to statins. The quality of patient-doctor communication in relation to online health information-seeking behavior remains critical for patient decision-making.

FAMILY PRACTICE (2023)

Article Computer Science, Information Systems

Event Detection on Twitter by Mapping Unexpected Changes in Streaming Data into a Spatiotemporal Lattice

Zubair Shah, Adam G. Dunn

Summary: This research proposes a new approach to event localization and ranking by modeling the use of language in tweets. It detects unexpected changes in language usage and identifies anomalies across cities, countries, and time periods.

IEEE TRANSACTIONS ON BIG DATA (2022)

Article Computer Science, Cybernetics

Hybrid Text Representation for Explainable Suicide Risk Identification on Social Media

Usman Naseem, Matloob Khushi, Jinman Kim, Adam G. Dunn

Summary: The article introduces a hybrid text representation method for explaining suicide risk identification on social media. The method achieves excellent results on a public suicide dataset and demonstrates advantages in clinical and public health practice.

IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS (2022)

No Data Available