Journal Article

Software tools to support title and abstract screening for systematic reviews in healthcare: an evaluation

Usher-Smith, Juliet A., Griffin, Simon J., Kuhn, Isla, Harrison, Hannah

2020

Overview

Background Systematic reviews are vital to the pursuit of evidence-based medicine within healthcare. Screening titles and abstracts (T&Ab) for inclusion in a systematic review is an intensive, and often collaborative, step. The use of appropriate tools is therefore important. In this study, we identified and evaluated the usability of software tools that support T&Ab screening for systematic reviews within healthcare research. Methods We identified software tools using three search methods: a web-based search; a search of the online “systematic review toolbox”; and screening of references in existing literature. We included tools that were accessible and available for testing at the time of the study (December 2018), do not require specific computing infrastructure and provide basic screening functionality for systematic reviews. Key properties of each software tool were identified using a feature analysis adapted for this purpose. This analysis included a weighting developed by a group of medical researchers, therefore prioritising the most relevant features. The highest scoring tools from the feature analysis were then included in a user survey, in which we further investigated the suitability of the tools for supporting T&Ab screening amongst systematic reviewers working in medical research. Results Fifteen tools met our inclusion criteria. They vary significantly in relation to cost, scope and intended user community. Six of the identified tools (Abstrackr, Colandr, Covidence, DRAGON, EPPI-Reviewer and Rayyan) scored higher than 75% in the feature analysis and were included in the user survey. Of these, Covidence and Rayyan were the most popular with the survey respondents. Their usability scored highly across a range of metrics, with all surveyed researchers ( n  = 6) stating that they would be likely (or very likely) to use these tools in the future. Conclusions Based on this study, we would recommend Covidence and Rayyan to systematic reviewers looking for suitable and easy to use tools to support T&Ab screening within healthcare research. These two tools consistently demonstrated good alignment with user requirements. We acknowledge, however, the role of some of the other tools we considered in providing more specialist features that may be of great importance to many researchers.

• Publisher: BioMed Central,BioMed Central Ltd,Springer Nature B.V,BMC
• Subject: Abstracting and Indexing - methods / Biomedical Research / Collaboration / Data collection / Delivery of Health Care / Discussion groups / Evaluation / Evidence-based medicine / Evidence-Based Medicine - methods / Feature analysis / Health Sciences / Humans / Market surveys / Medical research / Medical researchers / Medicine / Medicine & Public Health / Methods / quality / reporting / Research Article / Researchers / Screening / Search strategies / Secondary data analysis / Software / Software engineering / Software industry / Software tools / Statistical Theory and Methods / Statistics for Life Sciences / Surveys and Questionnaires / Systematic review / Systematic reviews / Systematic Reviews as Topic - methods / Technology application / Theory of Medicine/Bioethics / Time / Title and abstract / User surveys / Web applications