Explore the vast range of titles available.

The importance of adequate intervention descriptions in minimising research waste and improving research usability and reproducibility has gained attention in the past few years. Nearly all focus to date has been on intervention reporting in randomised trials. Yet clinicians are encouraged to use systematic reviews, whenever available, rather than single trials to inform their practice. This article explores the problem and implications of incomplete intervention details during the planning, conduct, and reporting of systematic reviews and makes recommendations for review authors, peer reviewers, and journal editors

The aim of the INveStigating ProblEmatic Clinical Trials in Systematic Reviews (INSPECT-SR) project is to develop a tool to identify problematic RCTs in systematic reviews. In stage 1 of the project, a list of potential trustworthiness checks was created. The checks on this list must be evaluated to determine which should be included in the INSPECT-SR tool. We attempted to apply 72 trustworthiness checks to randomized controlled trials (RCTs) in 50 Cochrane reviews. For each, we recorded whether the check was passed, failed, or possibly failed or whether it was not feasible to complete the check. Following application of the checks, we recorded whether we had concerns about the authenticity of each RCT. We repeated each meta-analysis after removing RCTs flagged by each check and again after removing RCTs where we had concerns about authenticity to estimate the impact of trustworthiness assessment. Trustworthiness assessments were compared to Risk of Bias and Grading of Recommendations Assessment, Development and Evaluation (GRADE) assessments in the reviews. Ninety-five RCTs were assessed. Following application of the checks, assessors had some or serious concerns about the authenticity of 25% and 6% of the RCTs, respectively. Removing RCTs with either some or serious concerns resulted in 22% of meta-analyses having no remaining RCTs. However, many checks proved difficult to understand or implement, which may have led to unwarranted skepticism in some instances. Furthermore, we restricted assessment to meta-analyses with no more than five RCTs (54% contained only 1 RCT), which will distort the impact on results. No relationship was identified between trustworthiness assessment and Risk of Bias or GRADE. This study supports the case for routine trustworthiness assessment in systematic reviews, as problematic studies do not appear to be flagged by Risk of Bias assessment. The study produced evidence on the feasibility and impact of trustworthiness checks. These results will be used, in conjunction with those from a subsequent Delphi process, to determine which checks should be included in the INSPECT-SR tool. Systematic reviews collate evidence from randomized controlled trials (RCTs) to find out whether health interventions are safe and effective. However, it is now recognized that the findings of some RCTs are not genuine, and some of these studies appear to have been fabricated. Various checks for these “problematic” RCTs have been proposed, but it is necessary to evaluate these checks to find out which are useful and which are feasible. We applied a comprehensive list of “trustworthiness checks” to 95 RCTs in 50 systematic reviews to learn more about them and to see how often performing the checks would lead us to classify RCTs as being potentially inauthentic. We found that applying the checks led to concerns about the authenticity of around 1 in three RCTs. However, we found that many of the checks were difficult to perform and could have been misinterpreted. This might have led us to be overly skeptical in some cases. The findings from this study will be used, alongside other evidence, to decide which of these checks should be performed routinely to try to identify problematic RCTs, to stop them from being mistaken for genuine studies and potentially being used to inform health care decisions. •An extensive list of potential checks for assessing study trustworthiness was assessed via an application to 95 randomized controlled trials (RCTs) in 50 Cochrane Reviews.•Following application of the checks, assessors had concerns about the authenticity of 32% of the RCTs.•If these RCTs were excluded, 22% of meta-analyses would have no remaining RCTs.•However, the study showed that some checks were frequently infeasible, and others could be easily misunderstood or misinterpreted.•The study restricted assessment to meta-analyses, including five or fewer RCTs, which might distort the impact of applying the checks.

The objective of this study was to assess current Cochrane Review practice in identifying and incorporating information from clinical trial registers. A cross-sectional study was conducted to assess a sample of new or updated intervention reviews from all Cochrane Review Groups up to February 1, 2017. Two assessors independently extracted data from each review using a pretested audit questionnaire. Data were analyzed relating to the frequency of reporting (1) the register source and search strategy; (2) the results of trial register searches; and (3) the use of trial register information in the review. Over 90% (236/260) of Cochrane Reviews reported searching a trial register (e.g., ClinicalTrials.gov or the WHO International Clinical Trials Registry Platform). In reviews that reported trial register searches, 39% (92/236) indicated the number of trial records retrieved and 56.8% (134/236) used information from the trial register records in the review. Trial record information was incorporated into the results (39.6%; 53/134), risk of bias assessments (53.7%; 72/134), and discussion (24.6%, 33/134) and conclusion sections (25.4%, 34/134). Most audited reviews used trial register information. Guidance may be needed to better incorporate information from these valuable resources in Cochrane Reviews to assist future research decisions made by funders and prospective study investigators.

To identify studies of influenza vaccination of HCWs and influenza in elderly residents in long-term care facilities. We searched seven electronic databases for randomised controlled trials (RCTs) and non-RCTs. Two reviewers independently extracted data and assessed trial quality. The key outcomes are serologically proven influenza, pneumonia, and deaths from pneumonia, and pooled data from three C-RCTs showed no effect. Pooled data from three C-RCTs showed lower resident all-cause mortality, but as influenza constituted less than 10% of all deaths even in epidemic years we question the appropriateness of this outcome measure. Pooled data from three C-RCTs showed vaccination of HCWs reduced ILI and data from one C-RCT that HCW vaccination reduced GP consultations for ILI, but as influenza constitutes less than 25% of ILI and we did not show that HCW influenza vaccination reduced serologically proven influenza we question whether this effect is due to confounding.

We evaluated the inter-rater reliability (IRR) of assessing the quality of evidence (QoE) using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach. On completing two training exercises, participants worked independently as individual raters to assess the QoE of 16 outcomes. After recording their initial impression using a global rating, raters graded the QoE following the GRADE approach. Subsequently, randomly paired raters submitted a consensus rating. The IRR without using the GRADE approach for two individual raters was 0.31 (95% confidence interval [95% CI] = 0.21–0.42) among Health Research Methodology students (n = 10) and 0.27 (95% CI = 0.19–0.37) among the GRADE working group members (n = 15). The corresponding IRR of the GRADE approach in assessing the QoE was significantly higher, that is, 0.66 (95% CI = 0.56–0.75) and 0.72 (95% CI = 0.61–0.79), respectively. The IRR further increased for three (0.80 [95% CI = 0.73–0.86] and 0.74 [95% CI = 0.65–0.81]) or four raters (0.84 [95% CI = 0.78–0.89] and 0.79 [95% CI = 0.71–0.85]). The IRR did not improve when QoE was assessed through a consensus rating. Our findings suggest that trained individuals using the GRADE approach improves reliability in comparison to intuitive judgments about the QoE and that two individual raters can reliably assess the QoE using the GRADE system.

Assessment of risk of bias is regarded as an essential component of a systematic review on the effects of an intervention. The most commonly used tool for randomised trials is the Cochrane risk-of-bias tool. We updated the tool to respond to developments in understanding how bias arises in randomised trials, and to address user feedback on and limitations of the original tool.

In 2019, only 7% of Cochrane systematic reviews (SRs) cited a core outcome set (COS) in relation to choosing outcomes, even though a relevant COS existed but was not mentioned (or cited) for a further 29% of SRs. Our objectives for the current work were to (1) examine the extent to which authors are currently considering COS to inform outcome choice in Cochrane protocols and completed SRs, and (2) understand author facilitators and barriers to using COS. We examined all completed Cochrane SRs published in the last 3 months of 2022 and all Cochrane protocols published in 2022 for the extent to which they: (a) cited a COS, (b) searched for COS, (c) used outcomes from existing COS, and (d) reported outcome inconsistency among included studies and/or noted the need for COS. One investigator extracted information; a second extractor verified all information, discussing discrepancies to achieve consensus. We then conducted an online survey of authors of the included SRs to assess awareness of COS and identify facilitators and barriers to using COS to inform outcome choice. Objective 1: We included 294 SRs of interventions (84 completed SRs and 210 published SR protocols), of which 13% cited specific COS and 5% did not cite but mentioned searching for COS. A median of 83% of core outcomes from cited COS (interquartile range [IQR] 57%–100%) were included in the corresponding SR. We identified a relevant COS for 39% of SRs that did not cite a COS. A median of 50% of core outcomes from noncited COS (IQR 35%–72%) were included in the corresponding SR. Objective 2: Authors of 236 (80%) of the 294 eligible SRs completed our survey. Seventy-seven percent of authors noted being aware of COS before the survey. Fifty-five percent of authors who did not cite COS but were aware of them reported searching for a COS. The most reported facilitators of using COS were author awareness of the existence of COS (59%), author positive perceptions of COS (52%), and recommendation in the Cochrane Handbook regarding COS use (48%). The most reported barriers related to matching of the scope of the COS and the SR: the COS target population was too narrow/broad relative to the SR population (29%) or the COS target intervention was too narrow/broad relative to the SR intervention (21%). Most authors (87%) mentioned that they would consider incorporating missing core outcomes in the SR/update. Since 2019, there is increasing consideration and awareness of COS when choosing outcomes for Cochrane SRs of interventions, but uptake remains low and can be improved further. Use of COS in SRs is important to improve outcome standardization, reduce research waste, and improve evidence syntheses of the relevant effects of interventions across health research.

IntroductionRandomised controlled trials (RCTs) inform healthcare decisions. It is now apparent that some published RCTs contain false data and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs that have been conducted on a given topic. While it is usual to assess methodological features of the RCTs in the process of undertaking a systematic review, it is not usual to consider whether the RCTs contain false data. Studies containing false data therefore go unnoticed and contribute to systematic review conclusions. The INveStigating ProblEmatic Clinical Trials in Systematic Reviews (INSPECT-SR) project will develop a tool to assess the trustworthiness of RCTs in systematic reviews of healthcare-related interventions.Methods and analysisThe INSPECT-SR tool will be developed using expert consensus in combination with empirical evidence, over five stages: (1) a survey of experts to assemble a comprehensive list of checks for detecting problematic RCTs, (2) an evaluation of the feasibility and impact of applying the checks to systematic reviews, (3) a Delphi survey to determine which of the checks are supported by expert consensus, culminating in, (4) a consensus meeting to select checks to be included in a draft tool and to determine its format and (5) prospective testing of the draft tool in the production of new health systematic reviews, to allow refinement based on user feedback. We anticipate that the INSPECT-SR tool will help researchers to identify problematic studies and will help patients by protecting them from the influence of false data on their healthcare.Ethics and disseminationThe University of Manchester ethics decision tool was used, and this returned the result that ethical approval was not required for this project (30 September 2022), which incorporates secondary research and surveys of professionals about subjects relating to their expertise. Informed consent will be obtained from all survey participants. All results will be published as open-access articles. The final tool will be made freely available.

The Grading of Recommendations Assessment, Development and Evaluation (GRADE) is widely used and reliable and accurate for assessing the certainty in the body of health evidence. The GRADE working group has provided detailed guidance for assessing the certainty in the body of evidence in systematic reviews and health technology assessments (HTAs) and how to grade the strength of health recommendations. However, there is limited advice regarding how to maximize transparency of these judgments, in particular through explanatory footnotes or explanations in Summary of Findings tables and Evidence Profiles (GRADE evidence tables). We conducted this study to define the essential attributes of useful explanations and to develop specific guidance for explanations associated with GRADE evidence tables. We used a sample of explanations according to their complexity, type of judgment involved, and appropriateness from a database of published GRADE evidence tables in Cochrane reviews and World Health Organization guidelines. We used an iterative process and group consensus to determine the attributes and develop guidance. Explanations in GRADE evidence tables should be concise, informative, relevant, easy to understand, and accurate. We provide general and domain-specific guidance to assist authors with achieving these desirable attributes in their explanations associated with GRADE evidence tables. Adhering to the general and GRADE domain-specific guidance should improve the quality of explanations associated with GRADE evidence tables, assist authors of systematic reviews, HTA reports, or guidelines with information that they can use in other parts of their evidence synthesis. This guidance will also support editorial evaluation of evidence syntheses using GRADE and provide a minimum quality standard of judgments across tables.

Publications of living systematic reviews (LSRs) are increasing rapidly. Guidance facilitating transparent, complete, and accurate reporting of LSRs is needed. This paper reports the development of an extension of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2020 statement for LSRs (PRISMA-LSR). The PRISMA-LSR extension includes the PRISMA-LSR checklist, the PRISMA-LSR flow diagram, reporting recommendations for the LSR status, and an explanation and elaboration document. This extension has been developed as an “add-on” to the PRISMA 2020 statement, meaning it should be used in addition to the PRISMA 2020 statement. The PRISMA-LSR extension is expected to benefit authors, editors, peer reviewers, and users of LSRs through transparent, complete, and accurate reporting of LSRs.