Explore the vast range of titles available.

The objectives of this study were to determine the interrater reliability (IRR) of assessment of multiple systematic reviews (AMSTAR) 2 for reviews of pharmacological or psychological interventions for the treatment of major depression, to compare it to that of AMSTAR and risk of bias in systematic reviews (ROBIS), and to assess the convergent validity between the appraisal tools. Two groups of four raters were each assigned one of two samples of 30 systematic reviews. All eight raters applied AMSTAR 2 to their sample. Each group also applied either AMSTAR or ROBIS. Fleiss' kappa and Gwet's AC1 were calculated, and agreement between the tools was assessed. The median kappa values as a measure of IRR indicated a moderate agreement for AMSTAR 2 (median = 0.51), a substantial agreement for AMSTAR (median = 0.62), and a fair agreement for ROBIS (median = 0.27). Validity results showed a positive association for AMSTAR and AMSTAR 2 (r = 0.91) as well as ROBIS and AMSTAR 2 (r = 0.84). For the overall rating, AMSTAR 2 showed a high concordance with ROBIS and a lower concordance with AMSTAR. The IRR of AMSTAR 2 was found to be slightly lower than the IRR of AMSTAR and higher than the IRR of ROBIS. Validity measurements indicate that AMSTAR 2 is closely related to both ROBIS and AMSTAR.

The objective of the study was to evaluate the inter-rater and intercenter reliability, usability, and utility of A MeaSurement Tool to Assess systematic Reviews (AMSTAR), AMSTAR 2, and Risk Of Bias In Systematic reviews (ROBIS). This is a prospective evaluation using 30 systematic reviews of randomized trials, undertaken at three international centers. Reviewers completed AMSTAR, AMSTAR 2, and ROBIS in median (interquartile range) 15.7 (11.3), 19.7 (12.1), and 28.7 (17.4) minutes and reached consensus in 2.6 (3.2), 4.6 (5.3), and 10.9 (10.8) minutes, respectively. Across all centers, inter-rater reliability was substantial to almost perfect for 8/11 AMSTAR, 9/16 AMSTAR 2, and 12/24 ROBIS items. Intercenter reliability was substantial to almost perfect for 6/11 AMSTAR, 12/16 AMSTAR 2, and 7/24 ROBIS items. Intercenter reliability for confidence in the results of the review or overall risk of bias was moderate (Gwet's first-order agreement coefficient (AC1) 0.58, 95% confidence intervals [CI]: 0.30 to 0.85) to substantial (AC1 0.74, 95% CI: 0.30 to 0.85) for AMSTAR 2 and poor (AC1 −0.21, 95% CI: −0.55 to 0.13) to moderate (AC1 0.56, 95% CI: 0.30 to 0.83) for ROBIS. It is not clear whether using the appraisals of any tool as an inclusion criterion would alter an overview's findings. Improved guidance may be needed to facilitate the consistent interpretation and application of the newer tools (especially ROBIS).

To compare A Measurement Tool to Assess Systematic Reviews (AMSTAR 2) with a tool to assess risk of bias in systematic reviews (ROBIS) in terms of validity, reliability, and applicability. We analyzed 30 systematic reviews (SRs) that included randomized and nonrandomized studies, with Cochrane and non-Cochrane SRs sampled in 1:1 ratio. Four reviewers assessed independently all 30 SRs with AMSTAR 2, followed by ROBIS. We calculated Fleiss’ Kappa as a measure of inter-rater reliability (IRR) across 4 raters. The IRR for scoring the overall confidence in the SRs with AMSTAR 2 and the overall domain in ROBIS was fair (AMSTAR 2: κ = 0.30, 95% [confidence interval] CI: 0.17 to 0.43; ROBIS: κ = 0.28, 95% CI: 0.13 to 0.42). AMSTAR 2 confidence in review ratings strongly correlated with the overall domain rating in ROBIS (Spearman rs = 0.84). Mean time for scoring AMSTAR 2 was slightly higher than for ROBIS (18 vs. 16 min), with huge differences between the reviewers. Both AMSTAR 2 and ROBIS can be applied to SRs including both randomized controlled trials (RCTs) and non-RCTs. Measurement properties of ROBIS seemed not to be much different when comparing with other studies that include only SRs of RCTs.

The aim of this study was to investigate the differences in main characteristics, reporting and methodological quality between prospectively registered and nonregistered systematic reviews. PubMed was searched to identify systematic reviews of randomized controlled trials published in 2015 in English. After title and abstract screening, potentially relevant reviews were divided into three groups: registered non-Cochrane reviews, Cochrane reviews, and nonregistered reviews. For each group, random number tables were generated in Microsoft Excel, and the first 50 eligible studies from each group were randomly selected. Data of interest from systematic reviews were extracted. Regression analyses were conducted to explore the association between total Revised Assessment of Multiple Systematic Review (R-AMSTAR) or Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) scores and the selected characteristics of systematic reviews. The conducting and reporting of literature search in registered reviews were superior to nonregistered reviews. Differences in 9 of the 11 R-AMSTAR items were statistically significant between registered and nonregistered reviews. The total R-AMSTAR score of registered reviews was higher than nonregistered reviews [mean difference (MD) = 4.82, 95% confidence interval (CI): 3.70, 5.94]. Sensitivity analysis by excluding the registration-related item presented similar result (MD = 4.34, 95% CI: 3.28, 5.40). Total PRISMA scores of registered reviews were significantly higher than nonregistered reviews (all reviews: MD = 1.47, 95% CI: 0.64-2.30; non-Cochrane reviews: MD = 1.49, 95% CI: 0.56-2.42). However, the difference in the total PRISMA score was no longer statistically significant after excluding the item related to registration (item 5). Regression analyses showed similar results. Prospective registration may at least indirectly improve the overall methodological quality of systematic reviews, although its impact on the overall reporting quality was not significant.

Reliable synthesis of the various rapidly expanding bodies of evidence is vital for the process of evidence-informed decision-making in environmental policy, practice and research. With the rise of evidence-base medicine and increasing numbers of published systematic reviews, criteria for assessing the quality of reporting have been developed. First QUOROM (Lancet 354:1896–1900, 1999 ) and then PRISMA (Ann Intern Med 151:264, 2009 ) were developed as reporting guidelines and standards to ensure medical meta-analyses and systematic reviews are reported to a high level of detail. PRISMA is now widely used by a range of journals as a pre-submission checklist. However, due to its development for systematic reviews in healthcare, PRISMA has limited applicability for reviews in conservation and environmental management. We highlight 12 key problems with the application of PRISMA to this field, including an overemphasis on meta-analysis and no consideration for other synthesis methods. We introduce ROSES (RepOrting standards for Systematic Evidence Syntheses), a pro forma and flow diagram designed specifically for systematic reviews and systematic maps in the field of conservation and environmental management. We describe how ROSES solves the problems with PRISMA. We outline the key benefits of our approach to designing ROSES, in particular the level of detail and inclusion of rich guidance statements. We also introduce the extraction of meta-data that describe key aspects of the conduct of the review. Collated together, this summary record can help to facilitate rapid review and appraisal of the conduct of a systematic review or map, potentially speeding up the peer-review process. We present the results of initial road testing of ROSES with systematic review experts, and propose a plan for future development of ROSES.

A measurement tool to assess systematic reviews (SRs) 2 (AMSTAR 2) allows for deriving the overall confidence in an SR. We investigated how authors derived the overall confidence rating and whether different schemes lead to different results. We compared three different schemes (original 7-item scheme, a self-developed 5-item scheme, and the AMSTAR Web site) to derive the overall confidence in AMSTAR 2 using two distinct samples of SRs. Multiple bibliographic databases were searched for articles to analyze how AMSTAR 2 was applied by others. In both samples (n = 60 and n = 58), the Friedman test revealed a significant difference between the schemes (P < 0.001). The Web site scheme was the least strict one, whereas between the 5-item and 7-item scheme, no differences were found in post hoc analyses. We included 53 publications applying AMSTAR 2 identified in our literature search. Only 37 of them (70%) used the original 7-item scheme. Less than half of them (18 of 37) reported how they derived the overall rating. Authors should clearly report how they have derived the overall rating when applying AMSTAR 2. Reporting should allow for reproducing the overall ratings for editors, peer reviewers, and readers.

Objective The purpose of this umbrella review was to assess the associations between sarcopenia and adverse health‐related outcomes. Design An umbrella review of meta‐analyses of observational studies. Setting and Participants Patients with sarcopenia and controls without sarcopenia were included. Measures The PubMed, Web of Science and Embase were searched for relevant systematic review and meta‐analysis. AMSTAR and GRADE system were used for methodological quality and evidence quality assessments, respectively. Results Totally 54 outcomes extracted from 30 meta‐analyses were analyzed. Twenty out of 21 prognostic outcomes indicated that sarcopenia was significantly associated with poorer prognosis of gastric cancer, hepatocellular cancer, urothelial cancer, head and neck cancer, hematological malignancy, pancreatic cancer, breast cancer, colorectal cancer, lung cancer, esophageal cancer, and ovarian cancer. Besides, 10 out of 16 postoperative outcomes suggested that sarcopenia significantly increased the risk of multiple postoperative complications and prolonged the length of hospitalization of patients with digestive cancer. In age‐related outcomes, sarcopenia significantly increased the risk of dysphagia, cognitive impairment, fractures, falls, hospitalization, and all‐cause mortality of elderly populations. Moreover, sarcopenia was also associated with higher level of albuminuria, risk of depression, and several metabolic diseases. Conclusions and Implications Sarcopenia significantly affected a wide range of adverse health‐related outcomes, particularly in patients of tumor and elderly populations. Because evidences of most outcomes were rated as “low” and “very low,” more prospective cohort studies are required in the future. Sarcopenia significantly affected a wide range of adverse health‐related outcomes, particularly in patients of tumor and elderly populations. Besides, associations between sarcopenia and risk of metabolic diseases, depression and albuminuria were also noticeable.

Background Systematic reviews (SRs) in the field of neuropathic pain (NeuP) are increasingly important for decision-making. However, methodological flaws in SRs can reduce the validity of conclusions. Hence, it is important to assess the methodological quality of NeuP SRs critically. Additionally, it remains unclear which assessment tool should be used. We studied the methodological quality of SRs published in the field of NeuP and compared two assessment tools. Methods We systematically searched 5 electronic databases to identify SRs of randomized controlled trials of interventions for NeuP available up to March 2015. Two independent reviewers assessed the methodological quality of the studies using the Assessment of Multiple Systematic Reviews (AMSTAR) and the revised AMSTAR (R-AMSTAR) tools. The scores were converted to percentiles and ranked into 4 grades to allow comparison between the two checklists. Gwet’s AC1 coefficient was used for interrater reliability assessment. Results The 97 included SRs had a wide range of methodological quality scores (AMSTAR median (IQR): 6 (5–8) vs. R-AMSTAR median (IQR): 30 (26–35)). The overall agreement score between the 2 raters was 0.62 (95% CI 0.39–0.86) for AMSTAR and 0.62 (95% CI 0.53–0.70) for R-AMSTAR. The 31 Cochrane systematic reviews (CSRs) were consistently ranked higher than the 66 non-Cochrane systematic reviews (NCSRs). The analysis of individual domains showed the best compliance in a comprehensive literature search (item 3) on both checklists. The results for the domain that was the least compliant differed: conflict of interest (item 11) was the item most poorly reported on AMSTAR vs. publication bias assessment (item 10) on R-AMSTAR. A high positive correlation between the total AMSTAR and R-AMSTAR scores for all SRs, as well as for CSRs and NCSRs, was observed. Conclusions The methodological quality of analyzed SRs in the field of NeuP was not optimal, and CSRs had a higher quality than NCSRs. Both AMSTAR and R-AMSTAR tools produced comparable quality ratings. Our results point out to weaknesses in the methodology of existing SRs on interventions for the management NeuP and call for future improvement by better adherence to analyzed quality checklists, either AMSTAR or R-AMSTAR.

To assess the methodological quality and the consideration of heterogeneity in systematic reviews (SRs). We conducted a methodological study (CRD42019134904) and searched three databases from January 2010 to July 2019. Interventional SRs with a statistically significant meta-analysis of at least four randomized controlled trials in advanced cancer patients were included. A MeaSurement Tool to Assess Systematic Reviews (AMSTAR) 2 was used to evaluate the SRs’ methodological quality. The consideration of heterogeneity was categorized in clinical or/and methodological heterogeneity and not explored. From 6234 identified references, 261 SRs were included. Most SRs had a critically low quality (230, 88.1%). The majority of them (209, 80.1%) was classified as critically low because of non-registration (222, 85.1%) combined with the non-reporting of excluded full-texts and missing justifications for exclusion (218, 83.5%). Heterogeneity in trial results was not explored at all in 51 (19.5%) SRs whereas clinical heterogeneity was considered in 117 (44.8%), methodological heterogeneity in 13 (5.0%), and both clinical and methodological heterogeneity in 80 (30.7%) SRs. The consideration of these findings in trainings for review authors and peer reviewers could improve the awareness of quality criteria and the quality of future SRs. PROSPERO-ID: CRD42019134904

The objective of the study was to assess the inter-rater reliability (IRR) of AMSTAR and ROBIS in judging individual domains and overall methodological quality/risk of bias of systematic reviews, the concurrent validity of the tools, and the time required to apply them. This is a cross-sectional study. Five raters independently read 31 systematic reviews and applied AMSTAR and ROBIS. Fleiss' k for multiple raters for individual domains and overall methodological quality/risk of bias was calculated. Similar domains assessed by both tools and final scores were matched to explore the concurrent validity, using the Kendall tau correlation. IRR ranged from fair to perfect for AMSTAR and from moderate to substantial for ROBIS. Kappa for overall quality/risk of bias was 0.73 (95% confidence interval [CI] 0.65–0.81) for AMSTAR and 0.64 (95% CI 0.54–0.74) for ROBIS. We judged most of the reviews at intermediate quality with AMSTAR (53%), while judgments were split in high (53%) and low (47%) risk of bias with ROBIS. The correlation between judgments on similar domains ranged from moderate to high, while it was fair on the overall judgment (K = 0.35, 95% CI 0.21–0.49). The mean time to complete ROBIS was about double that for AMSTAR. AMSTAR and ROBIS offer similar IRR but differ in their construct and applicability.