Explore the vast range of titles available.

Coronavirus Disease 2019 (COVID-19) is a rapidly evolving global pandemic for which more than a thousand clinical trials have been registered to secure therapeutic effectiveness, expeditiously. Most of these are single-center non-randomized studies rather than multi-center, randomized controlled trials. Single-arm trials have several limitations and may be conducted when spontaneous improvement is not anticipated, small placebo effect exists, and randomization to a placebo is not ethical. In an emergency where saving lives takes precedence, it is ethical to conduct trials with any scientifically proven design, however, safety must not be compromised. A phase II or III trial can be conducted directly in a pandemic with appropriate checkpoints and stopping rules. COVID-19 has two management paradigms- antivirals, or treatment of its complications. Simultaneous assessment of two different treatments can be done using 2 × 2 factorial schema. World Health Organization’s SOLIDARITY trial is a classic example of the global research protocol which can evaluate the preferred treatment to combat COVID-19 pandemic. Short of that, a trial design must incorporate the practicality of the intervention used, and an appropriate primary endpoint which should ideally be a clinical outcome. Collaboration between institutions is needed more than ever to successfully execute and accrue in randomized trials.

To assess the inter-rater reliability (IRR) and usability of the risk of bias in nonrandomized studies of interventions tool (ROBINS-I). We designed a cross-sectional study. Five raters independently applied ROBINS-I to the nonrandomized cohort studies in three systematic reviews on vaccines, opiate abuse, and rehabilitation. We calculated Fleiss' Kappa for multiple raters as a measure of IRR and discussed the application of ROBINS-I to identify difficulties and possible reasons for disagreement. Thirty one studies were included (195 evaluations). IRRs were slight for overall judgment (IRR 0.06, 95% CI 0.001 to 0.12) and individual domains (from 0.04, 95% CI −0.04 to 0.12 for the domain “selection of reported results” to 0.18, 95% CI 0.10 to 0.26 for the domain “deviation from intended interventions”). Mean time to apply the tool was 27.8 minutes (SD 12.6) per study. The main difficulties were due to poor reporting of primary studies, misunderstanding of the question, translation of questions into a final judgment, and incomplete guidance. We found ROBINS-I difficult and demanding, even for raters with substantial expertise in systematic reviews. Calibration exercises and intensive training before its application are needed to improve reliability.

Although the number of quasi-experiments conducted by health researchers has increased in recent years, there clearly remains unrealized potential for using these methods for causal evaluation of health policies and programs globally. This article proposes five prescriptions for capturing the full value of quasi-experiments for health research. First, new funding opportunities targeting proposals that use quasi-experimental methods should be made available to a broad pool of health researchers. Second, administrative data from health programs, often amenable to quasi-experimental analysis, should be made more accessible to researchers. Third, training in quasi-experimental methods should be integrated into existing health science graduate programs to increase global capacity to use these methods. Fourth, clear guidelines for primary research and synthesis of evidence from quasi-experiments should be developed. Fifth, strategic investments should be made to continue to develop new innovations in quasi-experimental methodologies. Tremendous opportunities exist to expand the use of quasi-experimental methods to increase our understanding of which health programs and policies work and which do not. Health researchers should continue to expand their commitment to rigorous causal evaluation with quasi-experimental methods, and international institutions should increase their support for these efforts.

Background: The major component of a systematic review is assessment of the methodologic quality and bias of randomized and nonrandomized trials. While there are multiple instruments available to assess the methodologic quality and bias for randomized controlled trials (RCTs), there is a lack of extensively utilized instruments for observational studies, specifically for interventional pain management (IPM) techniques. Even Cochrane review criteria for randomized trials is considered not to be a “gold standard,” but merely an indication of the current state of the art review methodology. Recently a specific instrument to assess the methodologic quality of randomized trials has been developed for interventional techniques. Objectives: Our objective was to develop an IPM specific instrument to assess the methodological quality of nonrandomized trials or observational studies of interventional techniques. Methods: The item generation for the instrument was based on a definition of quality, to the extent to which the design and conduct of the trial were congruent with the objectives of the study. Applicability was defined as the extent to which procedures produced by the study could be applied using contemporary IPM techniques. Multiple items based on Cochrane review criteria and Interventional Pain Management Techniques – Quality Appraisal of Reliability and Risk of Bias Assessment for Nonrandomized Studies (IPM-QRBNR) were utilized. Results: A total of 16 items were developed which formed the IPM-QRBNR tool. The assessment was performed in multiple stages. The final assessment was 4 nonrandomized studies. The inter-rater agreement was moderate to good for IPM-QRBNR criteria. Limitations: Limited validity or accuracy assessment of the instrument and the large number of items to be scored were limitations. Conclusion: We have developed a new comprehensive instrument to assess the methodological quality of nonrandomized studies of interventional techniques. This instrument provides extensive information specific to interventional techniques is useful in assessing the methodological quality and bias of observational studies of interventional techniques. Key words: Methodological quality assessment, evidence-based medicine, comparative effectiveness research, Cochrane Reviews, interventional techniques, risk of bias assessment, nonrandomized trials, observational studies

Some medical scientists argue that only data from randomized controlled trials (RCTs) are trustworthy. They claim data from natural experiments and administrative data sets are always spurious and cannot be used to evaluate health policies and other population-wide phenomena in the real world. While many acknowledge biases caused by poor study designs, in this article we argue that several valid designs using administrative data can produce strong findings, particularly the interrupted time series (ITS) design. Many policy studies neither permit nor require an RCT for cause-and-effect inference. Framing our arguments using Campbell and Stanley’s classic research design monograph, we show that several “quasi-experimental” designs, especially interrupted time series (ITS), can estimate valid effects (or non-effects) of health interventions and policies as diverse as public insurance coverage, speed limits, hospital safety programs, drug abuse regulation and withdrawal of drugs from the market. We further note the recent rapid uptake of ITS and argue for expanded training in quasi-experimental designs in medical and graduate schools and in post-doctoral curricula.

Randomized noncomparative trials (RNCTs) promise reduced accrual requirements vs randomized controlled comparative trials because RNCTs do not enroll a control group and instead compare outcomes to historical controls or prespecified estimates. We hypothesized that RNCTs often suffer from two methodological concerns: (1) lack of interpretability due to group-specific inferences in nonrandomly selected samples and (2) misinterpretation due to unlicensed between-group comparisons lacking prespecification. The purpose of this study was to characterize RNCTs and the incidence of these two methodological concerns. We queried PubMed and Web of Science on September 14, 2023, to conduct a meta-epidemiological analysis of published RNCTs in any field of medicine. Trial characteristics and the incidence of methodological concerns were manually recorded. We identified 70 RNCTs published from 2002 to 2023. RNCTs have been increasingly published over time (slope = 0.28, 95% CI 0.17–0.39, P < .001). Sixty trials (60/70, 86%) had a lack of interpretability for the primary endpoint due to group-specific inferences. Unlicensed between-group comparisons were present in 36 trials (36/70, 51%), including in the primary conclusion of 31 trials (31/70, 44%), and were accompanied by significance testing in 20 trials (20/70, 29%). Only five (5/70, 7%) trials were found to have neither of these flaws. Although RNCTs are increasingly published over time, the primary analysis of nearly all published RNCTs in the medical literature was unsupported by their fundamental underlying methodological assumptions. RNCTs promise group-specific inference, which they are unable to deliver, and undermine the primary advantage of randomization, which is comparative inference. The ongoing use of the RNCT design in lieu of a traditional randomized controlled comparative trial should therefore be reconsidered.

We aimed to review how ‘Risk of Bias In Non-randomized Studies–of Interventions’ (ROBINS-I), a Cochrane risk of bias assessment tool, has been used in recent systematic reviews. Database and citation searches were conducted in March 2020 to identify recently published reviews using ROBINS-I. Reported ROBINS-I assessments and data on how ROBINS-I was used were extracted from each review. Methodological quality of reviews was assessed using AMSTAR 2 (‘A MeaSurement Tool to Assess systematic Reviews’). Of 181 hits, 124 reviews were included. Risk of bias was serious/critical in 54% of assessments on average, most commonly due to confounding. Quality of reviews was mostly low, and modifications and incorrect use of ROBINS-I were common, with 20% reviews modifying the rating scale, 20% understating overall risk of bias, and 19% including critical-risk of bias studies in evidence synthesis. Poorly conducted reviews were more likely to report low/moderate risk of bias (predicted probability 57% [95% CI: 47–67] in critically low-quality reviews, 31% [19–46] in high/moderate-quality reviews). Low-quality reviews frequently apply ROBINS-I incorrectly, and may thus inappropriately include or give too much weight to uncertain evidence. Readers should be aware that such problems can lead to incorrect conclusions in reviews.

Backgrounds Syntheses of non-randomized studies of interventions (NRSIs) and randomized controlled trials (RCTs) are increasingly used in decision-making. This study aimed to summarize when NRSIs are included in evidence syntheses of RCTs, with a particular focus on the methodological issues associated with combining NRSIs and RCTs. Methods We searched PubMed to identify clinical systematic reviews published between 9 December 2017 and 9 December 2022, randomly sampling reviews in a 1:1 ratio of Core and non-Core clinical journals. We included systematic reviews with RCTs and NRSIs for the same clinical question. Clinical scenarios for considering the inclusion of NRSIs in eligible studies were classified. We extracted the methodological characteristics of the included studies, assessed the concordance of estimates between RCTs and NRSIs, calculated the ratio of the relative effect estimate from NRSIs to that from RCTs, and evaluated the impact on the estimates of pooled estimates when NRSIs are included. Results Two hundred twenty systematic reviews were included in the analysis. The clinical scenarios for including NRSIs were grouped into four main justifications: adverse outcomes ( n  = 140, 63.6%), long-term outcomes ( n  = 36, 16.4%), the applicability of RCT results to broader populations ( n  = 11, 5.0%), and other ( n  = 33, 15.0%). When conducting a meta-analysis, none of these reviews assessed the compatibility of the different types of evidence prior, 203 (92.3%) combined estimates from RCTs and NRSIs in the same meta-analysis. Of the 203 studies, 169 (76.8%) used crude estimates of NRSIs, and 28 (13.8%) combined RCTs and multiple types of NRSIs. Seventy-seven studies (35.5%) showed “qualitative disagree” between estimates from RCTs and NRSIs, and 101 studies (46.5%) found “important difference”. The integration of NRSIs changed the qualitative direction of estimates from RCTs in 72 out of 200 studies (36.0%). Conclusions Systematic reviews typically include NRSIs in the context of assessing adverse or long-term outcomes. The inclusion of NRSIs in a meta-analysis of RCTs has a substantial impact on effect estimates, but discrepancies between RCTs and NRSIs are often ignored. Our proposed recommendations will help researchers to consider carefully when and how to synthesis evidence from RCTs and NRSIs.

•First known systematic review reporting the impact of pharmacist immunizers on vaccination rates.•Limited number of randomized controlled trials and high quality studies.•Pharmacist involvement in immunization services increased vaccine coverage rates. Underutilization of vaccination programs remains a significant public health concern. Pharmacists serve as educators, facilitators, and in some jurisdictions, as administrators of vaccines. Though pharmacists have been involved with immunizations in various ways for many years, there has yet to be a systematic review assessing the impact of pharmacists as immunizers in these three roles. To complete a systematic review of the literature on the impact of pharmacists as educators, facilitators, and administrators of vaccines on immunization rates. We identified 2825 articles searching the following databases from inception until October 2015: PubMed, EMBASE, Cochrane Libraries, Cumulative Index to Nursing and Allied Health Literature, International Pharmaceutical Abstracts, Google Scholar. Grey literature was identified through use of the Canadian Agency for Drugs and Technology in Health “Grey Matters” search tool. Content from relevant journals and references of included studies were also searched. Inclusion criteria were clinical or epidemiologic studies in which pharmacists were involved in the immunization process. Studies were excluded if no comparator was reported. Two reviewers independently completed data extraction and bias assessments using standardized forms. Thirty-six studies were included in the review, 22 assessed the role of pharmacists as educators and/or facilitators and 14 assessed their role as administrators of vaccines. All studies reviewed found an increase in vaccine coverage when pharmacists were involved in the immunization process, regardless of role (educator, facilitator, administrator) or vaccine administered (e.g., influenza, pneumococcal), when compared to vaccine provision by traditional providers without pharmacist involvement. Limitations of the results include the large number of non-randomized trials and the heterogeneity between study designs. Pharmacist involvement in immunization, whether as educators, facilitators, or administrators of vaccines, resulted in increased uptake of immunizations. PROSPERO Registration: CRD42013005067.

Research comparing the adherence to food-based dietary guidelines (FBDGs) across countries with different socio-economic status is lacking, which may be a concern for developing nutrition policies. The aim was to report on the adherence to FBDGs in high-income (HIC) and low-and-middle-income countries (LMIC). A systematic review with searches in six databases was performed up to June 2020. English language articles were included if they investigated a population of healthy children and adults (7–65 years), using an observational or experimental design evaluating adherence to national FBDGs. Findings indicate that almost 40% of populations in both HIC and LMIC do not adhere to their national FBDGs. Fruit and vegetables (FV) were most adhered to and the prevalence of adhering FV guidelines was between 7% to 67.3%. HIC have higher consumption of discretionary foods, while results were mixed for LMIC. Grains and dairy were consumed below recommendations in both HIC and LMIC. Consumption of animal proteins (>30%), particularly red meat, exceeded the recommendations. Individuals from HIC and LMIC may be falling short of at least one dietary recommendation from their country’s guidelines. Future health policies, behavioral-change strategies, and dietary guidelines may consider these results in their development.