Explore the vast range of titles available.

The aim of this study was to evaluate how often the European Medicines Agency (EMA) has authorized drugs based on nonrandomized studies and whether there is an association between treatment effects and EMA preference for further testing in randomized clinical trials (RCTs). We reviewed all initial marketing authorizations in the EMA database on human medicines between 1995 and 2015 and included authorizations granted without randomized data. We extracted data on treatment effects and EMA preference for further testing in RCTs. Of 723 drugs, 51 were authorized based on nonrandomized data. These 51 drugs were licensed for 71 indications. In the 51 drug-indication pairs with no preference for further RCT testing, effect estimates were large [odds ratio (OR): 12.0 (95% confidence interval {CI}: 8.1–17.9)] compared to effect estimates in the 20 drug-indication pairs for which future RCTs were preferred [OR: 4.3 (95% CI 2.8–6.6)], with a significant difference between effects (P = 0.0005). Nonrandomized data were used for 7% of EMA drug approvals. Larger effect sizes were associated with greater likelihood of approval based on nonrandomized data alone. We did not find a clear treatment effect threshold for drug approval without RCT evidence.

Quasi-experimental designs are gaining popularity in epidemiology and health systems research—in particular for the evaluation of health care practice, programs, and policy—because they allow strong causal inferences without randomized controlled experiments. We describe the concepts underlying five important quasi-experimental designs: Instrumental Variables, Regression Discontinuity, Interrupted Time Series, Fixed Effects, and Difference-in-Differences designs. We illustrate each of the designs with an example from health research. We then describe the assumptions required for each of the designs to ensure valid causal inference and discuss the tests available to examine the assumptions.

There is a growing trend to include nonrandomized studies of interventions (NRSIs) in meta-analyses of randomized controlled trials (RCTs) for health decision-making. The study aimed to quantify the impact of integrating NRSI on the evidence derived from RCTs within the same systematic review. We searched PubMed for systematic reviews published between December 9, 2017, and December 9, 2022, that included both RCTs and NRSIs under the same outcome. Using the DerSimonian–Laird random-effects model, we reanalyzed the pooled estimates to compare those derived from RCTs with those from combined RCTs and NRSIs. We examined changes in point estimates, subgroup differences, statistical heterogeneity, and the weight of RCTs in pooled estimates. Results were defined as being in qualitative agreement if both estimates demonstrated statistical significance in the same direction or if neither achieved statistical significance. A total of 220 eligible systematic reviews were identified and 217 meta-analyses were reanalyzed. Qualitative disagreement between RCTs only and pooled estimates combining RCTs and NRSIs was observed in 78 meta-analyses (35.9%), of which 69 (88.5%) gained statistical significance after the inclusion of NRSIs. Point estimates in 58 meta-analyses (26.7%) failed to meet predefined agreement criteria, and statistically significant subgroup differences between RCTs and NRSIs were identified in 32 meta-analyses (14.8%). The incorporation of NRSIs raised the heterogeneity from 21.8% to 36.9%, whereas RCTs accounted for a median weight of 33.9% in the pooled estimates. These findings highlight the need for caution in conducting and interpreting meta-analyses combining RCTs and NRSIs, particularly in scenarios where RCTs yield nonsignificant results whereas the inclusion of NRSIs achieves statistical significance. Although randomized controlled trials (RCTs) remain the gold standard for clinical evidence, they are often insufficient to address complex clinical questions. Nonrandomized studies of interventions (NRSIs), leveraging real-world clinical data, are increasingly used to supplement RCT findings. Despite growing interest in integrating NRSIs into meta-analyses with RCTs, the clinical and statistical implications of this approach remain uncertain. To address this gap, we conducted a systematic evaluation of how NRSI inclusion impacts meta-analytic results by analyzing 220 systematic reviews that combined RCTs and NRSIs under the same outcome. Our analysis revealed that incorporating NRSIs altered effect estimates in over one-third of cases, with 88.5% of meta-analyses achieving statistical significance only after NRSI inclusion–a finding with critical implications for decision-making. In addition, NRSI integration elevated statistical heterogeneity, although RCTs accounted for less than one-third of the weight in pooled estimates. These findings collectively underscore the necessity for robust evaluation and cautious interpretation when merging NRSI data with RCTs in meta-analyses. [Display omitted] •Including NRSIs in meta-analyses of RCTs altered the estimates in more than one-third of the studies.•The inclusion of NRSIs increased statistical heterogeneity of the pooled estimates. •The median weight of the RCTs in the pooled estimates was approximately one-third.

Rigorous and transparent bias assessment is a core component of high-quality systematic reviews. We assess modifications to existing risk of bias approaches to incorporate rigorous quasi-experimental approaches with selection on unobservables. These are nonrandomized studies using design-based approaches to control for unobservable sources of confounding such as difference studies, instrumental variables, interrupted time series, natural experiments, and regression-discontinuity designs. We review existing risk of bias tools. Drawing on these tools, we present domains of bias and suggest directions for evaluation questions. The review suggests that existing risk of bias tools provide, to different degrees, incomplete transparent criteria to assess the validity of these designs. The paper then presents an approach to evaluating the internal validity of quasi-experiments with selection on unobservables. We conclude that tools for nonrandomized studies of interventions need to be further developed to incorporate evaluation questions for quasi-experiments with selection on unobservables.

We evaluated whether a diagnostic stewardship initiative consisting of ASP preauthorization paired with education could reduce false-positive hospital-onset (HO) Clostridioides difficile infection (CDI). Single center, quasi-experimental study. Tertiary academic medical center in Chicago, Illinois. Adult inpatients were included in the intervention if they were admitted between October 1, 2016, and April 30, 2018, and were eligible for C. difficile preauthorization review. Patients admitted to the stem cell transplant (SCT) unit were not included in the intervention and were therefore considered a contemporaneous noninterventional control group. The intervention consisted of requiring prescriber attestation that diarrhea has met CDI clinical criteria, ASP preauthorization, and verbal clinician feedback. Data were compared 33 months before and 19 months after implementation. Facility-wide HO-CDI incidence rates (IR) per 10,000 patient days (PD) and standardized infection ratios (SIR) were extracted from hospital infection prevention reports. During the entire 52 month period, the mean facility-wide HO-CDI-IR was 7.8 per 10,000 PD and the SIR was 0.9 overall. The mean ± SD HO-CDI-IR (8.5 ± 2.0 vs 6.5 ± 2.3; P < .001) and SIR (0.97 ± 0.23 vs 0.78 ± 0.26; P = .015) decreased from baseline during the intervention. Segmented regression models identified significant decreases in HO-CDI-IR (Pstep = .06; Ptrend = .008) and SIR (Pstep = .1; Ptrend = .017) trends concurrent with decreases in oral vancomycin (Pstep < .001; Ptrend < .001). HO-CDI-IR within a noninterventional control unit did not change (Pstep = .125; Ptrend = .115). A multidisciplinary, multifaceted intervention leveraging clinician education and feedback reduced the HO-CDI-IR and the SIR in select populations. Institutions may consider interventions like ours to reduce false-positive C. difficile NAAT tests.

To review the literature and obtain preferences and perceptions from experts regarding the role of randomized studies (RSs) and nonrandomized studies (NRSs) in systematic reviews of intervention effects. Scoping review and survey of experts. Using levels of certainty developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group, experts expressed their preferences about the use of RS and NRS in health syntheses. Of 189 respondents, 123 had the expertise required to answer the questionnaire; 116 provided their extent of agreement with approaches to use NRS with RS. Most respondents would include NRS when RS was unfeasible (83.6%) or unethical (71.5%) and a majority to maximize the body of evidence (66.3%), compare results in NRS and RS (53.5%) and to identify subgroups (51.7%). Sizable minorities would include NRS and RS to address the effect of randomization (29.5%) or because the question being addressed was a public-health intervention (36.5%). In summary of findings tables, most respondents would include both bodies of evidence–in two rows in the same table—when RS provided moderate, low, or very-low certainty evidence; even when RS provided high certainty evidence, a sizable minority (25%) would still present results from both bodies of evidence. Very few (3.6%) would, under realistic circumstances, pool RS and NRS results. Most experts would include both RS and NRS in the same review under a wide variety of circumstances, but almost all would present results of two bodies of evidence separately.

To outline issues of importance to analytic approaches to the synthesis of quasi-experiments (QEs) and to provide a statistical model for use in analysis. We drew on studies of statistics, epidemiology, and social-science methodology to outline methods for synthesis of QE studies. The design and conduct of QEs, effect sizes from QEs, and moderator variables for the analysis of those effect sizes were discussed. Biases, confounding, design complexities, and comparisons across designs offer serious challenges to syntheses of QEs. Key components of meta-analyses of QEs were identified, including the aspects of QE study design to be coded and analyzed. Of utmost importance are the design and statistical controls implemented in the QEs. Such controls and any potential sources of bias and confounding must be modeled in analyses, along with aspects of the interventions and populations studied. Because of such controls, effect sizes from QEs are more complex than those from randomized experiments. A statistical meta-regression model that incorporates important features of the QEs under review was presented. Meta-analyses of QEs provide particular challenges, but thorough coding of intervention characteristics and study methods, along with careful analysis, should allow for sound inferences.

To identify variables that must be coded when synthesizing primary studies that use quasi-experimental designs. All quasi-experimental (QE) designs. When designing a systematic review of QE studies, potential sources of heterogeneity—both theory-based and methodological—must be identified. We outline key components of inclusion criteria for syntheses of quasi-experimental studies. We provide recommendations for coding content-relevant and methodological variables and outlined the distinction between bivariate effect sizes and partial (i.e., adjusted) effect sizes. Designs used and controls used are viewed as of greatest importance. Potential sources of bias and confounding are also addressed. Careful consideration must be given to inclusion criteria and the coding of theoretical and methodological variables during the design phase of a synthesis of quasi-experimental studies. The success of the meta-regression analysis relies on the data available to the meta-analyst. Omission of critical moderator variables (i.e., effect modifiers) will undermine the conclusions of a meta-analysis.

PurposeTo compare the efficacy and safety of thulium laser VapoResection of the prostate (ThuVaRP) versus standard traditional transurethral resection of the prostate (TURP) or plasmakinetic resection of prostate (PKRP) for benign prostatic obstruction.MethodsSystematic searches were performed in the Medline, EMBASE, the Cochrane Library, Web of Science, and CNKI in December 2017. The outcomes of demographic and clinical characteristics, perioperative variables, complications, and postoperative efficacy including International Prostate Symptom Score (IPSS), quality of life (QoL), maximum flow rate (Qmax), and postvoid residual (PVR) were assessed.Results16 studies were selected in the meta-analysis including nine randomized controlled trials (RCTs) and seven non-RCTs. Among of them, nine studies compared ThuVaRP with PKRP, while seven studies compared ThuVaRP with TURP. It seemed that ThuVaRP needed longer operation time than TURP (WMD = 6.41, 95% CI 1.38–11.44, p = 0.01) and PKRP (WMD = 10.15, 95% CI 5.20–15.10, p < 0.0001). ThuVaRP was associated with less serum hemoglobin decreased, catheterization time, and the length of hospital stay compared with TURP (WMD = − 0.58, 95% CI − 0.77 to 0.38, p < 0.00001; WMD = − 1.89, 95% CI − 2.67 to 1.11, p < 0.00001; WMD = − 2.25, 95% CI − 2.91 to 1.60, p < 0.00001) and PKRP (WMD = − 0.28, 95% CI − 0.46 to 0.10, p = 0.002; WMD = − 1.88, 95% CI − 2.87 to 0.89, p = 0.0002; WMD = − 2.08, 95% CI − 2.63 to 1.54, p<0.00001). According to our assessment, there was no significantly difference in postoperative efficacy.ConclusionsThe pooled data indicated that ThuVaRP had a nearly efficacy to TURP and PKRP based on IPSS, QoL, Qmax, and PVR. Although ThuVaRP was associated with longer operation time, it got distinct superiority on serum hemoglobin decreased, catheterization time, and hospital stay.

Background Little evidence is available on searches for non-randomized studies (NRS) in bibliographic databases within the framework of systematic reviews. For instance, it is currently unclear whether, when searching for NRS, effective restriction of the search strategy to certain study types is possible. The following challenges need to be considered: 1) For non-randomized controlled trials (NRCTs): whether they can be identified by established filters for randomized controlled trials (RCTs). 2) For other NRS types (such as cohort studies): whether study filters exist for each study type and, if so, which performance measures they have. The aims of the present analysis were to identify and validate existing NRS filters in MEDLINE as well as to evaluate established RCT filters using a set of MEDLINE citations. Methods Our analysis is a retrospective analysis of study filters based on MEDLINE citations of NRS from Cochrane reviews. In a first step we identified existing NRS filters. For the generation of the reference set, we screened Cochrane reviews evaluating NRS, which covered a broad range of study types. The citations of the studies included in the Cochrane reviews were identified via the reviews’ bibliographies and the corresponding PubMed identification numbers (PMIDs) were extracted from PubMed. Random samples comprising up to 200 citations (i.e. 200 PMIDs) each were created for each study type to generate the test sets. Results A total of 271 Cochrane reviews from 41 different Cochrane groups were eligible for data extraction. We identified 14 NRS filters published since 2001. The study filters generated between 660,000 and 9.5 million hits in MEDLINE. Most filters covered several study types. The reference set included 2890 publications classified as NRS for the generation of the test sets. Twelve test sets were generated (one for each study type), of which 8 included 200 citations each. None of the study filters achieved sufficient sensitivity (≥ 92%) for all of the study types targeted. Conclusions The performance of current NRS filters is insufficient for effective use in daily practice. It is therefore necessary to develop new strategies (e.g. new NRS filters in combination with other search techniques). The challenges related to NRS should be taken into account.