Explore the vast range of titles available.

This paper analyzes 12 recent within-study comparisons contrasting causal estimates from a randomized experiment with those from an observational study sharing the same treatment group. The aim is to test whether different causal estimates result when a counterfactual group is formed, either with or without random assignment, and when statistical adjustments for selection are made in the group from which random assignment is absent. We identify three studies comparing experiments and regression-discontinuity (RD) studies. They produce quite comparable causal estimates at points around the RD cutoff. We identify three other studies where the quasi-experiment involves careful intact group matching on the pretest. Despite the logical possibility of hidden bias in this instance, all three cases also reproduce their experimental estimates, especially if the match is geographically local. We then identify two studies where the treatment and nonrandomized comparison groups manifestly differ at pretest but where the selection process into treatment is completely or very plausibly known. Here too, experimental results are recreated. Two of the remaining studies result in correspondent experimental and nonexperimental results under some circumstances but not others, while two others produce different experimental and nonexperimental estimates, though in each case the observational study was poorly designed and analyzed. Such evidence is more promising than what was achieved in past within-study comparisons, most involving job training. Reasons for this difference are discussed.

Theory predicts that regression discontinuity (RD) provides valid causal inference at the cutoff score that determines treatment assignment. One purpose of this paper is to test RD's internal validity across 15 studies. Each of them assesses the correspondence between causal estimates from an RD study and a randomized control trial (RCT) when the estimates are made at the same cutoff point where they should not differ asymptotically. However, statistical error, imperfect design implementation, and a plethora of different possible analysis options, mean that they might nonetheless differ. We test whether they do, assuming that the bias potential is greater with RDs than RCTs. A second purpose of this paper is to investigate the external validity of RD by exploring how the size of the bias estimates varies across the 15 studies, for they differ in their settings, interventions, analyses, and implementation details. Both Bayesian and frequentist meta-analysis methods show that the RD bias is below 0.01 standard deviations on average, indicating RD's high internal validity. When the study-specific estimates are shrunken to capitalize on the information the other studies provide, all the RD causal estimates fall within 0.07 standard deviations of their RCT counterparts, now indicating high external validity. With unshrunken estimates, the mean RD bias is still essentially zero, but the distribution of RD bias estimates is less tight, especially with smaller samples and when parametric RD analyses are used.

The folate pathway plays a crucial role in the regeneration and repair of the adult CNS after injury. Here, we have shown in rodents that such repair occurs at least in part through DNA methylation. In animals with combined spinal cord and sciatic nerve injury, folate-mediated CNS axon regeneration was found to depend on injury-related induction of the high-affinity folate receptor 1 (Folr1). The activity of folate was dependent on its activation by the enzyme dihydrofolate reductase (Dhfr) and a functional methylation cycle. The effect of folate on the regeneration of afferent spinal neurons was biphasic and dose dependent and correlated closely over its dose range with global and gene-specific DNA methylation and with expression of both the folate receptor Folr1 and the de novo DNA methyltransferases. These data implicate an epigenetic mechanism in CNS repair. Folic acid and possibly other nontoxic dietary methyl donors may therefore be useful in clinical interventions to promote brain and spinal cord healing. If indeed the benefit of folate is mediated by epigenetic mechanisms that promote endogenous axonal regeneration, this provides possible avenues for new pharmacologic approaches to treating CNS injuries.

In observational studies of patients hospitalized for heart failure (HHF), risk of death is highest immediately after discharge and decreases over time. It is unclear whether this population risk trajectory reflects (1) lowering of individual patient mortality risk with increasing time from index hospitalization or (2) temporal changes in population case-mix with earlier postdischarge death for “sicker” patients. Survival rate and longitudinal models were used to estimate temporal changes in postdischarge all-cause mortality risk in 3,993 HHF patients discharged alive in the Efficacy of Vasopressin Antagonism in Heart Failure Outcome Study with Tolvaptan (EVEREST) trial. After median follow-up of 9.9 months, 971 patients died (24.2%). Predicted mortality rate decreased from 15.9 per 100 patient-years immediately after discharge to 13.4 at 30 days and 12.8 at 90 days; mortality rate increased steadily thereafter. Risk variation between quintiles of risk was considerably larger than the temporal variation within risk strata. In a longitudinal model serially reassessing predicted patient mortality risk after each follow-up visit using data collected at these visits, predicted mortality risk increased during the 90 days preceding subsequent heart failure readmission and then followed a postdischarge trajectory similar to the index admission. In conclusion, although there is transiently elevated individual patient risk in the 90 days before and after discharge, the patient's individual risk profile, rather than temporal change in risk relative to hospitalization, remains the main determinant of mortality. For purposes of reducing all-cause mortality in HF patients, preventative and therapeutic measures may be best implemented as long-term interventions for high mortality risk patients based on serial risk assessments, irrespective of recent hospitalization.

Abstract Clearinghouses develop scientific criteria that they then use to vet existing research studies on a program to reach a verdict about how evidence-based it is. This verdict is then recorded on a website in hopes that stakeholders in science, public policy, the media, and even the general public, will consult it. This paper (1) compares the causal design and analysis preferences of 13 clearinghouses that assess the effectiveness of social and behavioral development programs, (2) estimates how consistently these clearinghouses rank the same program, and then (3) uses case studies to probe why their conclusions differ. Most clearinghouses place their highest value on randomized control trials, but they differ in how they treat program implementation, quasi-experiments, and whether their highest program ratings require effects of a given size that independently replicate or that temporally persist. Of the 2525 social and behavioral development programs sampled over clearinghouses, 82% (n = 2069) were rated by a single clearinghouse. Of the 297 programs rated by two clearinghouses, agreement about program effectiveness was obtained for about 55% (n = 164), but the clearinghouses agreed much more on program ineffectiveness than effectiveness. Most of the inconsistency is due to clearinghouses’ differences in requiring independently replicated and/or temporally sustained effects. Without scientific consensus about matters like these, “evidence-based” will remain more of an aspiration than achievement in the social and behavioral sciences.

A decade ago, the Society of Prevention Research (SPR) endorsed a set of standards for evidence related to research on prevention interventions. These standards (Flay et al., Prevention Science 6:151–175, 2005 ) were intended in part to increase consistency in reviews of prevention research that often generated disparate lists of effective interventions due to the application of different standards for what was considered to be necessary to demonstrate effectiveness. In 2013, SPR’s Board of Directors decided that the field has progressed sufficiently to warrant a review and, if necessary, publication of “the next generation” of standards of evidence. The Board convened a committee to review and update the standards. This article reports on the results of this committee’s deliberations, summarizing changes made to the earlier standards and explaining the rationale for each change. The SPR Board of Directors endorses “The Standards of Evidence for Efficacy, Effectiveness, and Scale-up Research in Prevention Science: Next Generation.”

To provide recommendations for the selection of comparators for randomized controlled trials of health-related behavioral interventions. The National Institutes of Health Office of Behavioral and Social Science Research convened an expert panel to critically review the literature on control or comparison groups for behavioral trials and to develop strategies for improving comparator choices and for resolving controversies and disagreements about comparators. The panel developed a Pragmatic Model for Comparator Selection in Health-Related Behavioral Trials. The model indicates that the optimal comparator is the one that best serves the primary purpose of the trial but that the optimal comparator's limitations and barriers to its use must also be taken into account. We developed best practice recommendations for the selection of comparators for health-related behavioral trials. Use of the Pragmatic Model for Comparator Selection in Health-Related Behavioral Trials can improve the comparator selection process and help resolve disagreements about comparator choices.

This study assessed some ways in which schools, neighborhoods, nuclear families, and friendship groups jointly contribute to positive change during early adolescence. For each context, existing theory was used to develop a multiattribute index that should promote successful development. Descriptive analyses showed that the four resulting context indices were only modestly intercorrelated at the individual student level (N = 12,398), but clustered more tightly at the school and neighborhood levels (N = 23 and 151 respectively). Only for aggregated units did knowing the developmental capacity of any one context strongly predict the corresponding capacity of the other contexts. Analyses also revealed that each context facilitated individual change in a success index that tapped into student academic performance, mental health, and social behavior. However, individual context effects were only modest in size over the 19 months studied and did not vary much by context. The joint influence of all four contexts was cumulatively large, however, and because it was generally additive in form, no constellation of contexts was identified whose total effect reliably surpassed the sum of its individual context main effects. These results suggest that achieving significant population changes in multidimensional student growth during early adolescence most likely requires both theory and interventions that are explicitly pan-contextual.

The sharp regression discontinuity design (RDD) has three key weaknesses compared to the randomized clinical trial (RCT). It has lower statistical power, it is more dependent on statistical modeling assumptions, and its treatment effect estimates are limited to the narrow subpopulation of cases immediately around the cutoff, which is rarely of direct scientific or policy interest. This paper examines how adding an untreated comparison to the basic RDD structure can mitigate these three problems. In the example we present, pretest observations on the posttest outcome measure are used to form a comparison RDD function. To assess its performance as a supplement to the basic RDD, we designed a within-study comparison that compares causal estimates and their standard errors for (1) the basic posttest-only RDD, (2) a pretest-supplemented RDD, and (3) an RCT chosen to serve as the causal benchmark. The two RDD designs are constructed from the RCT, and all analyses are replicated with three different assignment cutoffs in three American states. The results show that adding the pretest makes functional form assumptions more transparent. It also produces causal estimates that are more precise than in the posttest-only RDD, but that are nonetheless larger than in the RCT. Neither RDD version shows much bias at the cutoff, and the pretest-supplemented RDD produces causal effects in the region beyond the cutoff that are very similar to the RCT estimates for that same region. Thus, the pretest-supplemented RDD improves on the standard RDD in multiple ways that bring causal estimates and their standard errors closer to those of an RCT, not just at the cutoff, but also away from it.