Explore the vast range of titles available.

This meta-analysis synthesized recent research on strategy instruction (SI) effectiveness to estimate SI effects and their moderators for two domains: second/foreign language and self-regulated learning. A total of 37 studies (47 independent samples) for language domain and 16 studies (17 independent samples) for self-regulated learning domain contributed effect sizes for this meta-analysis. Findings indicate that the overall effects of SI were large, 0.78 and 0.87, for language and self-regulated learning, respectively. A number of context (e.g., educational level, script differences), treatment (e.g., delivery agent), and methodology (e.g., pretest) characteristics were found to moderate SI effectiveness. Notably, the moderating effects varied by language versus self-regulated learning domains. The overall results identify SI as a viable instructional tool for second/foreign language classrooms, highlight more effective SI design features, and suggest a need for a greater emphasis on self-regulated learning in SI interventions and research.

This article reports a systematic review and meta-analysis of studies that use regression discontinuity to examine the effects of placement into developmental education. Results suggest that placement into developmental education is associated with effects that are negative, statistically significant, andsubstantively large for three outcomes: (a) the probability of passing the college-level course in which remediation was needed, (b) college credits earned, and (c) attainment. Several sensitivity analyses suggest these results are not a function of particular stylized studies or the choices made in assembling the metaanalytic database. Two exploratory moderator analyses suggest that the negative effects of placement into developmental education are stronger for university students than for community college students and worse for students placed in reading or writing than in math. This work can inform debate and research on postsecondary policies and on alternative mechanisms for ensuring that college students have the skills needed to meet their goals.

In this article, the authors outline methods for using fixed and random effects power analysis in the context of meta-analysis. Like statistical power analysis for primary studies, power analysis for meta-analysis can be done either prospectively or retrospectively and requires assumptions about parameters that are unknown. The authors provide some suggestions for thinking about these parameters, in particular for the random effects variance component. The authors also show how the typically uninformative retrospective power analysis can be made more informative. The authors then discuss the value of confidence intervals, show how they could be used in addition to or instead of retrospective power analysis, and also demonstrate that confidence intervals can convey information more effectively in some situations than power analyses alone. Finally, the authors take up the question “How many studies do you need to do a meta-analysis?” and show that, given the need for a conclusion, the answer is “two studies,” because all other synthesis techniques are less transparent and/or are less likely to be valid. For systematic reviewers who choose not to conduct a quantitative synthesis, the authors provide suggestions for both highlighting the current limitations in the research base and for displaying the characteristics and results of studies that were found to meet inclusion criteria.

Non-randomised studies of the effects of interventions are critical to many areas of healthcare evaluation, but their results may be biased. It is therefore important to understand and appraise their strengths and weaknesses. We developed ROBINS-I (“Risk Of Bias In Non-randomised Studies - of Interventions”), a new tool for evaluating risk of bias in estimates of the comparative effectiveness (harm or benefit) of interventions from studies that did not use randomisation to allocate units (individuals or clusters of individuals) to comparison groups. The tool will be particularly useful to those undertaking systematic reviews that include non-randomised studies.

During the past decade, mentoring has proliferated as an intervention strategy for addressing the needs that young people have for adult support and guidance throughout their development. Currently, more than 5,000 mentoring programs serve an estimated three million youths in the United States. Funding and growth imperatives continue to fuel the expansion of programs as well as the diversification of mentoring approaches and applications. Important questions remain, however, about the effectiveness of these types of interventions and the conditions required to optimize benefits for young people who participate in them. In this article, we use metaanalysis to take stock of the current evidence on the effectiveness of mentoring programs for youth. As a guiding conceptual framework for our analysis, we draw on a developmental model of youth mentoring relationships (Rhodes, 2002, 2005). This model posits an interconnected set of processes (socialemotional, cognitive, identity) through which caring and meaningful relationships with nonparental adults (or older peers) can promote positive developmental trajectories. These processes are presumed to be conditioned by a range of individual, dyadic, programmatic, and contextual variables. Based on this model and related prior research, we anticipated that we wouldfind evidence for the effectiveness of mentoring as an approach for fostering healthy development among youth. We also expected that effectiveness would vary as a function of differences in both program practices and the characteristics of participating young people and their mentors. The meta-analysis encompassed 73 independent evaluations of mentoring programs directed toward children and adolescents published over the past decade (1999-2010). Overall, findings support the effectiveness of mentoring for improving outcomes across behavioral, social, emotional, and academic domains of young people's development. The most common pattern of benefits is for mentored youth to exhibit positive gains on outcome measures while nonmentored youth exhibit declines. It appears then that mentoring as an intervention strategy has the capacity to serve both promotion and prevention aims. Programs also show evidence of being able to affect multiple domains of youth functioning simultaneously and to improve selected outcomes of policy interest (e.g., academic achievement test scores). From a developmental standpoint, benefits of participation in mentoring programs are apparent from early childhood to adolescence and thus not confined to a particular stage of development. Similarly, although programs typically have utilized adult volunteers and focused on cultivating one-to-one relationships, those that have engaged older peers as mentors or used group formats show comparable levels of effectiveness. Collectively, these findings point toward the flexibility and broad applicability of mentoring as an approach for supporting positive youth development. Several other aspects of our findings, however, underscore a need for caution. These include a failure of evaluations to assess several key outcomes of policy interest (e.g., juvenile offending, obesity prevention) or to determine whether benefits for youth are sustained at later points in their development. More generally, we find that gains on outcome measures for the typical young person in a mentoring program have been modest (equivalent to a difference of 9 percentile points from scores of nonmentored youth on the same measures). This level of impact is within the range of effects observed for other types of interventions for children and adolescents but fails to reflect discernible improvement over the previous generation of mentoring programs (DuBois, Holloway, Valentine, & Cooper, 2002). Variability in program effectiveness, although less pronounced, also continues to be evident even after accounting for methodological differences in studies. In analyzing this variability, we find that programs have been more effective when (a) participating youth have either had preexisting difficulties (including problem behavior specifically) or been exposed to significant levels of environmental risk, (b) evaluation samples have included greater proportions of male youth, (c) there has been a good fit between the educational or occupational backgrounds of mentors and the goals of the program, (d) mentors and youth have been paired based on similarity of interests, and (e) programs have been structured to support mentors in assuming teaching or advocacy roles with youth. These findings suggest that effects may hinge to a noteworthy extent on decisions that are made regarding which youth and mentors to involve in a program and on the care with which mentoring relationships are established and then guided toward specific types of activities. Taking stock of the available evidence leads us to see value in continued support for youth mentoring programs. The argument for using mentoring as an intervention strategy is particularly strong when there is interest in promoting outcomes across multiple areas of a young person s development. For investments to yield optimal returns, however, there is a need for policy to be directed toward several critical areas of concern: (a) ensuring adherence to core practices (e.g., screening and training of mentors) that both research and common sense dictate to be essential elements of program quality, (b) facilitating ongoing refinement and strengthening of programs using the available evidence as a guide, and (c) fostering stronger collaborations between practitioners and researchers as a framework for evidence-driven dissemination and growth within the field. From a research standpoint, to support and inform these efforts there is a pressing need to (a) gauge the impact of mentoring interventions on key outcomes of policy interest and on the outcomes of participating youth at later points in their development; (b) utilize study designs and analyses that are capable of addressing the relative effectiveness of competing models and practices, the unique contributions of mentoring within more complex, multi-component interventions, and differences in youth responsiveness (including potential harmful effects for some youth); (c) investigate increasingly well-specified models of how different types of program practices and processes may be instrumental in shaping consequential features of mentoring relationships and ultimately, the realization ofparticular desired outcomes for youth; and (d) establish a research registry to improve the quality and synthesis of available evidence regarding the effectiveness of youth mentoring as an intervention strategy.

The first and most important decision in preparing any systematic review is to clearly frame the question the review team seeks to answer. However, this is not always straightforward, particularly if synthesis teams are interested in the effects of complex interventions. In this article, we discuss how to formulate good systematic review questions of complex interventions. We describe the rationale for developing well-formulated review questions and review the existing guidance on formulating review questions. We discuss that complex interventions can contain a mix of effective and ineffective (or even harmful) actions, which may interact synergistically or dysynergistically or be interdependent, and how these interactions and interdependencies need to be considered when formulating systematic review questions. We discuss complexity specifically in terms of how it relates to the type of question, the scope of the review (i.e., lumping vs. splitting debate), and specification of the intervention. We offer several recommendations to assist review authors in developing a definition for their complex intervention of interest, which is an essential first step in formulating the review question. We end by identifying areas in which future methodological research aimed at improving question formulation, especially as it relates to complex interventions, is needed.

Outcome-reporting bias occurs when primary studies do not include information about all outcomes measured in a study. When studies omit findings on important measures, efforts to synthesize the research using systematic review techniques will be biased and interpretations of individual studies will be incomplete. Outcome-reporting bias has been well documented in medicine and has been shown to lead to inaccurate assessments of the effects of medical treatments and, in some cases, to omission of reports of harms. This study examines outcome-reporting bias in educational research by comparing the reports of educational interventions from dissertations to their published versions. We find that nonsignificant outcomes were 30% more likely to be omitted from a published study than statistically significant ones.

This is the protocol for a Campbell systematic review. The objectives are as follows: To identify methods used to assess the risk of outcome reporting bias (ORB) in studies included in recent Campbell systematic reviews of intervention effects. The review will answer the following questions: What proportion of recent Campbell reviews included assessment of ORB? How did recent reviews define levels of risk of ORB (what categories, labels, and definitions did they use)? To what extent and how did these reviews use study protocols as sources of data on ORB? To what extent and how did reviews document reasons for judgments about risk of ORB? To what extent and how did reviews assess the inter‐rater reliability of ORB ratings? To what extent and how were issues of ORB considered in the review's , plain language summary, and conclusions?

Ever increasing demands for accountability, together with the proliferation of lists of evidence-based prevention programs and policies, led the Society for Prevention Research to charge a committee with establishing standards for identifying effective prevention programs and policies. Recognizing that interventions that are effective and ready for dissemination are a subset of effective programs and policies, and that effective programs and policies are a subset of efficacious interventions, SPR's Standards Committee developed overlapping sets of standards. We designed these Standards to assist practitioners, policy makers, and administrators to determine which interventions are efficacious, which are effective, and which are ready for dissemination. Under these Standards, an efficacious intervention will have been tested in at least two rigorous trials that (1) involved defined samples from defined populations, (2) used psychometrically sound measures and data collection procedures; (3) analyzed their data with rigorous statistical approaches; (4) showed consistent positive effects (without serious iatrogenic effects); and (5) reported at least one significant long-term follow-up. An effective intervention under these Standards will not only meet all standards for efficacious interventions, but also will have (1) manuals, appropriate training, and technical support available to allow third parties to adopt and implement the intervention; (2) been evaluated under real-world conditions in studies that included sound measurement of the level of implementation and engagement of the target audience (in both the intervention and control conditions); (3) indicated the practical importance of intervention outcome effects; and (4) clearly demonstrated to whom intervention findings can be generalized. An intervention recognized as ready for broad dissemination under these Standards will not only meet all standards for efficacious and effective interventions, but will also provide (1) evidence of the ability to \"go to scale\"; (2) clear cost information; and (3) monitoring and evaluation tools so that adopting agencies can monitor or evaluate how well the intervention works in their settings. Finally, the Standards Committee identified possible standards desirable for current and future areas of prevention science as the field develops. If successful, these Standards will inform efforts in the field to find prevention programs and policies that are of proven efficacy, effectiveness, or readiness for adoption and will guide prevention scientists as they seek to discover, research, and bring to the field new prevention programs and policies.

We used meta‐analysis to review 55 evaluations of the effects of mentoring programs on youth. Overall, findings provide evidence of only a modest or small benefit of program participation for the average youth. Program effects are enhanced significantly, however, when greater numbers of both theory‐based and empirically based “best practices” are utilized and when strong relationships are formed between mentors and youth. Youth from backgrounds of environmental risk and disadvantage appear most likely to benefit from participation in mentoring programs. Outcomes for youth at‐risk due to personal vulnerabilities have varied substantially in relation to program characteristics, with a noteworthy potential evident for poorly implemented programs to actually have an adverse effect on such youth. Recommendations include greater adherence to guidelines for the design and implementation of effective mentoring programs as well as more in‐depth assessment of relationship and contextual factors in the evaluation of programs.