Journal Article

A Multiple-Design, Experimental Strategy: Academic Probation Warning Letter's Impact on Student Achievement

Moss, Brian G., Yeaton, William H.

2020

Overview

We aimed to compare the findings of three research designs to bracket effect estimates of a strongly worded warning letter delivered by certified mail to students on academic probation. We embedded an experiment within a regression discontinuity design and calculated two achievement estimates, average GPA and percentage of students remaining on probation. Study participants attended a large Midwestern college. Cohen's d experimental effect size was .45. Regression discontinuity design results were validated by our experimental evidence, and outcome measures were generally statistically significant. We provided additional supportive evidence using comparative RD control group design logic. Regression point displacement design results were successfully replicated using a within-study comparison inside the experiment. In the context of probation, a diverse design, replicative approach provided considerable promise for more precise estimation of intervention effectiveness. We found no deleterious impact on reenrollment and concluded that the certified letter represents an inexpensive probation policy.

• Publisher: Routledge,Taylor & Francis, Ltd,Taylor & Francis Inc
• Subject: Academic Achievement / Academic Probation / college student achievement / College students / Community Colleges / comparative regression discontinuity / Control Groups / Cost Effectiveness / Delivery Systems / Electronic Mail / Enrollment Influences / Experiments / Grade Point Average / Intervention / Letters (Correspondence) / MEASUREMENT, STATISTICS, AND RESEARCH DESIGN / Negative Reinforcement / Parole & probation / Persuasive Discourse / randomized experiment / regression discontinuity design / regression point displacement design / Research design / School Policy / Student Attrition / Two Year College Students / within-study comparison