Explore the vast range of titles available.

This study examined a naturalistic, districtwide implementation of an automated writing evaluation (AWE) software program called MI Write in elementary schools. We specifically examined the degree to which aspects of MI Write were implemented, teacher and student attitudes towards MI Write, and whether MI Write usage along with other predictors like demographics and writing self-efficacy explained variability in students’ performance on a proximal and distal measure of writing performance. The participants included 1935 students in Grades 3–5 and 135 writing teachers from 14 elementary schools in a mid-Atlantic school district. Findings indicated that though MI Write was somewhat under-utilized, teachers and students held positive attitudes towards the AWE system. Usage of MI Write had a mixed and limited predictive effect on outcomes: The number of essays written had a small predictive effect on state test performance for Grades 3 and 5; gain on revision had a moderate predictive effect on posttest writing quality and a small predictive effect for Grade 5 state test performance. Students’ average AWE scores showed consistently moderate to large predictive effects for all outcomes. Interpreted in light of the underlying architecture of MI Write, findings have implications for other school districts considering implementing AWE as well as the design of AWE systems intended to support the teaching and learning of writing.

Although much attention has been given to rater effects in rater-mediated assessment contexts, little research has examined the overall stability of leniency and severity effects over time. This study examined longitudinal scoring data collected during three consecutive administrations of a large-scale, multi-state summative assessment program. Multilevel models were used to assess the overall extent of rater leniency/severity during scoring and examine the extent to which leniency/severity effects were stable across the three administrations. Model results were then applied to scaled scores to estimate the impact of the stability of leniency/severity effects on students' scores. Results showed relative scoring stability across administrations in mathematics. In English language arts, short constructed response items showed evidence of slightly increasing severity across administrations, while essays showed mixed results: evidence of both slightly increasing severity and moderately increasing leniency over time, depending on trait. However, when model results were applied to scaled scores, results revealed rater effects had minimal impact on students' scores.

Process-based approaches to writing tend to overlook the self-regulatory skills and motivational beliefs required for proficient writing (Harris, Santangelo, & Graham, 2008) and do not provide the support many students need to develop into effective writers (Graham, Harris, & Mason, 2005; Harris, Graham, & Mason, 2006). Additionally, restricted writing opportunities preclude the sustained deliberate practice students need to develop expertise in writing (Kellogg & Whiteford, 2009). This study examined an intervention that incorporated the self-regulated strategy development model (SRSD, Graham & Harris, 1993) with the automated writing evaluation (AWE) program NC Write. An embedded quasi-experimental mixed methods design was used to determine the impact of the intervention on students’ argumentative writing performance, knowledge, and self-efficacy. Middle school students (N=829) participated in one of three conditions: NC Write + traditional writing instruction, NC Write + SRSD instruction, or a comparison condition. Results of multi-level models that controlled for pretest performance and predicted posttest performance averaging across students and within teachers showed that students in the NC Write + SRSD instruction condition produced posttest essays that were of a higher quality, longer, and included more basic elements of argumentative essays than students in the other two conditions. Students in the NC Write + traditional writing instruction condition produced higher-quality essays than students in the comparison condition at posttest. Students in the NC Write + SRSD instruction condition identified more essay elements at posttest, though there were no between-condition differences in writing knowledge of substantive processes or in students’ writing self-efficacy at posttest. Additional multi-level models were specified to include all essays written by treatment condition students and examine the shape of growth in writing performance. Results showed that students’ growth in writing quality, essay length, and essay elements was best represented by a quadratic growth model. On average, students’ growth in writing performance reached a plateau following the fourth essay written during the intervention. Differences in rates of change and deceleration in writing quality, essay length, and essay elements were not significantly different between the two treatment conditions. Survey results showed students and teachers held generally favorable opinions of NC Write. Interview results determined that NC Write as well as the overall writing intervention had acceptable social validity. Qualitative data analysis revealed that NC Write provided a framework for deliberate writing practice. In this framework students’ growth in writing performance is supported by a cycle of learning, practice, and feedback. NC Write enabled deliberate practice by affording writing quality feedback, efficiency, and evidence of growth, and supporting teachers’ writing instruction and students’ intrinsic motivation. Limitations of the framework included some aspects of feedback, limited lesson data, and lack of a plagiarism scanner in NC Write. Implications from these findings support integrating SRSD instruction with an AWE program to support teacher implementation of the SRSD model and more efficiently provide students with the strategy instruction, practice opportunities, and feedback needed to develop proficiency in writing. Recommendations are provided for AWE programs to better support students’ maintenance of writing quality growth.