Journal Article

Different Tests, Different Answers: The Stability of Teacher Value-Added Estimates Across Outcome Measures

Papay, John P.

2011

Overview

Recently, educational researchers and practitioners have turned to value-added models to evaluate teacher performance. Although value-added estimates depend on the assessment used to measure student achievement, the importance of outcome selection has received scant attention in the literature. Using data from a large, urban school district, I examine whether value-added estimates from three separate reading achievement tests provide similar answers about teacher performance. I find moderate-sized rank correlations, ranging from 0.15 to 0.58, between the estimates derived from different tests. Although the tests vary to some degree in content, scaling, and sample of students, these factors do not explain the differences in teacher effects. Instead, test timing and measurement error contribute substantially to the instability of value-added estimates across tests.

• Publisher: SAGE Publications,American Educational Research Association
• Subject: Academic Achievement / Achievement Tests / Correlation / Correlation analysis / Correlations / Educational evaluation / Educational Policy / Educational research / Educational Researchers / Educational Testing / English teachers / Error of Measurement / Estimates / Evaluation Methods / Evaluation Problems / Health care outcome assessment / High Stakes Tests / Longitudinal Studies / Measurement / Measurement errors / Merit Pay / Modeling / Outcomes of education / Reading Achievement / Reading Improvement / Reading Tests / School districts / Standardized tests / Stanford Achievement Tests / Teacher Effectiveness / Teacher Evaluation / Teachers / Teaching, Learning, and Human Development / Test Reliability / Test scores / Test Selection / Testing Problems / Urban areas / Urban Schools / Value added / Value Added Models