Journal Article

Proof of Reliability Convergence to 1 at Rate of Spearman–Brown Formula for Random Test Forms and Irrespective of Item Pool Dimensionality

Sijtsma, Klaas, Ellis, Jules L.

2024

Overview

It is shown that the psychometric test reliability, based on any true-score model with randomly sampled items and uncorrelated errors, converges to 1 as the test length goes to infinity, with probability 1, assuming some general regularity conditions. The asymptotic rate of convergence is given by the Spearman–Brown formula, and for this it is not needed that the items are parallel, or latent unidimensional, or even finite dimensional. Simulations with the 2-parameter logistic item response theory model reveal that the reliability of short multidimensional tests can be positively biased, meaning that applying the Spearman–Brown formula in these cases would lead to overprediction of the reliability that results from lengthening a test. However, test constructors of short tests generally aim for short tests that measure just one attribute, so that the bias problem may have little practical relevance. For short unidimensional tests under the 2-parameter logistic model reliability is almost unbiased, meaning that application of the Spearman–Brown formula in these cases of greater practical utility leads to predictions that are approximately unbiased.

• Publisher: Springer US,Springer Nature B.V
• Subject: Accuracy / Assessment / Behavioral Science and Psychology / Classical test theory / Computer Simulation / Convergence / Cronbach's alpha / Humanities / Humans / Item Response Theory / Law / Logistic Models / Models, Statistical / Original Research / Psychology / Psychometrics / Quantitative psychology / Questionnaires / Reproducibility of Results / Statistical Theory and Methods / Statistics for Social Sciences / Test Length / Test Reliability / Testing and Evaluation / Textbooks