engleski

How erroneous are we in estimating the standard error of measurement?

Modern test theory, based on the characteristic item curve, introduces the concept of test information function and the related concept of conditional standard errors of measurement for scale scores. The IRT models offer measures of error for each individual ability parameter (&#61553; ) rather than the estimate of error in the total test provided by classical test theory. Test information function, for different points of the ability continuum, depends on the item slopes (discriminating power for specified levels of ability), item's difficulty (again for specified levels of ability), and the number of items. A simulation study with a factorial 3 x 3 x 3 experimental design was conducted. The influence of three main effects was examined: a) items difficulty for the specified levels of ability, b) items discriminating power, and c) total number of items in the composite. The known true ability parameter values were compared with the ability estimates obtained in different experimental conditions. Estimates of individual ability parameters (&#61553; ) were conducted under the 2-parameter logistic model of item response theory and under the assumptions of the classical test theory. Residuals between the known true individual ability scores and their estimates under the classical and IRT models are the first dependent variable in this study. In addition, as the second dependent variable we compared the standard deviation of the residuals with the estimates of the standard error of measurement under both models. Results indicate that IRT offers a biased estimate of error when tests are short (k<10) and the items are extremely easy/difficult for specified levels of ability. Under the classical theory the standard error of measurement is biased when we have short tests composed of highly discriminative items. In all situations when the items are far too easy/difficult for specified levels of ability the larger and more realistic estimate of standard error was done under the 2-PL model. In all other situations we found smaller estimates of standard error under the item response theory.

standard error of measurement; individual ability parameters; IRT; CTT

nije evidentirano