Marginal reliability coefficient of IRT
reliability(x)Estimated marginal reliability coefficients.
A model fit object from either IRTest_Dich, IRTest_Poly, IRTest_Cont, or IRTest_Mix.
Seewoo Li cu@yonsei.ac.kr
In accordance with the concept of reliability in classical test theory (CTT), this function calculates the IRT reliability coefficients.
The basic concept and formula of the reliability coefficient can be expressed as follows (Kim & Feldt, 2010):
An observed score of Item \(i\), \(X_i\), is decomposed as the sum of a true score \(T_i\) and an error \(e_i\). Then, with the assumption of \(\sigma_{T_{i}e_{j}}=\sigma_{e_{i}e_{j}}=0\), the reliability coefficient of a test is defined as; $$\rho_{TX}=\rho_{XX^{'}}=\frac{\sigma_{T}^{2}}{\sigma_{X}^{2}}=\frac{\sigma_{T}^{2}}{\sigma_{T}^{2}+\sigma_{e}^{2}}=1-\frac{\sigma_{e}^{2}}{\sigma_{X}^{2}}$$
See May and Nicewander (1994) for the specific formula used in this function.
For the coefficient on the \(\theta\) scale, this function calculates the parallel-forms reliability (Green et al., 1984; Kim, 2012): $$ \rho_{\hat{\theta} \hat{\theta}^{'}} =\frac{\sigma_{E\left(\hat{\theta}\mid \theta \right )}^{2}}{\sigma_{E\left(\hat{\theta}\mid \theta \right )}^{2}+E\left( \sigma_{\hat{\theta}|\theta}^{2} \right)} =\frac{1}{1+E\left(I\left(\hat{\theta}\right)^{-1}\right)} $$ This assumes that \(\sigma_{E\left(\hat{\theta}\mid \theta \right )}^{2}=\sigma_{\theta}^{2}=1\). Although the formula is often employed in several IRT studies and applications, the underlying assumption may not be true.
Green, B.F., Bock, R.D., Humphreys, L.G., Linn, R.L., & Reckase, M.D. (1984). Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement, 21(4), 347–360.
Kim, S. (2012). A note on the reliability coefficients for item response model-based ability estimates. Psychometrika, 77(1), 153-162.
Kim, S., Feldt, L.S. (2010). The estimation of the IRT reliability coefficient and its lower and upper bounds, with comparisons to CTT reliability statistics. Asia Pacific Education Review, 11, 179–188.
May, K., Nicewander, A.W. (1994). Reliability and information functions for percentile ranks. Journal of Educational Measurement, 31(4), 313-325.
# \donttest{
data <- DataGeneration(N=500, nitem_D = 10)$data_D
# Analysis
M1 <- IRTest_Dich(data)
# Reliability coefficients
reliability(M1)
# }
Run the code above in your browser using DataLab