CDM (version 4.991-1)

cdm.est.class.accuracy: Classification Reliability in a CDM

Description

This function computes the classification accuracy and consistency by the method of Cui, Gierl and Chang (2012) and by simulation. The simulation function now only works for models of class din (i.e. the mixed DINA and DINO model) while the analytical method also works for objects of class gdina.

Usage

cdm.est.class.accuracy(cdmobj, n.sims = 0, seed = 987)

Arguments

cdmobj

Object of class din or gdina

n.sims

Number of simulated persons. If n.sims=0, then the number of persons in the original data is used as the sample size. In case of missing item responses, everytime this sample size is used. Note that the simulation does (up to now) only work for CDMs of class din.

seed

An integer which indicates the simulation seed.

Value

A data frame for MLE, MAP and MAP (Skill 1, ... , Skill $K$) classification reliability for the whole latent class pattern and marginal skill classification with following columns:

P_a

Classification accuracy (Gierl et al., 2012)

P_a_sim

Classification accuracy based on simulated data (only for din models)

P_c

Classification consistency (Gierl et al., 2012)

P_c_sim

Classification consistency based on simulated data (only for din models)

Details

The item parameters and the probability distribution of latent classes is used as the basis of the simulation. Accuracy and consistency is estimated for both MLE and MAP classification estimators. In addition, classification accuracy measures are available for the separate classification of all skills.

References

Cui, Y., Gierl, M. J., & Chang, H.-H. (2012). Estimating classification consistency and accuracy for cognitive diagnostic assessment. Journal of Educational Measurement, 49, 19-38.

Examples

## Not run: 
# #############################################################################
# # EXAMPLE 1: DINO data example
# #############################################################################
# 
# #***
# # Model 1: estimate DINO model with din
# mod1 <- din( sim.dino , q.matrix = sim.qmatrix , rule="DINO")
# # estimate classification reliability
# cdm.est.class.accuracy(  mod1 , n.sims=5000)
#   ##                P_a P_a_sim   P_c P_c_sim
#   ##   MLE        0.668   0.661 0.583   0.541
#   ##   MAP        0.807   0.785 0.717   0.670
#   ##   MAP_Skill1 0.924      NA 0.860      NA
#   ##   MAP_Skill2 0.786      NA 0.746      NA
#   ##   MAP_Skill3 0.937      NA 0.901      NA
# 
# #***
# # Model 2: estimate DINO model with gdina
# mod2 <- gdina( sim.dino , q.matrix = sim.qmatrix , rule="DINO")
# # estimate classification reliability
# cdm.est.class.accuracy(  mod2 )
#   ##                P_a   P_c
#   ##   MLE        0.675 0.598
#   ##   MAP        0.832 0.739
#   ##   MAP_Skill1 0.960 0.925
#   ##   MAP_Skill2 0.629 0.618
#   ##   MAP_Skill3 0.823 0.729
# 
# m1 <- mod1$coef[ , c("guess" , "slip" ) ]
# m2 <- mod2$coef
# m2 <- cbind( m1 , m2[ seq(1,18,2) , "est" ]  , 
#           1 - m2[ seq(1,18,2) , "est" ]  - m2[ seq(2,18,2) , "est" ]  )
# colnames(m2) <- c("g.M1" , "s.M1" , "g.M2" , "s.M2" )
#   ##   > round( m2 , 3 )
#   ##          g.M1  s.M1  g.M2  s.M2
#   ##   Item1 0.109 0.192 0.109 0.191
#   ##   Item2 0.073 0.234 0.072 0.234
#   ##   Item3 0.139 0.238 0.146 0.238
#   ##   Item4 0.124 0.065 0.124 0.009
#   ##   Item5 0.125 0.035 0.125 0.037
#   ##   Item6 0.214 0.523 0.214 0.529
#   ##   Item7 0.193 0.514 0.192 0.514
#   ##   Item8 0.246 0.100 0.246 0.100
#   ##   Item9 0.201 0.032 0.195 0.032
# # Note that s (the slipping parameter) substantially differs for Item4 
# # for DINO estimation in 'din' and 'gdina'
# ## End(Not run)