sirt (version 0.36-30)

rm.hrm: Hierachical Rater Model Based on Signal Detection Theory

Description

This function estimates a version of the hierarchical rater model (HRM) based on signal detection theory (SDT; DeCarlo, 2005; DeCarlo, Kim & Johnson, 2011).

Usage

rm.hrm(dat, pid, rater, Qmatrix = NULL, theta.k = seq(-9, 9, len = 30), 
    est.a.item = FALSE, est.c.rater = "n", est.d.rater = "n", 
    d.min = 0.5, d.max = 100, d.start = 3, max.increment = 1, numdiff.parm = 0.001, 
    maxdevchange = 0.1, globconv = .001, maxiter = 1000, msteps = 4, mstepconv = 0.001)

## S3 method for class 'rm.hrm':
summary(object,...)

Arguments

dat

Original data frame. Ratings on variables must be in rows, i.e. every row corresponds to a person-rater combination.

pid

Person identifier.

rater

Rater identifier

Qmatrix

An optional Q-matrix. If this matrix is not provided, then by default the ordinary scoring of categories (from 0 to the maximum score of $K$) is used.

theta.k

A grid of theta values for the ability distribution.

est.a.item

Should item parameters $a_i$ be estimated?

est.c.rater

Type of estimation for item-rater parameters $c_{ir}$ in the signal detection model. Options are n (no estimation), e (set all parameters equal to each other), i (item wise estmation), r (rater wise

est.d.rater

Type of estimation of $d$ parameters. Options are the same as in est.c.rater.

d.min

Minimal $d$ parameter to be estimated

d.max

Maximal $d$ parameter to be estimated

d.start

Starting value of $d$ parameters

max.increment

Maximum increment of item parameters during estimation

numdiff.parm

Numerical differentiation step width

maxdevchange

Maximum relative deviance change as a convergence criterion

globconv

Maximum parameter change

maxiter

Maximum number of iterations

msteps

Maximum number of iterations during an M step

mstepconv

Convergence criterion in an M step

object

Object of class rm.hrm

...

Further arguments to be passed

Value

• A list with following entries:
• deviance
  Deviance
• ic
  Information criteria and number of parameters
• item
  Data frame with item parameters
• rater
  Data frame with rater parameters
• person
  Data frame with person parameters: EAP and corresponding standard errors
• EAP.rel
  EAP reliability
• sigma
  Standard deviation of the trait
• tau.item
  Item parameters $\tau_{ik}$
• se.tau.item
  Standard error of item parameters $\tau_{ik}$
• a.item
  Item slopes $a_i$
• se.a.item
  Standard error of item slopes $a_i$
• c.rater
  Rater parameters $c_{irk}$
• se.c.rater
  Standard error of rater severity parameter $c_{irk}$
• d.rater
  Rater slope parameter $d_{ir}$
• se.d.rater
  Standard error of rater slope parameter $d_{ir}$
• f.yi.qk
  Individual likelihood
• f.qk.yi
  Individual posterior distribution
• probs
  Item probabilities at grid theta.k
• n.ik
  Expected counts
• maxK
  Maximum number of categories
• procdata
  Processed data
• iter
  Number of iterations
• ...
  Further values

Details

The specification of the model follows DeCarlo et al. (2011). The second level models the ideal rating (latent response) $\eta =0, ...,K$ of person $p$ on item $i$ $$P( \eta_{pi} = \eta | \theta_p ) \propto exp( a_{i} q_{ik} \theta_p - \tau_{ik} )$$ At the first level, the ratings $X_{pir}$ for person $p$ on item $i$ and rater $r$ are modelled as a signal detection model $$P( X_{pir} \leq k | \eta_{pi} ) = G( c_{irk} - d_{ir} \eta_{pi} )$$ where $G$ is the logistic distribution function and the categories are $k=1,\ldots , K+1$. The thresholds $c_{irk}$ can be further restricted to $c_{irk} = c_{k}$ (est.c.rater="e"), $c_{irk} = c_{ik}$ (est.c.rater="i") or $c_{irk} = c_{ir}$ (est.c.rater="r"). The same holds for the rater precision parameters $d_{ir}$.

References

DeCarlo, L. T. (2005). A model of rater behavior in essay grading based on signal detection theory. Journal of Educational Measurement, 42, 53-76. DeCarlo, T., Kim, Y., & Johnson, M. S. (2011). A hierarchical rater model for constructed responses, with a signal detection rater model. Journal of Educational Measurement, 48, 333-356.

See Also

The facets rater model can be estimated with rm.facets.

Examples

#############################################################################
# EXAMPLE 1: HRM
#############################################################################
data(data.ratings1)
dat <- data.ratings1

# Model 1: Partial Credit Model: no rater effects
mod1 <- rm.hrm( dat[ , paste0( "k",1:5) ] , rater=dat$rater , 
            pid=dat$idstud , est.c.rater="n" , est.d.rater="n" , 
            d.start=100 , maxiter=15)
summary(mod1)
            
# Model 2: Generalized Partial Credit Model: no rater effects
mod2 <- rm.hrm( dat[ , paste0( "k",1:5) ] , rater=dat$rater , 
            pid=dat$idstud , est.c.rater="n" , est.d.rater="n" , 
            est.a.item =TRUE , d.start=100 , maxiter=15)
summary(mod2)
            
# Model 3: Equal effects in SDT
mod3 <- rm.hrm( dat[ , paste0( "k",1:5) ] , rater=dat$rater , 
            pid=dat$idstud , est.c.rater="e" , est.d.rater="e" , maxiter=15)
summary(mod3)

# Model 4: Rater effects in SDT
mod4 <- rm.hrm( dat[ , paste0( "k",1:5) ] , rater=dat$rater , 
            pid=dat$idstud , est.c.rater="r" , est.d.rater="r" , maxiter=15)
summary(mod4)