catIrt (version 0.5.1)

FI: Calculate Expected and Observed Fisher Information for IRT Models

Description

FI calculates expected and/or observed Fisher information for various IRT models given a vector of ability values, a vector/matrix of item parameters, and an IRT model. It also calculates test information and expected/observed standard error of measurement.

Usage

FI( params, theta, type = c("expected", "observed"), resp = NULL )
# S3 method for brm
FI( params, theta, type = c("expected", "observed"), resp = NULL )
# S3 method for grm
FI( params, theta, type = c("expected", "observed"), resp = NULL )

Arguments

params

numeric: a vector or matrix of item parameters. If specified as a matrix, the rows must index the items, and the columns must designate the item parameters. Furthermore, if calculating expected information, the number of rows must match the number of columns of resp. The class of params must match the model: either "brm" or "grm". For the binary response model, params must either be a 3-dimensional vector or a 3-column matrix. See Details for more information.

theta

numeric: a vector of ability values, one for each simulee. If calculating expected information, the length of theta must match the number of rows of resp, unless theta is a scalar, in which case resp could also be a vector of length nrow(params).

type

character: a character string indicating the type of information, either "expected" or "observed". For the 1-parameter and 2-parameter binary response model (of class "brm" with the third column of params set to 0), both "expected" and "observed" information are identical. See Details for more information.

resp

numeric: either a \(N \times J\) matrix (where \(N\) indicates the number of simulees and \(J\) indicates the number of items), a \(N\) length vector (if there is only one item) or a \(J\) length vector (if there is only one simulee). For the binary response model ("brm"), resp must solely contain 0s and 1s. For the graded response model ("grm"), resp must solely contain integers \(1, \ldots, K\), where \(K\) is the number of categories, as indicated by the dimension of params.

Value

FI, FI.brm, and FI.grm return a list of the following elements:

item

either: (1) a \(N \times J\) matrix of item information for each simulee to each item; (2) a \(J\)-length vector of item information for one simulee to each item; or (3) an \(N\)-length vector of item information for all simulees to one item, depending on the dimensions of params and theta.

test

an \(N\)-length vector of test information, one for each simulee. Test information is the sum of item information across items. See Details for more information.

sem

an \(N\)-length vector of expected or observed standard error of measurement for each simulee, which is the inverse-square-root of test information. See Details for more information.

type

either "observed" or "expected", indicating the type of information calculated.

Details

The function FI returns item information, test information, and standard error of measurement for the binary response model ("brm") or the graded response model ("grm"). If the log likelihood is twice differentiable, expected Fisher information is the negative, expected, second derivative of the log likelihood with respect to the parameter. For the binary response model, expected item information simplifies to the following:

$$ I(\theta_i | a_j, b_j, c_j) = \frac{\left(\frac{\partial p_{ij}}{\partial \theta_i}\right)^2}{p_{ij}(1 - p_{ij})} $$

where \(\partial p_{ij}/\partial \theta_i\) is the partial derivative of \(p_{ij}\) with respect to \(\theta\), and \(p_{ij}\) is the probability of response, as indicated in the help page for simIrt.

For the graded response model, expected item information simplifies to the following:

$$ I(\theta_i | a_j, b_{j1}, \ldots, b_{j(k - 1)}) = \sum_k\frac{\left(\frac{\partial P_{ijk}}{\partial \theta_i}\right)^2}{P_{ijk}}$$

where \(\partial P_{ijk}/\partial \theta_i\) is the partial derivative of \(P_{ijk}\) with respect to \(\theta\), and \(P_{ijk}\) is the probability of responding in category k as indicated in the help page for simIrt. See van der Linden and Pashley (2010).

Observed information is the negative second derivative of the log-likelihood. For the binary response model ("brm") with 2-parameters (such that the third column of the parameter matrix is set to 0), observed and expected information are identical because the second derivative of their log-likelihoods do not contain observed data. See Baker and Kim (2004), pp. 66 -- 69.

For all models, test information is defined as the following:

$$ T(\theta_i) = \sum_jI_j(\theta_i) $$

where \(I(\theta_i)_j\) is shorthand for Fisher information of simulee \(i\) on item \(j\). Finally, the standard error of measurement (SEM) is the inverse, square-root of test information. FI is frequently used to select items in a CAT and to estimate the precision of \(\hat{\theta}_i\) after test termination.

References

Baker, F. B., & Kim, S.-H. (2004). Item Response Theory: Parameter Estimation Techniques, Second Edition. New York, NY: Marcel Dekker, Inc.

Dodd, B. G., De Ayala, R. J., & Koch, W. R. (1995). Computerized adaptive testing with polytomous items. Applied Psychological Measurement, 19, 5 -- 22.

Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for Psychologists. Mahway, NJ: Lawrence Erlbaum Associates.

Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22, 79 -- 86.

van der Linden, W. J. & Pashley, P. J. (2010). Item selection and ability estimation in adaptive testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Elements of Adaptive Testing. New York, NY: Springer.

See Also

catIrt, KL, simIrt

Examples

# NOT RUN {
#########################
# Binary Response Model #
#########################

## 1 ##
set.seed(888)
# generating random theta:
theta <- rnorm(20)
# generating an item bank under a 2-parameter binary response model:
b.params <- cbind(a = runif(100, .5, 1.5), b = rnorm(100, 0, 2), c = 0)
# simulating responses using random theta:
b.mod <- simIrt(params = b.params, theta = theta, mod = "brm")

# you can indicate class of params or extract it from simIrt object:
class(b.params) <- "brm"

# calculating expected and observed information:
e.info <- FI(params = b.params, theta = theta, type = "expected")
o.info <- FI(params = b.params, theta = theta, type = "observed", resp = b.mod$resp)

# 2-parameter model, so e.info will be equal to o.info:
all(signif(e.info$item) == signif(o.info$item))


## 2 ##
# generating an item bank under a 3-parameter binary response model:
b.params2 <- cbind(a = runif(100, .5, 1.5), b = rnorm(100, 0, 2), c = .2)
# simulating responses using pre-specified thetas:
b.mod2 <- simIrt(params = b.params2, mod = "brm")

# calculating expected and observed information:
# (if you don't indicate class, you can extract from simIrt object)
e.info2 <- FI(params = b.params2, theta = b.mod2$theta, type = "expected")
o.info2 <- FI(params = b.params2, theta = b.mod2$theta, type = "observed",
                                  resp = b.mod2$resp)

# 3-parameter model, so e.info will not be equal to o.info:
all(signif(e.info2$item) == signif(o.info2$item))


## 3 ##
# if theta is a scalar, item will be a vector and test will be a scalar:
e.info3 <- FI(params = b.params2, theta = 0, type = "expected")
dim(e.info3$item)       # no dimension because it's a vector
length(e.info3$item)    # of length equal to the number of items

# if params is a vector, item will be a matrix with one row:
e.info4 <- FI(params = c(1, 2, 0), theta = c(1, 2), type = "expected")
dim(e.info4$item)

# if you don't class params, FI will assume a binary response model.


#########################
# Graded Response Model #
#########################
set.seed(999)
# generating random theta
theta <- rnorm(10)
# generating an item bank under a graded response model:
g.params <- cbind(a = runif(30, .5, 1.5), b1 = rnorm(30), b2 = rnorm(30),
                                          b3 = rnorm(30), b4 = rnorm(30))
# you can sort the parameters yourself:
g.params <- cbind(g.params[ , 1],
                  t(apply(g.params[ ,2:dim(g.params)[2]], MARGIN = 1,
                                                          FUN = sort)))
# simulating responses using random theta:
g.mod <- simIrt(params = g.params, theta = theta, mod = "grm")

# calculating expected and observed information:
class(g.params) <- "grm"   # always indicate model or extract from simulation.
e.info5 <- FI(params = g.params, theta = theta, type = "expected")
o.info5 <- FI(params = g.params, theta = theta, type = "observed", resp = g.mod$resp)

# grm, so e.info will not be equal to o.info:
all(signif(e.info5$item) == signif(o.info5$item))

# if thet is a vector and params is a vector, item will be a J x N matrix:
dim(e.info5$item)

# if you don't want to sort the parameters, you can extract from simIrt object:
e.info6 <- FI(params = g.mod$params[ , -1], theta = g.mod$theta, type = "expected")

# but you first need to remove column 1 (the item number column).
# }