catR (version 3.1)

simulateRespondents: Simulation of multiple examinees of adaptive tests

Description

This command runs a set of adaptive tests, for a given item bank, a set of ability levels, a possible matrix of item responses, and several lists of CAT parameters (starting items, stopping rule, provisional and final ability estimators).

Usage

simulateRespondents(thetas, itemBank, responsesMatrix = NULL, model = NULL, 
  	maxItems = 50, cbControl = NULL, rmax = 1, Mrmax = "restricted", 
  	start = list(fixItems = NULL, seed = NULL, nrItems = 1, theta = 0, 
  	halfRange = 2, startSelect = "MFI"),test = list(method = "BM", 
  	priorDist = "norm", priorPar = c(0,1), range = c(-4, 4), D = 1, 
  	parInt = c(-4, 4, 33), itemSelect = "MFI", infoType = "observed", 
  	randomesque = 1, AP = 1), stop = list(rule = "length", thr = 20, 
  	alpha = 0.05), final = list(method = "BM", priorDist = "norm", 
  	priorPar = c(0,1), range = c(-4, 4), D = 1, parInt = c(-4, 4, 33), 
  	alpha = 0.05), save.output = FALSE, output = c("", "catR", "csv"))
## S3 method for class 'catResult':
print(x, ...)
## S3 method for class 'catResult':
plot(x, type = "all", deciles = "theta", save.plot = FALSE, 
  	save.options = c("", "plot", "pdf"), res = 300, ...)

Arguments

thetas

numeric: a vector of true ability values for which a CAT must be generated for each component.

itemBank

numeric: a suitable matrix of item parameters (possibly augmented by group memebership for content balancing). See Details.

responsesMatrix

numeric: either NULL (default) or a suitable matrix of item responses. See Details.

model

either NULL (default) for dichotomous models, or any suitable acronym for polytomous models. Possible values are "GRM", "MGRM", "PCM", "GPCM", "RSM" and "NRM". See

maxItems

numeric: the maximal number of items to be administered (default is 50).

cbControl

either a list of accurate format to control for content balancing, or NULL. See Details.

rmax

numeric: the maximum exposure rate (default is 1).

Mrmax

character: the method for controling maximum exposure rate. Possible values are "restricted"(default) or "IE". See Details.

start

a list with the options for starting the adaptive test. See Details.

test

a list with the options for provisional ability estimation and next item selection. See Details.

stop

a list with the options of the stopping rule. See Details.

final

a list with the options for final ability estimation. See Details.

save.output

logical: should the output be saved in an external text file? (default is FALSE).

output

character: a vector of three components. The first component is either the file path to save the output or "" (default), the second component is either the initial part of the name of the output file or "catR", and the third comp

x

an object of class "cat", typically an output of simulateRespondents function.

type

character: the type of plot to display. Possible values are "all" (default), "trueEst", "expRate", "cumExpRate", "cumNumberItems", "expRatePara", "condBias",

deciles

wheter the deciles number ("deciles") or the mean ability level per decile ("theta") will be used in the axis of the plots.

save.plot

logical: should the plot be saved in an external figure? (default is FALSE).

save.options

character: a vector of three components. The first component is either the file path or "" (default), the second component is the name of the output file or ,"plot" (default), and the third component is the file extension, either

res

numeric: the resolution for JPEG figures (default value is 300).

...

other generic arguments to be passed to print and plot functions.

Value

• The function simulateRespondents returns a list of class "catResult" with the following arguments:
• thetas
  the value of the thetas argument.
• itemBank
  the value of the itemBank argument.
• responsesMatrix
  the value of the responsesMatrix argument.
• model
  the value of the model argument.
• maxItems
  the value of the maxItems argument.
• cbControl
  the value of the cbControl argument.
• rmax
  the value of the rmax argument.
• Mrmax
  the value of the Mrmax argument.
• start
  the value of the start argument.
• test
  the value of the test argument.
• stop
  the value of the stop argument.
• final
  the value of the final argument.
• save.output
  the value of the save.output argument.
• output
  the value of the output argument.
• estimatedThetas
  a vector with (final) estimated ability levels.
• correlation
  the correlation between the thetas vector and estimated ability levels.
• bias
  the value of the bias between true and estimated ability levels.
• RMSE
  the value of the RMSE between true and estimated ability levels.
• thrOK
  a vector indicating wether the respondents finished the test satisfying (1) or not (0) the stop criteria.
• exposureRates
  a vector with empirical exposure rates of all item in the bank.
• testLength
  the mean test length.
• overlap
  the item overlap rate.
• numberItems
  a vector with the lengths of each adaptive test (i.e. the number of items administered).
• condTheta
  a vector with the mean ability level per decile.
• condBias
  a vector with conditional mean bias per decile.
• condRMSE
  a vector with conditional RMSE per decile.
• condnItems
  a vector with conditional mean test length per decile.
• condSE
  a vector with conditional mean standard error per decile.
• condthrOK
  a vector with conditional proportion of respondents that finish the test satisfying the stop criteria per decile.
• ndecile
  a vector with the number of respondents per decile.
• final.values.df
  a data frame with true ability levels, final ability estimates and standard errors, and test lengths.
• responses.df
  a data.frame with all items administered, all item responses and all provisional ability estimates. -99 is displayed when the actual test length is smaller than maxItems for those item positions where no item was administered as the stop criterion was already reached.
• start.time
  the CPU time at the start of the CAT generation.
• finish.time
  the CPU time at the end of all CAT generations.

code

randomCAT

enumerate

1. "trueEst": the scatterplot of true vs. estimated ability levels.

item

• "expRate": the exposure rates of the items, having ranked them according to these exposure rates.
• "cumExpRate": the cumulative exposure rates of the items, having ranked them according to these exposure rates.
• "cumNumberItems": the test length as a function of cumulative percent of examinees. This plot is not available when rule is 'length'.
• "expRatePara": the scatterplot of item exposure rates vs. item discrimination parameters. This plot is not available when model is 'PCM' or 'NRM', as in those IRT models there are no discrimination parameters.
• "condBias": the conditional bias of ability estimation as a function of the deciles of the true ability levels.
• "condRMSE": the conditional RMSE of ability estimation as a function of the deciles of the true ability levels.
• "numberItems": the conditional test length as a function of the deciles of the true ability levels. This plot is not available when rule is 'length'.
• "sError": the conditional standard error of ability estimation as a function of the deciles of the true ability levels.
• "condThr": the conditional proportions of CATs satisfying the 'precision' or 'classification' stopping rule, as a function of the deciles of the true ability levels. This plot is not available when rule is 'length'.

bold

Examples

Details

The simulateRespondents function permits to generate several adaptive tests to a set of respondents defined by their ability levels. It makes a repeated call to an adaptive test using an item bank specified by arguments itemBank model, and with the same start, test, stop and final lists. Content balancing can also be controled for each respondent with the cbControl argument. All arguments of simulateRespondents are used in exactly the same manner as in randomCAT (so refer to this function for further information), except the following four. First, thetas is now a vector of ability levels, and a CAT will be generated for each component of thetas. If responsesMatrix is NULL, item responses are generated from the IRT model, item bank parameters, and the ability levels. In this case, thetas can be considered as real ability levels. Otherwise, responsesMatrix must be provided as a matrix with as many rows as the length of thetas and as many columns as the number of items in itemBank. Each row contains the response pattern of one examinee whose ability level is given by the value of the corresponding component of thetas. Note that only allowable item responses can be included in responsesMatrix and missing values are not accepted. The option of providing a response matrix through responsesMatrix is considered for two possible uses:

1. post-hoc simulations: examinees provided responses to the full item bank and one wants to test the performance of a CAT with those responses,

simulations considering item parameter estimation errors: responses to the full bank are generated with the real parameters and the CAT is run is the estimated parameters and the responses from the correct model.

References

Barrada, J. R., Abad, F. J., and Veldkamp, B. P. (2009). Comparison of methods for controlling maximum exposure rates in computerized adaptive testing. Psicothema, 21, 313-320. Magis, D., and Raiche, G. (2012). Random Generation of Response Patterns under Computerized Adaptive Testing with the R Package catR. Journal of Statistical Software, 48 (8), 1-31. URL http://www.jstatsoft.org/v48/i08/ Revuelta, J., and Ponsoda, V. (1998). A comparison of item exposure control methods in computerized adaptive testing. Journal of Educational Measurement, 35, 311-327. doi: 10.1111/j.1745-3984.1998.tb00541.x van der Linden, W.J., & Veldkamp, B.P. (2004). Constraining item exposure in computerized adaptive testing with shadow tests. Journal of Educational & Behavioral Statistics, 29, 273-291. doi: 10.3102/10769986029003273

See Also

randomCAT

Examples

## Dichotomous IRT model ##

 # Loading the 'tcals' parameters 
 data(tcals)
 bank <- as.matrix(tcals[,1:4])
 
 # Creation of a starting list: 3 items, initial theta 0, bw 2
 start <- list(nrItems = 3, theta = 0, halfRange = 2)

 # Creation of 'test' list: maximum likelihood estimation and
 # progressive method
 test <- list(method = "ML", itemSelect = "progressive")

 # Creation of a stopping rule: precision criterion, standard
 # error to be reached 0.3
 stop <- list(rule = "precision", thr = 0.3)

 # Creation of 'final' list: ML estimation of final ability
 final <- list(method = "ML")

 # Generation of ten respondents
 set.seed(1)
 thetas <- rnorm(10)

 # Default CAT generations, output not saved
 res <- simulateRespondents(thetas, bank, start = start, test = test, stop = stop, 
  final = final)

 # Maximum exposure restricted to 0.8
 res2 <- simulateRespondents(thetas, bank, start = start, test = test, stop = stop, 
  final = final, rmax = 0.8)

 # Output saved
 res3 <- simulateRespondents(thetas, bank, start = start, test = test, stop = stop, 
  final = final, save.output = TRUE, output = c("C:/Program Files/", "out", "txt"))

 # With content balancing #

 # Creation of an appropriate list for content balancing
 # Equal proprotions across subgroups of items
 cbList <- list(names = c("Audio1", "Audio2", "Written1", "Written2", "Written3"), 
        props = c(0.1, 0.2, 0.2, 0.2, 0.3))

 # CAT test (same options as above)
 res4 <- simulateRespondents(thetas, tcals, start = start, test = test, stop = stop, 
  final = final, cbControl = cbList)

 # Plotting and saving output #

 # Plotting all possible panels
 plot(res)
 plot(res, deciles = "deciles")

 # Saving the plot in the "fig" pdf file in "c:/Program Files/"
 plot(res, save.plot = TRUE, save.options = c("c:/Program Files/", "fig", "pdf"))

 # Plotting the 'trueEst' type of plot
 plot(res, type = "trueEst")