The simulateRespondents function permits to generate several adaptive tests to a set of respondents defined by their ability levels. It makes a repeated call to an adaptive test using an item bank specified by arguments itemBank model, and with the same start, test, stop and final lists. Content balancing can also be controlled for each respondent with the cbControl argument. All arguments of simulateRespondents are used in exactly the same manner as in randomCAT (so refer to this function for further information), except the following four.First, thetas is now a vector of ability levels, and a CAT will be generated for each component of thetas. If responsesMatrix is NULL, item responses are generated from the IRT model, item bank parameters, and the ability levels. In this case, thetas can be considered as real ability levels. Otherwise, responsesMatrix must be provided as a matrix with as many rows as the length of thetas and as many columns as the number of items in itemBank. Each row contains the response pattern of one examinee whose ability level is given by the value of the corresponding component of thetas. Note that only allowable item responses can be included in responsesMatrix and missing values are not accepted. Fixing the random seed can also be done with the genSeed argument. The latter must hold as many components as the vector thetas, otherwise an error message is returned. Each component of genSeed is used to fix the seed for ach pattern generation.
The option of providing a response matrix through responsesMatrix is considered for two possible uses:
- post-hoc simulations: examinees provided responses to the full item bank and one wants to test the performance of a CAT with those responses,
- simulations considering item parameter estimation errors: responses to the full bank are generated with the real parameters and the CAT is run is the estimated parameters and the responses from the correct model.
Note that if thetas holds a single value, then the function simply calls randomCAT and returns its output instead.
Second, rmax fixes the desired maximum exposure rate for all items. Default value is 1, allowing thus items to be administered to all respondents without restrictions.
Third, the Mrmax argument fixes the method to constraint exposure rates to be smaller than the maximum allowed rate. Possible methods are the restricted method ("restricted"; Revuelta and Ponsoda, 1998) and the item-eligibility method ("IE"; van der Linden and Veldkamp, 2004). A description of both methods can be found in Barrada, Abad and Veldkamp (2009)
Fourth, if the length of thetas is greater than 1, save.output for randomCAT is fixed to "FALSE". Otherwise, the same file would be overwritten for each new respondent.
The output of simulateRespondents, as displayed by the print.catResult function, proposes summary statistics related to overall accuracy (bias, RMSE, etc), conditional accuracy per decile, and item exposure control (minimum and maximum exposure rates, test overlap rate, etc) among all generated CATs. This output can be saved when save.output is set to TRUE, and if so, three outpout files are returned: (1) one with the main summary statistics, with overall results and conditional on decile results; (2) the file with respondents' patterns, items administered and provisional ability estimates; and (3) a table with true and estimated ability levels, final standard errors and numbers of items administered per respondent. Specific information provided and saved depends on the rule used in stop.
This output can also be graphically displayed with the plot.catResult function. In addition to the output of the function and the item bank, it takes the argument type to determine which plot should be returned. Ten different single plots can be displayed:
"trueEst": the scatterplot of true vs. estimated ability levels.
"expRate": the exposure rates of the items, having ranked them according to these exposure rates.
"cumExpRate": the cumulative exposure rates of the items, having ranked them according to these exposure rates.
"cumNumberItems": the test length as a function of cumulative percent of examinees. This plot is not available when rule is 'length'.
"expRatePara": the scatterplot of item exposure rates vs. item discrimination parameters. This plot is not available when model is 'PCM' or 'NRM', as in those IRT models there are no discrimination parameters.
"condBias": the conditional bias of ability estimation as a function of the deciles of the true ability levels.
"condRMSE": the conditional RMSE of ability estimation as a function of the deciles of the true ability levels.
"numberItems": the conditional test length as a function of the deciles of the true ability levels. This plot is not available when rule is 'length'.
"sError": the conditional standard error of ability estimation as a function of the deciles of the true ability levels.
"condThr": the conditional proportions of CATs satisfying the 'precision' or 'classification' stopping rule, as a function of the deciles of the true ability levels. This plot is not available when rule is 'length'.
In addition, the value "all" (default value) displays several available plots in a single panel. Displayed plots depend on the rule used in stop: (a) with 'length', "trueEst", "condBias", "condRMSE", "expRate", "cumExpRate", "expRatePara"; (b) with 'precision' and 'classification', all the plots but "cumExpRate".
These plots can be saved as external PDF or JPEG files, by setting save.plot to TRUE and defining the arguments of save.options accurately (see randomCAT for further explanations and the Examples section below).