randomCAT: Random generation of adaptive tests

Description

This command generates a response pattern to an adaptive test, for a given item bank, a true ability level, and several lists of CAT parameters (starting items, stopping rule, provisional and final ability estimators).

Usage

randomCAT(trueTheta, itemBank, maxItems=50, cbControl=NULL,
 	start=list(fixItems=NULL, seed=NULL, nrItems=1, theta=0, 
 	halfRange=2, startSelect="bOpt"), test=list(method="BM", 
 	priorDist="norm", priorPar=c(0,1), range=c(-4,4), D=1, 
 	parInt=c(-4,4,33), itemSelect="MFI", infoType="observed",
 	randomesque=1), stop=list(rule="length", thr=20,
 	alpha=0.05), final=list(method="BM", priorDist="norm",
 	priorPar=c(0,1), range=c(-4,4),D=1, parInt=c(-4,4,33), 
 	alpha=0.05), save.output=FALSE, output=c("out","default"))
## S3 method for class 'cat':
print(x, ...)
## S3 method for class 'cat':
plot(x, ci=FALSE, alpha=0.05, trueTh=TRUE, classThr=NULL, 
 	save.plot=FALSE, save.options=c("plot","default","pdf"),...)

Arguments

trueTheta

numeric: the value of the true ability level.

itemBank

an item bank, i.e. a list of class itBank, typically an output of the function createItemBank.

maxItems

numeric: the maximal number of items to be administered (default is 50).

cbControl

either a list of accurate format to control for content balancing, or NULL. See Details.

start

a list with the options for starting the adaptive test. See Details.

test

a list with the options for provisional ability estimation and next item selection. See Details.

stop

a list with the options of the stopping rule. See Details.

final

a list with the options for final ability estimation. See Details.

save.output

logical: should the output be saved in an external text file? (default is FALSE).

output

character: a vector of two components. The first component is the name of the output file, the second component is either the file path or "default" (default). See Details.

an object of class "cat", typically an output of randomCAT function.

logical: should the confidence intervals be plotted for each provisional ability estimate? (default is TRUE).

alpha

numeric: the significance level for provisional confidence intervals (default is 0.05). Ignored if ci is FALSE.

trueTh

logical: should the true ability level be drawn by a horizontal line? (default is TRUE).

classThr

either a numeric value giving the classification threshold to be displayed, or NULL.

save.plot

logical: should the plot be saved in an external figure? (default is FALSE).

save.options

character: a vector of three components. The first component is the name of the output file, the second component is either the file path or "default" (default), and the third component is the file extension, either

...

other generic arguments to be passed to print and plot functions.

Value

The function randomCAT returns a list of class "cat" with the following arguments:
trueThetathe value of the trueTheta argument.
maxItemsthe value of the maxItems argument.
testItemsa vector with the items that were administered during the test.
itemPara matrix with the parameters of the items administered during the test.
patternthe generated response pattern (as vector of 0 and 1 entries).
thetaProva vector with the provisional ability estimates.
seprova vector with the standard errors of the provisional ability estimates.
thFinalthe final ability estimate.
seFinalthe standrad error of the final ability estimate.
ciFinalthe confidence interval of the final ability estimate.
startFixItemsthe value of the start$fixItems argument (or its default value if missing).
startSeedthe value of the start$seed argument (or its default value if missing).
startNrItemsthe value of the start$nrItems argument (or its default value if missing).
startThetathe value of the start$theta argument (or its default value if missing).
startHalfRangethe value of the start$halfRange argument (or its default value if missing).
startThStartthe starting ability values used for selecting the first items of the test.
startSelectthe value of the start$startSelect argument (or its default value if missing).
provMethodthe value of the test$method argument (or its default value if missing).
provDistthe value of the test$priorDist argument (or its default value if missing).
provParthe value of the test$priorPar argument (or its default value if missing).
provRangethe value of the test$range argument (or its default value if missing).
provDthe value of the test$D argument (or its default value if missing).
itemSelectthe value of the test$itemSelect argument (or its default value if missing).
infoTypethe value of the test$infoType argument (or its default value if missing).
randomesquethe value of the test$randomesque argument (or its default value if missing).
cbControlthe value of the cbControl argument (or its default value if missing).
cbGroupthe value of the itemBank$cbGroup element of the item bank itemBank.
stopRulethe value of the stop$rule argument (or its default value if missing).
stopThrthe value of the stop$thr argument (or its default value if missing).
stopAlphathe value of the stop$alpha argument (or its default value if missing).
endWarninga logical indactor indicating whether the adaptive test stopped because the stopping rule was satisfied or not.
finalMethodthe value of the final$method argument (or its default value if missing).
finalDistthe value of the final$priorDist argument (or its default value if missing).
finalParthe value of the final$priorPar argument (or its default value if missing).
finalRangethe value of the final$range argument (or its default value if missing).
finalDthe value of the final$D argument (or its default value if missing).
finalAlphathe value of the final$alpha argument (or its default value if missing).
save.outputthe value of the save.output argument.
outputthe value of the output argument.
The function print.cat returns similar (but differently organized) results.

code

Examples

itemize

rule: a character string specifying the type of stopping rule. Possible values are:"length"(default), to stop the test after a pre-specified number of items administered;"precision", to stop the test when the provisional standard error of ability becomes less than or equal to the pre-specified value; and"classification", for which the test ends whenever the provisional confidence interval (set by thealphaargument) does not hold the classification threshold anymore.

item

priorDist: a character string which sets the prior distribution. Possible values are: "norm" (default) for normal distribution, "unif" for uniform distribution, and "Jeffreys" for Jeffreys' noninformative prior distribution (Jeffreys, 1939, 1946). ignored if method is neither "BM" nor "EAP".
priorPar: a vector of two numeric components, which sets the parameters of the prior distribution. If (method="BM" or method=="EAP") and priorDist="norm", the components of priorPar are respectively the mean and the standard deviation of the prior normal density. If (method="BM" or method=="EAP") and priorDist="unif", the components of priorPar are respectively the lower and upper bound of the prior uniform density. Ignored in all other cases. By default, priorPar takes the parameters of the prior standard normal distribution (i.e., priorPar=c(0,1)). In addition, priorPar also provides the prior parameters for the comoutation of MLWI and MPWI values for next item selection (see nextItem for further details).
range: the maximal range of ability levels, set as a vector of two numeric components. The ability estimate will always lie to this interval (set by default to [-4, 4]). Ignored if method=="EAP".
D: the value of the metric constant. Default is D=1 for logistic metric. Setting D=1.702 yields approximately the normal metric (Haley, 1952).
parInt: a numeric vector of three components, holding respectively the values of the arguments lower, upper and nqp of the eapEst, eapSem and MWI commands. It specifies the range of quadrature points for numerical integration, and is used for computing the EAP estimate, its standard error, and the MLWI and MPWI values for next item selection. Default vector is (-4, 4, 33), thus setting the range from -4 to 4 by steps of 0.25. Ignored if method is not "EAP" and if itemSelect is neither "MLWI" nor "MPWI".
itemSelect: the rule for next item selecion, with possible values "MFI" (default) for maximum Fisher information criterion; "Urry" for Urry's procedure; "MLWI" and "MPWI" for respectively maximum likelihood and posterior weighted information criterion; "MEPV" for minimum expected posterior variance; "MEI" for maximum expected information; and "random" for random selection. For further details, see nextItem.
infoType: character: the type of information function to be used for next item selection. Possible values are "observed" (default) for observed information function, and "Fisher" for Fisher information function. Ignored if itemselect is not "MEI".
randomesque: integer: the number of items to be chosen from the next item selection rule, among those the next item to be administered will be randomly picked up. Default value is 1 and leads to usual selection of the optimal item (Kingsbury and Zara, 1989).
thr: a numeric value fixing the threshold of the stopping rule. If rule="length", thr is the maximal number of items to be administered (in practice, it is replaced by the value of the maxItems argument if the latter is smaller than thr). If rule="precision", thr is the precision level (i.e. the standard error) to be reached before stopping. Finally, if rule="classification", thr corresponds to the ability level which serves as a classification rule (i.e. which must not be covered by the provisional confidence interval).
alpha: the significance (or $\alpha$) level for computing the priovisional confidence interval of ability. Ignored if rule is not "classification".

eqn

$se(\hat{\theta})$

deqn

[\hat{θ} - z_{1 - α / 2} s e (\hat{θ}); \hat{θ} + z_{1 - α / 2} s e (\hat{θ})]

bold

Examples

Details

The randomCAT function generates an adaptive test using an item bank specified by argument itemBank, and for a given true ability level specified by argument trueTheta. The maximal length of the test can be fixed through the maxItems argument, with a default value of 50 items. Content balancing can be controlled by the cbControl argument. See the nextItem function for further details on how to specify cbControl properly and under which conditions it is operational. By default, content balancing is not controlled (see Kingsbury and Zara, 1989, for further details on content balancing). The test specification is made by means of four lists of options: one list for the selection of the starting items, one list with the options for provisional ability estimation, one list to define the stopping rule, and one list with the options for final ability estimation. These lists are specified respectively by the arguments start, test, stop and final. The start list can contain one or several of the following arguments:

fixItems: either a vector of integer values, setting the items to be administered as first items, orNULL(default) to let the function select the items.

seed: either a numeric value to fix the random seed for item selection, or NULL (default) to select the items on the basis of their difficulty level. Ignored if fixItems is not NULL. nrItems: the number of first items to be selected (default is 1). Ignored if fixItems is not NULL. theta: the central initial ability value, used to define the range of ability levels for selecting the first items (default is 0). Ignored if either fixItems or seed is not NULL. See startItems for further details. halfRange: the half range of starting ability levels for selecting the first items (default is 2). Ignored if either fixItems or seed is not NULL. See startItems for further details. startSelect: the method for selecting the first items of the test, with possible values "bOpt" (default) and "MFI". Ignored if either fixItems or seed is not NULL. See startItems for further details.

References

Birnbaum, A. (1969). Statistical theory for logistic mental test models with a prior distribution of ability. Journal of Mathematical Psychology, 6, 258-276. Bock, R. D., and Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444. Haley, D.C. (1952). Estimation of the dosage mortality relationship when the dose is subject to error. Technical report no 15. Palo Alto, CA: Applied Mathematics and Statistics Laboratory, Stanford University. Jeffreys, H. (1939). Theory of probability. Oxford, UK: Oxford University Press. Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 186, 453-461. Kingsbury, G. G., and Zara, A. R. (1989). Procedures for selecting items for computerized adaptive tests. Applied Measurement in Education, 2, 359-375. Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum. Magis, D., and Raiche, G. (in press). Random generation of response patterns under computerized adaptive testing with the R package catR. Journal of Statistical Software. Urry, V. W. (1970). A Monte Carlo investigation of logistic test models. Unpublished doctoral dissertation. West Lafayette, IN: Purdue University. van der Linden, W. J. (1998). Bayesian item selection criteria for adaptive testing. Psychometrika, 63, 201-216. Veerkamp, W. J. J., and Berger, M. P. F. (1997). Some new item selection criteria for adaptive testing. Journal of Educational and Behavioral Statistics, 22, 203-226. Warm, T.A. (1989). Weighted likelihood estimation of ability in item response models. Psychometrika, 54, 427-450.

Examples

Run this code

# Loading the 'tcals' parameters 
 data(tcals)
 
 # Item bank creation with 'tcals' item parameters
 bank <- createItemBank(tcals)

 # Creation of a starting list: 5 items, initial theta 0, bw 2
 start <- list(nrItems=5, theta=0, halfRange=2)

 # Creation of 'test' list: weighted likelihood
 # estimation of provisional ability, and MEI criterion 
 # for next item selection
 test <- list(method="WL", itemSelect="MEI")

 # Creation of 'final' list: EAP estimation of final
 # ability
 final <- list(method="EAP")

 # Creation of a stopping rule: precision criterion, standard
 # error to be reached 0.3
 stop <- list(rule="precision", thr=0.3)

 # CAT test
 res <- randomCAT(0, bank, start=start, test=test, stop=stop,
 	final=final)

 # New 'test' and 'final' rules (BM and EAP estimation
 # with Jeffreys' prior, randomesque value 5)
 test2 <- list(method="BM", priorDist="Jeffreys", randomesque=5)
 final2 <- list(method="EAP", priorDist="Jeffreys")

 # New stopping rule: classification criterion, with 
 # classification threshold 0 and alpha level 0.05
 stop2 <- list(rule="classification", thr=0, alpha=0.05)

 # CAT test with new 'test', 'stop' and 'final' rules
 res2 <- randomCAT(0, bank, start=start, test=test2, stop=stop2,
 	final=final2)

 # New stopping rule: classification criterion, with 
 # classification threshold 0.5 and alpha level 0.05
 stop3 <- list(rule="classification", thr=0.5, alpha=0.05)

 # CAT test with new 'stop' rule
 res3 <- randomCAT(0, bank, start=start, test=test2, stop=stop3,
 	final=final2)

 # new 'test' and 'stop' rule for next item selection
 test3 <- list(method="WL", itemSelect="MLWI")
 stop4 <- list(rule="length",thr=10)
 res4 <- randomCAT(0, bank, start=start, test=test3, stop=stop4,
 	final=final2)

 # Item bank creation for content balancing
 bank2 <- createItemBank(tcals, cb=TRUE)

 # Creation of the 'cbList' list with arbitrary proportions
 cbList <- list(names=c("Audio1","Audio2","Written1","Written2",
                "Written3"), props=c(0.1,0.2,0.2,0.2,0.3))

 # CAT test with 'start', 'test2', 'stop4' and 'final2' lists
 # and content balancing using 'cbList'
 res5 <- randomCAT(0, bank2, start=start, test=test2, stop=stop4,
 	final=final2, cbControl=cbList)

 # Saving the output in the external 'ou' text file within folder
 # 'Program Files' of hard drive 'C'
 res5 <- randomCAT(0, bank2, start=start, test=test2, stop=stop4,
 	final=final2, cbControl=cbList, save.output = TRUE,
      output=c("out","c:/Program Files/"))

 # Plotting results
 plot(res)
 plot(res, ci=TRUE)
 plot(res, ci=TRUE, trueTh=FALSE)
 plot(res, ci=TRUE, classThr=1)

 # Saving last figure into PDF file 'figure' within folder
 # 'C:/Program Files/'
 plot(res, ci=TRUE, classThr=1, save.plot=TRUE,
      save.options=c("figure","c:/Program Files","pdf"))

 # With mistake
 plot(res, ci=0.05)
 plot(res, classThr=TRUE)

Run the code above in your browser using DataLab

Join us for
RADAR: AI Edition

randomCAT: Random generation of adaptive tests

Description

Usage

Arguments

Value

code

itemize

item

eqn

deqn

bold

Details

References

See Also

Examples

Join us for RADAR: AI Edition

Description

Usage

Arguments

Value

code

itemize

item

eqn

deqn

bold

Details

References

See Also

Examples

Join us for
RADAR: AI Edition