catR (version 2.2)

randomCAT: Random generation of adaptive tests

Description

This command generates a response pattern to an adaptive test, for a given item bank, a true ability level, and several lists of CAT parameters (starting items, stopping rule, provisional and final ability estimators).

Usage

randomCAT(trueTheta, itemBank, maxItems=50, cbControl=NULL,
 	start=list(fixItems=NULL, seed=NULL, nrItems=1, theta=0, 
 	halfRange=2, startSelect="bOpt"), test=list(method="BM", 
 	priorDist="norm", priorPar=c(0,1), range=c(-4,4), D=1, 
 	parInt=c(-4,4,33), itemSelect="MFI", infoType="observed",
 	randomesque=1), stop=list(rule="length", thr=20,
 	alpha=0.05), final=list(method="BM", priorDist="norm",
 	priorPar=c(0,1), range=c(-4,4),D=1, parInt=c(-4,4,33), 
 	alpha=0.05), save.output=FALSE, output=c("out","default"))
## S3 method for class 'cat':
print(x, ...)
## S3 method for class 'cat':
plot(x, ci=FALSE, alpha=0.05, trueTh=TRUE, classThr=NULL, 
 	save.plot=FALSE, save.options=c("plot","default","pdf"),...)

Arguments

trueTheta

numeric: the value of the true ability level.

itemBank

an item bank, i.e. a list of class itBank, typically an output of the function createItemBank.

maxItems

numeric: the maximal number of items to be administered (default is 50).

cbControl

either a list of accurate format to control for content balancing, or NULL. See Details.

start

a list with the options for starting the adaptive test. See Details.

test

a list with the options for provisional ability estimation and next item selection. See Details.

stop

a list with the options of the stopping rule. See Details.

final

a list with the options for final ability estimation. See Details.

save.output

logical: should the output be saved in an external text file? (default is FALSE).

output

character: a vector of two components. The first component is the name of the output file, the second component is either the file path or "default" (default). See Details.

x

an object of class "cat", typically an output of randomCAT function.

ci

logical: should the confidence intervals be plotted for each provisional ability estimate? (default is TRUE).

alpha

numeric: the significance level for provisional confidence intervals (default is 0.05). Ignored if ci is FALSE.

trueTh

logical: should the true ability level be drawn by a horizontal line? (default is TRUE).

classThr

either a numeric value giving the classification threshold to be displayed, or NULL.

save.plot

logical: should the plot be saved in an external figure? (default is FALSE).

save.options

character: a vector of three components. The first component is the name of the output file, the second component is either the file path or "default" (default), and the third component is the file extension, either

...

other generic arguments to be passed to print and plot functions.

Value

• The function randomCAT returns a list of class "cat" with the following arguments:
• trueTheta
  the value of the trueTheta argument.
• maxItems
  the value of the maxItems argument.
• testItems
  a vector with the items that were administered during the test.
• itemPar
  a matrix with the parameters of the items administered during the test.
• pattern
  the generated response pattern (as vector of 0 and 1 entries).
• thetaProv
  a vector with the provisional ability estimates.
• seprov
  a vector with the standard errors of the provisional ability estimates.
• thFinal
  the final ability estimate.
• seFinal
  the standrad error of the final ability estimate.
• ciFinal
  the confidence interval of the final ability estimate.
• startFixItems
  the value of the start$fixItems argument (or its default value if missing).
• startSeed
  the value of the start$seed argument (or its default value if missing).
• startNrItems
  the value of the start$nrItems argument (or its default value if missing).
• startTheta
  the value of the start$theta argument (or its default value if missing).
• startHalfRange
  the value of the start$halfRange argument (or its default value if missing).
• startThStart
  the starting ability values used for selecting the first items of the test.
• startSelect
  the value of the start$startSelect argument (or its default value if missing).
• provMethod
  the value of the test$method argument (or its default value if missing).
• provDist
  the value of the test$priorDist argument (or its default value if missing).
• provPar
  the value of the test$priorPar argument (or its default value if missing).
• provRange
  the value of the test$range argument (or its default value if missing).
• provD
  the value of the test$D argument (or its default value if missing).
• itemSelect
  the value of the test$itemSelect argument (or its default value if missing).
• infoType
  the value of the test$infoType argument (or its default value if missing).
• randomesque
  the value of the test$randomesque argument (or its default value if missing).
• cbControl
  the value of the cbControl argument (or its default value if missing).
• cbGroup
  the value of the itemBank$cbGroup element of the item bank itemBank.
• stopRule
  the value of the stop$rule argument (or its default value if missing).
• stopThr
  the value of the stop$thr argument (or its default value if missing).
• stopAlpha
  the value of the stop$alpha argument (or its default value if missing).
• endWarning
  a logical indactor indicating whether the adaptive test stopped because the stopping rule was satisfied or not.
• finalMethod
  the value of the final$method argument (or its default value if missing).
• finalDist
  the value of the final$priorDist argument (or its default value if missing).
• finalPar
  the value of the final$priorPar argument (or its default value if missing).
• finalRange
  the value of the final$range argument (or its default value if missing).
• finalD
  the value of the final$D argument (or its default value if missing).
• finalAlpha
  the value of the final$alpha argument (or its default value if missing).
• save.output
  the value of the save.output argument.
• output
  the value of the output argument.
• The function print.cat returns similar (but differently organized) results.

code

Examples

itemize

• rule: a character string specifying the type of stopping rule. Possible values are:"length"(default), to stop the test after a pre-specified number of items administered;"precision", to stop the test when the provisional standard error of ability becomes less than or equal to the pre-specified value; and"classification", for which the test ends whenever the provisional confidence interval (set by thealphaargument) does not hold the classification threshold anymore.

item

• priorDist: a character string which sets the prior distribution. Possible values are: "norm" (default) for normal distribution, "unif" for uniform distribution, and "Jeffreys" for Jeffreys' noninformative prior distribution (Jeffreys, 1939, 1946). ignored if method is neither "BM" nor "EAP".
• priorPar: a vector of two numeric components, which sets the parameters of the prior distribution. If (method="BM" or method=="EAP") and priorDist="norm", the components of priorPar are respectively the mean and the standard deviation of the prior normal density. If (method="BM" or method=="EAP") and priorDist="unif", the components of priorPar are respectively the lower and upper bound of the prior uniform density. Ignored in all other cases. By default, priorPar takes the parameters of the prior standard normal distribution (i.e., priorPar=c(0,1)). In addition, priorPar also provides the prior parameters for the comoutation of MLWI and MPWI values for next item selection (see nextItem for further details).
• range: the maximal range of ability levels, set as a vector of two numeric components. The ability estimate will always lie to this interval (set by default to [-4, 4]). Ignored if method=="EAP".
• D: the value of the metric constant. Default is D=1 for logistic metric. Setting D=1.702 yields approximately the normal metric (Haley, 1952).
• parInt: a numeric vector of three components, holding respectively the values of the arguments lower, upper and nqp of the eapEst, eapSem and MWI commands. It specifies the range of quadrature points for numerical integration, and is used for computing the EAP estimate, its standard error, and the MLWI and MPWI values for next item selection. Default vector is (-4, 4, 33), thus setting the range from -4 to 4 by steps of 0.25. Ignored if method is not "EAP" and if itemSelect is neither "MLWI" nor "MPWI".
• itemSelect: the rule for next item selecion, with possible values "MFI" (default) for maximum Fisher information criterion; "Urry" for Urry's procedure; "MLWI" and "MPWI" for respectively maximum likelihood and posterior weighted information criterion; "MEPV" for minimum expected posterior variance; "MEI" for maximum expected information; and "random" for random selection. For further details, see nextItem.
• infoType: character: the type of information function to be used for next item selection. Possible values are "observed" (default) for observed information function, and "Fisher" for Fisher information function. Ignored if itemselect is not "MEI".
• randomesque: integer: the number of items to be chosen from the next item selection rule, among those the next item to be administered will be randomly picked up. Default value is 1 and leads to usual selection of the optimal item (Kingsbury and Zara, 1989).
• thr: a numeric value fixing the threshold of the stopping rule. If rule="length", thr is the maximal number of items to be administered (in practice, it is replaced by the value of the maxItems argument if the latter is smaller than thr). If rule="precision", thr is the precision level (i.e. the standard error) to be reached before stopping. Finally, if rule="classification", thr corresponds to the ability level which serves as a classification rule (i.e. which must not be covered by the provisional confidence interval).
• alpha: the significance (or $\alpha$) level for computing the priovisional confidence interval of ability. Ignored if rule is not "classification".

eqn

$se(\hat{\theta})$

deqn

$$[\hat{\theta}-z_{1-\alpha/2} \; se(\hat{\theta}) ; \hat{\theta}+z_{1-\alpha/2} \; se(\hat{\theta})]$$

bold

Examples

Details

The randomCAT function generates an adaptive test using an item bank specified by argument itemBank, and for a given true ability level specified by argument trueTheta. The maximal length of the test can be fixed through the maxItems argument, with a default value of 50 items. Content balancing can be controlled by the cbControl argument. See the nextItem function for further details on how to specify cbControl properly and under which conditions it is operational. By default, content balancing is not controlled (see Kingsbury and Zara, 1989, for further details on content balancing). The test specification is made by means of four lists of options: one list for the selection of the starting items, one list with the options for provisional ability estimation, one list to define the stopping rule, and one list with the options for final ability estimation. These lists are specified respectively by the arguments start, test, stop and final. The start list can contain one or several of the following arguments:

• fixItems: either a vector of integer values, setting the items to be administered as first items, orNULL(default) to let the function select the items.

seed: either a numeric value to fix the random seed for item selection, or NULL (default) to select the items on the basis of their difficulty level. Ignored if fixItems is not NULL. nrItems: the number of first items to be selected (default is 1). Ignored if fixItems is not NULL. theta: the central initial ability value, used to define the range of ability levels for selecting the first items (default is 0). Ignored if either fixItems or seed is not NULL. See startItems for further details. halfRange: the half range of starting ability levels for selecting the first items (default is 2). Ignored if either fixItems or seed is not NULL. See startItems for further details. startSelect: the method for selecting the first items of the test, with possible values "bOpt" (default) and "MFI". Ignored if either fixItems or seed is not NULL. See startItems for further details.

References

Birnbaum, A. (1969). Statistical theory for logistic mental test models with a prior distribution of ability. Journal of Mathematical Psychology, 6, 258-276. Bock, R. D., and Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444. Haley, D.C. (1952). Estimation of the dosage mortality relationship when the dose is subject to error. Technical report no 15. Palo Alto, CA: Applied Mathematics and Statistics Laboratory, Stanford University. Jeffreys, H. (1939). Theory of probability. Oxford, UK: Oxford University Press. Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 186, 453-461. Kingsbury, G. G., and Zara, A. R. (1989). Procedures for selecting items for computerized adaptive tests. Applied Measurement in Education, 2, 359-375. Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum. Magis, D., and Raiche, G. (in press). Random generation of response patterns under computerized adaptive testing with the R package catR. Journal of Statistical Software. Urry, V. W. (1970). A Monte Carlo investigation of logistic test models. Unpublished doctoral dissertation. West Lafayette, IN: Purdue University. van der Linden, W. J. (1998). Bayesian item selection criteria for adaptive testing. Psychometrika, 63, 201-216. Veerkamp, W. J. J., and Berger, M. P. F. (1997). Some new item selection criteria for adaptive testing. Journal of Educational and Behavioral Statistics, 22, 203-226. Warm, T.A. (1989). Weighted likelihood estimation of ability in item response models. Psychometrika, 54, 427-450.

See Also

testList, startItems, nextItem, thetaEst, semTheta, eapEst, eapSem, MWI, MEI

Examples

# Loading the 'tcals' parameters 
 data(tcals)
 
 # Item bank creation with 'tcals' item parameters
 bank <- createItemBank(tcals)

 # Creation of a starting list: 5 items, initial theta 0, bw 2
 start <- list(nrItems=5, theta=0, halfRange=2)

 # Creation of 'test' list: weighted likelihood
 # estimation of provisional ability, and MEI criterion 
 # for next item selection
 test <- list(method="WL", itemSelect="MEI")

 # Creation of 'final' list: EAP estimation of final
 # ability
 final <- list(method="EAP")

 # Creation of a stopping rule: precision criterion, standard
 # error to be reached 0.3
 stop <- list(rule="precision", thr=0.3)

 # CAT test
 res <- randomCAT(0, bank, start=start, test=test, stop=stop,
 	final=final)

 # New 'test' and 'final' rules (BM and EAP estimation
 # with Jeffreys' prior, randomesque value 5)
 test2 <- list(method="BM", priorDist="Jeffreys", randomesque=5)
 final2 <- list(method="EAP", priorDist="Jeffreys")

 # New stopping rule: classification criterion, with 
 # classification threshold 0 and alpha level 0.05
 stop2 <- list(rule="classification", thr=0, alpha=0.05)

 # CAT test with new 'test', 'stop' and 'final' rules
 res2 <- randomCAT(0, bank, start=start, test=test2, stop=stop2,
 	final=final2)

 # New stopping rule: classification criterion, with 
 # classification threshold 0.5 and alpha level 0.05
 stop3 <- list(rule="classification", thr=0.5, alpha=0.05)

 # CAT test with new 'stop' rule
 res3 <- randomCAT(0, bank, start=start, test=test2, stop=stop3,
 	final=final2)

 # new 'test' and 'stop' rule for next item selection
 test3 <- list(method="WL", itemSelect="MLWI")
 stop4 <- list(rule="length",thr=10)
 res4 <- randomCAT(0, bank, start=start, test=test3, stop=stop4,
 	final=final2)

 # Item bank creation for content balancing
 bank2 <- createItemBank(tcals, cb=TRUE)

 # Creation of the 'cbList' list with arbitrary proportions
 cbList <- list(names=c("Audio1","Audio2","Written1","Written2",
                "Written3"), props=c(0.1,0.2,0.2,0.2,0.3))

 # CAT test with 'start', 'test2', 'stop4' and 'final2' lists
 # and content balancing using 'cbList'
 res5 <- randomCAT(0, bank2, start=start, test=test2, stop=stop4,
 	final=final2, cbControl=cbList)

 # Saving the output in the external 'ou' text file within folder
 # 'Program Files' of hard drive 'C'
 res5 <- randomCAT(0, bank2, start=start, test=test2, stop=stop4,
 	final=final2, cbControl=cbList, save.output = TRUE,
      output=c("out","c:/Program Files/"))

 # Plotting results
 plot(res)
 plot(res, ci=TRUE)
 plot(res, ci=TRUE, trueTh=FALSE)
 plot(res, ci=TRUE, classThr=1)

 # Saving last figure into PDF file 'figure' within folder
 # 'C:/Program Files/'
 plot(res, ci=TRUE, classThr=1, save.plot=TRUE,
      save.options=c("figure","c:/Program Files","pdf"))

 # With mistake
 plot(res, ci=0.05)
 plot(res, classThr=TRUE)