Learn R Programming

mirtCAT (version 1.2)

mirtCAT: Generate an adaptive or non-adaptive test HTML interface

Description

Provides tools to generate an HTML interface for creating adaptive and non-adaptive educational and psychological tests using the shiny package. Suitable for applying unidimensional and multidimensional computerized adaptive tests using item response theory methodology. Test scoring is performed using the mirt package. However, if no scoring is required (i.e., a standard survey) then defining a mirt object may be omitted.

Usage

mirtCAT(df = NULL, mo = NULL, method = "MAP", criteria = "seq", start_item = 1, local_pattern = NULL, design_elements = FALSE, cl = NULL, progress = FALSE, primeCluster = TRUE, design = list(), shinyGUI = list(), preCAT = list(), ...)
"print"(x, ...)
"summary"(object, sort = TRUE, ...)
"plot"(x, pick_theta = NULL, true_thetas = TRUE, SE = 1, main = NULL, par.strip.text = list(cex = 0.7), par.settings = list(strip.background = list(col = "#9ECAE1"), strip.border = list(col = "black")), ...)

Arguments

df
a data.frame or list object containing the character vector inputs required to generate GUI questions through shiny. If factors are supplied instead of character vectors then the inputs will be coerced using the as.character() function (set stringsAsFactors = FALSE when defining a data.frame to avoid this). Each row in the object corresponds to a unique item. The object supports the follow column name combinations as inputs to specify the type of response format, questions, options, answers, and stems:

mo
single group object defined by the mirt::mirt() function. This is required if the test is to be scored adaptively or non-adaptively, but not required for general questionnaires. The object can be constructed by using the generate.mirt_object function if population parameters are known or by including a calibrated model estimated from the mirt function with real data.
method
argument passed to mirt::fscores() for computing new scores in the CAT stage, with the addition of a 'fixed' input to keep the latent trait estimates fixed at the previous values. Default is 'MAP'
criteria
adaptive criteria used, default is to administer each item sequentially using criteria = 'seq'.

Possible inputs for unidimensional adaptive tests include: 'MI' for the maximum information, 'MEPV' for minimum expected posterior variance, 'MLWI' for maximum likelihood weighted information, 'MPWI' for maximum posterior weighted information, 'MEI' for maximum expected information, and 'IKLP' as well as 'IKL' for the integration based Kullback-Leibler criteria with and without the prior density weight, respectively, and their root-nitems administered weighted counter-parts, 'IKLn' and 'IKLPn'. Possible inputs for multidimensional adaptive tests include: 'Drule' for the maximum determinant of the information matrix, 'Trule' for the maximum (potentially weighted) trace of the information matrix, 'Arule' for the minimum (potentially weighted) trace of the asymptotic covariance matrix, 'Erule' for the minimum value of the information matrix, and 'Wrule' for the weighted information criteria. For each of these rules, the posterior weight for the latent trait scores can also be included with the 'DPrule', 'TPrule', 'APrule', 'EPrule', and 'WPrule', respectively. As a safety precaution, if the selected criteria do not weight by the posterior (and therefore do not exist for extreme response styles) and less than 5 items have been administered then the method is temporarily switched to the posterior weighting until a variable response pattern is observed. Applicable to both unidimensional and multidimensional tests are the 'KL' and 'KLn' for point-wise Kullback-Leibler divergence and point-wise Kullback-Leibler with a decreasing delta value (delta*sqrt(n), where n is the number of items previous answered), respectively. The delta criteria is defined in the design object Non-adaptive methods applicable even when no mo is passed are: 'random' to randomly select items, and 'seq' for selecting items sequentially.

start_item
two possible inputs to determine the starting item are available. Passing a single number will indicate the specific item to be used as the start item; default is 1, which selects the first item in the defined test/survey. If a character string is passed then the item will be selected from one of the item selections criteria available (see the criteria argument)
local_pattern
a character/numeric matrix of response patterns used to run the CAT application without generating the GUI interface. This option requires complete response pattern(s) to be supplied. local_pattern is required to be numeric if no questions are supplied, and the responses must be within a valid range of the defined mo object. Otherwise, it must contain character values of plausible responses which corresponds to the answer key and/or options supplied in df. If the object contains an attribute 'Theta' then these values will be stored within the respective returned objects. See generate_pattern to generate response patterns for Monte Carlo simulations
design_elements
logical; return an object containing the test, person, and design elements? Primarily this is to be used with the findNextItem function
cl
an object definition to be passed to the parallel package (see ?parallel::parLapply for details). If defined, and if nrow(local_pattern) > 1, then each row will be run in parallel to help decrease estimation times in simulation work
progress
logical; print a progress bar to the console with the pbapply package for given response patterns? Useful for guaging how long Monte Carlo simulations will take to finish
primeCluster
logical; when a cl object is supplied, should the cluster be primed first before running the simulations in parallel? Setting to TRUE will ensure that using the cluster will be optimal every time a new cl is defined. Default is TRUE
design
a list of design based control parameters for adaptive and non-adaptive tests. These can be

shinyGUI
a list of GUI based parameters to be over-written. These can be

preCAT
a list object which can be used to specify a pre-CAT block in which different test properties may be applied prior to beginning the CAT session. If the list is empty, no preCAT block will be used. All of the following elements are required to use the preCAT input:
...
additional arguments to be passed to mirt, fscores, runApp, or lattice
x
object of class 'mirtCAT'
object
object of class 'mirtCAT'
sort
logical; sort the response patterns based on the order they were administered? If FALSE, the raw response patterns containing NAs will be returned for items that were not administered
pick_theta
a number indicating which theta to plot (only applicable for multidimensional tests). The default is to facet each theta on one plot, but to plot only the first factor pass pick_theta = 1
true_thetas
logical; include a horizontal line indicating where the population-level theta values are? Only applicable to Monte Carlo simulations because this value would not be known otherwise
SE
size of the standard errors to plot. The default is 1, and therefore plots the standard error. To obtain the 95% interval use SE = 1.96 (from the z-distribution)
main
title of the plot. Will default to 'CAT Standard Errors' or 'CAT ##% Confidence Intervals' depending on the SE input
par.strip.text
plotting argument passed to lattice
par.settings
plotting argument passed to lattice

Value

Returns a list object of class 'Person' containing the following elements:

HTML help files, exercises, and examples

To access examples, vignettes, and exercise files that have been generated with knitr please visit https://github.com/philchalmers/mirtCAT/wiki.

Modifying the <code>design</code> object directly through <code>customNextItem()</code> (advanced)

In addition to providing a completely defined item-selection map via the customNextItem() function, users may also wish to control some of the more fine-grained elements of the design object to adjust the general control parameters of the CAT (e.g., modifying the maximum number of items to administer, stopping the CAT if something peculiar has been detected in the response patterns, etc). Note that this feature is rarely required for most applications, though more advanced users may wish to modify these various low-level elements of the design object directly to change the flow of the CAT to suit their specific needs. While the person object is defined as a Reference Class (see setRefClass) the design object is generally considered a fixed S4 class, meaning that, unlike the person object, it's elements are not mutable. Therefore, in order to make changes directly to the design object the users should follow these steps:
  1. Within the defined customNextItem function, the design object slots are first modified (e.g., design@max_items <- 20L).
  2. Along with the desired next item scalar value from customNextItem(), the scalar object should also contain an attribute with the name 'design' which holds the newly defined design object (e.g., attr(ret, 'design') <- design; return(ret)).
Following the above process the work-flow in mirtCAT will use the new design object in place of the old one, even in Monte Carlo simulations.

Details

All tests will stop once the 'min_SEM' criteria has been reached or classification above or below the specified cutoffs can be made. If all questions should be answered, users should specify an extremely small 'min_SEM' or, equivalently, a large 'min_items' criteria to the design list input.

References

Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications. Journal of Statistical Software, 71(5), 1-39. doi:10.18637/jss.v071.i05

See Also

generate_pattern, generate.mirt_object, extract.mirtCAT, findNextItem

Examples

Run this code
## Not run: 
# 
# ### unidimensional scored example with generated items
# 
# # create mo from estimated parameters
# set.seed(1234)
# nitems <- 50
# itemnames <- paste0('Item.', 1:nitems)
# a <- matrix(rlnorm(nitems, .2, .3))
# d <- matrix(rnorm(nitems))
# dat <- simdata(a, d, 1000, itemtype = 'dich')
# mod <- mirt(dat, 1)
# coef(mod, simplify=TRUE)
# 
# # alternatively, define mo from population values (not run)
# pars <- data.frame(a1=a, d=d)
# mod2 <- generate.mirt_object(pars, itemtype='2PL')
# coef(mod2, simplify=TRUE)
# 
# # simple math items
# questions <- answers <- character(nitems)
# choices <- matrix(NA, nitems, 5)
# spacing <- floor(d - min(d)) + 1 #easier items have more variation in the options
# 
# for(i in 1:nitems){
#     n1 <- sample(1:50, 1)
#     n2 <- sample(51:100, 1)
#     ans <- n1 + n2
#     questions[i] <- paste0(n1, ' + ', n2, ' = ?')
#     answers[i] <- as.character(ans)
#     ch <- ans + sample(c(-5:-1, 1:5) * spacing[i,], 5)
#     ch[sample(1:5, 1)] <- ans
#     choices[i, ] <- as.character(ch)
# }
# 
# df <- data.frame(Question=questions, Option=choices, 
#                               Type = 'radio', stringsAsFactors = FALSE)
# head(df)
# 
# (res <- mirtCAT(df)) #collect response only (no scoring or estimating thetas)
# summary(res)
# 
# # include scoring by providing Answer key
# df$Answer <- answers
# (res_seq <- mirtCAT(df, mod)) #sequential scoring 
# (res_random <- mirtCAT(df, mod, criteria = 'random')) #random
# (res_MI <- mirtCAT(df, mod, criteria = 'MI', start_item = 'MI')) #adaptive, MI starting item
# 
# summary(res_seq)
# summary(res_random)
# summary(res_MI)
# 
# #-----------------------------------------
# 
# # run locally, random response pattern given Theta
# set.seed(1)
# pat <- generate_pattern(mod, Theta = 0, df=df)
# head(pat)
# 
# # seq scoring with character pattern for the entire test (adjust min_items)
# res <- mirtCAT(df, mod, local_pattern=pat, design = list(min_items = 50)) 
# summary(res)
# 
# # same as above, but using special input vector that doesn't require df input
# set.seed(1)
# pat2 <- generate_pattern(mod, Theta = 0)
# head(pat2)
# print(mirtCAT(mo=mod, local_pattern=pat2))
# 
# # run CAT, and save results to object called person (start at 10th item)
# person <- mirtCAT(df, mod, item_answers = answers, criteria = 'MI', 
#                   start_item = 10, local_pattern = pat)
# print(person)
# summary(person)
# 
# # plot the session
# plot(person) #standard errors
# plot(person, SE=1.96) #95 percent confidence intervals
# 
# #-----------------------------------------
# 
# ### save response object to temp directory in case session ends early
# wdf <- paste0(getwd(), '/temp_file.rds')
# res <- mirtCAT(df, mod, shinyGUI = list(temp_file = wdf))
# 
# # resume test this way if test was stopped early (and temp files were saved)
# res <- mirtCAT(df, mod, shinyGUI = list(temp_file = wdf))
# print(res)
# 
# ## End(Not run)

Run the code above in your browser using DataLab