Imagine that there are people that play in tournaments of more than one board game. For example, the computer player AlphaZero (Silver et al. 2018) has trained to play chess, shogi, and Go. We can take the tournament match outcome data and find rankings among the players for each of these games. We may also suspect that there is a latent board game skill that accounts for some proportion of the variance in the per-board game rankings.
generateFactorItems(df, prop, th = 0.5, scale = 1, name)
a data frame with pairs of vertices given in columns pa1
and pa2
, and item response data in other columns
the number of items or a vector of proportions of variance
a vector of thresholds
the scaling constant
a vector of item names
The pairwise comparison item response model has thresholds and a scale parameter similar to the partial credit model (Masters, 1982). The model is cumbersome to describe in traditional mathematical notation, but the R code is fairly straightforward,
softmax <- function(y) exp(y) / sum(exp(y))cmp_probs <- function(scale, pa1, pa2, thRaw) { th <- cumsum(thRaw) diff <- scale * (pa2 - pa1) unsummed <- c(0, c(diff + rev(th)), c(diff - th), use.names = FALSE) softmax(cumsum(unsummed)) }
The function cmp_probs
takes a scale
constant, the
latent scores for two objects pa1
and pa2
, and a
vector of thresholds thRaw
. The thresholds are parameterized
as the difference from the previous threshold. For example,
thresholds c(0.5, 0.5) are not at the same location but are at
locations c(0.5, 1.0). Thresholds are symmetric. If there is one
thresholds then the model admits three possible response outcomes
(e.g. win, tie, and lose). Responses are always stored centered
with zero representing a tie. Therefore, it is necessary to add one
plus the number of thresholds to response data to index into the
vector returned by cmp_probs
. For example, if our response
data (-1, 0, 1) has one threshold then we would add 2 (1 + 1
threshold) to obtain the indices (1, 2, 3).
Use itemModelExplorer
to explore the item model. In
this shiny app, the discrimination parameter does what is
customary in item response models. However, it is not difficult to
show that discrimination is a function of thresholds and
scale. That is, discrimination is not an independent parameter and
cannot be estimated. In pairwise comparison models, discrimination
and measurement error are confounded.
Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149<U+2013>174. doi: 10.1007/BF02296272
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., ... & Lillicrap, T. (2018). A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419), 1140-1144.
Other item generators: generateCovItems
,
generateItem
# NOT RUN {
df <- twoLevelGraph(letters[1:10], 100)
df <- generateFactorItems(df, 3)
# }
Run the code above in your browser using DataLab