Generate pairwise comparison data with a common factor that accounts for some proportion of the variance

Imagine that there are people that play in tournaments of more than one board game. For example, the computer player AlphaZero (Silver et al. 2018) has trained to play chess, shogi, and Go. We can take the tournament match outcome data and find rankings among the players for each of these games. We may also suspect that there is a latent board game skill that accounts for some proportion of the variance in the per-board game rankings.

generateFactorItems(df, prop, th = 0.5, scale = 1, name)

a data frame with pairs of vertices given in columns pa1 and pa2, and item response data in other columns


the number of items or a vector of proportions of variance


a vector of thresholds


the scaling constant


a vector of item names


The pairwise comparison item response model has thresholds and a scale parameter similar to the partial credit model (Masters, 1982). The model is cumbersome to describe in traditional mathematical notation, but the R code is fairly straightforward,

softmax <- function(y) exp(y) / sum(exp(y))

cmp_probs <- function(scale, pa1, pa2, thRaw) { th <- cumsum(thRaw) diff <- scale * (pa2 - pa1) unsummed <- c(0, c(diff + rev(th)), c(diff - th), use.names = FALSE) softmax(cumsum(unsummed)) }

The function cmp_probs takes a scale constant, the latent scores for two objects pa1 and pa2, and a vector of thresholds thRaw. The thresholds are parameterized as the difference from the previous threshold. For example, thresholds c(0.5, 0.5) are not at the same location but are at locations c(0.5, 1.0). Thresholds are symmetric. If there is one thresholds then the model admits three possible response outcomes (e.g. win, tie, and lose). Responses are always stored centered with zero representing a tie. Therefore, it is necessary to add one plus the number of thresholds to response data to index into the vector returned by cmp_probs. For example, if our response data (-1, 0, 1) has one threshold then we would add 2 (1 + 1 threshold) to obtain the indices (1, 2, 3).

Use itemModelExplorer to explore the item model. In this shiny app, the discrimination parameter does what is customary in item response models. However, it is not difficult to show that discrimination is a function of thresholds and scale. That is, discrimination is not an independent parameter and cannot be estimated. In pairwise comparison models, discrimination and measurement error are confounded.


Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149<U+2013>174. doi: 10.1007/BF02296272

Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., ... & Lillicrap, T. (2018). A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419), 1140-1144.

See Also

Other item generators: generateCovItems, generateItem

  • generateFactorItems
df <- twoLevelGraph(letters[1:10], 100)
df <- generateFactorItems(df, 3)
# }
Documentation reproduced from package pcFactorStan, version 0.11, License: GPL (>= 3)

Community examples

Looks like there are no examples yet.