rater (version 1.3.2)

loo.rater_fit: Compute the PSIS LOO CV - a measure of model fit - of a rater fit object.

Description

Compute the PSIS LOO CV - a measure of model fit - of a rater fit object.

Usage

# S3 method for rater_fit
loo(x, ..., cores = getOption("mc.cores", 1))

Value

A loo object.

Arguments

x

A rater_fit object. All model types are currently supported except the basic Dawid-Skene model fit with grouped data.

...

Other arguments passed.

cores

The number of cores to use when calling the underlying functions. By default the value of the mc.cores option.

Details

This function is somewhat experimental; model comparison is always difficult and choosing between variants of the Dawid-Skene model should be largely guided by considerations of data size and what is known about the characteristics of the raters. loo is, however, one of the leading methods for Bayesian model comparison and should provide a helpful guide in many situations.

When calculating loo we always use the relative effective sample size, calculated using loo::relaive_eff to improve the estimates of the PSIS effective sample sizes and Monte Carlo error.

For further information about the details of loo and PSIS please consult the provided references.

References

Vehtari, A., Gelman, A., and Gabry, J. (2017a). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing. 27(5), 1413--1432. doi:10.1007/s11222-016-9696-4 (journal version, preprint arXiv:1507.04544).

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2019). Pareto smoothed importance sampling. preprint arXiv:1507.02646

Examples

# \donttest{
fit_ds <- rater(anesthesia, "dawid_skene", verbose = FALSE, chains = 1)
fit_ccds <- rater(anesthesia, "class_conditional_dawid_skene",
                  verbose = FALSE, chains = 1)

loo_ds <- loo(fit_ds)
loo_ccds <- loo(fit_ccds)

# To compare the loos easily we can use the loo_compare function from the
# loo package:
library(loo)

loo_compare(loo_ds, loo_ccds)

# The documentation of the loo package contains more information about how
# the output should be interpreted.
# }