TreeBUGS (version 1.5.0)

WAIC: WAIC: Widely Applicable Information Criterion

Description

Implementation of the WAIC for model comparison.

Usage

WAIC(
  fittedModel,
  n.adapt = 1000,
  n.chains = 3,
  n.iter = 10000,
  n.thin = 1,
  summarize = FALSE
)
# S3 method for waic
print(x, ...)
# S3 method for waic_difference
print(x, ...)
# S3 method for waic
-(e1, e2)

Value

Function WAIC() returns an object of class waic, which is basically a list containing three vectors p_waic, deviance, and waic, with separate values for each observed node (i.e., for all combinations of persons and free categories).

For these objects, a print() method exists, which also calculates the standard error of the estimate of WAIC.

For backwards compatibility, if WAIC() is called with summarize = TRUE, a vector with values p_waic, deviance, waic, and se_waic is returned.

WAIC values from two models can be compared by using the - operator; the result is an object of class waic_difference.

Arguments

fittedModel

fitted latent-trait or beta MPT model (traitMPT, betaMPT)

n.adapt

number of adaptation samples.

n.chains

number of chains (no parallel computation).

n.iter

number of iterations after burnin.

n.thin

Thinning rate.

summarize

deprecated argument only available for backwards compatibility

x

An object of class waic or waic_difference to be printed.

...

Further arguments that may be passed to print methods.

e1, e2

Two objects of class waic to be compared.

Details

WAIC provides an approximation of predictive accuracy with respect to out-of-sample deviance. The uncertainty of the WAIC for the given number of observed nodes (i.e., number of free categories times the number of participants) is quantified by the standard error of WAIC "se_waic" (cf. Vehtari et al., 2017). In contrast, to assess whether the approximation uncertainty due to MCMC sampling (not sample size) is sufficiently low, it is a good idea to fit each model twice and compute WAIC again to assess the stability of the WAIC values.

For more details, see Vehtari et al. (2017) and the following discussion about the JAGS implementation (which is currently an experimental feature of JAGS 4.3.0):

https://sourceforge.net/p/mcmc-jags/discussion/610036/thread/8211df61/

References

Vehtari, A., Gelman, A., & Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27(5), 1413–1432. doi:10.1007/s11222-016-9696-4

Examples

if (FALSE) {

#### WAIC for a latent-trait MPT model:
fit <- traitMPT(...)
WAIC(fit)


#### pairwise comparison of two models:

# (1) compute WAIC per model
waic1 <- WAIC(fit1)
waic2 <- WAIC(fit2)

# (2) WAIC difference
waic1 - waic2
}