report (version 0.5.9)

report.compare.loo: Reporting Bayesian Model Comparison

Description

Automatically report the results of Bayesian model comparison using the loo package.

Usage

# S3 method for compare.loo
report(x, include_IC = TRUE, include_ENP = FALSE, ...)

Value

Objects of class report_text().

Arguments

x

An object of class brms::loo_compare.

include_IC

Whether to include the information criteria (IC).

include_ENP

Whether to include the effective number of parameters (ENP).

...

Additional arguments (not used for now).

Details

The rule of thumb is that the models are "very similar" if |elpd_diff| (the absolute value of elpd_diff) is less than 4 (Sivula, Magnusson and Vehtari, 2020). If superior to 4, then one can use the SE to obtain a standardized difference (Z-diff) and interpret it as such, assuming that the difference is normally distributed. The corresponding p-value is then calculated as 2 * pnorm(-abs(Z-diff)). However, note that if the raw ELPD difference is small (less than 4), it doesn't make much sense to rely on its standardized value: it is not very useful to conclude that a model is much better than another if both models make very similar predictions.

Examples

if (FALSE) { # require("brms", quietly = TRUE)
# \donttest{
library(brms)

m1 <- brms::brm(mpg ~ qsec, data = mtcars)
m2 <- brms::brm(mpg ~ qsec + drat, data = mtcars)
m3 <- brms::brm(mpg ~ qsec + drat + wt, data = mtcars)

x <- brms::loo_compare(
  brms::add_criterion(m1, "loo"),
  brms::add_criterion(m2, "loo"),
  brms::add_criterion(m3, "loo"),
  model_names = c("m1", "m2", "m3")
)
report(x)
report(x, include_IC = FALSE)
report(x, include_ENP = TRUE)
# }
}