performance (version 0.10.7)

model_performance.stanreg: Performance of Bayesian Models

Description

Compute indices of model performance for (general) linear models.

Usage

# S3 method for stanreg
model_performance(model, metrics = "all", verbose = TRUE, ...)
# S3 method for BFBayesFactor
model_performance(
  model,
  metrics = "all",
  verbose = TRUE,
  average = FALSE,
  prior_odds = NULL,
  ...
)

Value

A data frame (with one row) and one column per "index" (see metrics).

Arguments

model

Object of class stanreg or brmsfit.

metrics

Can be "all", "common" or a character vector of metrics to be computed (some of c("LOOIC", "WAIC", "R2", "R2_adj", "RMSE", "SIGMA", "LOGLOSS", "SCORE")). "common" will compute LOOIC, WAIC, R2 and RMSE.

verbose

Toggle off warnings.

...

Arguments passed to or from other methods.

average

Compute model-averaged index? See bayestestR::weighted_posteriors().

prior_odds

Optional vector of prior odds for the models compared to the first model (or the denominator, for BFBayesFactor objects). For data.frames, this will be used as the basis of weighting.

Details

Depending on model, the following indices are computed:

• ELPD: expected log predictive density. Larger ELPD values mean better fit. See looic().

• LOOIC: leave-one-out cross-validation (LOO) information criterion. Lower LOOIC values mean better fit. See looic().

• WAIC: widely applicable information criterion. Lower WAIC values mean better fit. See ?loo::waic.

• R2: r-squared value, see r2_bayes().

• R2_adjusted: LOO-adjusted r-squared, see r2_loo().

• RMSE: root mean squared error, see performance_rmse().

• SIGMA: residual standard deviation, see insight::get_sigma().

• LOGLOSS: Log-loss, see performance_logloss().

• SCORE_LOG: score of logarithmic proper scoring rule, see performance_score().

• SCORE_SPHERICAL: score of spherical proper scoring rule, see performance_score().

• PCP: percentage of correct predictions, see performance_pcp().

References

Gelman, A., Goodrich, B., Gabry, J., and Vehtari, A. (2018). R-squared for Bayesian regression models. The American Statistician, The American Statistician, 1-6.

See Also

r2_bayes

Examples

if (FALSE) { # require("rstanarm") && require("rstantools") && require("BayesFactor")
# \donttest{
model <- suppressWarnings(rstanarm::stan_glm(
  mpg ~ wt + cyl,
  data = mtcars,
  chains = 1,
  iter = 500,
  refresh = 0
))
model_performance(model)

model <- suppressWarnings(rstanarm::stan_glmer(
  mpg ~ wt + cyl + (1 | gear),
  data = mtcars,
  chains = 1,
  iter = 500,
  refresh = 0
))
model_performance(model)

model <- BayesFactor::generalTestBF(carb ~ am + mpg, mtcars)

model_performance(model)
model_performance(model[3])
model_performance(model, average = TRUE)
# }
}