performance (version 0.8.0)

model_performance.stanreg: Performance of Bayesian Models

Description

Compute indices of model performance for (general) linear models.

Usage

# S3 method for stanreg
model_performance(model, metrics = "all", verbose = TRUE, ...)
# S3 method for BFBayesFactor
model_performance(
  model,
  metrics = "all",
  verbose = TRUE,
  average = FALSE,
  prior_odds = NULL,
  ...
)

Arguments

model

Object of class stanreg or brmsfit.

metrics

Can be "all", "common" or a character vector of metrics to be computed (some of c("LOOIC", "WAIC", "R2", "R2_adj", "RMSE", "SIGMA", "LOGLOSS", "SCORE")). "common" will compute LOOIC, WAIC, R2 and RMSE.

verbose

Toggle off warnings.

...

Arguments passed to or from other methods.

average

Compute model-averaged index? See bayestestR::weighted_posteriors().

prior_odds

Optional vector of prior odds for the models compared to the first model (or the denominator, for BFBayesFactor objects). For data.frames, this will be used as the basis of weighting.

Value

A data frame (with one row) and one column per "index" (see metrics).

Details

Depending on model, the following indices are computed:

• ELPD expected log predictive density. Larger ELPD values mean better fit. See looic().

• LOOIC leave-one-out cross-validation (LOO) information criterion. Lower LOOIC values mean better fit. See looic().

• WAIC widely applicable information criterion. Lower WAIC values mean better fit. See ?loo::waic.

• R2 r-squared value, see r2_bayes().

• R2_adjusted LOO-adjusted r-squared, see r2_loo().

• RMSE root mean squared error, see performance_rmse().

• SIGMA residual standard deviation, see insight::get_sigma().

• LOGLOSS Log-loss, see performance_logloss().

• SCORE_LOG score of logarithmic proper scoring rule, see performance_score().

• SCORE_SPHERICAL score of spherical proper scoring rule, see performance_score().

• PCP percentage of correct predictions, see performance_pcp().

References

Gelman, A., Goodrich, B., Gabry, J., & Vehtari, A. (2018). R-squared for Bayesian regression models. The American Statistician, The American Statistician, 1-6.

See Also

r2_bayes

Examples

# NOT RUN {
if (require("rstanarm") && require("rstantools")) {
  model <- stan_glm(mpg ~ wt + cyl, data = mtcars, chains = 1, iter = 500, refresh = 0)
  model_performance(model)

  model <- stan_glmer(
    mpg ~ wt + cyl + (1 | gear),
    data = mtcars,
    chains = 1,
    iter = 500,
    refresh = 0
  )
  model_performance(model)
}

if (require("BayesFactor") && require("rstantools")) {
  model <- generalTestBF(carb ~ am + mpg, mtcars)

  model_performance(model)
  model_performance(model[3])

  model_performance(model, average = TRUE)
}
# }