loo (version 0.1.5)

loo: Leave-one-out cross-validation (LOO)

Description

Efficient approximate leave-one-out cross-validation for Bayesian models.

Usage

loo(x, ...)
## S3 method for class 'matrix':
loo(x, ...)
## S3 method for class 'function':
loo(x, ..., args)

Arguments

x

A log-likelihood matrix or function. See the Methods (by class) section below for a detailed description.

...

Optional arguments to pass to psislw. Possible arguments and their defaults are: [object Object],[object Object],[object Object]

We recommend using the default values for the psislw argument

args

Only required if x is a function. A list containing the data required to specify the arguments to the function. See the Methods (by class) section below for how args should be specified.

Value

• A named list with class 'loo' and components:

  [object Object],[object Object],[object Object],[object Object],[object Object]

Methods (by class)

• matrix: An$S$by$N$matrix, where$S$is the size of the posterior sample (the number of simulations) and$N$is the number of data points. Typically (but not restricted to be) the object returned byextract_log_lik.
• function: A function$f$that takes argumentsi,data, anddrawsand returns a vector containing the log-likelihood for theith observation evaluated at each posterior draw.

  Theargsargument must also be specified and should be a named list with the following components:

  • draws: An object containing the posterior draws for any parameters needed to compute the pointwise log-likelihood.
  • data: An object containing data (e.g. observed outcome and predictors) needed to compute the pointwise log-likelihood.datashould be in an appropriate form so that$f$(i=i, data=data[i,,drop=FALSE], draws=draws)returns theS-vector of log-likelihoods for theith observation.
  • N: The number of observations.
  • S: The size of the posterior sample.

Details

PSIS-LOO{ We approximate LOO using Pareto Smoothed Importance Sampling (PSIS). See loo-package for details. } Memory{ For models fit to very large datasets we recommend the loo.function method, which is much more memory efficient than the loo.matrix method. However, the loo.matrix method is typically more convenient, so it is usually worth trying loo.matrix and then switching to loo.function if memory is an issue. }

See Also

loo-package for details on the Pareto Smoothed Importance Sampling (PSIS) procedure used for approximating LOO.

print.loo for the print and plot methods for 'loo' objects.

compare for model comparison.

Examples

### Usage with stanfit objects
log_lik1 <- extract_log_lik(stanfit1) # see ?extract_log_lik
loo1 <- loo(log_lik1)
print(loo1, digits = 3)

log_lik2 <- extract_log_lik(stanfit2)
(loo2 <- loo(log_lik2))
compare(loo1, loo2)

### Using log-likelihood function instead of matrix
set.seed(024)

# Simulate data and draw from posterior
N <- 50; K <- 10; S <- 100; a0 <- 3; b0 <- 2
p <- rbeta(1, a0, b0)
y <- rbinom(N, size = K, prob = p)
a <- a0 + sum(y); b <- b0 + N * K - sum(y)
draws <- rbeta(S, a, b)
data <- data.frame(y,K)

llfun <- function(i, data, draws) {
  dbinom(data$y, size = data$K, prob = draws, log = TRUE)
}
loo_with_fn <- loo(llfun, args = nlist(data, draws, N, S), cores = 1)

# Check that we get same answer if using log-likelihood matrix
log_lik_mat <- sapply(1:N, function(i) llfun(i, data[i,, drop=FALSE], draws))
loo_with_mat <- loo(log_lik_mat, cores = 1)
all.equal(loo_with_mat, loo_with_fn)