loo: Leave-one-out cross-validation (LOO)

Description

Efficient approximate leave-one-out cross-validation

Usage

loo(x, ...)
## S3 method for class 'matrix':
loo(x, ...)
## S3 method for class 'function':
loo(x, ..., args)

Arguments

A log-likelihood matrix or function. See the Methods (by class) section below for a detailed description.

...

Optional arguments to pass to psislw. Possible arguments and their defaults are: [object Object],[object Object],[object Object]

We recommend using the default values for the psislw argument

args

Only required if x is a function. A list containing the data required to specify the arguments to the function. See the Methods (by class) section below for how args should be specified.

Value

A named list with class 'loo' and components:
[object Object],[object Object],[object Object],[object Object],[object Object]

Methods (by class)

matrix: An$S$by$N$matrix, where$S$is the size of the posterior sample (the number of simulations) and$N$is the number of data points. Typically (but not restricted to be) the object returned byextract_log_lik.
function: A function$f$that takes argumentsi,data, anddrawsand returns a vector containing the log-likelihood for theith observation evaluated at each posterior draw.
Theargsargument must also be specified and should be a named list with the following components:
- draws: An object containing the posterior draws for any parameters needed to compute the pointwise log-likelihood.
- data: An object containing data (e.g. observed outcome and predictors) needed to compute the pointwise log-likelihood.datashould be in an appropriate form so that$f$(i=i, data=data[i,,drop=FALSE], draws=draws)returns theS-vector of log-likelihoods for theith observation.
- N: The number of observations.
- S: The size of the posterior sample.

Examples

Run this code

log_lik1 <- extract_log_lik(stanfit1)
loo1 <- loo(log_lik1)
print(loo1, digits = 3)

log_lik2 <- extract_log_lik(stanfit2)
loo2 <- loo(log_lik2)
loo2

loo_diff <- compare(loo1, loo2)
loo_diff
print(loo_diff, digits = 5)

### Example using log-likelihood function instead of matrix
set.seed(024)

# fake data and posterior draws
N <- 50; K <- 10; S <- 100; a0 <- 3; b0 <- 2
p <- rbeta(1, a0, b0)
y <- rbinom(N, size = K, prob = p)
a <- a0 + sum(y); b <- b0 + N * K - sum(y)
draws <- rbeta(S, a, b)
data <- data.frame(y,K)
llfun <- function(i, data, draws) {
  dbinom(data$y, size = data$K, prob = draws, log = TRUE)
}
loo_with_fn <- loo(llfun, args = nlist(data, draws, N, S), cores = 1)

# Check that we get same answer if using log-likelihood matrix
log_lik_mat <- sapply(1:N, function(i) llfun(i, data[i,, drop=FALSE], draws))
loo_with_mat <- loo(log_lik_mat, cores = 1)
all.equal(loo_with_mat, loo_with_fn)

Run the code above in your browser using DataLab

Get 50% off unlimited learning