A vector or matrix with class 'compare.loo'. If ... has
more than two objects then a matrix is returned. This matrix summarizes the
objects and also reports model weights (the posterior probability that each
model has the best expected out-of-sample predictive accuracy). If
... contains exactly two objects then the difference in expected
predictive accuracy and the standard error of the difference are returned
(see Details) in addition to model weights.
Details
When comparing two fitted models, we can estimate the difference in
their expected predictive accuracy by the difference in elpd_waic or
elpd_loo (multiplied by -2, if desired, to be on the deviance
scale). To compute the standard error of this difference we can use a
paired estimate to take advantage of the fact that the same set of $N$
data points is being used to fit both models. We would think that these
calculations would be most useful when $N$ is large, because then
non-normality of the distribution is not such an issue when estimating the
uncertainty of these sums. In any case, we suspect that these standard
errors, for all their flaws, should give a better sense of uncertainty than
what is obtained using the current standard approach of comparing
differences of deviances to a Chi-squared distribution, a practice derived
for Gaussian linear models or asymptotically and which only applies to
nested models in any case.