abctools (version 0.3-2)

mc.ci: Diagnostic plots for model choice coverage output

Description

Plots credible interval estimates for raw model choice output from cov.mc. This is used to investigate whether the coverage property holds and validate whether diagnostic statistics are acting as intended.

Usage

mc.ci(raw, tol, eps, modname, modtrue, nbins=5, bintype=c("interval",
"quantile"), bw=FALSE, ...)

Details

This function provides a plot which can be used as an informal test of the model choice coverage hypothesis for a particular value of eps or tol and choice of model. The plot is more flexible than the diagnostics, but not suitable as the basis of a formal test.

For each pseudo-observed data set, the ABC probability that the model is modname is taken from raw, and the true model is taken from modtrue. The probabilities are binned into nbins intervals, either of equal length or based on nbins+1 equally spaced empirical quantiles. The function estimates the observed probability of modname within each bin using Bayesian inference for a binomial proportion using a uniform prior. The plot shows the mean and 95% credible interval plotted against predicted probabilities. Informally, the coverage property should be rejected if predicted values are too unlikely given the observed values.

References

Prangle D., Blum M. G. B., Popovic G., Sisson S. A. (2013) Diagnostic tools of approximate Bayesian computation using the coverage property. (preprint) arxiv.org/abs/1301.3166

See Also

cov.mc to produce the input for this function

Examples

##The examples below are chosen to run relatively quickly (<5 mins)
  ##and do not represent recommended tuning choices.
  index <- sample(1:2, 1E4, replace=TRUE)
  sumstat <- ifelse(index==1, rnorm(1E4,0,1), rnorm(1E4,0,rexp(1E4,1)))
  sumstat <- data.frame(ss=sumstat)
  covdiag <- cov.mc(index=index, sumstat=sumstat, testsets=1:100, tol=seq(0.1,1,by=0.1),
  diagnostics=c("freq"))
  mc.ci(covdiag$raw, tol=0.5, modname=1, modtrue=index[1:100])