BAS (version 1.0.6)

bas.lm: Bayesian Adaptive Sampling Without Replacement for Variable Selection in Linear Models

Description

Sample without replacement from a posterior distribution on models

Usage

bas.lm(formula, data, n.models=NULL,  prior="ZS-null", alpha=NULL,
 modelprior=beta.binomial(1,1),
 initprobs="Uniform", method="BAS", update=NULL,
 bestmodel = NULL, bestmarg = NULL, prob.local = 0.0, prob.rw=0.5,
 MCMC.iterations = NULL, lambda = NULL, delta = 0.025,thin=1)

Arguments

formula

linear model formula for the full model with all predictors, Y ~ X. All code assumes that an intercept will be included in each model and that the X's will be centered.

data

data frame

n.models

number of models to sample either without replacement (method="BAS" or "MCMC+BAS") or with replacement (method="MCMC"). If NULL, BAS with method="BAS" will try to enumerate all 2^p models. If enumeration is not possible (memory or time) th

prior

prior distribution for regression coefficients. Choices include "AIC", "BIC", "g-prior", "ZS-null", "ZS-full", "hyper-g", "hyper-g-laplace", "hyper-g-n", "EB-local", and "EB-global"

alpha

optional hyperparameter in g-prior or hyper g-prior. For Zellner's g-prior, alpha = g, for the Liang et al hyper-g or hyper-g-n method, recommended choice is alpha are between (2 < alpha < 4), with alpha = 3 recommended.

modelprior

Family of prior distribution on the models. Choices include uniform Bernoulli or beta.binomial

initprobs

Vector of length p or a character string for approach used to create the vector. This is used to order variables for sampling all methods for potentially more efficient storage while sampling and provides the initial inclusion probabili

method

A character variable indicating which sampling method to use: method="BAS" uses Bayesian Adaptive Sampling (without replacement) using the sampling probabilities given in initprobs; method="MCMC" samples with replacement via a MCMC algorithm b

update

number of iterations between potential updates of the sampling probabilities for method "BAS". If NULL do not update, otherwise the algorithm will update using the marginal inclusion probabilities as they change while sampling takes place.

bestmodel

optional binary vector representing a model to initialize the sampling. If NULL sampling starts with the null model

bestmarg

optional value for the log marginal associated with the bestmodel

prob.local

A future option to allow sampling of models "near" the median probability model. Not used at this time.

prob.rw

For any of the MCMC methods, probability of using the random-walk proposal; otherwise use a random "flip" move to propose a new model.

MCMC.iterations

Number of iterations for the MCMC sampler; the default is n.models*10 if not set by the user.

lambda

Parameter in the AMCMC algorithm.

delta

truncation parameter to prevent sampling probabilities to degenerate to 0 or 1 prior to enumeration for sampling withour replacent.

thin

For "MCMC", thin the MCMC every "thin" iterations; default is no thinning.

Value

• bas returns an object of class BMA

  An object of class BAS is a list containing at least the following components:

• postprob
  the posterior probabilities of the models selected
• priorprobs
  the prior probabilities of the models selected
• namesx
  the names of the variables
• R2
  R2 values for the models
• logmarg
  values of the log of the marginal likelihood for the models
• n.vars
  total number of independent variables in the full model, including the intercept
• size
  the number of independent variables in each of the models, includes the intercept
• which
  a list of lists with one list per model with variables that are included in the model
• probne0
  the posterior probability that each variable is non-zero computed using the renormalized marginal likelihoods of sampled models. This may be biased if the number of sampled models is much smaller than the total number of models. Unbiased estimates may be obtained using method "MCMC".
• mle
  list of lists with one list per model giving the MLE (OLS) estimate of each (nonzero) coefficient for each model. The intercept is the mean of Y as each column of X has been centered by subtracting its mean.
• mle.se
  list of lists with one list per model giving the MLE (OLS) standard error of each coefficient for each model
• prior
  the name of the prior that created the BMA object
• alpha
  value of hyperparameter in prior used to create the BMA object.
• modelprior
  the prior distribution on models that created the BMA object
• Y
  response
• X
  matrix of predictors
• mean.x
  vector of means for each column of X (used in predict.bas)
• The function summary.bas, is used to print a summary of the results. The function plot.bma is used to plot posterior distributions for the coefficients and image.bma provides an image of the distribution over models. Posterior summaries of coefficients can be extracted using coefficients.bma. Fitted values and predictions can be obtained using the S3 functions fitted.bas and predict.bas. BA objects may be updated to use a different prior (without rerunning the sampler) using the function update.bma.

Details

BAS provides several search algorithms to find high probability models for use in Bayesian Model Averaging or Bayesian model selection. For p less than 20-25, BAS can enumerate all models depending on memory availability, for larger p, BAS samples without replacement using random or deterministic sampling. The Bayesian Adaptive Sampling algorithm of Clyde, Ghosh, Littman (2010) samples models without replacement using the initial sampling probabilities, and will optionally update the sampling probabilities every "update" models using the estimated marginal inclusion probabilties. BAS uses different methods to obtain the initprobs, which may impact the results in high-dimensional problems. The deterinistic sampler provides a list of the top models in order of an approximation of independence using the provided initprobs. This may be effective after running the other algorithms to identify high probability models and works well if the correlations of variables are small to modest. The priors on coefficients include Zellner's g-prior, the Hyper-g prior (Liang et al 2008, the Zellner-Siow Cauchy prior, Empirical Bayes (local and gobal) g-priors. AIC and BIC are also included.

References

Clyde, M. Ghosh, J. and Littman, M. (2010) Bayesian Adaptive Sampling for Variable Selection and Model Averaging. Journal of Computational Graphics and Statistics. 20:80-101 http://dx.doi.org/10.1198/jcgs.2010.09049 Clyde, M. and George, E. I. (2004) Model Uncertainty. Statist. Sci., 19, 81-94. http://dx.doi.org/10.1214/088342304000000035 Clyde, M. (1999) Bayesian Model Averaging and Model Search Strategies (with discussion). In Bayesian Statistics 6. J.M. Bernardo, A.P. Dawid, J.O. Berger, and A.F.M. Smith eds. Oxford University Press, pages 157-185.

Hoeting, J. A., Madigan, D., Raftery, A. E. and Volinsky, C. T. (1999) Bayesian model averaging: a tutorial (with discussion). Statist. Sci., 14, 382-401. http://www.stat.washington.edu/www/research/online/hoeting1999.pdf Liang, F., Paulo, R., Molina, G., Clyde, M. and Berger, J.O. (2008) Mixtures of g-priors for Bayesian Variable Selection. Journal of the American Statistical Association. 103:410-423. http://dx.doi.org/10.1198/016214507000001337

Zellner, A. (1986) On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti, pp. 233-243. North-Holland/Elsevier. Zellner, A. and Siow, A. (1980) Posterior odds ratios for selected regression hypotheses. In Bayesian Statistics: Proceedings of the First International Meeting held in Valencia (Spain), pp. 585-603.

See Also

summary.bas, coefficients.bma, print.bas, predict.bas, fitted.bas plot.bma, image.bma, eplogprob, update.bma

Examples

demo(BAS.hald)
demo(BAS.USCrime)