monomvn (version 1.2)

blasso: Bayesian Lasso Regression

Description

Ordinary least squares and Lasso regression by sampling from the Bayesian posterior distribution via Gibbs Sampling

Usage

blasso(X, y, T = 100, thin = 10, lambda2 = 1, s2 = 1,
       tau2i = rep(1, ncol(X)), r = 1, delta = 1, a = 0, b = 0,
       rao.s2 = TRUE, normalize = TRUE, verb = 1)

Arguments

X

data.frame, matrix, or vector of inputs X

y

vector of output responses y of length equal to the leading dimension (rows) of X, i.e., length(y) == nrow(X)

T

Total number of MCMC samples to be collected

thin

number of MCMC samples to skip before a sample is collected (via thinning)

lambda2

square of the initial lasso penalty parameter. If zero, the least squares regressions are used

s2

initial variance parameter

tau2i

initial vector of lasso latent-variables along the diagonal of the covariance matrix in the prior for beta

r

alpha (shape) parameter to the gamma distribution prior for the lasso parameter lambda

delta

beta (rate) parameter to the gamma distribution prior for the lasso parameter lambda

a

alpha (shape) parameter for the inverse-gamma distribution prior for the variance parameter s2

b

beta (scale) parameter for the inverse-gamma distribution prior for the variance parameter s2

rao.s2

indicates whether to use Rao-Blackwellized samples for $\sigma^2$ should be used (default TRUE), see the details section, below

normalize

if TRUE, each variable is standardized to have unit L2 norm, otherwise it is left alone. Default is TRUE

verb

verbosity level; currently only verb = 0 and verb = 1 are supported

Value

• blasso returns an object of class "blasso", which is a list containing a copy of all of the input arguments as well as of the components listed below.
• call
  a copy of the function call as used
• mu
  a vector of T samples from the (un-penalized) intercept parameter
• beta
  a T*ncol(X) matrix of T samples from the (penalized) regression coefficients
• s2
  a vector of T samples from the variance parameter
• lambda2
  a vector of T samples from the penalty parameter
• tau2i
  a T*ncol(X) matrix of T samples from the (latent) inverse diagonal of the prior covariance matrix for beta

Details

The Bayesian lasso model and Gibbs Sampling algorithm is described in detail in Park & Casella (2008). The algorithm implemented by this function is identical to that described therein., with the exception of the option to use a Rao-Blackwellized sample of $\sigma^2$ (with $\beta$ integrated out) for improved mixing. When input argument lambda2 = 0 is supplied, the model is a simple hierarchical linear model where the prior for $\beta$ has mean zero and diagonal covariance matrix with diagonal 1/tau2i

References

Park, T., Casella, G. (2008). The Bayesian Lasso, (unpublished) http://www.stat.ufl.edu/~casella/Papers/bayeslasso.pdf

http://www.statslab.cam.ac.uk/~bobby/monomvn.html

See Also

lm , lars in the lars package, regress

Examples

## following the lars diabetes example
data(diabetes)
attach(diabetes)

## Ordinary Least Squares regression
reg.ols <- regress(x, y)

## Lasso regression
reg.las <- regress(x, y, method="lasso")

## Bayesian Lasso regression
reg.blas <- blasso(x, y, T=1000)

## summarize the beta (regression coefficients) estimates
plot(reg.blas, burnin=200)
points(drop(reg.las$b), col=2, pch=19)
points(drop(reg.ols$b), col=3, pch=18)
abline(h=0, lty=2, lwd=2)
legend("topleft", c("lasso", "lsr"), col=2:3, pch=19:18)

## summarize s2
plot(reg.blas, burnin=200, which="s2")

## summarize the posterior distribution of lambda2 and s2
detach(diabetes)