Spike-and-Slab LASSO is a spike-and-slab refinement of the LASSO procedure, using a mixture of Laplace priors indexed by lambda0 (spike) and lambda1 (slab).
The SSLASSO procedure fits coefficients paths for Spike-and-Slab LASSO-penalized
linear regression models over a grid of values for the regularization
parameter lambda0. The code has been adapted from the SSLASSO package (Rockova, V. and Moran, G. (2019). Package <U+2018>SSLASSO<U+2019>.) such that now it does NOT normalize each column and allows specifying initialization value).
SSLASSO_2(X, y, initial.beta, penalty = c("adaptive", "separable"),
variance = c("fixed", "unknown"), lambda1, lambda0, nlambda = 100,
theta = 0.5, sigma = 1, a = 1, b, eps = 0.001, max.iter = 500,
counter = 10, warn = FALSE)The design matrix (n x p), without an intercept. SSLASSO
standardizes the data by default.
Vector of continuous responses (n x 1). The responses will be centered by default.
Initial value for beta when searching for the solution.
The penalty to be applied to the model. Either "separable"
(with a fixed theta) or "adaptive" (with a random theta, where theta ~ B(a,p)). The default is "adaptive".
Whether the error variance is also estimated. Either "fixed" (with a fixed sigma) or "unknown" (with a random sigma, where p(sigma) ~ 1/sigma). The default is "fixed".
Slab variance parameter. Needs to be less than lambda0. The default is lambda0 = 1.
Spike penalty parameters (L x 1). Either a numeric value for a single run (L=1) or a sequence of increasing values for dynamic posterior exploration. The default is lambda0 = seq(1, nrow(X), length.out = 100).
The number of lambda0 values. Default is 100.
Prior mixing proportion. For "separable" penalty, this value is fixed. For "adaptive" penalty, this value is used as a starting value.
Error variance. For "fixed" variance, this value is fixed. For "unknown" variance, this value is used as a starting value.
Hyperparameter of the beta prior B(a,b) for the adaptive penalty (default a = 1).
Hyperparameter of the beta prior B(a,b) for the adaptive penalty (default b = ncol(X)).
Convergence criterion: converged when difference in regression coefficients is less than eps (default eps = 0.001).
Maximum number of iterations. Default is 500.
Applicable only for the adaptive penalty. Determines how often the parameter theta is updated throughout the cycles of coordinate ascent. Default is 10.
TRUE if warnings should be printed; FALSE by default
An object with S3 class "SSLASSO" containing:
The fitted matrix of coefficients (p x L). The number of rows is
equal to the number of coefficients p, and the number of columns is
equal to L (the length of lambda0).
A vector of length L containing the intercept for each value of lambda0. The intercept is intercept = mean(y) - crossprod(XX, beta), where XX is the centered design matrix.
A vector of length L containing the number
of iterations until convergence at each value of lambda0.
The sequence of regularization parameter values in the path.
Same as above.
A vector of length L containing the hyper-parameter values theta (the same as theta for "separable" penalty).
A vector of length L containing the values sigma (the same as the initial sigma for "known" variance).
A (p x L) binary matrix indicating which variables were selected along the solution path.
A single model chosen after the stabilization of the regularization path.
The sequence of models indexed by the regularization parameter
lambda0 is fitted using a coordinate descent algorithm. The algorithm uses
screening rules for discarding irrelevant predictors along the lines of Breheny (2011).
Rockova, V. and George, E.I. (2018) The Spike-and-Slab LASSO. Journal of the American Statistical Association.
Moran, G., Rockova, V. and George, E.I. (2018) On variance estimation for Bayesian variable selection. <https://arxiv.org/abs/1801.03019>.
Nie, L., & Ro<U+010D>kov<U+00E1>, V. (2020). Bayesian Bootstrap Spike-and-Slab LASSO. arXiv:2011.14279.
# NOT RUN {
## Linear regression, where p > n
p <- 1000
n <- 100
X <- matrix(rnorm(n*p), nrow = n, ncol = p)
beta <- c(1, 2, 3, rep(0, p-3))
y = X[,1] * beta[1] + X[,2] * beta[2] + X[,3] * beta[3] + rnorm(n)
# Oracle SSLASSO with known variance
result1 <- SSLASSO_2(X, y, penalty = "separable", theta = 3/p, initial.beta = rep(0,p))
plot(result1)
# Adaptive SSLASSO with known variance
result2 <- SSLASSO_2(X, y, initial.beta = rep(0,p))
plot(result2)
# }
Run the code above in your browser using DataLab