Fits generalized linear models by penalized maximum likelihood estimation.
The coefficients path is computed for the regression model over a grid of the regularization parameter lambda.
Fits Gaussian (linear), binomial Logit (logistic), Poisson, multinomial Logit regression models, and
Cox proportional hazard model with various non-convex penalties.
ncpen.reg(formula, data, family = c("gaussian", "linear", "binomial",
"logit", "multinomial", "cox", "poisson"), penalty = c("scad", "mcp",
"tlp", "lasso", "classo", "ridge", "sridge", "mbridge", "mlog"),
x.standardize = TRUE, intercept = TRUE, lambda = NULL,
n.lambda = NULL, r.lambda = NULL, w.lambda = NULL, gamma = NULL,
tau = NULL, alpha = NULL, df.max = 50, cf.max = 100,
proj.min = 10, add.max = 10, niter.max = 30, qiter.max = 10,
aiter.max = 100, b.eps = 1e-07, k.eps = 1e-04, c.eps = 1e-06,
cut = TRUE, local = FALSE, local.initial = NULL)(formula) regression formula. To include/exclude intercept, use intercept option
instead of using the "0 +" option in the formula.
The y value must be 0,1 for binomial and 1,2,..., for multinomial.
(numeric matrix or data.frame) contains both y and X. Each row is an observation vector.
The censoring indicator must be included at the last column of the data for cox.
(character) regression model. Supported models are
gaussian (or linear),
binomial (or logit),
poisson,
multinomial,
and cox.
Default is gaussian.
(character) penalty function.
Supported penalties are
scad (smoothly clipped absolute deviation),
mcp (minimax concave penalty),
tlp (truncated LASSO penalty),
lasso (least absolute shrinkage and selection operator),
classo (clipped lasso = mcp + lasso),
ridge (ridge),
sridge (sparse ridge = mcp + ridge),
mbridge (modified bridge) and
mlog (modified log).
Default is scad.
(logical) whether to standardize x.mat prior to fitting the model (see details).
The estimated coefficients are always restored to the original scale.
(logical) whether to include an intercept in the model.
(numeric vector) user-specified sequence of lambda values.
Default is supplied automatically from samples.
(numeric) the number of lambda values.
Default is 100.
(numeric) ratio of the smallest lambda value to largest.
Default is 0.001 when n>p, and 0.01 for other cases.
(numeric vector) penalty weights for each coefficient (see references). If a penalty weight is set to 0, the corresponding coefficient is always nonzero.
(numeric) additional tuning parameter for controlling shrinkage effect of classo and sridge (see references).
Default is half of the smallest lambda.
(numeric) concavity parameter of the penalties (see reference).
Default is 3.7 for scad, 2.1 for mcp, classo and sridge, 0.001 for tlp, mbridge and mlog.
(numeric) ridge effect (weight between the penalty and ridge penalty) (see details).
Default value is 1. If penalty is ridge and sridge then alpha is set to 0.
(numeric) the maximum number of nonzero coefficients.
(numeric) the maximum of absolute value of nonzero coefficients.
(numeric) the projection cycle inside CD algorithm (largely internal use. See details).
(numeric) the maximum number of variables added in CCCP iterations (largely internal use. See references).
(numeric) maximum number of iterations in CCCP.
(numeric) maximum number of quadratic approximations in each CCCP iteration.
(numeric) maximum number of iterations in CD algorithm.
(numeric) convergence threshold for coefficients vector.
(numeric) convergence threshold for KKT conditions.
(numeric) convergence threshold for KKT conditions (largely internal use).
(logical) convergence threshold for KKT conditions (largely internal use).
(logical) whether to use local initial estimator for path construction. It may take a long time.
(numeric vector) initial estimator for local=TRUE.
An object with S3 class ncpen.
response vector.
design matrix.
regression model.
penalty.
whether to standardize x.mat=TRUE.
whether to include the intercept.
scale factor for x.standardize.
sequence of lambda values.
penalty weights.
extra shrinkage parameter for classo and sridge only.
ridge effect.
whether to use local initial estimator.
local initial estimator for local=TRUE.
fitted coefficients. Use coef.ncpen for multinomial since the coefficients are represented as vectors.
the number of non-zero coefficients.
The sequence of models indexed by lambda is fit
by using concave convex procedure (CCCP) and coordinate descent (CD) algorithm (see references).
The objective function is $$ (sum of squared residuals)/2n + [alpha*penalty + (1-alpha)*ridge] $$ for gaussian
and $$ (log-likelihood)/n - [alpha*penalty + (1-alpha)*ridge] $$ for the others,
assuming the canonical link.
The algorithm applies the warm start strategy (see references) and tries projections
after proj.min iterations in CD algorithm, which makes the algorithm fast and stable.
x.standardize makes each column of x.mat to have the same Euclidean length
but the coefficients will be re-scaled into the original.
In multinomial case, the coefficients are expressed in vector form. Use coef.ncpen.
Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American statistical Association, 96, 1348-60. Zhang, C.H. (2010). Nearly unbiased variable selection under minimax concave penalty. The Annals of statistics, 38(2), 894-942. Shen, X., Pan, W., Zhu, Y. and Zhou, H. (2013). On constrained and regularized high-dimensional regression. Annals of the Institute of Statistical Mathematics, 65(5), 807-832. Kwon, S., Lee, S. and Kim, Y. (2016). Moderately clipped LASSO. Computational Statistics and Data Analysis, 92C, 53-67. Kwon, S. Kim, Y. and Choi, H.(2013). Sparse bridge estimation with a diverging number of parameters. Statistics and Its Interface, 6, 231-242. Huang, J., Horowitz, J.L. and Ma, S. (2008). Asymptotic properties of bridge estimators in sparse high-dimensional regression models. The Annals of Statistics, 36(2), 587-613. Zou, H. and Li, R. (2008). One-step sparse estimates in nonconcave penalized likelihood models. Annals of statistics, 36(4), 1509. Lee, S., Kwon, S. and Kim, Y. (2016). A modified local quadratic approximation algorithm for penalized optimization problems. Computational Statistics and Data Analysis, 94, 275-286.
# NOT RUN {
### linear regression with scad penalty
sam = sam.gen.ncpen(n=200,p=5,q=5,cf.min=0.5,cf.max=1,corr=0.5,family="gaussian")
x.mat = sam$x.mat; y.vec = sam$y.vec
data = cbind(y.vec, x.mat)
colnames(data) = c("y", paste("xv", 1:ncol(x.mat), sep = ""))
fit1 = ncpen.reg(formula = y ~ xv1 + xv2 + xv3 + xv4 + xv5, data = data,
family="gaussian", penalty="scad")
fit2 = ncpen(y.vec=y.vec,x.mat=x.mat);
# }
Run the code above in your browser using DataLab