Rdimtools (version 0.3.2)

do.enet: Elastic Net Regularization

Description

Elastic Net is a regularized regression method by solving $$\textrm{min}_{\beta} ~ \frac{1}{2}\|X\beta-y\|_2^2 + \lambda_1 \|\beta \|_1 + \lambda_2 \|\beta \|_2^2$$ where \(y\) iis response variable in our method. The method can be used in feature selection like LASSO.

Usage

do.enet(X, response, ndim = 2, preprocess = c("null", "center", "scale",
  "cscale", "decorrelate", "whiten"), ycenter = FALSE, lambda1 = 1,
  lambda2 = 1)

Arguments

X

an \((n\times p)\) matrix or data frame whose rows are observations and columns represent independent variables.

response

a length-\(n\) vector of response variable.

ndim

an integer-valued target dimension.

preprocess

an additional option for preprocessing the data. Default is "null". See also aux.preprocess for more details.

ycenter

a logical; TRUE to center the response variable, FALSE otherwise.

lambda1

\(\ell_1\) regularization parameter in \((0,\infty)\).

lambda2

\(\ell_2\) regularization parameter in \((0,\infty)\).

Value

a named list containing

Y

an \((n\times ndim)\) matrix whose rows are embedded observations.

featidx

a length-\(ndim\) vector of indices with highest scores.

trfinfo

a list containing information for out-of-sample prediction.

projection

a \((p\times ndim)\) whose columns are basis for projection.

References

zou_regularization_2005ADMM

Examples

# NOT RUN {
## generate swiss roll with auxiliary dimensions
## it follows reference example from LSIR paper.
n = 123
theta = runif(n)
h     = runif(n)
t     = (1+2*theta)*(3*pi/2)
X     = array(0,c(n,10))
X[,1] = t*cos(t)
X[,2] = 21*h
X[,3] = t*sin(t)
X[,4:10] = matrix(runif(7*n), nrow=n)

## corresponding response vector
y = sin(5*pi*theta)+(runif(n)*sqrt(0.1))

## try different regularization parameters
out1 = do.enet(X, y, lambda1=0.1, lambda2=0.1)
out2 = do.enet(X, y, lambda1=1,   lambda2=0.1)
out3 = do.enet(X, y, lambda1=10,  lambda2=0.1)
out4 = do.enet(X, y, lambda1=0.1, lambda2=1)
out5 = do.enet(X, y, lambda1=1,   lambda2=1)
out6 = do.enet(X, y, lambda1=10,  lambda2=1)
out7 = do.enet(X, y, lambda1=0.1, lambda2=10)
out8 = do.enet(X, y, lambda1=1,   lambda2=10)
out9 = do.enet(X, y, lambda1=10,  lambda2=10)

## visualize
## ( , ) denotes two regularization parameters
par(mfrow=c(3,3))
plot(out1$Y[,1], out1$Y[,2], main="ENET::(0.1,0.1)")
plot(out2$Y[,1], out2$Y[,2], main="ENET::(1,  0.1)")
plot(out3$Y[,1], out3$Y[,2], main="ENET::(10, 0.1)")
plot(out4$Y[,1], out4$Y[,2], main="ENET::(0.1,1)")
plot(out5$Y[,1], out5$Y[,2], main="ENET::(1,  1)")
plot(out6$Y[,1], out6$Y[,2], main="ENET::(10, 1)")
plot(out7$Y[,1], out7$Y[,2], main="ENET::(0.1,10)")
plot(out8$Y[,1], out8$Y[,2], main="ENET::(1,  10)")
plot(out9$Y[,1], out9$Y[,2], main="ENET::(10, 10)")
# }
# NOT RUN {
# }