VDA (version 1.01)

VDA_LE: Multicategory Vertex Discriminant Analysis (VDA) For High-Dimensional Data

Description

The method of vertex discriminant analysis (VDA) is ideally suited to handle multiple categories and an excess of predictors over training cases. VDA_LE is an elaboration of VDA that simultaneously conducts classification of $k$ possible categories and variable selection of p features, based on a data set of n cases. Variable selection is imposed using L1 (Lasso) and group Euclidean penalties.

Usage

VDA_LE(x, y, lambda1, lambda2)

Arguments

x

n x p matrix or data frame containing the cases for each feature. The rows correspond to cases and the columns to the features. Intercept column is not included in this.

y

n x 1 vector representing the outcome variable. Each element denotes which one of the k classes that case belongs to

lambda1

Tuning parameter to control the lasso penalty. The default value is $1/n$. For determining the optimal lambda1, refer to cv.VDA_LE.

lambda2

Tuning parameter to control the Euclidean penalty. The default value is 0.01. For determining the optimal lambda1, refer to cv.VDA_LE

Value

• feature
  Feature matrix x with an intercept vector added as the first column. All entries in the first column should equal 1.
• stand.feature
  The feature matrix where the all columns are standardized, with the exception of the intercept column which is left unstandardized.
• class
  Class vector y. All elements should be integers between 1 and classes.
• cases
  Number of cases, n.
• classes
  Number of classes, k.
• features
  Number of feautres, p.
• lambda
  Vector of tuning constants where the first component is lambda1 and the second is lambda2
• predicted
  Vector of predicted category values based on VDA.
• coefficient
  The estimated coefficient matrix where the columns represent the coefficients for each predictor variable corresponding to k-1 outcome categories. The coefficient matrix is used for classifying new cases.
• training_error_rate
  The percentage of instances in the training set where the predicted outcome category is not equal to the case's true category.
• nonzeros
  Number of feature coefficients retained in the model. Is equal to p - number of features eliminated by penalization.
• selected
  An integer vector which represents the attributes that were selected after penalization.
• call
  The matched call.

Details

VDA_LE carries out cyclic coordinate descent in the context of VDA to minimize the loss function. By adding lasso ($L1$-norm) and group Euclidean penalties to the VDA loss function, unnecessary predictors are elimnated, adding parsimony and making the model more interpretable. Lasso penalties are applied to each predictor coefficient separately, while Euclidean penalties couples the coefficients of a single predictor and penalize the group collectively. If lambda1=0, then the overall penalty reduces to only group penalties. When lambda2=0, then the overall penalty reduces to the lasso. With these penalties in place, cyclic coordinate descent accelerates estimation of all coefficients.

References

Wu, T.T. and Lange, K. (2010) Multicategory Vertex Discriminant Analysis for High-Dimensional Data. Annals of Applied Statistics, Volume 4, No 4, 1698-1721. Lange, K. and Wu, T.T. (2008) An MM Algorithm for Multicategory Vertex Discriminant Analysis. Journal of Computational and Graphical Statistics, Volume 17, No 3, 527-544.

See Also

For determining the optimal values for lambda1 and lambda2, see cv.VDA_LE

For VDA without variable selection, see VDA_R.

Examples

#load dataset from package
data(zoo)

#matrix containing all predictor vectors
x <- zoo[,2:17]

#outcome class vector
y <- zoo[,18]

#run VDA
out<-VDA_LE(x,y)