SZVDcv: Cross-validation of sparse zero variance discriminant analysis

Description

Applies alternating direction methods of multipliers to solve sparse zero variance discriminant analysis.

Usage

SZVDcv(Atrain, ...)
# S3 method for default
SZVDcv(
  Atrain,
  Aval,
  k,
  num_gammas,
  g_mults,
  D,
  sparsity_pen,
  scaling,
  penalty,
  beta,
  tol,
  ztol,
  maxits,
  quiet,
  ...
)

Value

SZVDcv returns an object of class "SZVDcv" including a list with the following named components:

DVs: Discriminant vectors for the best choice of gamma.
all_DVs: Discriminant vectors for all choices of gamma.
l0_DVs: Discriminant vectors for gamma minimizing cardinality.
mc_DVs: Discriminant vector minimizing misclassification.
gamma: Choice of gamma minimizing validation criterion.
gammas: Set of all gammas trained on.
max_g: Maximum value of gamma guaranteed to yield a nontrivial solution.
ind: Index of best gamma.
w0: unpenalized zero-variance discriminants (initial solutions) plus B and W, etc. from ZVD

NULL

Arguments

Atrain: Training data set.
...: Parameters passed to SZVD.default.
Aval: Validation set.
k: Number of classes within training and validation sets.
num_gammas: Number of gammas to train on.
g_mults: Parameters defining range of gammas to train, g_max*(c_min, c_max). Note that it is an array/vector with two elements.
D: Penalty dictionary basis matrix.
sparsity_pen: weight defining validation criteria as weighted sum of misclassification error and cardinality of discriminant vectors.
scaling: Whether to rescale data so each feature has variance 1.
penalty: Controls whether to apply reweighting of l1-penalty (using sigma = within-class std devs)
beta: Parameter for augmented Lagrangian term in the ADMM algorithm.
tol: Stopping tolerances for the ADMM algorithm, must have tol$rel and tol$abs.
ztol: Threshold for truncating values in DVs to zero.
maxits: Maximum number of iterations used in the ADMM algorithm.
quiet: Controls display of intermediate results.

Details

This function might require a wrapper similar to ASDA.

Examples

Run this code

  P <- 300 # Number of variables
  N <- 50 # Number of samples per class

  # Mean for classes, they are zero everywhere except the first 3 coordinates
  m1 <- rep(0,P)
  m1[1] <- 3

  m2 <- rep(0,P)
  m2[2] <- 3

  m3 <- rep(0,P)
  m3[3] <- 3

  # Sample dummy data
  Xtrain <- rbind(MASS::mvrnorm(n=N,mu = m1, Sigma = diag(P)),
                 MASS::mvrnorm(n=N,mu = m2, Sigma = diag(P)),
                MASS::mvrnorm(n=N,mu = m3, Sigma = diag(P)))
 Xval <- rbind(MASS::mvrnorm(n=N,mu = m1, Sigma = diag(P)),
                 MASS::mvrnorm(n=N,mu = m2, Sigma = diag(P)),
                MASS::mvrnorm(n=N,mu = m3, Sigma = diag(P)))

  # Generate the labels
  Ytrain <- rep(1:3,each=N)
  Yval <- rep(1:3,each=N)


  # Train the classifier and increase the sparsity parameter from the default
  # so we penalize more for non-sparse solutions.

  res <- accSDA::SZVDcv(cbind(Ytrain,Xtrain),cbind(Yval,Xval),num_gammas=4,
                        g_mults = c(0,1),beta=2.5,
                        D=diag(P), maxits=100,tol=list(abs=1e-3,rel=1e-3), k = 3,
                        ztol=1e-4,sparsity_pen=0.3,quiet=FALSE,penalty=TRUE,scaling=TRUE)