bmrm (version 3.3)

nrbm: Convex and non-convex risk minimization with L2 regularization and limited memory

Description

Use algorithm of Do and Artieres, JMLR 2012 to find w minimizing: f(w) = 0.5*LAMBDA*l2norm(w) + riskFun(w) where riskFun is either a convex or a non-convex risk function.

Usage

nrbm(riskFun, LAMBDA = 1, MAX_ITER = 1000L, EPSILON_TOL = 0.01, w0 = 0,
  maxCP = 100L, convexRisk = TRUE, LowRankQP.method = "LU")

Arguments

riskFun

the risk function to use in the optimization (e.g.: hingeLoss, softMarginVectorLoss). The function must evaluate the loss value and its gradient for a given point vector (w).

LAMBDA

control the regularization strength in the optimization process. This is the value used as coefficient of the regularization term.

MAX_ITER

the maximum number of iteration to perform. The function stop with a warning message if the number of iteration exceed this value

EPSILON_TOL

control optimization stoping criteria: the optimization end when the optimization gap is below this threshold

w0

initial weight vector where optimization start

maxCP

mximal number of cutting plane to use to limit memory footprint

convexRisk

a length 1 logical telling if the risk function riskFun is convex. If TRUE, use CRBM algorithm; if FALSE use NRBM algorithm from Do and Artieres, JMLR 2012

LowRankQP.method

a single character value defining the method used by LowRankQP (should be either "LU" or "CHOL")

Value

the optimal weight vector (w)

References

Do and Artieres Regularized Bundle Methods for Convex and Non-Convex Risks JMLR 2012

Examples

# NOT RUN {
  set.seed(123)
  X <- matrix(rnorm(4000*200), 4000, 200)
  beta <- c(rep(1,ncol(X)-4),0,0,0,0)
  Y <- X%*%beta + rnorm(nrow(X))
  w <- nrbm(ladRegressionLoss(X/100,Y/100),maxCP=50)
  layout(1)
  barplot(w)
# }