The SVM has a regularization or cost parameter C, which controls the amount by which points overlap their soft margins. Typically either a default large value for C is chosen (allowing minimal overlap), or else a few values are compared using a validation set. This algorithm computes the entire regularization path (i.e. for all possible values of C for which the solution changes), with a cost a small (~3) multiple of the cost of fitting a single model.
svmpath(x, y, K, kernel.function = poly.kernel, param.kernel = 1, trace,
plot.it, eps = 1e-10, Nmoves = 3 * n, digits = 6, lambda.min = 1e-04,ridge=0, ...)
the data matrix (n x p) with n rows (observations) on p variables (columns)
The "-1,+1" valued response variable.
a n x n kernel matrix, with default value K= kernel.function(x, x)
This is a user-defined function. Provided are
poly.kernel
(the default, with parameter set to default to a
linear kernel) and radial.kernel
parameter(s) of the kernels
if TRUE
, a progress report is printed as the
algorithm runs; default is FALSE
a flag indicating whether a plot should be produced
(default FALSE
; only usable with p=2
a small machine number which is used to identify minimal step sizes
the maximum number of moves
the number of digits in the printout
The smallest value of lambda = 1/C
; default
is lambda=10e-4
, or C=10000
Sometimes the algorithm encounters singularities; in this
case a small value of ridge, around 1e-12, can help. Default is ridge=0
additional arguments to some of the functions called by
svmpath. One such argument that can be passed is ridge
(default
is 1e-10). This is used to produce "stable" solutions to linear equations.
a "svmpath" object is returned, for which there are print, summary, coef and predict methods.
Currently the algorithm can get into machine errors if
epsilon
is too small, or if lambda.min
is too
small. Increasing either from their defaults should make the problems
go away, by terminating the algorithm slightly early.
The algorithm used in svmpath()
is described in detail in
"The Entire Regularization Path for the Support Vector Machine" by
Hastie, Rosset, Tibshirani and Zhu (2004). It exploits the fact that
the "hinge" loss-function is piecewise linear, and the penalty term is
quadratic. This means that in the dual space, the lagrange multipliers
will be pieceise linear (c.f. lars
).
The paper http://www-stat.stanford.edu/~hastie/Papers/svmpath.pdf, as well as the talk http://www-stat.stanford.edu/~hastie/TALKS/svmpathtalk.pdf.
print, coef, summary, predict, and FilmPath
# NOT RUN {
data(svmpath)
attach(unbalanced.separated)
svmpath(x,y,trace=TRUE,plot=TRUE)
detach(2)
# }
# NOT RUN {
svmpath(x,y,kernel=radial.kernel,param.kernel=.8)
# }
Run the code above in your browser using DataLab