Gradient boosting for optimizing loss functions with componentwise linear, smoothing splines, tree models as base learners.
bst(x, y, cost = 0.5, family = c("gaussian", "hinge", "hinge2", "binom", "expo",
"poisson", "tgaussianDC", "thingeDC", "tbinomDC", "binomdDC", "texpoDC", "tpoissonDC",
"huber", "thuberDC", "clossR", "clossRMM", "closs", "gloss", "qloss", "clossMM",
"glossMM", "qlossMM", "lar"), ctrl = bst_control(), control.tree = list(maxdepth = 1),
learner = c("ls", "sm", "tree"))
# S3 method for bst
print(x, ...)
# S3 method for bst
predict(object, newdata=NULL, newy=NULL, mstop=NULL,
type=c("response", "all.res", "class", "loss", "error"), ...)
# S3 method for bst
plot(x, type = c("step", "norm"),...)
# S3 method for bst
coef(object, which=object$ctrl$mstop, ...)
# S3 method for bst
fpartial(object, mstop=NULL, newdata=NULL)
An object of class bst
with print
, coef
,
plot
and predict
methods are available for linear models.
For nonlinear models, methods print
and predict
are available.
These are input variables and parameters
the input ctrl
with possible updated fk
if family="thingeDC", "tbinomDC", "binomdDC"
predicted function estimates
a list of length mstop
. Each element is a fitted model to the pseudo residuals, defined as negative gradient of loss function at the current estimated function
the last element of ens
a vector of length mstop
. Each element is the variable selected in each boosting step when applicable
selected variables in mstop
estimated coefficients in each iteration. Used internally only
a data frame containing the variables in the model.
vector of responses. y
must be in {1, -1} for family
= "hinge".
price to pay for false positive, 0 < cost
< 1; price of false negative is 1-cost
.
A variety of loss functions.
family
= "hinge" for hinge loss and family
="gaussian" for squared error loss.
Implementing the negative gradient corresponding
to the loss function to be minimized. For hinge loss, +1/-1 binary responses is used.
an object of class bst_control
.
control parameters of rpart.
a character specifying the component-wise base learner to be used:
ls
linear models,
sm
smoothing splines,
tree
regression trees.
class of bst
.
new data for prediction with the same number of columns as x
.
new response.
boosting iteration for prediction.
at which boosting mstop
to extract coefficients.
additional arguments.
Zhu Wang
Boosting algorithms for classification and regression problems. In a classification problem, suppose \(f\) is a classifier for a response \(y\). A cost-sensitive or weighted loss function is $$L(y,f,cost)=l(y,f,cost)\max(0, (1-yf))$$ For family="hinge"
,
$$l(y,f,cost)=
1-cost, if \, y= +1;
\quad cost, if \, y= -1$$ For family="hinge2"
,
l(y,f,cost)= 1, if y = +1 and f > 0 ; = 1-cost, if y = +1 and f < 0; = cost, if y = -1 and f > 0; = 1, if y = -1 and f < 0.
For twin boosting if twinboost=TRUE
, there are two types of adaptive boosting if learner="ls"
: for twintype=1
, weights are based on coefficients in the first round of boosting; for twintype=2
, weights are based on predictions in the first round of boosting. See Buehlmann and Hothorn (2010).
Zhu Wang (2011), HingeBoost: ROC-Based Boost for Classification and Variable Selection. The International Journal of Biostatistics, 7(1), Article 13.
Peter Buehlmann and Torsten Hothorn (2010), Twin Boosting: improved feature selection and prediction, Statistics and Computing, 20, 119-138.
cv.bst
for cross-validated stopping iteration. Furthermore see
bst_control
x <- matrix(rnorm(100*5),ncol=5)
c <- 2*x[,1]
p <- exp(c)/(exp(c)+exp(-c))
y <- rbinom(100,1,p)
y[y != 1] <- -1
x <- as.data.frame(x)
dat.m <- bst(x, y, ctrl = bst_control(mstop=50), family = "hinge", learner = "ls")
predict(dat.m)
dat.m1 <- bst(x, y, ctrl = bst_control(twinboost=TRUE,
coefir=coef(dat.m), xselect.init = dat.m$xselect, mstop=50))
dat.m2 <- rbst(x, y, ctrl = bst_control(mstop=50, s=0, trace=TRUE),
rfamily = "thinge", learner = "ls")
predict(dat.m2)
Run the code above in your browser using DataLab