Model-based Gradient Boosting
Gradient boosting for optimizing arbitrary loss functions, where component-wise models are utilized as base-learners.
mboost(formula, data = list(), baselearner = c("bbs", "bols", "btree", "bss", "bns"), ...) mboost_fit(blg, response, weights = NULL, offset = NULL, family = Gaussian(), control = boost_control())
- a symbolic description of the model to be fit.
- a data frame containing the variables in the model.
- a character specifying the component-wise base
learner to be used:
bbsmeans P-splines with a B-spline basis (see Schmid and Hothorn 2008),
- a list of objects of class
blg, as returned by all base-learners.
- the response variable.
- an optionally numeric vector of weights.
- an optionally numeric vector to be used as offset.
- a list of parameters controlling the algorithm.
- additional arguments passed to
The function implements component-wise functional gradient boosting
in a generic way. Basically,
the algorithm is initialized with a function for computing
the negative gradient of the loss function (via its
and one or more base-learners (given as
The algorithm minimized the in-sample empirical risk defined as
the weighted sum (by
weights) of the loss function (corresponding
to the negative gradient) evaluated at the data.
The structure of the model is determined by the structure of the base-learners. If more than one base-learner is given, the model is additive in these components.
Base-learners can be specified via a formula interface
mboost) or as a list of objects of class
Peter Buehlmann and Bin Yu (2003), Boosting with the L2 loss: regression and classification. Journal of the American Statistical Association, 98, 324--339.
Peter Buehlmann and Torsten Hothorn (2007), Boosting algorithms: regularization, prediction and model fitting. Statistical Science, 22(4), 477--505.
Yoav Freund and Robert E. Schapire (1996), Experiments with a new boosting algorithm. In Machine Learning: Proc. Thirteenth International Conference, 148--156.
Jerome H. Friedman (2001), Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29, 1189--1232.
glmboost for boosted linear models and
blackboost for boosted trees. See e.g.
for possible base-learners. See
cross-validated stopping iteration. Furthermore see
data("bodyfat", package = "mboost") ### formula interface: additive Gaussian model with ### a non-linear step-function in `age', a linear function in `waistcirc' ### and a smooth non-linear smooth function in `hipcirc' mod <- mboost(DEXfat ~ btree(age) + bols(waistcirc) + bbs(hipcirc), data = bodyfat) layout(matrix(1:6, nc = 3, byrow = TRUE)) plot(mod, ask = FALSE, main = "formula") ### the same with(bodyfat, mod <- mboost_fit(list(btree(age), bols(waistcirc), bbs(hipcirc)), response = DEXfat)) plot(mod, ask = FALSE, main = "base-learner")