Model-based Gradient Boosting

Gradient boosting for optimizing arbitrary loss functions, where component-wise models are utilized as base-learners.

models, nonlinear
mboost(formula, data = list(), 
       baselearner = c("bbs", "bols", "btree", "bss", "bns"), ...) 
mboost_fit(blg, response, weights = NULL, offset = NULL,
           family = Gaussian(), control = boost_control())
a symbolic description of the model to be fit.
a data frame containing the variables in the model.
a character specifying the component-wise base learner to be used: bbs means P-splines with a B-spline basis (see Schmid and Hothorn 2008), bols linea
a list of objects of class blg, as returned by all base-learners.
the response variable.
an optionally numeric vector of weights.
an optionally numeric vector to be used as offset.
a Family object.
a list of parameters controlling the algorithm.
additional arguments passed to mboost.

The function implements component-wise functional gradient boosting in a generic way. Basically, the algorithm is initialized with a function for computing the negative gradient of the loss function (via its family argument) and one or more base-learners (given as blg).

The algorithm minimized the in-sample empirical risk defined as the weighted sum (by weights) of the loss function (corresponding to the negative gradient) evaluated at the data.

The structure of the model is determined by the structure of the base-learners. If more than one base-learner is given, the model is additive in these components.

Base-learners can be specified via a formula interface (function mboost) or as a list of objects of class bl, see bols.

Note that the more convenient modelling interfaces gamboost, glmboost and blackboost all call mboost directly.



Peter Buehlmann and Bin Yu (2003), Boosting with the L2 loss: regression and classification. Journal of the American Statistical Association, 98, 324--339.

Peter Buehlmann and Torsten Hothorn (2007), Boosting algorithms: regularization, prediction and model fitting. Statistical Science, 22(4), 477--505.

Yoav Freund and Robert E. Schapire (1996), Experiments with a new boosting algorithm. In Machine Learning: Proc. Thirteenth International Conference, 148--156.

Jerome H. Friedman (2001), Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29, 1189--1232.

See Also

glmboost for boosted linear models and blackboost for boosted trees. See e.g. bbs for possible base-learners. See cvrisk for cross-validated stopping iteration. Furthermore see boost_control, Family and methods

  • mboost
  • mboost_fit
data("bodyfat", package = "mboost")

  ### formula interface: additive Gaussian model with
  ### a non-linear step-function in `age', a linear function in `waistcirc'
  ### and a smooth non-linear smooth function in `hipcirc'
  mod <- mboost(DEXfat ~ btree(age) + bols(waistcirc) + bbs(hipcirc), 
                data = bodyfat)
  layout(matrix(1:6, nc = 3, byrow = TRUE))
  plot(mod, ask = FALSE, main = "formula")

  ### the same
       mod <- mboost_fit(list(btree(age), bols(waistcirc), bbs(hipcirc)), 
                         response = DEXfat))
  plot(mod, ask = FALSE, main = "base-learner")
Documentation reproduced from package mboost, version 2.0-1, License: GPL-2

Community examples

Looks like there are no examples yet.