methods

0th

Percentile

Methods for Gradient Boosting Objects

Methods for models fitted by boosting algorithms.

Keywords
methods
Usage
## S3 method for class 'glmboost':
print(x, ...)
## S3 method for class 'mboost':
print(x, ...)

## S3 method for class 'mboost': summary(object, ...)

## S3 method for class 'mboost': coef(object, which = NULL, aggregate = c("sum", "cumsum", "none"), ...) ## S3 method for class 'glmboost': coef(object, which = NULL, aggregate = c("sum", "cumsum", "none"), off2int = FALSE, ...)

## S3 method for class 'mboost': [(x, i, return = TRUE, ...)

## S3 method for class 'mboost': AIC(object, method = c("corrected", "classical", "gMDL"), df = c("trace", "actset"), ..., k = 2)

## S3 method for class 'mboost': mstop(object, ...) ## S3 method for class 'gbAIC': mstop(object, ...) ## S3 method for class 'cvrisk': mstop(object, ...)

## S3 method for class 'mboost': predict(object, newdata = NULL, type = c("link", "response", "class"), which = NULL, aggregate = c("sum", "cumsum", "none"), ...) ## S3 method for class 'glmboost': predict(object, newdata = NULL, type = c("link", "response", "class"), which = NULL, aggregate = c("sum", "cumsum", "none"), ...)

## S3 method for class 'mboost': fitted(object, ...)

## S3 method for class 'mboost': residuals(object, ...) ## S3 method for class 'mboost': resid(object, ...)

## S3 method for class 'mboost': extract(object, what = c("design", "penalty", "lambda", "df", "coefficients", "residuals", "bnames", "offset", "nuisance", "weights", "index", "control"), which = NULL, ...) ## S3 method for class 'glmboost': extract(object, what = c("design", "coefficients", "residuals", "bnames", "offset", "nuisance", "weights", "control"), which = NULL, asmatrix = FALSE, ...) ## S3 method for class 'blg': extract(object, what = c("design", "penalty", "index"), asmatrix = FALSE, expand = FALSE, ...)

## S3 method for class 'mboost': logLik(object, ...) ## S3 method for class 'gamboost': hatvalues(model, ...) ## S3 method for class 'glmboost': hatvalues(model, ...)

## S3 method for class 'mboost': selected(object, ...)

## S3 method for class 'mboost': nuisance(object)

Arguments
object
objects of class glmboost, gamboost, blackboost or gbAIC.
x
objects of class glmboost or gamboost.
model
objects of class mboost
newdata
optionally, a data frame in which to look for variables with which to predict. In case the model was fitted using the matrix interface to glmboost, newdata mus
which
a subset of base-learners to take into account for computing predictions or coefficients. If which is given (as an integer vector or characters corresponding to base-learners) a list or matrix
type
the type of prediction required. The default is on the scale of the predictors; the alternative "response" is on the scale of the response variable. Thus for a binomial model the default predictions are of log-
aggregate
a character specifying how to aggregate predictions or coefficients of single base-learners. The default returns the prediction or coefficient for the final number of boosting iterations. "
off2int
logical indicating whether the offset should be added to the intercept (if there is any) or if the offset is returned as attribute of the coefficient (default).
i
integer. Index specifying the model to extract. If i is smaller than the initial mstop, a subset is used. If i is larger than the initial mstop, additional boosting st
return
a logical indicating whether the changed object is returned.
method
a character specifying if the corrected AIC criterion or a classical (-2 logLik + k * df) should be computed.
df
a character specifying how degrees of freedom should be computed: trace defines degrees of freedom by the trace of the boosting hat matrix and actset uses the number of non-zero coefficients
k
numeric, the penalty per parameter to be used; the default k = 2 is the classical AIC. Only used when method = "classical".
what
a character specifying the quantities to extract. Depending on object this can be a subset of "design" (default; design matrix), "penalty" (penalty matrix),
asmatrix
a logical indicating whether the the returned matrix should be coerced to a matrix (default) or if the returned object stays as it is (i.e., potentially a sparse matrix). This option is only applicable if ext
expand
a logical indicating whether the design matrix should be expanded (default: FALSE). This is useful if ties where taken into account either manually (via argument index in a base-learner) or automatically
...
additional arguments passed to callies.
Details

These functions can be used to extract details from fitted models. print shows a dense representation of the model fit and summary gives a more detailed representation.

The function coef extracts the regression coefficients of a linear model fitted using the glmboost function or an additive model fitted using the gamboost. Per default, only coefficients of selected base-learners are returned. However, any desired coefficient can be extracted using the which argument (see examples for details). Per default, the coefficient of the final iteration is returned (aggregate = "sum") but it is also possible to return the coefficients from all iterations simultaniously (aggregate = "cumsum"). If aggregate = "none" is specified, the coefficients of the selected base-learners are returned (see examples below). For models fitted via glmboost with option center = TRUE the intercept is rarely selected. However, it is implicitly estimated through the centering of the design matrix. In this case the intercept is always returned except which is specified such that the intercept is not selected. See examples below.

The predict function can be used to predict the status of the response variable for new observations whereas fitted extracts the regression fit for the observations in the learning sample. For predict newdata can be specified, otherwise the fitted values are returned. If which is specified, marginal effects of the corresponding base-learner(s) are returned. The argument type can be used to make predictions on the scale of the link (i.e., the linear predictor $X\beta$), the response (i.e. $h(X\beta)$, where h is the response function) or the class (in case of classification). Furthermore, the predictions can be aggregated analogously to coef by setting aggregate to either sum (default; predictions of the final iteration are given), cumsum (predictions of all iterations are returned simultaniously) or none (change of prediction in each iteration). If applicable the offset is added to the predictions. If marginal predictions are requested the offset is attached to the object via attr(..., "offset") as adding the offset to one of the marginal predictions doesn't make much sense.

The residuals function can be used to extract the residuals (i.e., the negative gradient of the current iteration). resid is is an alias for residuals.

The [.mboost function can be used to enhance or restrict a given boosting model to the specified boosting iteration i. Note that in both cases the original x will be changed to reduce the memory footprint. If the boosting model is enhanced by specifying an index that is larger than the initial mstop, only the missing i - mstop steps are fitted. If the model is restricted, the spare steps are not dropped, i.e., if we increase i again, these boosting steps are immediately available.

The generic extract function can be used to extract various characteristics of a fitted model or a base-learner. Note that the sometimes a penalty function is returned (e.g. by extract(bols(x), what = "penalty")) even if the estimation is unpenalized. However, in this case the penalty paramter lambda is set to zero. If a matrix is returned by extract one can to set asmatrix = TRUE if the returned matrix should be coerced to class matrix. If asmatrix = FALSE one might get a sparse matrix as implemented in package Matrix. If one requests the design matrix (what = "design") expand = TRUE expands the resulting matrix by taking the duplicates handeled via index into account.

The ids of base-learners selected during the fitting process can be extracted using selected(). The nuisance() method extracts nuisance parameters from the fit that are handled internally by the corresponding family object, see "boost_family".

For (generalized) linear and additive models, the AIC function can be used to compute both the classical AIC (only available for familiy = Binomial() and familiy = Poisson()) and corrected AIC (Hurvich et al., 1998, only available when family = Gaussian() was used). Details on the used approximations for the hat matrix can be found in Buehlmann and Hothorn (2007). The AIC is useful for the determination of the optimal number of boosting iterations to be applied (which can be extracted via mstop). The degrees of freedom are either computed via the trace of the boosting hat matrix (which is rather slow even for moderate sample sizes) or the number of variables (non-zero coefficients) that entered the model so far (faster but only meaningful for linear models fitted via gamboost (see Hastie, 2007)).

In addition, the general Minimum Description Length criterion (Buehlmann and Yu, 2006) can be computed using function AIC.

Note that logLik and AIC only make sense when the corresponding Family implements the appropriate loss function.

Note

The [.mboost function changes the original object, i.e. gbmodel[10] changes gbmodel directly!

Warning

The coefficients resulting from boosting with family Binomial are $1/2$ of the coefficients of a logit model obtained via glm. This is due to the internal recoding of the response to $-1$ and $+1$ (see Binomial).

References

Clifford M. Hurvich, Jeffrey S. Simonoff and Chih-Ling Tsai (1998), Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. Journal of the Royal Statistical Society, Series B, 20(2), 271--293.

Peter Buehlmann and Torsten Hothorn (2007), Boosting algorithms: regularization, prediction and model fitting. Statistical Science, 22(4), 477--505.

Trevor Hastie (2007), Discussion of ``Boosting algorithms: Regularization, prediction and model fitting'' by Peter Buehlmann and Torsten Hothorn. Statistical Science, 22(4), 505.

Peter Buehlmann and Bin Yu (2006), Sparse boosting. Journal of Machine Learning Research, 7, 1001--1024.

See Also

gamboost, glmboost and blackboost for model fitting. See cvrisk for cross-validated stopping iteration.

Aliases
  • print.glmboost
  • print.mboost
  • summary.mboost
  • coef.mboost
  • coef.glmboost
  • [.mboost
  • AIC.mboost
  • mstop
  • mstop.gbAIC
  • mstop.mboost
  • mstop.cvrisk
  • predict.mboost
  • predict.gamboost
  • predict.blackboost
  • predict.glmboost
  • fitted.mboost
  • residuals.mboost
  • resid.mboost
  • extract
  • extract.mboost
  • extract.gamboost
  • extract.glmboost
  • extract.blackboost
  • extract.blg
  • extract.bl_lin
  • extract.bl_tree
  • logLik.mboost
  • hatvalues.gamboost
  • hatvalues.glmboost
  • selected
  • selected.mboost
  • nuisance
  • nuisance.mboost
Examples
### a simple two-dimensional example: cars data
  cars.gb <- glmboost(dist ~ speed, data = cars,
                      control = boost_control(mstop = 2000),
                      center = FALSE)
  cars.gb

  ### initial number of boosting iterations
  mstop(cars.gb)

  ### AIC criterion
  aic <- AIC(cars.gb, method = "corrected")
  aic

  ### extract coefficients for glmboost
  coef(cars.gb)
  coef(cars.gb, off2int = TRUE)        # offset added to intercept
  coef(lm(dist ~ speed, data = cars))  # directly comparable

  cars.gb_centered <- glmboost(dist ~ speed, data = cars,
                               center = TRUE)
  selected(cars.gb_centered)           # intercept never selected
  coef(cars.gb_centered)               # intercept implicitly estimated
                                       # and thus returned
  ## intercept is internally corrected for mean-centering
  - mean(cars$speed) * coef(cars.gb_centered, which="speed") # = intercept
  # not asked for intercept thus not returned
  coef(cars.gb_centered, which="speed")
  # explicitly asked for intercept
  coef(cars.gb_centered, which=c("Intercept", "speed"))

  ### enhance or restrict model
  cars.gb <- gamboost(dist ~ speed, data = cars,
                      control = boost_control(mstop = 100, trace = TRUE))
  cars.gb[10]
  cars.gb[100, return = FALSE] # no refitting required
  cars.gb[150, return = FALSE] # only iterations 101 to 150
                               # are newly fitted

  ### coefficients for optimal number of boosting iterations
  coef(cars.gb[mstop(aic)])
  plot(cars$dist, predict(cars.gb[mstop(aic)]),
       ylim = range(cars$dist))
  abline(a = 0, b = 1)

  ### example for extraction of coefficients
  set.seed(1907)
  n <- 100
  x1 <- rnorm(n)
  x2 <- rnorm(n)
  x3 <- rnorm(n)
  x4 <- rnorm(n)
  int <- rep(1, n)
  y <- 3 * x1^2 - 0.5 * x2 + rnorm(n, sd = 0.1)
  data <- data.frame(y = y, int = int, x1 = x1, x2 = x2, x3 = x3, x4 = x4)

  model <- gamboost(y ~ bols(int, intercept = FALSE) +
                        bbs(x1, center = TRUE, df = 1) +
                        bols(x1, intercept = FALSE) +
                        bols(x2, intercept = FALSE) +
                        bols(x3, intercept = FALSE) +
                        bols(x4, intercept = FALSE),
                    data = data, control = boost_control(mstop = 500))

  coef(model) # standard output (only selected base-learners)
  coef(model,
       which = 1:length(variable.names(model))) # all base-learners
  coef(model, which = "x1") # shows all base-learners for x1

  cf1 <- coef(model, which = c(1,3,4), aggregate = "cumsum")
  tmp <- sapply(cf1, function(x) x)
  matplot(tmp, type = "l", main = "Coefficient Paths")

  cf1_all <- coef(model, aggregate = "cumsum")
  cf1_all <- lapply(cf1_all, function(x) x[, ncol(x)]) # last element
  ## same as coef(model)

  cf2 <- coef(model, aggregate = "none")
  cf2 <- lapply(cf2, rowSums) # same as coef(model)

  ### example continued for extraction of predictions

  yhat <- predict(model) # standard prediction; here same as fitted(model)
  p1 <- predict(model, which = "x1") # marginal effects of x1
  orderX <- order(data$x1)
  ## rowSums needed as p1 is a matrix
  plot(data$x1[orderX], rowSums(p1)[orderX], type = "b")

  ## better: predictions on a equidistant grid
  new_data <- data.frame(x1 = seq(min(data$x1), max(data$x1), length = 100))
  p2 <- predict(model, newdata = new_data, which = "x1")
  lines(new_data$x1, rowSums(p2), col = "red")

  ### extraction of model characteristics
  extract(model, which = "x1")  # design matrices for x1
  extract(model, what = "penalty", which = "x1") # penalty matrices for x1
  extract(model, what = "lambda", which = "x1") # df and corresponding lambda for x1
       ## note that bols(x1, intercept = FALSE) is unpenalized

  ### extract from base-learners
  extract(bbs(x1), what = "design")
  extract(bbs(x1), what = "penalty")
  ## weights and lambda can only be extracted after using dpp
  weights <- rep(1, length(x1))
  extract(bbs(x1)$dpp(weights), what = "lambda")
Documentation reproduced from package mboost, version 2.1-0, License: GPL-2

Community examples

Looks like there are no examples yet.