gfboost (version 0.1.1)

CMB: CMB aggregation function

Description

Aggregates the selection frequencies of multiple SingBoost models. May be used with caution since there are not yet recommendations about good hyperparameters.

Usage

CMB(
  D,
  nsing,
  Bsing = 1,
  alpha = 1,
  singfam = Gaussian(),
  evalfam = Gaussian(),
  sing = FALSE,
  M = 10,
  m_iter = 100,
  kap = 0.1,
  LS = FALSE,
  best = 1,
  wagg,
  robagg = FALSE,
  lower = 0,
  ...
)

Arguments

D

Data matrix. Has to be an \(n \times (p+1)-\)dimensional data frame in the format \((X,Y)\). The \(X-\)part must not contain an intercept column containing only ones since this column will be added automatically.

nsing

Number of observations (rows) used for the SingBoost submodels.

Bsing

Number of subsamples based on which the SingBoost models are validated. Default is 1. Not to confuse with parameter B for the Stability Selection.

alpha

Optional real number in \(]0,1]\). Defines the fraction of best SingBoost models used in the aggregation step. Default is 1 (use all models).

singfam

A SingBoost family. The SingBoost models are trained based on the corresponding loss function. Default is Gaussian() (squared loss).

evalfam

A SingBoost family. The SingBoost models are validated according to the corresponding loss function. Default is Gaussian() (squared loss).

sing

If sing=FALSE and the singfam family is a standard Boosting family that is contained in the package mboost, the CMB aggregation procedure is executed for the corresponding standard Boosting models.

M

An integer between 2 and m_iter. Indicates that in every \(M-\)th iteration, a singular iteration will be performed. Default is 10.

m_iter

Number of SingBoost iterations. Default is 100.

kap

Learning rate (step size). Must be a real number in \(]0,1]\). Default is 0.1 It is recommended to use a value smaller than 0.5.

LS

If a singfamily object that is already provided by mboost is used, the respective Boosting algorithm will be performed in the singular iterations if Ls is set to TRUE. Default is FALSE.

best

Needed in the case of localized ranking. The parameter K of the localized ranking loss will be computed by \(best \cdot n\) (rounded to the next larger integer). Warning: If a parameter K is inserted into the LocRank family, it will be ignored when executing SingBoost.

wagg

Type of row weight aggregation. 'weights1' indicates that the selection frequencies of the (best) SingBoost models are averaged. 'weights2' respects the validation losses for each model and downweights the ones with higher validation losses.

robagg

Optional. If setting robagg=TRUE, the best SingBoost models are ignored when executing the aggregation to avoid inlier effects. Only reasonable in combination with lower.

lower

Optional argument. Only reasonable when setting robagg=TRUE. lower is a real number in \([0,1[\) (a rather small number is recommended) and indicates that the aggregation ignores the SingBoost models with the best performances to avoid possible inlier effects.

...

Optional further arguments

Value

Column measure

Aggregated column measure as \((p+1)-\)dimensional vector.

Selected variables

Names of the variables with positive aggregated column measure.

Variables names

Names of all variables including the intercept.

Row measure

Aggregated row measure as \(n-\)dimensional vector.

Details

SingBoost is designed to detect variables that standard Boosting procedures may not but which may be relevant w.r.t. the target loss function. However, one may try to stabilize this ''singular part'' of the column measure by aggregating several SingBoost models in the sense that they are evaluated on a validation set and that the selection frequencies are averaged, maybe in a weighted manner according to the validation losses. Warning: This procedure does not replace a Stability Selection!

References

Werner, T., Gradient-Free Gradient Boosting, PhD Thesis, Carl von Ossietzky University Oldenburg, 2020

Examples

# NOT RUN {
firis<-as.formula(Sepal.Length~.)
Xiris<-model.matrix(firis,iris)
Diris<-data.frame(Xiris[,-1],iris$Sepal.Length)
colnames(Diris)[6]<-"Y"
set.seed(19931023)
cmb1<-CMB(Diris,nsing=100,Bsing=50,alpha=0.8,singfam=Rank(),
evalfam=Rank(),sing=TRUE,M=10,m_iter=100,
kap=0.1,LS=TRUE,wagg='weights1',robagg=FALSE,lower=0)
cmb1
set.seed(19931023)
cmb2<-CMB(Diris,nsing=100,Bsing=50,alpha=0.8,singfam=Rank(),
evalfam=Rank(),sing=TRUE,M=2,m_iter=100,
kap=0.1,LS=TRUE,wagg='weights1',robagg=FALSE,lower=0)
cmb2[[1]]
set.seed(19931023)
cmb3<-CMB(Diris,nsing=100,Bsing=50,alpha=0.8,singfam=Rank(),
evalfam=Rank(),sing=TRUE,M=10,m_iter=100,
kap=0.1,LS=TRUE,wagg='weights2',robagg=FALSE,lower=0)
cmb3[[1]]
# }