See Friedman & Popescu (2008) for a description of the general RuleFit algorithm. This method uses XGBoost to fit a tree ensemble, extracts a ruleset as the conjunction of tree traversals, and fits a sparse linear model to the resulting feature set (including the original feature set) using glmnet.
# S3 method for formula
xrf(
object,
data,
family,
xgb_control = list(nrounds = 100, max_depth = 3),
glm_control = list(type.measure = "deviance", nfolds = 5),
sparse = TRUE,
prefit_xgb = NULL,
deoverlap = FALSE,
...
)
a formula prescribing features to use in the model. transformation of the response variable is not supported. when using transformations on the input features (not suggested in general) it is suggested to set sparse=F
a data frame with columns corresponding to the formula
the family of the fitted model. one of 'gaussian', 'binomial', 'multinomial'
a list of parameters for xgboost. must supply an nrounds argument
a list of parameters for the glmnet fit. must supply a type.measure and nfolds arguments (for the lambda cv)
whether a sparse design matrix should be used
an xgboost model (of class xgb.Booster) to be used instead of the model that xrf
would normally fit
if true, the tree derived rules are deoverlapped, in that the deoverlapped rule set contains no overlapped rules
ignored arguments
Friedman, J. H., & Popescu, B. E. (2008). Predictive learning via rule ensembles. The Annals of Applied Statistics, 2(3), 916-954.
m <- xrf(Petal.Length ~ ., iris,
xgb_control = list(nrounds = 2, max_depth = 2),
family = 'gaussian')
Run the code above in your browser using DataLab