MachineShop (version 1.1.0)

resample: Resample Estimation of Model Performance

Description

Estimation of the predictive performance of a model estimated and evaluated on training and test samples generated from an observed data set.

Usage

Resamples(...)
resample(x, ...)
# S3 method for formula
resample(x, data, model, control = CVControl, ...)
# S3 method for ModelFrame
resample(x, model, control = CVControl, ...)
# S3 method for recipe
resample(x, model, control = CVControl, ...)

Arguments

...

named or unnamed resample output to combine together with the Resamples constructor.

x

defined relationship between model predictors and an outcome. May be a ModelFrame containing a formula, data, and optionally case weights; a formula; or a recipe.

data

data.frame containing observed predictors and outcomes.

model

MLModel object, constructor function, or character string naming a constructor function that returns an MLModel object.

control

MLControl object, control function, or character string naming a control function defining the resampling method to be employed.

Value

Resamples class object.

Details

Output being combined from more than one model with the Resamples constructor must have been generated with the same resampling control object.

Stratified resampling is performed for the formula method according to values of the response variable; i.e. categorical levels for factor, continuous for numeric, and event status Surv.

User-specified stratification variables may be specified for ModelFrames upon creation with the strata argument in its constructor. Resampling of this class is unstratified by default.

Variables in a recipe may be used for stratification by defining a "case_strata" role for them. Resampling will be unstratified if no variables have that role.

See Also

ModelFrame, recipe, modelinfo, MLControl, performance, metricinfo, plot, summary

Examples

# NOT RUN {
## Factor response example

fo <- Species ~ .
control <- CVControl()

gbmres1 <- resample(fo, iris, GBMModel(n.trees = 25), control)
gbmres2 <- resample(fo, iris, GBMModel(n.trees = 50), control)
gbmres3 <- resample(fo, iris, GBMModel(n.trees = 100), control)

summary(gbmres1)
plot(gbmres1)

res <- Resamples(GBM1 = gbmres1, GBM2 = gbmres2, GBM3 = gbmres3)
summary(res)
plot(res)

# }