MachineShop (version 2.0.0)

resample: Resample Estimation of Model Performance

Description

Estimation of the predictive performance of a model estimated and evaluated on training and test samples generated from an observed data set.

Usage

resample(x, ...)
# S3 method for formula
resample(x, data, model, control = MachineShop::settings("control"), ...)
# S3 method for matrix
resample(x, y, model, control = MachineShop::settings("control"), ...)
# S3 method for ModelFrame
resample(x, model, control = MachineShop::settings("control"), ...)
# S3 method for recipe
resample(x, model, control = MachineShop::settings("control"), ...)
# S3 method for MLModel
resample(x, ...)
# S3 method for MLModelFunction
resample(x, ...)

Arguments

x

defines a relationship between model predictor and response variables. May be a formula, design matrix of predictors, ModelFrame, SelectedModelFrame, untrained recipe, SelectedRecipe, or TunedRecipe object. Alternatively, a model function or call may be given first followed by objects defining the predictor and response relationship and the control value.

...

arguments passed to other methods.

data

data frame containing observed predictors and outcomes.

model

model function, function name, or call.

control

control function, function name, or call defining the resampling method to be employed.

y

response variable.

Value

Resamples class object.

Details

Stratified resampling is performed for the formula method according to values of the response variable; i.e. categorical levels for factor, continuous for numeric, and event status Surv.

User-specified stratification variables may be specified for ModelFrames upon creation with the strata argument in its constructor. Resampling of this class is unstratified by default.

Variables in recipe specifications may be designated as case strata with the role_case function. Resampling will be unstratified otherwise.

See Also

c, metrics, performance, plot, summary

Examples

# NOT RUN {
## Factor response example

fo <- Species ~ .
control <- CVControl()

gbm_res1 <- resample(fo, iris, GBMModel(n.trees = 25), control)
gbm_res2 <- resample(fo, iris, GBMModel(n.trees = 50), control)
gbm_res3 <- resample(fo, iris, GBMModel(n.trees = 100), control)

summary(gbm_res1)
plot(gbm_res1)

res <- c(GBM1 = gbm_res1, GBM2 = gbm_res2, GBM3 = gbm_res3)
summary(res)
plot(res)

# }