mlr (version 2.2)

crossval: Fit models according to a resampling strategy.

Description

resample: Given a resampling strategy, which defines sets of training and test indices, fits the selected learner using the training sets and performs predictions for the training/test sets. This depends on what you selected in the resampling strategy, see parameter predict in makeResampleDesc.

Then performance measures are calculated on all respective data sets and aggregated.

You are able to return all fitted models (parameter models) or extract specific parts of the models (parameter extract) as returning all of them completely might be memory intensive.

For construction of the resampling strategies use the factory methods makeResampleDesc and makeResampleInstance.

The remaining functions on this page are convenience wrappers for the various existing resampling strategies.

Usage

crossval(learner, task, iters = 10L, stratify = FALSE, measures,
  models = FALSE, ..., show.info = getMlrOption("show.info"))
repcv(learner, task, folds = 10L, reps = 10L, stratify = FALSE, measures,
  models = FALSE, ..., show.info = getMlrOption("show.info"))
holdout(learner, task, split = 2/3, stratify = FALSE, measures,
  models = FALSE, ..., show.info = getMlrOption("show.info"))
subsample(learner, task, iters = 30, split = 2/3, stratify = FALSE,
  measures, models = FALSE, ..., show.info = getMlrOption("show.info"))
bootstrapOOB(learner, task, iters = 30, stratify = FALSE, measures,
  models = FALSE, ..., show.info = getMlrOption("show.info"))
bootstrapB632(learner, task, iters = 30, stratify = FALSE, measures,
  models = FALSE, ..., show.info = getMlrOption("show.info"))
bootstrapB632plus(learner, task, iters = 30, stratify = FALSE, measures,
  models = FALSE, ..., show.info = getMlrOption("show.info"))
resample(learner, task, resampling, measures, weights = NULL,
  models = FALSE, extract, show.info = getMlrOption("show.info"))

Value

List of:measures.test [data.frame]Gives you access to performance measurements on the individual test sets. Rows correspond to sets in resampling iterations, columns to performance measures.measures.train [data.frame]Gives you access to performance measurements on the individual training sets. Rows correspond to sets in resampling iterations, columns to performance measures. Usually not available, only if specifically requested, see general description above.aggr [numeric]Named vector of aggregated performance values. Names are coded like this ..err.msgs [data.frame]Number of rows equals resampling iterations and columns are: iter, train, predict. Stores error messages generated during train or predict, if these were caught via configureMlr.pred [ResamplePrediction]Container for all predictions during resampling.models [list of WrappedModel]List of fitted models or NULL.extract [list]List of extracted parts from fitted models or NULL.

See Also

Other resample: ResampleDesc, makeResampleDesc; ResampleInstance, makeResampleInstance; ResamplePrediction

Examples

task = makeClassifTask(data = iris, target = "Species")
rdesc = makeResampleDesc("CV", iters = 2)
r = resample(makeLearner("classif.qda"), task, rdesc)
print(r$aggr)
print(r$measures.test)
print(r$pred)