mlr3 (version 0.1.1)

resample: Resample a Learner on a Task

Description

Runs a resampling (possibly in parallel).

Usage

resample(task, learner, resampling, ctrl = list())

Arguments

task

:: Task.

learner

:: Learner.

resampling

:: Resampling.

ctrl

:: named list() Object to control learner execution. See mlr_control() for details.

Value

ResampleResult.

Syntactic Sugar

The mlr3 package provides some shortcuts to ease the creation of its objects.

First, instead of an object, it is possible to pass a string identifier which is used to lookup the object in a mlr3misc::Dictionary:

• Task in mlr_tasks.

• Learner in mlr_learners.

• Resampling in mlr_resamplings.

• Measure in mlr_measures.

Additionally, each task type has an associated default measure (stored in mlr_reflections) which is used as a fallback if no other measure is provided. Classification tasks default to the classification error in "classif.ce", regression tasks to the mean squared error in "regr.mse".

Parallelization

This function can be parallelized with the future package. One job is one resampling iteration, and all jobs are send to an apply function from future.apply in a single batch. To select a parallel backend, use future::plan().

Examples

# NOT RUN {
task = mlr_tasks$get("iris")
learner = mlr_learners$get("classif.rpart")
resampling = mlr_resamplings$get("cv")

# explicitly instantiate the resampling for this task for reproduciblity
set.seed(123)
resampling$instantiate(task)

rr = resample(task, learner, resampling)
print(rr)

# retrieve performance
rr$performance("classif.ce")
rr$aggregate("classif.ce")

# merged prediction objects of all resampling iterations
pred = rr$prediction
pred$confusion

# Repeat resampling with featureless learner
rr.featureless = resample(task, "classif.featureless", resampling)

# Combine the ResampleResults into a BenchmarkResult
bmr = rr$combine(rr.featureless)
print(bmr)
# }