Measure: Measure Class

Description

This is the abstract base class for measures like MeasureClassif and MeasureRegr.

Measures are classes around tailored around two functions:

A function score which quantifies the performance by comparing true and predicted response.
A function aggregator which combines multiple performance values returned by calculate to a single numeric value.

In addition to these two functions, meta-information about the performance measure is stored.

Predefined measures are stored in the Dictionary mlr_measures, e.g. classif.auc or time_train.

Arguments

Format

R6::R6Class object.

Construction

Note: This object is typically constructed via a derived classes, e.g. MeasureClassif or MeasureRegr.

m = Measure$new(id, task_type, range, minimize, predict_type = "response",
     task_properties = character(), na_score = FALSE, packages = character())

id :: character(1) Identifier for the measure.
task_type :: character(1) Type of the task the measure can operator on. E.g., "classif" or "regr".
range :: numeric(2) Feasible range for this measure as c(lower_bound, upper_bound). Both bounds may be infinite.
minimize :: logical(1) Set to TRUE if good predictions correspond to small values, and to FALSE if good predictions correspond to large values. If set to NA, tuning with this measure is not possible.
aggregator :: function(x) Function to aggregate individual performance values x where x is a numeric vector. If NULL, defaults to mean().
predict_type :: character(1) Required predict type of the Learner. Possible values are stored in mlr_reflections$learner_predict_types.
task_properties :: character() Required task properties, see Task.
na_score :: logical(1) Is the measure expected to return NA in some cases? Default is FALSE.
packages :: character() Set of required packages. Note that these packages will be loaded via requireNamespace(), and are not attached.

Fields

id :: character(1) Identifier of the measure.
minimize :: logical(1) Is TRUE if the best value is reached via minimization and FALSE by maximization.
packages :: character() Stores the names of required packages.
range :: numeric(2) Stores the feasible range of the measure.
task_type :: character(1) Stores the required type of the Task.
task_properties :: character() Stores required properties of the Task.

Methods

aggregate(rr) ResampleResult -> numeric(1) Aggregates multiple performance scores into a single score using the aggregator function of the measure. Operates on a ResampleResult as returned by resample.
score(prediction, task = NULL, learner = NULL) (Prediction, Task, Learner) -> numeric(1) Takes a Prediction and calculates a numeric score. If the measure if flagged with the properties "requires_task" or "requires_learner", you must additionally pass the respective Task or the Learner for the measure to extract information from these objects.