This is the abstract base class for measures like MeasureClassif and MeasureRegr.
Measures are classes around tailored around two functions:
A function score
which quantifies the performance by comparing true and predicted response.
A function aggregator
which combines multiple performance values returned by
calculate
to a single numeric value.
In addition to these two functions, meta-information about the performance measure is stored.
Predefined measures are stored in the Dictionary mlr_measures,
e.g. classif.auc
or time_train
.
R6::R6Class object.
Note: This object is typically constructed via a derived classes, e.g. MeasureClassif or MeasureRegr.
m = Measure$new(id, task_type, range, minimize, predict_type = "response", task_properties = character(), na_score = FALSE, packages = character())
id
:: character(1)
Identifier for the measure.
task_type
:: character(1)
Type of the task the measure can operator on. E.g., "classif"
or "regr"
.
range
:: numeric(2)
Feasible range for this measure as c(lower_bound, upper_bound)
.
Both bounds may be infinite.
minimize
:: logical(1)
Set to TRUE
if good predictions correspond to small values,
and to FALSE
if good predictions correspond to large values.
If set to NA
, tuning with this measure is not possible.
aggregator
:: function(x)
Function to aggregate individual performance values x
where x
is a numeric vector.
If NULL
, defaults to mean()
.
predict_type
:: character(1)
Required predict type of the Learner.
Possible values are stored in mlr_reflections$learner_predict_types.
task_properties
:: character()
Required task properties, see Task.
na_score
:: logical(1)
Is the measure expected to return NA
in some cases? Default is FALSE
.
packages
:: character()
Set of required packages.
Note that these packages will be loaded via requireNamespace()
, and are not attached.
id
:: character(1)
Identifier of the measure.
minimize
:: logical(1)
Is TRUE
if the best value is reached via minimization and FALSE
by maximization.
packages
:: character()
Stores the names of required packages.
range
:: numeric(2)
Stores the feasible range of the measure.
task_type
:: character(1)
Stores the required type of the Task.
task_properties
:: character()
Stores required properties of the Task.
aggregate(rr)
ResampleResult -> numeric(1)
Aggregates multiple performance scores into a single score using the aggregator
function of the measure.
Operates on a ResampleResult as returned by resample.
score(prediction, task = NULL, learner = NULL)
(Prediction, Task, Learner) -> numeric(1)
Takes a Prediction and calculates a numeric score.
If the measure if flagged with the properties "requires_task"
or "requires_learner"
, you must additionally
pass the respective Task or the Learner for the measure to extract information from these objects.
Other Measure: MeasureClassif
,
MeasureRegr
, mlr_measures