mlr (version 2.10)

makeCustomResampledMeasure: Construct your own resampled performance measure.

Description

Construct your own performance measure, used after resampling. Note that individual training / test set performance values will be set to NA, you only calculate an aggregated value. If you can define a function that makes sense for every single training / test set, implement your own Measure.

Usage

makeCustomResampledMeasure(measure.id, aggregation.id, minimize = TRUE,
  properties = character(0L), fun, extra.args = list(), best = NULL,
  worst = NULL, measure.name = measure.id,
  aggregation.name = aggregation.id, note = "")

Arguments

measure.id
[character(1)] Short name of measure.
aggregation.id
[character(1)] Short name of aggregation.
minimize
[logical(1)] Should the measure be minimized? Default is TRUE.
properties
[character] Set of measure properties. For a list of values see Measure. Default is character(0).
fun
[function(task, group, pred, extra.args)] Calculates performance value from ResamplePrediction object. For rare cases you can also use the task, the grouping or the extra arguments extra.args.
task [Task]
The task.
group [factor]
Grouping of resampling iterations. This encodes whether specific iterations 'belong together' (e.g. repeated CV).
pred [Prediction]
Prediction object.
extra.args [list]
See below.
extra.args
[list] List of extra arguments which will always be passed to fun. Default is empty list.
best
[numeric(1)] Best obtainable value for measure. Default is -Inf or Inf, depending on minimize.
worst
[numeric(1)] Worst obtainable value for measure. Default is Inf or -Inf, depending on minimize.
measure.name
[character(1)] Long name of measure. Default is measure.id.
aggregation.name
[character(1)] Long name of the aggregation. Default is aggregation.id.
note
[character] Description and additional notes for the measure. Default is “”.

Value

[Measure].

See Also

Other performance: ConfusionMatrix, calculateConfusionMatrix, calculateROCMeasures, estimateRelativeOverfitting, makeCostMeasure, makeMeasure, measures, performance