Base class for confidence interval measures. See section Inheriting on how to add a new method.
alpha :: numeric(1)
The desired alpha level.
This is initialized to $0.05$.
within_range :: logical(1)
Whether to restrict the confidence interval within the range of possible values.
This is initialized to TRUE.
To define a new CI method, inherit from the abstract base class and implement the private method:
ci: function(tbl: data.table, rr: ResampleResult, param_vals: named list()) -> numeric(3)
If requires_obs_loss is set to TRUE, tbl contains the columns loss, row_id and iteration, which are the pointwise loss,
Otherwise, tbl contains the result of rr$score() with the name of the loss column set to "loss".
the identifier of the observation and the resampling iteration.
It should return a vector containing the estimate, lower and upper boundary in that order.
In case the confidence interval is not of the form (estimate, estimate - z * se, estimate + z * se)
it is also necessary to implement the private method:
.trafo: function(ci: numeric(3), measure: Measure) -> numeric(3)
Which receives a confidence interval for a pointwise loss (e.g. squared-error) and transforms it according
to the transformation measure$trafo (e.g. sqrt to go from mse to rmse).
mlr3::Measure -> MeasureAbstractCi
resamplings(character())
On which resampling classes this method can operate.
measure(Measure)
Inherited methods
new()Creates a new instance of this R6 class.
MeasureAbstractCi$new(
measure = NULL,
param_set = ps(),
packages = character(),
resamplings,
label,
delta_method = FALSE,
requires_obs_loss = TRUE,
man = NA
)measure(Measure)
The measure for which to calculate a confidence interval. Must have $obs_loss.
param_set(ParamSet)
Set of hyperparameters.
packages(character())
Set of required packages.
A warning is signaled by the constructor if at least one of the packages is not installed,
but loaded (not attached) later on-demand via requireNamespace().
resamplings(character())
To which resampling classes this measure can be applied.
label(character(1))
Label for the new instance.
delta_method(logical(1))
Whether to use the delta method for measures (such RMSE) that have a trafo.
requires_obs_loss(logical(1))
Whether the inference method requires a pointwise loss function.
man(character(1))
Manual page.
aggregate()Obtain a point estimate, as well as lower and upper CI boundary.
MeasureAbstractCi$aggregate(rr)rr(ResampleResult)
The resample result.
named numeric(3)
clone()The objects of this class are cloneable with this method.
MeasureAbstractCi$clone(deep = FALSE)deepWhether to make a deep clone.
The aggregator of the wrapped measure is ignored, as the inheriting CI dictates how the point
estimate is constructed. If a measure for which to calculate a CI has $obs_loss but also a $trafo,
(such as RMSE), the delta method is used to obtain confidence intervals.