mlr3inferr (version 0.2.1)

mlr_measures_ci.ncv: Nested CV CI

Description

Confidence Intervals based on ResamplingNestedCV, including bias-correction. This inference method can only be applied to decomposable losses.

Point Estimation

The point estimate uses a bias correction term as described in Bates et al. (2024). Therefore, the results of directly applying a measure $aggregate(msr(<key>)) will be different from the point estimate of $aggregate(msr("ci", <key>)), where the point estimate is obtained by averaging over the outer CV results.

Parameters

Those from MeasureAbstractCi, as well as:

• bias :: logical(1)
  Whether to do bias correction. This is initialized to TRUE. If FALSE, the outer iterations are used for the point estimate and no bias correction is applied.

Super classes

mlr3::Measure -> mlr3inferr::MeasureAbstractCi -> MeasureCiNestedCV

Methods

Public methods

• MeasureCiNestedCV$new()

• MeasureCiNestedCV$clone()

Inherited methods

• mlr3::Measure$format()
• mlr3::Measure$help()
• mlr3::Measure$print()
• mlr3::Measure$score()
• mlr3inferr::MeasureAbstractCi$aggregate()

---

Method new()

Creates a new instance of this R6 class.

Usage

MeasureCiNestedCV$new(measure)

Arguments

measure

(Measure or character(1))
A measure of ID of a measure.

---

Method clone()

The objects of this class are cloneable with this method.

Usage

MeasureCiNestedCV$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

References

Bates, Stephen, Hastie, Trevor, Tibshirani, Robert (2024). “Cross-validation: what does it estimate and how well does it do it?” Journal of the American Statistical Association, 119(546), 1434--1445.

Examples

ci_ncv = msr("ci.ncv", "classif.acc")
ci_ncv