tailor (version 0.1.0)

adjust_equivocal_zone: Apply an equivocal zone to a binary classification model.

Description

Equivocal zones describe intervals of predicted probabilities that are deemed too uncertain or ambiguous to be assigned a hard class. Rather than predicting a hard class when the probability is very close to a threshold, tailors using this adjustment predict "[EQ]".

Usage

adjust_equivocal_zone(x, value = 0.1, threshold = NULL)

Value

An updated tailor() containing the new operation.

Arguments

x

A tailor().

value

A numeric value (between zero and 1/2) or hardhat::tune(). The value is the size of the buffer around the threshold.

threshold

A numeric value (between zero and one) or hardhat::tune(). Defaults to adjust_probability_threshold(threshold) if previously set in x, or 1 / 2 if not.

Data Usage

This adjustment doesn't require estimation and, as such, the same data that's used to train it with fit() can be predicted on with predict(); fitting this adjustment just collects metadata on the supplied column names and does not risk data leakage.

Details

This function transforms the class prediction column estimate to have type class_pred from probably::class_pred(). You can loosely think of this column type as a factor, except there's a possible entry "[EQ]" that is not a level and will be excluded from performance metric calculations. As a result, the output column has the same number of levels as the input, except now has a possible entry "[EQ]" that tidymodels functions know to exclude from further analyses.

Examples

if (FALSE) { # rlang::is_installed(c("probably", "modeldata"))
library(dplyr)
library(modeldata)

head(two_class_example)

# `predicted` gives hard class predictions based on probabilities
two_class_example |> count(predicted)

# when probabilities are within (.25, .75), consider them equivocal
tlr <-
  tailor() |>
  adjust_equivocal_zone(value = 1 / 4)

tlr

# fit by supplying column names.
tlr_fit <- fit(
  tlr,
  two_class_example,
  outcome = c(truth),
  estimate = c(predicted),
  probabilities = c(Class1, Class2)
)

tlr_fit

# adjust hard class predictions
predict(tlr_fit, two_class_example) |> count(predicted)
}