scoringutils (version 2.3.0)

impute_missing_scores: Impute missing scores

Description

Fills in scores for forecast-target combinations that are missing from the data, using a user-specified imputation strategy. This is useful to ensure all models are evaluated on the same set of targets, which avoids bias when summarising scores.

Missing combinations are identified by comparing each value of the compare column against the union of targets observed across all values. The strategy is then called to fill the metric columns for those rows.

An .imputed column is added to the output indicating which rows were imputed (TRUE) and which are original (FALSE).

Usage

impute_missing_scores(scores, strategy, compare = "model")

Value

A scores object with an additional .imputed

column. Rows that were imputed have .imputed = TRUE.

Arguments

scores

An object of class scores (a data.table with an additional metrics attribute as produced by score()).

strategy

A strategy function with signature function(scores, missing_rows, metrics, compare) that returns missing_rows with the metric columns filled. Built-in options are impute_worst_score(), impute_mean_score(), impute_na_score(), and impute_model_score(). Custom strategies are also supported.

compare

Character string (default "model") naming the column whose values are compared against each target to identify missing combinations.

See Also

impute_worst_score(), impute_mean_score(), impute_na_score(), impute_model_score(), vignette("handling-missing-forecasts")

Examples

# \dontshow{
  data.table::setDTthreads(2)
# }
scores <- example_quantile |>
  as_forecast_quantile() |>
  score()

# Impute with NA values
impute_missing_scores(scores, strategy = impute_na_score())