IBLM (version 1.0.2)

get_pinball_scores: Calculate Pinball Scores for IBLM and Additional Models

Description

Computes Poisson deviance and pinball scores for an IBLM model alongside homogeneous, GLM, and optional additional models.

Usage

get_pinball_scores(
  data,
  iblm_model,
  trim = NA_real_,
  additional_models = list()
)

Value

Data frame with 3 columns:

• "model" - will be homog, glm, iblm and any other models specified in `additional_models`

• "`family`_deviance" - the value from the loss function based on the family of the glm function

• "pinball_score" - The more positive the score, the better the model than a basic homog model (i.e. all predictions are mean value). A negative score indicates worse than homog model.

Arguments

data

Data frame. If you have used `split_into_train_validate_test()` this will be the "test" portion of your data.

iblm_model

Fitted IBLM model object of class "iblm"

trim

Numeric trimming parameter for IBLM predictions. Default is `NA_real_`.

additional_models

(Named) list of fitted models for comparison. These models MUST be fitted on the same data as `iblm_model` for sensible results. If unnamed, models are labeled by their class.

Details

Pinball scores are calculated relative to a homogeneous model (i.e. a simple mean prediction of training data). Higher scores indicate better predictive performance.

Examples

df_list <- freMTPLmini |> split_into_train_validate_test(seed = 9000)

iblm_model <- train_iblm_xgb(
  df_list,
  response_var = "ClaimRate",
  family = "poisson"
)

get_pinball_scores(data = df_list$test, iblm_model = iblm_model)