givitiCalibrationTest: Calibration Test

Description

givitiCalibrationTest performs the calibration test associated to the calibration belt.

Usage

givitiCalibrationTest(o, e, devel, subset = NULL, thres = 0.95, maxDeg = 4)

Arguments

A numeric vector representing the binary outcomes. The elements must assume only the values 0 or 1. The predictions in e must represent the probability of the event coded as 1.

A numeric vector containing the probabilities of the model under evaluation. The elements must be numeric and between 0 and 1. The lenght of the vector must be equal to the length of the vector o.

devel

A character string specifying if the model has been fit on the same dataset under evaluation (internal) or if the model has been developed on an external sample (external). See also the 'Details' section.

subset

An optional boolean vector specifying the subset of observations to be considered.

thres

A numeric scalar between 0 and 1 representing 1 - the significance level adopted in the forward selection. By default is set to 0.95.

maxDeg

The maximum degree considered in the forward selection. By default is set to 4.

Value

A list of class htest containing the following components:

Details

The calibration belt and the associated test can be used both to evaluate the calibration of the model in external samples or in the development dataset. However, the two cases have different requirements. When a model is evaluated on independent samples, the calibration belt and the related test can be applied whatever is the method used to fit the model. Conversely, they can be used on the development set only if the model is fitted with logistic regression.

Examples

Run this code

#Random by-construction well calibrated model
e <- runif(100)
o <- rbinom(100, size = 1, prob = e)
givitiCalibrationTest(o, e, "external")

#Random by-construction poorly calibrated model
e <- runif(100)
o <- rbinom(100, size = 1, prob = logistic(logit(e)+2))
givitiCalibrationTest(o, e, "external")