surveillance (version 1.12.1)

calibrationTest: Calibration Test for Poisson or Negative Binomial Predictions

Description

The implemented calibration tests for Poisson or negative binomial predictions of count data are based on proper scoring rules and described in detail in Wei and Held (2014). The following proper scoring rules are available: Dawid-Sebastiani score ("dss"), logarithmic score ("logs"), ranked probability score ("rps").

Usage

calibrationTest(x, ...)
## S3 method for class 'default':
calibrationTest(x, mu, size = NULL,
                which = c("dss", "logs", "rps"),
                tolerance = 1e-4, method = 2, ...)
dss(x, mu, size = NULL)
logs(x, mu, size = NULL)
rps(x, mu, size = NULL, k = 40, tolerance = sqrt(.Machine$double.eps))

Arguments

x

a numeric vector of observed counts. All involved functions are vectorized and also accept matrices or arrays. The score functions preserve the dimensions.

mu

a numeric vector of means of the predictive distributions for the observations x.

size

either NULL (indicating Poisson predictions with mean mu) or a numeric vector of dispersion parameters of the negative binomial distributions for the observations x.

which

a character string indicating which proper scoring rule to apply.

tolerance

absolute tolerance for the finite sum approximation for rps and for the null expectation and variance of "logs" and "rps". Unused for which = "dss" (closed form).

method

selection of the $z$-statistic: method = 2 refers to the alternative test statistic $Z_s^*$ of Wei and Held (2014, Discussion), which has been recommended for low counts. method = 1 corresponds to Equation 5 in Wei an

k

scalar argument controlling the finite sum approximation for the rps with truncation at ceiling(mu + k*sd).

...

unused (argument of the generic).

Value

• an object of class "htest", which is a list with the following components:
• method
  a character string indicating the type of test performed (including which scoring rule).
• data.name
  a character string naming the supplied x argument.
• statistic
  the $z$-statistic of the test.
• parameter
  the number of predictions underlying the test, i.e., length(x).
• p.value
  the p-value for the test.

References

Wei, W. and Held, L. (2014): Calibration tests for count data. Test, 23, 787-805.

Examples

## simulated example
mu <- c(0.1, 1, 3, 6, pi, 100)
size <- 0.1
set.seed(1)
y <- rnbinom(length(mu), mu = mu, size = size)
calibrationTest(y, mu = mu, size = size) # p = 0.99
calibrationTest(y, mu = mu, size = 1) # p = 4.3e-05
calibrationTest(y, mu = 1, size = 0.1) # p = 0.6959
calibrationTest(y, mu = 1, size = 0.1, which = "rps") # p = 0.1286


## a univariate surveillance time series
data("salmonella.agona")
salmonella <- disProg2sts(salmonella.agona)

## fit a hhh4() model
model <- list(end = list(f = addSeason2formula(~1 + t)),
              ar = list(f = ~1),
              family = "NegBin1")
fit <- hhh4(salmonella, model)

## do sequential one-step-ahead predictions for the last 5 weeks
pred <- oneStepAhead(fit, nrow(salmonella)-5, type="rolling",
                     which.start="final", verbose=FALSE)
pred

## test if the model is calibrated
with(pred, calibrationTest(x = observed, mu = pred, size = exp(psi)))
## p = 0.8746