accuracy.default: Accuracy measures for a cross-validation model and a conformal prediction model

Description

Return range of summary measures of the out-of-sample forecast accuracy. If x is given, the function also measures test set forecast accuracy. If x is not given, the function only produces accuracy measures on validation set.

Usage

# S3 method for default
accuracy(
  object,
  x,
  CV = TRUE,
  period = NULL,
  measures = interval_measures,
  byhorizon = FALSE,
  ...
)

Value

A matrix giving mean out-of-sample forecast accuracy measures.

Arguments

object: An object of class "cvforecast" or "cpforecast".
x: An optional numerical vector containing actual values of the same length as mean in object.
CV: If TRUE, the cross-validation forecast accuracy will be returned.
period: The seasonal period of the data.
measures: A list of accuracy measure functions to compute (such as point_measures or interval_measures).
byhorizon: If TRUE, accuracy measures will be calculated for each individual forecast horizon h separately.
...: Additional arguments depending on the specific measure.

Details

The measures calculated are:

ME: Mean Error
MAE: Mean Absolute Error
MSE: Mean Squared Error
RMSE: Root Mean Squared Error
MPE: Mean Percentage Error
MAPE: Mean Absolute Percentage Error
MASE: Mean Absolute Scaled Error
RMSSE: Root Mean Squared Scaled Error
winkler_score: Winkler Score
MSIS: Mean Scaled Interval Score

Examples

Run this code

# Simulate time series from an AR(2) model
library(forecast)
series <- arima.sim(n = 200, list(ar = c(0.8, -0.5)), sd = sqrt(1))

# Cross-validation forecasting with a rolling window
far2 <- function(x, h, level) {
  Arima(x, order = c(2, 0, 0)) |>
    forecast(h = h, level)
}
fc <- cvforecast(series, forecastfun = far2, h = 3, level = 95,
                 forward = TRUE, initial = 1, window = 50)

# Out-of-sample forecast accuracy on validation set
accuracy(fc, measures = point_measures, byhorizon = TRUE)
accuracy(fc, measures = interval_measures, level = 95, byhorizon = TRUE)

# Out-of-sample forecast accuracy on test set
accuracy(fc, x = c(1, 0.5, 0), measures = interval_measures,
         level = 95, byhorizon = TRUE)