FoReco (version 0.2.6)

score_index: Measuring accuracy in a rolling forecast experiment

Description

Function to calculate the accuracy indices of the reconciled point forecasts of a cross-temporal (not only, see examples) system (more in Average relative accuracy indices). (Experimental version)

Usage

score_index(recf, base, test, m, nb, nl, type = "mse", compact = TRUE)

Value

It returns a summary table called Avg_mat (if compact option is TRUE, default), otherwise it returns a list of six tables (more in Average relative accuracy indices).

Arguments

recf

list of q (forecast origins) reconciled forecasts' matrices ([n h(k^ + m)] in the cross-temporal case, [h n] in the cross-sectional case, and vectors of length [h(k^ m)] in the temporal framework).

base

list of q (forecast origins) base forecasts' matrices ([n h(k^ + m)] in the cross-temporal case, [h n] in the cross-sectional case, and vectors of length [h(k^ m)] in the temporal framework).

test

list of q (forecast origins) test observations' matrices ([n h(k^ + m)] in the cross-temporal case, [h n] in the cross-sectional case, and vectors of length [h(k^ m)] in the temporal framework).

m

Highest available sampling frequency per seasonal cycle (max. order of temporal aggregation, m), or a subset of p factors of m.

nb

number of bottom time series in the cross-sectional framework.

nl

(L 1) vector containing the number of time series in each cross-sectional level of the hierarchy (nl[1] = 1).

type

type of accuracy measure ("mse" Mean Square Error, "rmse" Root Mean Square Error or "mae" Mean Absolute Error).

compact

if TRUE returns only the summary matrix.

References

Di Fonzo, T., and Girolimetto, D. (2023), Cross-temporal forecast reconciliation: Optimal combination method and heuristic alternatives, International Journal of Forecasting, 39(1), 39-57.

See Also

Other utilities: Cmatrix(), FoReco2ts(), agg_ts(), arrange_hres(), commat(), ctf_tools(), hts_tools(), lcmat(), oct_bounds(), residuals_matrix(), shrink_estim(), thf_tools()

Examples

# \donttest{
data(FoReco_data)

# Cross-temporal framework
oct_recf <- octrec(FoReco_data$base, m = 12, C = FoReco_data$C,
                   comb = "bdshr", res = FoReco_data$res)$recf
oct_score <- score_index(recf = oct_recf,
                         base = FoReco_data$base,
                         test = FoReco_data$test, m = 12, nb = 5)

# Cross-sectional framework
# monthly base forecasts
mbase <- FoReco2matrix(FoReco_data$base, m = 12)$k1
# monthly test set
mtest <- FoReco2matrix(FoReco_data$test, m = 12)$k1
# monthly residuals
mres <- FoReco2matrix(FoReco_data$res, m = 12)$k1
# monthly reconciled forecasts
mrecf <- htsrec(mbase, C = FoReco_data$C, comb = "shr", res = mres)$recf
# score
hts_score <- score_index(recf = mrecf, base = mbase, test = mtest, nb = 5)

# Temporal framework
# top ts base forecasts ([lowest_freq' ...  highest_freq']')
topbase <- FoReco_data$base[1, ]
# top ts residuals ([lowest_freq' ...  highest_freq']')
topres <- FoReco_data$res[1, ]
# top ts test ([lowest_freq' ...  highest_freq']')
toptest <- FoReco_data$test[1, ]
# top ts recf ([lowest_freq' ...  highest_freq']')
toprecf <- thfrec(topbase, m = 12, comb = "acov", res = topres)$recf
# score
thf_score <- score_index(recf = toprecf, base = topbase, test = toptest, m = 12)
# }