FoReco (version 0.2.4)

score_index: Measuring accuracy in a rolling forecast experiment

Description

Function to calculate the accuracy indices of the reconciled point forecasts of a cross-temporal (not only, see examples) system (more in Average relative accuracy indices). (Experimental version)

Usage

score_index(recf, base, test, m, nb, nl, type = "mse", compact = TRUE)

Arguments

recf

list of q (forecast origins) reconciled forecasts' matrices ([n h(k^ + m)] in the cross-temporal case, [h n] in the cross-sectional case, and vectors of length [h(k^ m)] in the temporal framework).

base

list of q (forecast origins) base forecasts' matrices ([n h(k^ + m)] in the cross-temporal case, [h n] in the cross-sectional case, and vectors of length [h(k^ m)] in the temporal framework).

test

list of q (forecast origins) test observations' matrices ([n h(k^ + m)] in the cross-temporal case, [h n] in the cross-sectional case, and vectors of length [h(k^ m)] in the temporal framework).

m

Highest available sampling frequency per seasonal cycle (max. order of temporal aggregation, m), or a subset of p factors of m.

nb

number of bottom time series in the cross-sectional framework.

nl

(L 1) vector containing the number of time series in each cross-sectional level of the hierarchy (nl[1] = 1).

type

type of accuracy measure ("mse" Mean Square Error, "rmse" Root Mean Square Error or "mae" Mean Absolute Error).

compact

if TRUE returns only the summary matrix.

Value

It returns a summary table called Avg_mat (if compact option is TRUE, default), otherwise it returns a list of six tables (more in Average relative accuracy indices).

References

Di Fonzo, T., and Girolimetto, D. (2021), Cross-temporal forecast reconciliation: Optimal combination method and heuristic alternatives, International Journal of Forecasting, in press.

See Also

Other utilities: Cmatrix(), FoReco2ts(), commat(), ctf_tools(), hts_tools(), lcmat(), oct_bounds(), shrink_estim(), thf_tools()

Examples

# NOT RUN {
data(FoReco_data)

# Cross-temporal framework
oct_recf <- octrec(FoReco_data$base, m = 12, C = FoReco_data$C,
                   comb = "bdshr", res = FoReco_data$res)$recf
oct_score <- score_index(recf = oct_recf,
                         base = FoReco_data$base,
                         test = FoReco_data$test, m = 12, nb = 5)

# Cross-sectional framework
# monthly base forecasts
id <- which(simplify2array(strsplit(colnames(FoReco_data$base), split = "_"))[1, ] == "k1")
mbase <- t(FoReco_data$base[, id])
# monthly test set
mtest <- t(FoReco_data$test[, id])
# monthly residuals
id <- which(simplify2array(strsplit(colnames(FoReco_data$res), split = "_"))[1, ] == "k1")
mres <- t(FoReco_data$res[, id])
# monthly reconciled forecasts
mrecf <- htsrec(mbase, C = FoReco_data$C, comb = "shr", res = mres)$recf
# score
hts_score <- score_index(recf = mrecf, base = mbase, test = mtest, nb = 5)

# Temporal framework
# top ts base forecasts ([lowest_freq' ...  highest_freq']')
topbase <- FoReco_data$base[1, ]
# top ts residuals ([lowest_freq' ...  highest_freq']')
topres <- FoReco_data$res[1, ]
# top ts test ([lowest_freq' ...  highest_freq']')
toptest <- FoReco_data$test[1, ]
# top ts recf ([lowest_freq' ...  highest_freq']')
toprecf <- thfrec(topbase, m = 12, comb = "acov", res = topres)$recf
# score
thf_score <- score_index(recf = toprecf, base = topbase, test = toptest, m = 12)
# }
# NOT RUN {
# }