GAS (version 0.1.1)

BacktestDensity: Backtest a series of one step ahead density predictions.

Description

The BacktestDensity() function accepts an object of the class uGASRoll, and returns a list with two elements: (i) the averages Negative Log Score (NLS) and weighted Continuous Ranked Probability Score (wCRPS) introduced by Gneiting and Ranjan (2012), and (ii) their values at each point in time. The wCRPS is evaluated using 5 weight functions, see the Details.

Usage

BacktestDensity(Roll, lower, upper, K = 1000, a = 0.0, b = 1.0)

Arguments

Roll

an object of the class uGASRoll.

lower

numeric the lower bound used to approximate the wCRSP by Monte Carlo integration as detailed in Gneiting and Ranjan (2012). This coincides with $y_{l}$ in Equation 16 of Gneiting and Ranjan (2012).

upper

numeric the upper bound used to approximate the wCRSP by Monte Carlo integration as detailed in Gneiting and Ranjan (2012). This coincides with $y_{u}$ in Equation 16 of Gneiting and Ranjan (2012).

K

numeric integer representing the number of points used to discretize the wCRPS integral. This is $I$ in Equation 16 of Gneiting and Ranjan (2012). By default, K = 1000.

a

numeric. mean of the normal distribution used in the weight function. By default a = 0.0

b

numeric. standard deviation of the normal distribution used in the weight function.. By default b = 1.0

Value

list with elements: average, series. The element "average" is a named vector with the averages NLS and wCRSP. The element "series" is a list: the fist element, LS, contains the out of sample Log Score (note, not with the negative sign), the second element, WCRPS, contains a matrix with the wCRPS series. The columns of this matrix are named: "uniform", "center", "tails", "tail_r", "tail_l" , which are associated with the wCRSP with emphasis on : Uniform, Center, Tails, Right tail and Left tail, respectively.

Details

The average Negative Log Score (NLS) is computed as the negative of the mean of the log scores evaluated during the out of sample period. The average weighted Continuous Ranked Probability Score (wCRPS) is computed as the mean of the wCRPS evaluated during the out of sample period, see Gneiting and Ranjan (2012).

The wCRPS is evaluated using Equation 16 of Gneiting and Ranjan (2012). The weights functions implemented are:

• $w(z) = 1$: Uniform,
• $w(z) = \phi_{a,b}(z)$: Center,
• $w(z) = 1 - \phi_{a,b}(z)$: Tails,
• $w(z) = \Phi_{a,b}(z)$: Right tail,
• $w(z) = 1 - \Phi_{a,b}(z)$: Left tail.

where $\phi_{a,b}(z)$ and $\Phi_{a,b}(z)$ are the pdf and cdf of a Gaussian distribution with mean $a$ and standard deviation $b$, respectively. The label "Uniform" represents the case where equal emphasis is given to all the parts of the distribution.

References

Gneiting T, Ranjan R (2012). "Comparing density forecasts using threshold- and quantile-weighted scoring rules."Journal of Business & Economic Statistics. \Sexpr[results=rd,stage=build]{tools:::Rd_expr_doi("#1")}doi:10.1198/jbes.2010.08110http://doi.org/10.1198/jbes.2010.08110doi:\ifelse{latex}{\out{~}}{ }latex~ 10.1198/jbes.2010.08110 .

Examples

data("cpichg")

GASSpec   = UniGASSpec(Dist = "std", ScalingType = "Identity",
                       GASPar = list(location = TRUE, scale = TRUE,
                                      shape = FALSE))

Roll = UniGASRoll(cpichg, GASSpec, ForecastLength = 50,
                  RefitEvery = 10, RefitWindow = c("moving"))

BackTest =  BacktestDensity(Roll, lower = -100, upper = 100)

BackTest$average