Run traffic light tests for value at risk (VaR) and expected shortfall (ES) as well as a selection of coverage and independence tests for VaR.
# S4 method for fEGarch_risk
trafflight_test(object, silent = FALSE, ...)# S4 method for fEGarch_risk
uncond_cov_test(object, silent = FALSE, ...)
# S4 method for fEGarch_risk
indep_test(object, silent = FALSE, ...)
# S4 method for fEGarch_risk
cond_cov_test(object, silent = FALSE, ...)
# S4 method for fEGarch_risk
cov_tests(object, silent = FALSE, ...)
# S4 method for fEGarch_risk
backtest_suite(object, silent = FALSE, ...)
# S4 method for fEGarch_risk
WAD(object, silent = FALSE, ...)
All methods return a list invisibly. The elements of the list differ
slightly depending on the method. Moreover, for silent = FALSE,
the default, test results are printed to the console.
an object of class "fEGarch_risk".
a logical value indicating whether or not to print test results in a nicely formatted manner to the console.
currently without use.
backtest_suite runs all the other backtesting methods.
cov_tests runs all of uncond_cov_test,
indep_test and cond_cov_test.
Traffic light tests (trafflight_test):
Given an input object object of class "fEGarch_risk",
traffic light tests for value at risk (VaR) and expected shortfall (ES)
are applied to the individual risk measure series in the object. Note
that in order for a traffic light test in context of ES being applicable,
the corresponding VaR series of the same confidence level must also
be present in object. If this is not fulfilled,
messages will be printed to the console, making the user aware of
this issue.
Let the number of test observations be denoted by \(n\in \mathbb{N}\) and let \(\{r_t\}\), \(t=1,\dots,n\), be the test returns. \(\{\text{VaR}_t\}\) are the (one-step rolling) VaR point forecasts for the same period with confidence level \(\alpha\). Denote by \(I_t\) an indicator that equals 1, whenever \(r_t < \text{VaR}_t\), and 0 otherwise, and define \(K_1 = \sum_{t=1}^{n}I_t\). \(I_t\) are assumed to follow a binomial distribution with probability \(P = \alpha\) for any \(I_t = 0\). Then \(C\) is computed as the cumulative probability of observing \(K_1\) under \(P\). The forecasted VaR series is then classified following \(C\). If \(C < 0.95\), then it is sorted into the green zone, if \(0.95 \leq C < 0.9999\), then the series belongs to the yellow zone, and if \(C \geq 0.9999\), then the class of the VaR series is the red zone (Basel Committee on Banking Supervision, 1996).
The traffic light test for the ES (Costanzino and Curran, 2018) uses a similar classification system based on the severity of breaches $$B = \sum_{t = 1}^{n} \frac{1-F(\hat \eta_t)-\alpha}{1-\alpha}I_t,$$ where \(F\) is the (fitted) cumulative distribution function of the standardized innovations and with \(\hat \eta_t\) as the standardized residuals of a fitted GARCH-type model (or of its semiparametric extension). Then \(B \overset{a}{\sim}N(\mu_{\text{ES}}, \sigma^2_{\text{ES}})\) with \(\mu_{\text{ES}} = 0.5(1-\alpha)n\) and \(\sigma^2_{\text{ES}} = (1-\alpha)[(1+3\alpha) / 12]\). The cumulative probability of observing a severity of breaches of \(B\) or less can be computed and classified in the same way as for the VaR traffic light test using this asymptotic distribution.
Weighted Absolute Deviation (WAD) (WAD):
Following the standard computation of the 99
97.5
into account and summarizes them into one numeric value. Let \(N_1\) be the
observed breaches for the 99
corresponding expected number of breaches. \(N_2\) and \(\mu_2\) are to
understood analogously for the 97.5
breaches of the 97.5
is \(\mu_{\text{ES}}\) from before. Then
$$\text{WAD} = \frac{|N_1-\mu_1|}{\mu_1} + \frac{|N_2-\mu_2|}{\mu_2} + \frac{|N_3-\mu_3|}{\mu_3}.$$
See also Letmathe et al. (2022) for further information.
Coverage and independence tests (cov_tests):
Following Christoffersen (1998), the backtesting suite also includes a selection of coverage and independence tests regarding the VaR. Let the number of test observations be denoted by \(n\in \mathbb{N}\) and let \(\{r_t\}\), \(t=1,\dots,n\), be the test returns. \(\{\text{VaR}_t\}\) are the (one-step rolling) VaR point forecasts for the same period with confidence level \(\alpha\). Furthermore, define \(I_t\) to be an indicator that equals \(1\), whenever \(r_t < \text{VaR}_t\) and zero otherwise. Let \(K_1 = \sum_{t=1}^{n}I_t\) and \(K_0 = n - K_1\). Furthermore, \(\hat z_1 = K_1 / (K_0 + K_1)\) and \(\hat z_0 = K_0 / (K_0 + K_1)\) as well as $$L_{\hat z} = \hat z_0^{K_0} \hat z_1^{K_1}$$ and $$L_{\alpha} = \alpha^{K_0}(1-\alpha)^{K_1}.$$
In addition, we require \(I^{*}_{i,j}(t)\), \(t = 2,\dots,n\) and \(i,j \in \{0,1\}\), to be other indicators that equal 1, whenever \(I_t=j\) and simultaneously \(I_{t-1} = i\). Per consequence, \(K_{i,j}=\sum_{t=2}^{n} I^{*}_{i,j}(t)\) and \(\hat z_{i,j} = K_{i,j} / (K_{i,0} + K_{i, 1})\). Moreover, \(\hat z_1^{*} = (K_{0,1}+K_{1,1}) / (n - 1)\) and \(\hat z_0^{*} = 1-\hat z_1^{*}\). Now, $$L_{\hat z_{0,0}} = \hat z_{0,0}^{K_{0,0}} \hat z_{0,1}^{K_{0,1}} \hat z_{1,0}^{K_{1,0}} \hat z_{1,1}^{K_{1,1}}$$ and $$L_{\hat z^{*}} = (\hat z_{0}^{*})^{(K_{0,0} + K_{1,0})} (\hat z_{1}^{*})^{(K_{0,1} + K_{1,1})}.$$
Ultimately, $$L_{\alpha^{*}} = \alpha^{(K_{0,0} + K_{1,0})}(1-\alpha)^{(K_{0,1} + K_{1, 1})}.$$
The three test statistics following Christoffersen (1998) are then $$S_{\text{uc}} = -2 \ln\left[L_{\alpha} / L_{\hat{z}}\right] \overset{a}{\sim} \chi^2 (1),$$ $$S_{\text{ind}} = -2 \ln\left[L_{\hat z^{*}} / L_{\hat{z}_{0,0}}\right] \overset{a}{\sim} \chi^2 (1), \hspace{4mm} \text{and}$$ $$S_{\text{cc}} = -2 \ln\left[L_{\alpha^{*}} / L_{\hat{z}_{0,0}}\right] \overset{a}{\sim} \chi^2 (2),$$ where \(S_{\text{uc}}\) is the test statistic of the unconditional coverage test, \(S_{\text{ind}}\) is that of the independence test and \(S_{\text{cc}}\) is that of the conditional coverage test.
Basel Committee on Banking Supervision (1996). Supervisory Framework For The Use of "Backtesting" in Conjunction With The Internal Models Approach to Market Risk Capital Requirements. URL: https://www.bis.org/publ/bcbs22.pdf.
Christoffersen, P. F. (1998). Evaluating Interval Forecasts. International Economic Review, 39(4): 841-862. DOI: 10.2307/2527341.
Costanzino, N., & Curran, M. (2018). A Simple Traffic Light Approach to Backtesting Expected Shortfall. Risks, 6(1). DOI: 10.3390/risks6010002.
Letmathe, S., Feng, Y., & Uhde, A. (2022). Semiparametric GARCH models with long memory applied to Value at Risk and Expected Shortfall. Journal of Risk, 25(2). DOI: 10.21314/JOR.2022.044.
window.zoo <- get("window.zoo", envir = asNamespace("zoo"))
rt <- window.zoo(SP500, end = "2002-12-31")
model <- fEGarch(egarch_spec(), rt, n_test = 250)
fcast <- predict_roll(model)
risk <- measure_risk(fcast, measure = c("VaR", "ES"), level = c(0.95, 0.975, 0.99))
trafflight_test(risk)
cov_tests(risk)
backtest_suite(risk)
Run the code above in your browser using DataLab