scoringRules (version 0.9.3)

scores_sample_univ: Scoring Rules for Simulated Forecast Distributions

Description

Calculate scores (CRPS, LogS) given observations and draws from the predictive distributions.

Usage

crps_sample(y, dat, method = "edf", w = NULL, bw = NULL,
  num_int = FALSE, show_messages = TRUE)
logs_sample(y, dat, bw = NULL, show_messages = TRUE)

Arguments

y

vector of realized values.

dat

vector or matrix (depending on y; see details) of simulation draws from forecast distribution.

method

string; approximation method. Options: "edf" (empirical distribution function) and "kde" (kernel density estimation).

w

vector or matrix (matching dat) of weights for method "edf".

bw

vector (matching y) of bandwidths for kernel density estimation; see details.

num_int

logical; if TRUE numerical integration is used for method "kde".

show_messages

logical; display of messages (does not affect warnings and errors).

Value

Value of the score. A lower score indicates a better forecast.

Details

For a vector y of length n, dat should be given as a matrix with n rows. If y has length 1, then dat may be a vector.

crps_sample employs an empirical version of the quantile decomposition of the CRPS (Laio and Tamea, 2007) when using method = "edf". For method = "kde", it uses kernel density estimation using a Gaussian kernel. The logarithmic score always uses kernel density estimation.

The bandwidth (bw) for kernel density estimation can be specified manually, in which case it must be a positive number. If bw == NULL, the bandwidth is selected using the core function bw.nrd. Numerical integration may speed up computation for crps_sample in case of large samples dat.

References

Evaluating simulation based forecast distributions:

Krueger, F., Lerch, S., Thorarinsdottir, T.L. and T. Gneiting (2016): `Probabilistic forecasting and comparative model assessment based on Markov Chain Monte Carlo output', working paper, Heidelberg Institute for Theoretical Studies, available at http://arxiv.org/abs/1608.06802.

Empirical quantile decomposition of the CRPS:

Laio, F. and S. Tamea (2007): `Verification tools for probabilistic forecasts of continuous hydrological variables', Hydrology and Earth System Sciences, 11, 1267-1277.

Examples

# NOT RUN {
# y has length greater than 1
y <- 1:2
sample <- matrix(rnorm(20), nrow = 2)
crps_sample(y = y, dat = sample)
logs_sample(y = y, dat = sample)

y <- 1:2
sample <- rnorm(10)
crps_sample(y = y, dat = sample) # error

# y has length 1
y <- 1
sample <- rnorm(10)
crps_sample(y = y, dat = sample)

sample <- matrix(rnorm(10), nrow = 1)
crps_sample(y = y, dat = sample)

sample <- matrix(rnorm(20), nrow = 2)
crps_sample(y = y, dat = sample) # error
# }
# NOT RUN {
# }