scoringRules (version 1.0.1)

scores_sample_multiv: Multivariate Scoring Rules for Simulated Forecast Distributions

Description

Compute multivariate scores of the form \(S(y, dat)\), where \(S\) is a proper scoring rule, \(y\) is a d-dimensional realization vector and \(dat\) is a simulated sample of multivariate forecasts. Available are the energy score and the variogram score of order \(p\).

Usage

es_sample(y, dat)
vs_sample(y, dat, w = NULL, p = 0.5)

Value

Value of the score. A lower score indicates a better forecast.

Arguments

y

realized values (numeric vector of length d).

dat

numeric matrix of data (columns are simulation draws from multivariate forecast distribution).

w

numeric matrix of weights for dat used in the variogram score. If no weights are specified, constant weights with \(w = 1\) are used.

p

order of variogram score. Standard choices include \(p = 1\) and \(p = 0.5\).

Author

Maximiliane Graeter, Sebastian Lerch, Fabian Krueger

Details

In the input matrix dat each column is expected to represent a sample from the multivariate forecast distribution, the number of rows of dat thus has to match the length of the observation vector y, and the number of columns of dat is the number of simulated samples.

In vs_sample it is possible to specify a matrix w of non-negative weights that allow to emphasize or downweight pairs of component combinations based on subjective expert decisions. w is a square matrix with dimensions equal to the length of the input vector y, and the entry in the \(i\)-th row and \(j\)-th column of w corresponds to the weight assigned to the combination of the corresponding \(i\)-th and \(j\)-th component. For details and examples, see Scheuerer and Hamill (2015).

References

Energy score

Gneiting, T., Stanberry, L.I., Grimit, E.P., Held, L. and N.A. Johnson (2008): `Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds', TEST, 17, 211-235. tools:::Rd_expr_doi("10.1007/s11749-008-0114-x")

Variogram-based proper scoring rules

Scheuerer, M. and T.M. Hamill (2015): `Variogram-based proper scoring rules for probabilistic forecasts of multivariate quantities', Monthly Weather Review, 143, 1321-1334. tools:::Rd_expr_doi("10.1175/mwr-d-14-00269.1")

Examples

d <- 10  # number of dimensions
m <- 50  # number of samples from multivariate forecast distribution

mu0 <- rep(0, d)
mu <- rep(1, d)
S0 <- S <- diag(d)
S[S==0] <- 0.1
S0[S0==0] <- 0.2

# generate samples from multivariate normal distributions
obs <- drop(mu0 + rnorm(d) %*% chol(S0))
fc_sample <- replicate(m, drop(mu + rnorm(d) %*% chol(S)))

es_sample(y = obs, dat = fc_sample)

# weighting matrix for variogram score
w_vs <- matrix(NA, nrow = d, ncol = d)
for(d1 in 1:d){for(d2 in 1:d){w_vs[d1,d2] <- 0.5^abs(d1-d2)}}

vs_sample(y = obs, dat = fc_sample) 
vs_sample(y = obs, dat = fc_sample, w = w_vs) 
vs_sample(y = obs, dat = fc_sample, w = w_vs, p = 1)