bride (version 1.3)

bride: Brier Score Decomposition

Description

Decomposes the empirical Brier score into reliability, resolution and uncertainty. Two different estimators for the components are provided: The original estimators proposed by Murphy (1974), and the bias-corrected estimators proposed by Ferro and Fricker (2012). Sampling variances of all the components are estimated. This package applies only to probabilistic predictions of binary events.

Usage

bride(p, y, n.bins=10, ...)

Arguments

p

a vector of forecast probabilities. No default.

y

a vector of binary event indicators. No default.

n.bins

number of bins used for calculating the calibration function. Default n.bins = 10.

...

possible additional arguments. Not used at the moment.

Value

An object of class bride, essentially a list containing:

p,y

the objects of the original request.

n.bins

number of equidistant, exhaustive bins used to calculate the cross table.

rel,res,unc

reliability, resolution, uncertainty estimates derived by Murphy (1970).

rel2,res2,unc2

bias-corrected reliability, resolution, uncertainty estimates derived by Ferro and Fricker (2012).

rel.var,res.var,unc.var,rel2.var,res2.var,unc2.var

variance estimators derived by Siegert (2013).

Details

The values of the forecast probabilities in p are binned into n.bins bins of equal length on the unit interval. All probabilities are replaced with their in-bin average. Based on this binning, the calibration function $P(y=1|p)$ is estimated, which is required to estimate the components of the Brier Score decomposition of p.

References

Murphy, AH (1974) A new vector partition of the probability score, Journal of Applied Meteorology, 12:595-600 Ferro CAT, Fricker TE (2012) A bias-corrected decomposition of the Brier Score, Quarterly Journal of the Royal Meteorological Society, 138(668): 1954-1960 Siegert, S. (2013) Variance estimation for Brier Score decomposition, http://arxiv.org/abs/1303.6182

Examples

## number of forecasts and verifications
N <- 100
## produce some forecasts
p <- runif(n=N)
## produce events y[i] that occur with probability p[i]
y <- rbinom(n=N, size=1, prob=p)
## perform brier score decomposition
b <- bride(p=p,y=y)
## print
print(b)