s2dverification (version 2.10.3)

BrierScore: Compute Brier Score And Its Decomposition And Brier Skill Score

Description

Computes the Brier score (BS) and the components of its standard decomposition as well with the two within-bin components described in Stephenson et al., (2008). It also returns the bias-corrected decomposition of the BS (Ferro and Fricker, 2012). BSS having the climatology as the reference forecast. .BrierScore provides the same functionality, but taking a matrix of ensemble members (exp) as input.

Usage

BrierScore(obs, pred, thresholds = seq(0, 1, 0.1))
.BrierScore(exp, obs, thresholds = seq(0, 1, 0.1))

Arguments

obs

Vector of binary observations (1 or 0).

pred

Vector of probablistic predictions with values in the range [0,1].

thresholds

Values used to bin the forecasts. By default the bins are [0,0.1), [0.1, 0.2), ... [0.9, 1].

exp

Matrix of predictions with values in the range [0,1] for the .BrierScore function

Value

Both BrierScore and .Brier score provide the same outputs:

• $relstandard reliability

• $resstandard resolution

• $uncstandard uncertainty

• $bsBrier score

• $bs_check_resrel-res+unc

• $bss_resres-rel/unc

• $gresgeneralized resolution

• $bs_check_gresrel-gres+unc

• $bss_gresgres-rel/unc

• $rel_bias_correctedbias-corrected rel

• $gres_bias_correctedbias-corrected gres

• $unc_bias_correctedbias-corrected unc

• $bss_bias_correctedgres_bias_corrected-rel_bias_corrected/unc_bias_corrected

• $nknumber of forecast in each bin

• $fkbaraverage probability of each bin

• $okbarrelative frequency that the observed event occurred

• $binsbins used

• $predvalues with which the forecasts are verified

• $obsprobability forecasts of the event

References

Wilks (2006) Statistical Methods in the Atmospheric Sciences. Stephenson et al. (2008). Two extra components in the Brier score decomposition. Weather and Forecasting, 23: 752-757. Ferro and Fricker (2012). A bias-corrected decomposition of the BS. Quarterly Journal of the Royal Meteorological Society, DOI: 10.1002/qj.1924.

Examples

# NOT RUN {
# Minimalist examples with BrierScore
a <- runif(10)
b <- round(a)
x <- BrierScore(b, a)
x$bs - x$bs_check_res
x$bs - x$bs_check_gres
x$rel_bias_corrected - x$gres_bias_corrected + x$unc_bias_corrected
 
# }
# NOT RUN {
a <- runif(10)
b <- cbind(round(a),round(a)) # matrix containing 2 identical ensemble members...
x2 <- BrierScore(a, b)
 
# }
# NOT RUN {
# Example of BrierScore using UltimateBrier
# See ?UltimateBrier for more information
example(Load)
clim <- Clim(sampleData$mod, sampleData$obs)
ano_exp <- Ano(sampleData$mod, clim$clim_exp)
ano_obs <- Ano(sampleData$obs, clim$clim_obs)
bs <- UltimateBrier(ano_exp, ano_obs, thr = c(1/3, 2/3))

 
# }
# NOT RUN {
# Example of .BrierScore with veriApply
require(easyVerification)
BrierScore2 <- s2dverification:::.BrierScore
bins_ano_exp <- ProbBins(ano_exp, thr = c(1/3, 2/3), posdates = 3, posdim = 2)
bins_ano_obs <- ProbBins(ano_obs, thr = c(1/3, 2/3), posdates = 3, posdim = 2)
bs2 <- veriApply("BrierScore2", bins_ano_exp, Mean1Dim(bins_ano_ob,s 3), 
                tdim = 2, ensdim = 3)
 
# }