SpecsVerification (version 0.5-0)

ReliabilityDiagram: Reliability diagram for probability forecasts

Description

Reliability diagram for probability forecasts

Usage

ReliabilityDiagram(probs, obs, bins = 10, nboot = 500, plot = FALSE, plot.refin = TRUE, cons.probs = 0.95, attributes = FALSE, handle.na = "na.fail")

Arguments

probs

vector of N probability forecasts for the event obs=1

obs

vector of N binary observations, event/no event are coded as 0/1

bins

binning to estimate the calibration function (see Details), default: 10

nboot

number of bootstrap resamples to calculate the consistency bars, default: 500

plot

logical, whether to plot the reliability diagram, default: FALSE

plot.refin

Whether to add the frequency distribution of the forecasts to the reliability diagram. default: TRUE

cons.probs

The width of the consitency intervals. default: 0.95

attributes

locical, whether attributes lines are included in the diagram. default: FALSE

handle.na

how should missing values be handled; possible values are 'na.fail' and 'use.pairwise.complete'; default: 'na.fail'

Value

a data.frame with nrows equal to the number of bins (given by the `bins` argument), with columns: average forecast probability per bin, conditional event frequency per bin, lower and upper limit of the consistency bar per bin, number of forecast probabilities per bin, lower and upper bin limit

Details

To estimate the reliability curve, the unit line is categorised into discrete bins, provided by the `bins` argument. If `bins` is a single number, it specifies the number of equidistant bins. If `bins` is a vector of values between zero and one, these values are used as the bin-breaks.

References

Jolliffe IT, Stephenson DB, eds. (2012): Forecast verification: A practitioner's guide in atmospheric science. John Wiley & Sons, 2012. ISBN: 978-0-470-66071-3 Broecker J, Smith LA (2007): Increasing the Reliability of Reliability Diagrams. Wea. Forecasting, 22, 651--661. DOI: 10.1175/WAF993.1.

Examples

data(eurotempforecast)
p <- rowMeans(ens.bin)
ReliabilityDiagram(p, obs.bin, plot=TRUE)