scoring (version 0.5-1)

brierscore: Calculate Brier Scores And Decompositions

Description

Calculate Brier scores, average Brier scores by a grouping variable, and Brier score decompositions for two-alternative forecasts.

Usage

brierscore(object, data, group = NULL, decomp = FALSE, bounds = NULL, reverse = FALSE)

Arguments

object

an object of class "formula", of the form outcome ~ forecast. See calcscore() documentation for further details.

data

an optional data frame or list containing the variables in the formula. If not found in data, the variables are taken from the environment from which calcscore is called.

group

the name of a grouping variable within data, which is used to calculate average Brier score by group.

decomp

if TRUE, Brier score decompositions are calculated.

bounds

a vector of length 2 corresponding to the desired minimum and maximum Brier score, respectively.

reverse

if FALSE (default), smaller scores imply better forecasts. If TRUE, larger scores imply better forecasts.

Value

Depending on input arguments, brierscore may return an object of class numeric containing raw Brier scores. It may also return a list containing the objects below.

rawscores

an object of class numeric containing raw Brier scores for each forecast.

mnbrier

an object of class numeric containing mean Brier scores for each unique value of group.

decomp

an object of class matrix containing Brier score decompositions and mean Brier scores for each unique value of group.

Details

If decomp=TRUE or group is supplied, the function returns a list (see value section). Otherwise, the function returns a numeric vector containing the Brier score associated with each forecast.

References

Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78, 1-3.

Murphy, A. H. (1973). A new vector partition of the probability score. Journal of Applied Meteorology, 12, 595-600.

Yates, J. F. (1982). External correspondence: Decompositions of the mean probability score. Organizational Behavior and Human Performance, 30, 132-156.

See Also

calcscore

Examples

data("WorldEvents")
## Raw Brier scores
brier1 <- brierscore(answer ~ forecast, data=WorldEvents)
## Raw Brier scores plus group means and decompositions
brier2 <- brierscore(answer ~ forecast, data=WorldEvents,
                     group="forecaster", decomp=TRUE)
## Obtain Brier scores via calcscore
brier3 <- calcscore(answer ~ forecast, data=WorldEvents,
                    param=2, fam="pow")
all.equal(brier1, brier3)