MetaculR (version 0.4.1)

MetaculR_brier: Calculate Brier statistics on MetaculR_questions object

Description

Calculate Brier statistics on MetaculR_questions object

Usage

MetaculR_brier(MetaculR_questions, me = TRUE, thresholds = seq(0, 1, 0.1))

Arguments

MetaculR_questions

A MetaculR_questions object

me

Show my scores alongside Metaculus scores

thresholds

Thresholds to bin questions

Value

A list of Brier statistics for you and Metaculus.

brier_me, brier_Metaculus, brier_community

baseline.tf

Logical indicator of whether climatology was provided.

bs

Brier score

bs.baseline

Brier Score for climatology

ss

Skill score

bs.reliability

Reliability portion of Brier score.

bs.resolution

Resolution component of Brier score.

bs.uncert

Uncertainty component of Brier score.

y.i

Forecast bins -- described as the center value of the bins.

obar.i

Observation bins -- described as the center value of the bins.

prob.y

Proportion of time using each forecast.

obar

Forecast based on climatology or average sample observations.

thresholds

The thresholds for the forecast bins.

check

Reliability - resolution + uncertainty should equal brier score.

Other

ss_me_Metaculus, ss_me_community, ss_Metaculus_community

Skill score, me vs. Metaculus, etc.

count_questions

Number of total questions included.

brier_df: Used for plotting Brier score statistics

ID

Predictor.

name

Name of value, see above.

value

Value.

brier_bins_df: Used for plotting histogram and calibration plots.

ID

Predictor.

centers

y.i, see above.

freqs

prob.y, see above.

obars

obar.i, see above.

ideal

Ideal calibration where centers equals obars.

ci_low

Low end of 95% confidence interval for obar.i.

ci_high

High end of 95% confidence interval for obar.i.

Examples

# NOT RUN {
brier_me <-
  MetaculR_brier(
    questions_myPredictions_resolved)
# }