apply_metrics

This helper function applies scoring rules (stored as a list of
functions) to a data table of forecasts. <code>apply_metrics</code> is used within
<code>score()</code> to apply all scoring rules to the data.
Scoring rules are wrapped in <code>run_safely()</code> to catch errors and to make
sure that only arguments are passed to the scoring rule that are actually
accepted by it. A warning is issued if any column names in the input
data match names in the metrics list, as these will be overwritten.

internal

Facilitate the evaluation of forecasts in a convenient
framework based on data.table. It allows user to to check their forecasts
and diagnose issues, to visualise forecasts and missing data, to transform
data before scoring, to handle missing forecasts, to aggregate scores, and
to visualise the results of the evaluation. The package mostly focuses on
the evaluation of probabilistic forecasts and allows evaluating several
different forecast types and input formats. Find more information about the
package in the Vignettes as well as in the accompanying paper,
<doi:10.48550/arXiv.2205.07090>.

Nikos Bosse

scoringutils

Utilities for Scoring and Assessing Predictions

Sam Abbott

Hugo Gruson

Johannes Bracher

Toshiaki Asakura 

James Mba Azam

Sebastian Funk

Michael Chirico

apply_metrics function

<dl><dt>forecast</dt>
<dd>A forecast object (a validated data.table with predicted and
observed values).</dd>
<dt>metrics</dt>
<dd>A named list of scoring functions. Each element should be a
function reference, not a function call. For example, use
<code>list("crps" = crps_sample)</code> rather than <code>list("crps" = crps_sample())</code>.
Names will be used as column names in the output. See <code>get_metrics()</code> for
more information on the default metrics used. See the Customising metrics
section below for information on how to pass custom arguments to scoring
functions.</dd>
<dt>...</dt>
<dd>Additional arguments to be passed to the scoring rules. Note that
this is currently not used, as all calls to <code>apply_scores</code> currently
avoid passing arguments via <code>...</code> and instead expect that the metrics
directly be modified using <code><a href="https://purrr.tidyverse.org/reference/partial.html">purrr::partial()</a></code>.</dd></dl>

Arguments

Apply a list of functions to a data table of forecasts — apply_metrics

<dl>

<dt>forecast</dt>
<dd>A forecast object (a validated data.table with predicted and
observed values).</dd>


<dt>metrics</dt>
<dd>A named list of scoring functions. Each element should be a
function reference, not a function call. For example, use
<code>list("crps" = crps_sample)</code> rather than <code>list("crps" = crps_sample())</code>.
Names will be used as column names in the output. See <code>get_metrics()</code> for
more information on the default metrics used. See the Customising metrics
section below for information on how to pass custom arguments to scoring
functions.</dd>


<dt>...</dt>
<dd>Additional arguments to be passed to the scoring rules. Note that
this is currently not used, as all calls to <code>apply_scores</code> currently
avoid passing arguments via <code>...</code> and instead expect that the metrics
directly be modified using <code><a href='https://purrr.tidyverse.org/reference/partial.html'>purrr::partial()</a></code>.</dd>

</dl>

Apply a list of functions to a data table of forecasts

apply_metrics: Apply a list of functions to a data table of forecasts

Description

Usage

Value

Arguments