reltest2_plot

Plots 9 diagnostics related to predictive probability matching.

Generates predictive distributions based on calibrating priors
for various commonly used statistical models, including models with
predictors. Routines for densities, probabilities, quantiles, random
deviates and the parameter posterior are provided. The predictions are
generated from the Bayesian prediction integral, with priors chosen to
give good reliability (also known as calibration). For homogeneous models,
the prior is set to the right Haar prior, giving predictions which are
exactly reliable. As a result, in repeated testing, the frequencies of
out-of-sample outcomes and the probabilities from the predictions agree.
For other models, the prior is chosen to give good reliability. Where
possible, the Bayesian prediction integral is solved exactly. Where exact
solutions are not possible, the Bayesian prediction integral is solved
using the Datta-Mukerjee-Ghosh-Sweeting (DMGS) asymptotic expansion.
Optionally, the prediction integral can also be solved using posterior
samples generated using Paul Northrop's ratio of uniforms sampling package
('rust'). Results are also generated based on maximum likelihood, for
comparison purposes. Various model selection diagnostics and testing
routines are included. Based on "Reducing reliability bias in assessments
of extreme weather risk using calibrating priors", Jewson, S., Sweeting, T.
and Jewson, L. (2024); <doi:10.5194/ascmo-11-1-2025>.

Stephen Jewson

fitdistcp

Distribution Fitting with Calibrating Priors for Commonly Used
Distributions

reltest2_plot function

<dl><dt>model</dt>
<dd>which distribution to test. Possibles values are
"gev",
"gpd",
"gev_p1".</dd>
<dt>ntrials</dt>
<dd>the number of trials o run. 5000 typically gives good results.</dd>
<dt>nrepeats</dt>
<dd>the number of entire repeats of the test to run, to check for convergence</dd>
<dt>nx</dt>
<dd>the length of the training data.</dd>
<dt>params</dt>
<dd>values for the parameters for the specified distribution</dd>
<dt>nmethods</dt>
<dd>the number of methods being tested</dd>
<dt>alpha</dt>
<dd>the values of alpha being tested</dd>
<dt>freqexceeded</dt>
<dd>the exceedance counts</dd>
<dt>case</dt>
<dd>there are 3 cases (must be set to case=1 except for my testing)</dd></dl>

Arguments

Plotting routine for reltest2 — reltest2_plot

<dl>

<dt>model</dt>
<dd>which distribution to test. Possibles values are
"gev",
"gpd",
"gev_p1".</dd>


<dt>ntrials</dt>
<dd>the number of trials o run. 5000 typically gives good results.</dd>


<dt>nrepeats</dt>
<dd>the number of entire repeats of the test to run, to check for convergence</dd>


<dt>nx</dt>
<dd>the length of the training data.</dd>


<dt>params</dt>
<dd>values for the parameters for the specified distribution</dd>


<dt>nmethods</dt>
<dd>the number of methods being tested</dd>


<dt>alpha</dt>
<dd>the values of alpha being tested</dd>


<dt>freqexceeded</dt>
<dd>the exceedance counts</dd>


<dt>case</dt>
<dd>there are 3 cases (must be set to case=1 except for my testing)</dd>

</dl>

reltest2_plot: Plotting routine for reltest2

Description

Usage

Value

Arguments