model_assessment: Model Assessment

Description

Computes various metrics for assessing the quality of a fitted flood frequency model. Metrics include accuracy (residual statistics), fitting efficiency (information criteria), and uncertainty (coverage based metrics using confidence intervals).

For NS-FFA: The metrics are computed from the normalized empirical/theoretical quantiles, which are determined based on the selected distribution family. Therefore, metrics from stationary and nonstationary models should not be compared.

Usage

model_assessment(
  data,
  distribution,
  method,
  prior = NULL,
  ns_years = NULL,
  ns_structure = NULL,
  alpha = 0.05,
  pp_formula = "Weibull",
  ci = NULL
)

Value

List containing the results of model assessment:

data: The data argument.
q_theoretical: The theoretical return level estimates based on the plotting positions. Normalized for nonstationary models.
q_empirical: The empirical return levels. Normalized for nonstationary models.
metrics: A list of model assessment metrics (see details).

Arguments

data

Numeric vector of observed annual maximum series values. Must be strictly positive, finite, and not missing.

distribution

A three-character code indicating the distribution family. Must be "GUM", "NOR", "LNO", "GEV", "GLO", "GNO", "PE3", "LP3", or "WEI".

method

Character scalar specifying the estimation method. Must be "L-moments", "MLE", or "GMLE".

prior

Numeric vector of length 2. Specifies the parameters of the Beta prior for the shape parameter \(\kappa\).

ns_years

For NS-FFA only: Numeric vector of observation years corresponding to data. Must be the same length as data and strictly increasing.

ns_structure

For NS-FFA only: Named list indicating which distribution parameters are modeled as nonstationary. Must contain two logical scalars:

location: If TRUE, the location parameter has a linear temporal trend.
scale: If TRUE, the scale parameter has a linear temporal trend.

alpha

Numeric scalar in \([0.01, 0.1]\). The significance level for confidence intervals or hypothesis tests. Default is 0.05.

pp_formula

Character string specifying the plotting position formula. Must "Weibull" (default), "Blom", "Cunnane", "Gringorten", or "Hazen".

ci

For S-FFA only. Dataframe containing return periods (in the column periods) and confidence intervals (in the columns lower and upper). Dataframes in this format can be generated with uncertainty_bootstrap(), uncertainty_rfpl(), or uncertainty_rfgpl().

Details

These metrics are are computed for all models:

AIC_MLL: Akaike Information Criterion, computed using the maximum log-likelihood.
BIC_MLL: Bayesian Information Criterion, computed using the maximum log-likelihood.
R2: Coefficient of determination from linear regression of estimates vs. data.
RMSE: Root mean squared error of quantile estimates.
bias: Mean bias of quantile estimates.
AIC_RMSE: Akaike Information Criterion, computed using the RMSE.
BIC_RMSE: Bayesian Information Criterion, computed using the RMSE.

These metrics are only computed if the ci argument is provided:

AW: Average width of the confidence interval(s).
POC: Percent of observations covered by the confidence interval(s).
CWI: Confidence width index, a metric that combines AW and POC.

Examples

Run this code

# Initialize example data and params
data <- rnorm(n = 100, mean = 100, sd = 10)
params <- c(100, 10)

# Perform uncertainty analysis
ci <- uncertainty_bootstrap(data, "NOR", "L-moments")$ci

# Run model assessment
model_assessment(data, "NOR", "L-moments", ci = ci)