hmer (version 1.6.2)

validation_diagnostics: Emulator Diagnostics

Description

Performs the standard set of validation diagnostics on emulators.

Usage

validation_diagnostics(
  emulators,
  targets = NULL,
  validation = NULL,
  which_diag = c("cd", "ce", "se"),
  analyze = TRUE,
  diagnose = "expectation",
  ...
)

Value

A data.frame containing points that failed one or more diagnostic tests.

Arguments

emulators

A list of Emulator objects.

targets

The list of observations for the outputs

validation

The validation set, containing all inputs and outputs.

which_diag

Which diagnostics should be performed (see description)

analyze

Should plotting and/or failing points be returned?

diagnose

For bimodal systems, should the expectation or variance be considered?

...

Any additional parameters to pass to the diagnostics (eg sd, cutoff, ...)

Details

All the diagnostics here can be performed with or without a validation (or `holdout') set of data. The presence of a set of targets is optional for some checks but mandatory for others: the appropriate warnings will be given in the event that some checks cannot be applied.

The current options for diagnostics (with the codes for which_diag) are:

Standardised Errors (se)

Comparison Diagnostics (cd)

Classification Errors (ce)

All of the above (all)

For details of each of the tests, see the help file for get_diagnostic.

See Also

Other diagnostic functions: analyze_diagnostic(), classification_diag(), comparison_diag(), get_diagnostic(), individual_errors(), residual_diag(), standard_errors(), summary_diag()

Examples

validation_diagnostics(SIREmulators$ems, SIREmulators$targets, SIRSample$validation)
# data.frame of failed points (empty) and a 3x3 set of plots
validation_diagnostics(SIREmulators$ems, SIREmulators$targets, SIRSample$validation,
 c('ce','cd'))
# empty data.frame and a 3x2 set of plots
validation_diagnostics(SIREmulators$ems, SIREmulators$targets, SIRSample$validation,
 cutoff = 2, sd = 2)
# k-fold (with k = 3)
validation_diagnostics(SIREmulators$ems, SIREmulators$targets, k = 3)