rtemis (version 1.4.3)

validate_config: Validate an rtemis config, against the schema and optionally against data

Description

Checks a config document -- as parsed from JSON, or already resolved -- and reports what is wrong with it. With no data, only the schema is checked: whether the document reconstructs into a valid rtemis config object. With data, the config is additionally checked against the dataset it would run on, catching the errors a schema cannot see: an outcome column that is not there, more folds than the rarer class has cases, predictors that never vary.

Usage

validate_config(config, data = NULL, outcome = NULL, step = NULL)

Value

Diagnostics object. Empty when nothing is wrong.

Arguments

config

Named list parsed from a config document (carrying its $schema), or a resolved rtemis config object. A resolved object is valid by construction, so the schema check is skipped for it.

data

Optional tabular data: The dataset the config would run on. NULL checks the schema only.

outcome

Optional Character: Name of the outcome column in data. NULL takes rtemis's convention, the last column -- but only for a config that designates an outcome at all; a decomposition or clustering config has no outcome and every column is a feature.

step

Optional Integer [1, Inf): Position of this config in the plan it came from, recorded on every finding. NULL when the config stands alone.

Author

EDG

Details

Nothing is thrown. Every finding is a Diagnostic carrying a stable code, a severity, a technical message, a plain-language plain text, the evidence behind it, and -- where a deterministic one exists -- a fix, an RFC 6902 JSON Patch that resolves it. An empty result means the config is clean.

Each data check runs only when the config carries the part it is about: the resampling checks need a resampler, the outcome checks need an outcome. A config carrying neither is checked for what it does declare and reports nothing about what it does not, so a preprocessing step and a training step each get the checks that apply to them.

Examples

# A config that does not reconstruct: one finding, and no data checks.
bad <- list(
  `$schema` = "https://schema.rtemis.org/supervised/r/v1/schema.json",
  n_foldz = 10L
)
validate_config(bad)

# A resolved config with nothing wrong: an empty Diagnostics.
validate_config(setup_SuperConfig(hyperparameters = setup_LightRF()))