topolow (version 1.0.0)

parameter_sensitivity_analysis: Parameter Sensitivity Analysis

Description

Analyzes the sensitivity of model performance (MAE) to changes in a parameter. Uses binning to identify the minimum MAE across parameter ranges and calculates thresholds for acceptable parameter values.

Usage

parameter_sensitivity_analysis(
  param,
  samples,
  bins = 30,
  mae_col = "Holdout_MAE",
  threshold_pct = 5,
  min_samples = 1
)

Value

Object of class "parameter_sensitivity" containing:

param_values

Vector of parameter bin midpoints

min_mae

Vector of minimum MAE values per bin

param_name

Name of analyzed parameter

threshold

Threshold value (default: Topolow min. +5%)

min_value

Minimum MAE value across all bins

sample_counts

Number of samples per bin

Arguments

param

Character name of parameter to analyze

samples

Data frame containing parameter samples and performance metrics

bins

Integer number of bins for parameter range (default: 40)

mae_col

Character name of column containing MAE values (default: "Holdout_MAE")

threshold_pct

Numeric percentage above minimum for threshold calculation (default: 5)

min_samples

Integer minimum number of samples required in a bin (default: 1)

Details

The function performs these steps:

1. Cleans the input data using MAD-based outlier detection

2. Bins the parameter values into equal-width bins

3. Calculates the minimum MAE within each bin. Analogous to "poorman's likelihood" approach, minimum MAE within each bin is an empirical estimate of the performance surface at this parameter value when other parameters are at their optimal values.

4. Identifies a threshold of acceptable performance (default: Topolow min. +5% MAE)

5. Returns an object for visualization and further analysis