RemixAutoML (version 0.11.0)

EvalPlot: EvalPlot automatically builds calibration plots for model evaluation

Description

This function automatically builds calibration plots and calibration boxplots for model evaluation using regression, quantile regression, and binary and multinomial classification

Usage

EvalPlot(data, PredictionColName = c("PredictedValues"),
  TargetColName = c("ActualValues"), GraphType = c("calibration"),
  PercentileBucket = 0.05, aggrfun = function(x) mean(x, na.rm = TRUE))

Arguments

data

Data containing predicted values and actual values for comparison

PredictionColName

String representation of column name with predicted values from model

TargetColName

String representation of column name with target values from model

GraphType

Calibration or boxplot - calibration aggregated data based on summary statistic; boxplot shows variation

PercentileBucket

Number of buckets to partition the space on (0,1) for evaluation

aggrfun

The statistics function used in aggregation, listed as a function

Value

Calibration plot or boxplot

See Also

Other Model Evaluation and Interpretation: ParDepCalPlots, RedYellowGreen, threshOptim

Examples

# NOT RUN {
Correl <- 0.85
data <- data.table::data.table(Target = runif(100))
data[, x1 := qnorm(Target)]
data[, x2 := runif(100)]
data[, Independent_Variable1 := log(pnorm(Correl * x1 +
                                            sqrt(1-Correl^2) * qnorm(x2)))]
data[, Predict := (pnorm(Correl * x1 +
                           sqrt(1-Correl^2) * qnorm(x2)))]
EvalPlot(data,
         PredictionColName = "Predict",
         TargetColName = "Target",
         GraphType = "calibration",
         PercentileBucket = 0.05,
         aggrfun = function(x) quantile(x, probs = 0.50, na.rm = TRUE))
# }