MachineShop (version 1.3.0)

performance: Model Performance Metrics

Description

Compute measures of model performance.

Usage

performance(x, ...)
# S3 method for Resamples
performance(x, ...)
# S3 method for factor
performance(x, y, metrics = c(Brier =
  MachineShop::brier, Accuracy = MachineShop::accuracy, Kappa =
  MachineShop::kappa2, `Weighted Kappa` = MachineShop::weighted_kappa2,
  ROCAUC = MachineShop::roc_auc, Sensitivity = MachineShop::sensitivity,
  Specificity = MachineShop::specificity), cutoff = 0.5, na.rm = TRUE,
  ...)
# S3 method for matrix
performance(x, y, metrics = c(RMSE = MachineShop::rmse,
  R2 = MachineShop::r2, MAE = MachineShop::mae), na.rm = TRUE, ...)
# S3 method for numeric
performance(x, y, metrics = c(RMSE = MachineShop::rmse,
  R2 = MachineShop::r2, MAE = MachineShop::mae), na.rm = TRUE, ...)
# S3 method for Surv
performance(x, y, metrics = c(CIndex =
  MachineShop::cindex, Brier = MachineShop::brier, ROCAUC =
  MachineShop::roc_auc, Accuracy = MachineShop::accuracy), cutoff = 0.5,
  na.rm = TRUE, ...)
# S3 method for Confusion
performance(x, ...)
# S3 method for ConfusionMatrix
performance(x, metrics = c(Accuracy =
  MachineShop::accuracy, Kappa = MachineShop::kappa2), ...)

Arguments

x

observed responses or class containing observed and predicted responses.

...

arguments passed from the Resamples method to the response type-specific methods or from the method for Confusion to ConfusionMatrix.

y

predicted responses.

metrics

function, one or more function names, or list of named functions to include in the calculation of performance metrics.

cutoff

threshold above which binary factor probabilities are classified as events and below which survival probabilities are classified.

na.rm

logical indicating whether to remove observed or predicted responses that are NA when calculating metrics.

See Also

response, predict, resample, confusion, metrics, plot, summary

Examples

# NOT RUN {
res <- resample(Species ~ ., data = iris, model = GBMModel)
(perf <- performance(res))
summary(perf)
plot(perf)

## Survival response example
library(survival)
library(MASS)

fo <- Surv(time, status != 2) ~ sex + age + year + thickness + ulcer
gbmfit <- fit(fo, data = Melanoma, model = GBMModel)

obs <- response(gbmfit, newdata = Melanoma)
pred <- predict(gbmfit, newdata = Melanoma, type = "prob")
performance(obs, pred)

# }