MachineShop (version 1.1.0)

performance: Model Performance Metrics

Description

Compute measures of model performance.

Usage

performance(x, ...)
# S3 method for Resamples
performance(x, ..., na.rm = TRUE)
# S3 method for ConfusionMatrix
performance(x, metrics = c(Accuracy =
  MachineShop::accuracy, Kappa = MachineShop::kappa2), ...)
# S3 method for factor
performance(x, y, metrics = c(Accuracy =
  MachineShop::accuracy, Kappa = MachineShop::kappa2, ROCAUC =
  MachineShop::roc_auc, Sensitivity = MachineShop::sensitivity, Specificity
  = MachineShop::specificity, Brier = MachineShop::brier), cutoff = 0.5,
  ...)
# S3 method for matrix
performance(x, y, metrics = c(R2 = MachineShop::r2, RMSE
  = MachineShop::rmse, MAE = MachineShop::mae), ...)
# S3 method for numeric
performance(x, y, metrics = c(R2 = MachineShop::r2,
  RMSE = MachineShop::rmse, MAE = MachineShop::mae), ...)
# S3 method for Surv
performance(x, y, metrics = c(CIndex =
  MachineShop::cindex, ROCAUC = MachineShop::roc_auc, Brier =
  MachineShop::brier), cutoff = 0.5, times = numeric(), ...)
modelmetrics(...)

Arguments

x

observed responses or class containing observed and predicted responses.

...

arguments passed from the Resamples method to the others and from deprecated function modelmetrics to performance.

na.rm

logical indicating whether to remove observed or predicted responses that are NA when calculating metrics.

metrics

function, one or more function names, or list of named functions to include in the calculation of performance metrics.

y

predicted responses.

cutoff

threshold above which binary factor probabilities are classified as events and below which survival probabilities are classified.

times

numeric vector of follow-up times at which survival events were predicted.

See Also

response, predict, resample, confusion, metrics

Examples

# NOT RUN {
res <- resample(Species ~ ., data = iris, model = GBMModel)
(perf <- performance(res))
summary(perf)
plot(perf)

## Survival response example
library(survival)
library(MASS)

fo <- Surv(time, status != 2) ~ sex + age + year + thickness + ulcer
gbmfit <- fit(fo, data = Melanoma, model = GBMModel)

obs <- response(fo, data = Melanoma)
pred <- predict(gbmfit, newdata = Melanoma, type = "prob")
performance(obs, pred)

# }