rtemis (version 1.4.3)

classification_metrics: Classification Metrics

Description

Classification Metrics

Usage

classification_metrics(
  true_labels,
  predicted_labels,
  predicted_prob = NULL,
  binclasspos = 2L,
  calc_auc = TRUE,
  calc_brier = TRUE,
  auc_method = "lightAUC",
  sample = NULL,
  verbosity = 0L
)

Value

ClassificationMetrics object.

Arguments

true_labels

Factor: True labels.

predicted_labels

Factor: predicted values.

predicted_prob

Optional Numeric vector or matrix [0, 1]: Predicted probabilities, one column per class. A single column is the positive class's probability and enables the probability-based metrics (AUC, Brier score); a multiclass matrix has no single score per case, so those are not computed.

binclasspos

Integer: Factor level position of the positive class in binary classification.

calc_auc

Logical: If TRUE, calculate AUC. May be slow in very large datasets.

calc_brier

Logical: If TRUE, calculate Brier score.

auc_method

Character: "lightAUC", "pROC", "ROCR".

sample

Optional Character {"Training", "Validation", "Test", "Calibrated Training", "Calibrated Validation", "Calibrated Test"}: Sample the metrics describe.

verbosity

Integer: Verbosity level.

Author

EDG

Details

auc_method = "pROC" is the only method that outputs an AUC when one or more predicted probabilities are NA.

Examples

# Assume positive class is "b"
true_labels <- factor(c("a", "a", "a", "b", "b", "b", "b", "b", "b", "b"))
predicted_labels <- factor(c("a", "b", "a", "b", "b", "a", "b", "b", "b", "a"))
predicted_prob <- c(0.3, 0.55, 0.45, 0.75, 0.57, 0.3, 0.8, 0.63, 0.62, 0.39)

classification_metrics(true_labels, predicted_labels, predicted_prob)
classification_metrics(true_labels, predicted_labels, 1 - predicted_prob, binclasspos = 1L)