imbalanced.rfsrc: Random Forests for Imbalanced Two-Class Classification

Description

Fits a random forest for a two-class outcome using the random forest quantile classifier (RFQ), balanced random forest (BRF), or standard classification. RFQ changes the probability threshold used to assign class labels; BRF changes the sampling used to grow the trees. The splitting rule, performance measure, and variable importance can be specified separately.

Companion helpers summarize performance, select a probability threshold, or extract the class-frequency threshold from response labels.

Usage

# S3 method for rfsrc
imbalanced(formula, data, ntree = 3000, 
  method = c("rfq", "brf", "standard"), splitrule = "auc",
  perf.type = NULL, block.size = NULL, fast = FALSE,
  ratio = NULL, ...)
get.imbalanced.performance(obj, prob = NULL, threshold = NULL,
                           confusion = FALSE, robust = FALSE)
get.imbalanced.optimize(obj, prob = NULL, newdata = NULL,
                        measure = c("gmean", "F1", "F1mod", "F1modgmean"),
                        ngrid = 1000, plot.it = TRUE)
get.rfq.threshold(y)
# S3 method for imbalanced.performance
print(x, digits = 4,
                                      show.confusion = TRUE, ...)

Value

imbalanced returns an object from the selected forest grow function. Usual components include the class-probability matrices

predicted

and predicted.oob, class assignments class and

class.oob, and requested performance and importance values. Availability depends on the selected grow interface and sampling options; see rfsrc and rfsrc.fast.

Performance and threshold helpers

get.imbalanced.performance: A named numeric vector of class counts, imbalance ratio, resolved numeric threshold, and performance measures. With confusion = TRUE, a named list also includes confusion, the confusion matrix with a class.error column. Both forms have class "imbalanced.performance" and a "rand" attribute containing the random-reference metrics and expected confusion counts.
get.imbalanced.optimize: A one-row data frame of performance measures at the selected threshold. Extract the cutoff from the threshold column; the row name is its grid index.
get.rfq.threshold: The smaller class count divided by the total observed class count, as a numeric scalar. Returns NULL unless the class-frequency table has exactly two entries. No forest or probability predictions are required.

The performance and optimization helpers return NULL when the evaluation response is not a two-level factor.

print.imbalanced.performance displays the summaries and returns

x invisibly.

Arguments

formula: A model formula with one factor response having two levels and one or more predictors.
data: A data frame containing the response and predictors. Data-frame subclasses are converted to a plain data frame.
ntree: Number of trees to grow. The default is 3000.
method: Classification method. "rfq", the default, uses the RFQ decision rule of O'Brien and Ishwaran (2019). "brf" uses class-weighted sampling with replacement and a reduced tree sample size. "standard" uses ordinary random forest classification without enabling the RFQ rule. See Details.
splitrule: Splitting rule passed to the forest grow function. The default is "auc". Other choices include "gini" and "entropy"; see rfsrc for supported classification rules. This setting is separate from perf.type.
perf.type: Performance measure used by the forest and its variable importance calculations. The default is "gmean" for RFQ and BRF, and "default" (misclassification error) for standard classification. Choices accepted by this wrapper are "none", "default", "standard", "misclass", "brier", "gmean", and the legacy spelling "g.mean". The latter is converted to "gmean". Use "none" to omit forest performance calculations.
block.size: Block size passed to the selected forest grow function for performance and VIMP calculations; see rfsrc. This wrapper passes NULL explicitly when the argument is not set. Specify the same block size when comparing VIMP across methods.
fast: Logical. Use rfsrc.fast for the RFQ or standard method when TRUE. The BRF branch always calls rfsrc and does not use this argument.
ratio: Optional, experimental majority-class undersampling for the RFQ and standard methods. Supply a number between zero and one. The majority class is sampled without replacement, while the minority class is bootstrapped with replacement. Small requested fractions may be increased by the sampling helper. This is a majority-class sampling fraction, rather than a requested final minority prevalence. Ignored for BRF. See Details for OOB support.
obj: For the performance and optimization helpers, a grow or prediction object with a single two-class outcome and observed responses, or a two-level response factor when prob is supplied.
prob: Optional probabilities aligned with the responses in obj. A numeric vector or one-column matrix or data frame contains minority-class probabilities. A two-column matrix or data frame contains both class probabilities; named columns are matched to factor levels, and unnamed columns follow factor-level order. NULL uses the object's predictions as described in Details.
threshold: Cutoff for minority-class probabilities in get.imbalanced.performance. NULL, the default, uses the minority proportion among the observed evaluation responses. "auto" uses that prevalence for an RFQ object and $0.5$ otherwise. "bayes" always uses $0.5$. A single finite numeric value specifies the cutoff directly; values outside $[0,1]$ are clipped to that interval. With direct response and probability inputs, "auto" uses $0.5$ because no fitted classifier is supplied.
confusion: Return a list including the confusion matrix instead of the default named vector of performance measures?
robust: Add one to the diagonal counts when calculating the four classification rates and their derived summaries? See Details.
newdata: Optional threshold-selection data for get.imbalanced.optimize, supplied with a grow object and prob = NULL. Must contain the predictors and observed response. NULL uses predictions already in obj. Ignored when prob is supplied.
measure: Performance measure to maximize during threshold selection: "gmean" (the default), "F1", "F1mod", or "F1modgmean". See Details for definitions.
ngrid: Number of equally spaced thresholds from zero to one, including both endpoints. Must be an integer of at least two.
plot.it: Draw the four threshold-search diagnostic panels?
y: Class-label vector for get.rfq.threshold, typically a two-level factor. Missing responses are omitted when counting classes.
...: Additional arguments passed to rfsrc or, when selected, rfsrc.fast. Examples include mtry, nodesize, nsplit, importance, and na.action. BRF sets its own case.wt, sampsize, and samptype. When ratio is supplied, the wrapper constructs samp and sets bootstrap = "by.user". Additional arguments to the print method are currently unused.
x: For the print method, the result of get.imbalanced.performance.
digits: Number of decimal places used in the performance display.
show.confusion: Print model and expected random-reference confusion tables when a confusion matrix is present in x? Obtain that matrix with confusion = TRUE in get.imbalanced.performance. This option does not recompute it.

Author

Hemant Ishwaran and Udaya B. Kogalur

Details

Choice of classifier

Let $\pi$ be the training proportion of the minority class and $\widehat p(x)$ its predicted probability. RFQ assigns an observation to the minority class when $\widehat p(x) \geq \pi$. The class-frequency threshold can identify minority observations whose predicted probability is below one half. The standard method instead uses ordinary class assignment based on the largest predicted probability. RFQ here is a classification rule, distinct from the continuous-response quantile estimation performed by quantreg.

BRF calls the grow function with replacement sampling, a tree sample size of twice the smaller observed class count, and case weights that give the two classes equal total sampling weight. The wrapper does not construct separate fixed-size samples from each class. Both fast and ratio are ignored in this branch.

Performance and variable importance

G-mean is the geometric mean of minority sensitivity and majority specificity, $\sqrt{\mathrm{sensitivity}\, \mathrm{specificity}}$. Larger values indicate better balance between the two class-specific rates. Misclassification and Brier scores are error measures for which smaller values are better. Changing perf.type selects the forest performance measure; it does not select the RFQ decision rule.

Request permutation VIMP explicitly with importance = "permute". For comparisons, use a common perf.type, splitting rule, and block.size, unless the comparison is intended to study those settings. Increase ntree as needed to assess the stability of OOB performance and VIMP.

Sampling and missing values

With ratio = NULL, the RFQ and standard branches pass missing-data handling to the selected grow function. BRF and calls using ratio first apply complete-case omission to the supplied data frame. This omission currently includes columns that are not selected by the formula and occurs before the grow function handles na.action.

With ratio supplied, each tree uses a bootstrap sample of the minority observations and a without-replacement sample of majority observations. At ratio = 1, every majority observation is in-bag for every tree, so majority-class OOB performance is unavailable. A single minority observation is also always in-bag in this sampling scheme. Use a separate evaluation sample when OOB support is absent.

Detailed performance summaries

get.imbalanced.performance calculates performance from observed responses and predicted probabilities.

Probability inputs.

Supply a grow or prediction object, or a two-level response factor with prob. With prob = NULL, the helper uses predicted.oob when present and not entirely missing; otherwise it uses predicted. When only some OOB predictions are missing, those entries remain missing.

A numeric vector or one-column matrix or data frame supplies minority-class probabilities; no column name is required. Two-column inputs supply both class probabilities; named columns are matched to factor levels, and unnamed columns follow factor-level order. Rows must align with the responses.

Evaluation classes.

All observed evaluation responses determine class counts, the minority class, and its proportion, before excluding unavailable predictions. The first factor level is the minority when counts tie.

Probability checks.

Finite probabilities below zero are set to zero, and those above one are set to one. Finite two-column rows must sum to one within numerical tolerance. Scoring excludes rows with missing responses or nonfinite probabilities; both columns must be finite for two-column inputs. Unavailable scores are NA.

Classification threshold.

An observation is assigned to the minority class when its predicted minority-class probability is at least the threshold, and to the majority class otherwise. The threshold choices are:

NULL: The evaluation minority proportion for every method (default).

"auto"

The evaluation minority proportion when forest$rfq is TRUE; $0.5$ otherwise, including direct response/probability inputs.

"bayes"

$0.5$ for every method.

Numeric

A single finite cutoff. Values below zero are set to zero, and values above one are set to one.

For RFQ test predictions, NULL and "auto" use evaluation prevalence, not training prevalence. The RFQ flag, rather than splitrule or perf.type, determines the "auto" choice. Changing the threshold preserves the evaluation minority definition and the fitted forest.

Performance measures.

The summary includes sensitivity (sens), specificity (spec), precision (prec), negative predictive value (npv), G-mean, misclass, brier, brier.norm, auc, logloss, and pr.auc. For complementary binary probabilities, brier is the mean squared error of the minority probability; brier.norm is four times this value. Log loss excludes infinite losses without clipping zero probabilities.

F-measures.

F1 is the harmonic mean of precision and sensitivity; F1mod is the harmonic mean of all four classification rates. F1gmean averages F1 and G-mean; F1modgmean averages F1mod and G-mean. A harmonic mean is zero if any component rate is zero and all are defined; it is NA if any required rate is undefined.

Robust summaries.

robust = TRUE adds one to each diagonal confusion count when calculating the four rates and their derived summaries. Returned confusion counts, misclassification error, and probability-based scores are unchanged.

Confusion matrix.

confusion = TRUE includes the confusion matrix: rows are observed classes and columns are predicted classes, with 0 denoting the majority and 1 the minority.

Random-score reference

The performance summary has a "rand" attribute containing reference metrics and expected confusion counts. This reference uses a score $U$ uniformly distributed on $[0,1]$, independent of class, and predicts the minority class when $U$ reaches the chosen threshold. For threshold $t$, its sensitivity is $1-t$, specificity is $t$, and ROC AUC is $1/2$. The separately returned PR-AUC reference is the proportion of positive observations among the rows scored by the PR helper. Its unnormalized Brier, normalized Brier, and log-loss references are $1/3$, $4/3$, and $1$, respectively. These differ from a predictor that always returns probability $1/2$.

Nonlinear rate summaries are computed from the reference rates (or smoothed expected counts under robust = TRUE); they are not finite-sample expectations of those nonlinear statistics. The performance print method compares the model and reference with Delta and percentage Gain, oriented so that positive values indicate improvement. The fitted-forest printer has its own probability reference; it need not use the same baseline.

Threshold selection

get.imbalanced.optimize searches ngrid equally spaced thresholds from zero to one, using G-mean, F1, F1mod, or F1modgmean as selected by measure. It uses the first grid point attaining the largest available value. It returns one row of performance metrics at the selected threshold and does not change the forest or its stored class predictions. plot.it = FALSE suppresses its four diagnostic panels. ngrid must be an integer of at least two. When the chosen measure is unavailable at every grid point, the function stops with an explanatory error. A valid optimum of zero is retained. The caller's graphics settings are restored after plotting.

A grow object ordinarily supplies OOB probabilities for this search. Supplying newdata searches using that data's observed responses and predictions, making it a threshold-selection sample. For an independent assessment of the chosen threshold, apply it unchanged to a separate test sample. The examples select a threshold from training OOB predictions and carry it forward to the test data.

Printing performance

Printing a performance result displays two metric tables. The first contains classification rates and their F1 and G-mean combinations; the second contains overall performance, including G-mean, AUC, PR-AUC, misclassification, Brier scores, and log loss. The same G-mean row appears in both tables; F1-related combinations appear only in the first.

Delta and percentage Gain compare the model with its random-score reference, with positive values indicating improvement. Larger rates, G-mean, and AUC values are better; smaller errors and losses are better. digits controls the display precision, and show.confusion = FALSE omits the optional confusion tables. These settings do not change the stored performance values.

References

Chen, C., Liaw, A. and Breiman, L. (2004). Using random forest to learn imbalanced data. University of California, Berkeley, Technical Report 110.

Kubat, M., Holte, R. and Matwin, S. (1997). Learning when negative examples abound. Machine Learning, ECML-97: 146-153.

O'Brien, R. and Ishwaran, H. (2019). A random forests quantile classifier for class imbalanced data. Pattern Recognition, 90, 232-249.

Examples

Run this code

# \donttest{
## ------------------------------------------------------------
## RFQ, BRF, and standard classification
## ------------------------------------------------------------
data(breast, package = "randomForestSRC")
dta <- na.omit(breast)
f <- status ~ .

set.seed(19)
rfq <- imbalanced(f, dta, ntree = 300)
print(rfq)
print(get.imbalanced.performance(rfq, confusion = TRUE))

## Extract the class-frequency threshold from the observed responses.
print(get.rfq.threshold(rfq$yvar))

brf <- imbalanced(f, dta, ntree = 300, method = "brf")
standard <- imbalanced(f, dta, ntree = 300, method = "standard")

## Compare all three using the helper's default evaluation rule.
metrics <- c("sens", "spec", "gmean", "misclass", "brier", "auc")
comparison <- rbind(
  RFQ = as.numeric(get.imbalanced.performance(rfq)[metrics]),
  BRF = as.numeric(get.imbalanced.performance(brf)[metrics]),
  Standard = as.numeric(get.imbalanced.performance(standard)[metrics])
)
colnames(comparison) <- metrics
print(comparison)

## Supply a cutoff explicitly to inspect another probability rule.
print(get.imbalanced.performance(standard, threshold = .5))

## Let the saved classifier setting select prevalence or 0.5.
print(get.imbalanced.performance(rfq, threshold = "auto"))
print(get.imbalanced.performance(standard, threshold = "auto"))
print(get.imbalanced.performance(brf, threshold = "auto"))

## Request the 0.5 Bayes cutoff explicitly, including for RFQ.
print(get.imbalanced.performance(rfq, threshold = "bayes"))

## Vectors, one-column objects, and named two-column matrices.
class.frequency <- table(rfq$yvar)
minority <- names(class.frequency)[which.min(class.frequency)]
p <- rfq$predicted.oob[, minority]
print(get.imbalanced.performance(rfq$yvar, p))
print(get.imbalanced.performance(rfq$yvar, data.frame(p = p)))
print(get.imbalanced.performance(
  rfq$yvar, rfq$predicted.oob[, rev(levels(rfq$yvar)), drop = FALSE]
))

## ------------------------------------------------------------
## Train/test assessment with a training-selected threshold
## ------------------------------------------------------------
set.seed(23)
trn.id <- sort(unlist(lapply(
  split(seq_len(nrow(dta)), dta$status),
  function(idx) idx[sample.int(length(idx),
    size = floor(2 * length(idx) / 3))]
)))
trn <- dta[trn.id, , drop = FALSE]
tst <- dta[-trn.id, , drop = FALSE]
print(rbind(training = table(trn$status), test = table(tst$status)))

fit <- imbalanced(f, trn, ntree = 300)
selected <- get.imbalanced.optimize(fit, measure = "gmean",
                                     ngrid = 501, plot.it = FALSE)
print(selected)
threshold.selected <- selected$threshold

pred <- predict(fit, newdata = tst)
print(get.imbalanced.performance(pred))
print(get.imbalanced.performance(pred, threshold = threshold.selected))

## Predicting labels for new rows does not require their outcomes.
class.frequency <- table(trn$status)
minority <- names(class.frequency)[which.min(class.frequency)]
majority <- setdiff(levels(trn$status), minority)
pred.x <- predict(fit, newdata = tst[, fit$xvar.names, drop = FALSE])
selected.class <- factor(
  ifelse(pred.x$predicted[, minority] >= threshold.selected,
         minority, majority),
  levels = levels(trn$status)
)
print(head(selected.class))

## ------------------------------------------------------------
## Explicit permutation VIMP and blocking
## ------------------------------------------------------------
rfq.vimp <- imbalanced(f, dta, ntree = 300,
                       importance = "permute", block.size = 1)
rfq.block <- imbalanced(f, dta, ntree = 300,
                        importance = "permute", block.size = 10)
print(cbind(unblocked = rfq.vimp$importance[, 1],
            blocked = rfq.block$importance[, 1]))

## Small B is for illustration; increase it for final inference.
smp <- subsample(rfq.block, B = 25, verbose = FALSE)
print(extract.subsample(smp)$var.sel.Z)
plot.subsample(smp, cex.axis = .7, alpha = .5)

## ------------------------------------------------------------
## Optional sampling and grow-interface choices
## ------------------------------------------------------------
rfq.sample <- imbalanced(f, dta, ntree = 300, ratio = .25)
print(get.imbalanced.performance(rfq.sample))
rfq.fast <- imbalanced(f, dta, ntree = 300, fast = TRUE)
print(get.imbalanced.performance(rfq.fast))
# }

Run the code above in your browser using DataLab