rfsrcSyn: Synthetic Random Forests

Description

Grows a synthetic random forest (RF) using RF machines as synthetic features. Applies only to regression and classification settings. Used for prediction only.

Usage

## S3 method for class 'rfsrc':
rfsrcSyn(formula, data, object, newdata,
  ntree = 1000,
  mtry = NULL,
  mtrySeq = NULL,
  nodesize = 5,
  nodesizeSeq = c(1:10,20,30,50,100),
  nsplit = 0,
  min.node = 3,
  use.org.features = TRUE,
  na.action = c("na.omit", "na.impute"),
  verbose = TRUE,
  ...)

Arguments

formula

A symbolic description of the model to be fit. Must be specified unless object is given.

data

Data frame containing the y-outcome and x-variables in the model. Must be specified unless object is given.

object

An object of class (rfsrc, synthetic). Not required when formula and data are supplied.

newdata

Test data used for prediction. If missing, predicted values for the training data are returned.

ntree

Number of trees.

mtry

mtry value for synthetic forest.

mtrySeq

Sequence of mtry values used for fitting the collection of RF machines. If NULL, set to the default value p/3.

nodesize

Nodesize value for the synthetic forest.

nodesizeSeq

Sequence of nodesize values used for the fitting the collection of RF machines.

nsplit

If non-zero, nsplit-randomized splitting is used which can significantly increase speed.

min.node

Minimum forest averaged number of nodes a RF machine must exceed in order to be used as a synthetic feature.

use.org.features

In addition to synthetic features, should the original features be used when fitting synthetic forests?

na.action

Missing value action. The default na.omit removes the entire record if even one of its entries is NA. The action na.impute pre-imputes the data using fast imputation via impute.rfsrc.

verbose

Set to TRUE for verbose output.

...

Further arguments to be passed to the rfsrc function used for fitting the synthetic forest.

Value

A list with the following components:
rfMachinesRF machines used to construct the synthetic features.
rfSynThe synthetic RF.
syntheticList containing the synthetic features.
predictedThe synthetic forest predicted value.
err.rateOOB error rate or test set error rate. For regression, MSE values are reported. For classification, the Brier score.
opt.machineOptimal machine: RF machine with smallest OOB error rate

Details

A collection of random forests are fit using different nodesize values. The predicted values from these machines are then used as synthetic features (called RF machines) to fit a synthetic random forest (the original features are also used when fitting the synthetic forest). Applies only to regression and classification settings. Used for prediction only.

Note that synthetic features are constructed using out-of-bag (OOB) data in order to avoid double dipping into training data. Nevertheless, the internal OOB error rate for the synthetic forest may be biased and thus it is recommended that cross-validation be used for determining test set performance.

If mtrySeq is set, RF machines are constructed for each combination of nodesize and mtry values specified by nodesizeSeq mtrySeq. However, a sequence of values for mtrySeq does not work as well as using a fixed value. Generally, performance gains are observed when one of the two sequences is fixed: mtrySeq is fixed and nodesizeSeq is varied, or nodesizeSeq is fixed and mtrySeq is varied.

References

TBD

Examples

Run this code

## ------------------------------------------------------------
## compare synthetic forests to regular forest (classification)
## ------------------------------------------------------------

## rfsrc and rfsrcSyn calls
if (library("mlbench", logical.return = TRUE)) {
  ring <- data.frame(mlbench.ringnorm(250, 20))
  ringRF <- rfsrc(classes ~., data = ring)
  ringSyn <- rfsrcSyn(classes ~., data = ring)

  ## OOB performance
  print(ringRF)
  print(ringSyn)
}

## ------------------------------------------------------------
## compare MSE of synthetic forest to regular forest (regression)
## 1) m-fold validation
## 2) oob MSE
## ------------------------------------------------------------

## synthetic data
n <- 250
d <- 50
std <- 0.1
x <- matrix(runif(n * d, -1, 1), ncol = d)
y <- 1 * (x[,1] + x[,4]^3 + x[,9] + sin(x[,12]*x[,18]) + rnorm(n, sd = std)>.38)
dat <- data.frame(x = x, y = y)

## define the cv folds
nfolds <- 10
n <- nrow(dat)
cvFolds <- split(sample(1:n), rep(1:nfolds, length = n))
mse <- matrix(NA, nfolds, 2)
rownames(mse) <- paste("fold", 1:nfolds)
colnames(mse) <- c("forest", "synthetic")

## cv loop
for (k in 1:nfolds) {

  ## train/test split
  cat("mfold:", k, "\n")
  test <- cvFolds[[k]]
  ytest <- dat$y[test]

  ## regular forest
  regF <- rfsrc(y ~ ., data = dat[-test, ])

  ## synthetic forest
  synF <- rfsrcSyn(y ~ ., data = dat[-test, ],
         newdata = dat[test, ], verbose = FALSE)

  ## extract predicted values
  pred.regF <- predict(regF, dat[test, ])$predicted
  pred.synF <- synF$predicted

  ## MSE performance
  mse[k, 1] <- mean((ytest - pred.regF)^2, na.rm = TRUE)
  mse[k, 2] <- mean((ytest - pred.synF)^2, na.rm = TRUE)
}

## output MSE performance
mse <- rbind(mse,
             avg = apply(mse, 2, mean, na.rm = TRUE),
             se  = sqrt(apply(mse, 2, var, na.rm = TRUE) / nfolds))
print(round(mse, 4))


## compare with OOB MSE
regF  <- rfsrc(y ~ ., data = dat, ntree = 5000)
synF <- rfsrcSyn(y ~ ., data = dat, ntree = 5000)
cat("forest    MSE:", regF$err.rate[5000], "\n")
cat("synthetic MSE:", synF$err.rate, "\n")

Run the code above in your browser using DataLab