VSURF (version 0.8.2)

VSURF.default: Variable Selection Using Random Forests

Description

Three steps variable selection procedure based on random forests for supervised classification and regression problems. First step ("thresholding step") is dedicated to eliminate irrelevant variables from the dataset. Second step ("interpretation step") aims to select all variables related to the response for interpretation prupose. Third step ("prediction step") refines the selection by eliminating redundancy in the set of variables selected by the second step, for prediction prupose.

Usage

## S3 method for class 'default':
VSURF(x, y, ntree = 2000, mtry = max(floor(ncol(x)/3), 1),
  nfor.thres = 50, nmin = 1, nfor.interp = 25, nsd = 1,
  nfor.pred = 25, nmj = 1, ...)
## S3 method for class 'formula':
VSURF(formula, data, ..., na.action = na.fail)
## S3 method for class 'default':
VSURF.parallel(x, y, ntree = 2000,
  mtry = max(floor(ncol(x)/3), 1), nfor.thres = 50, nmin = 1,
  nfor.interp = 25, nsd = 1, nfor.pred = 25, nmj = 1,
  clusterType = "PSOCK", ncores = detectCores() - 1, ...)
## S3 method for class 'formula':
VSURF.parallel(formula, data, ..., na.action = na.fail)

Arguments

data

a data frame containing the variables in the model.

na.action

A function to specify the action to be taken if NAs are found. (NOTE: If given, this argument must be named, and as randomForest it is only used with the formula-type call.)

x,formula

A data frame or a matrix of predictors, the columns represent the variables. Or a formula describing the model to be fitted.

y

A response vector (must be a factor for classification problems and numeric for regression ones).

ntree

Number of trees in each forests grown. Standard parameter of randomForest.

mtry

Number of variables randomly sampled as candidates at each split. Standard parameter of randomForest.

nfor.thres

Number of forests grown for "thresholding step" (first of the three steps).

nmin

Number of times the "minimum value" is multiplied to set threshold value.

nfor.interp

Number of forests grown for "intepretation step" (second of the three steps).

nsd

Number of times the standard deviation of the minimum value of err.interp is multiplied.

nfor.pred

Number of forests grown for "prediction step" (last of the three steps).

nmj

Number of times the mean jump is multiplied.

clusterType

Type of the multiple cores cluster used to run VSURF in parallel. Must be chosen among "PSOCK" (default: SOCKET cluster available locally on all OS), "FORK" (local too, only available for Linux and Mac OS) and "MPI" (can be used on a remote cluster, which

ncores

Number of cores to use. Default is set to the number of cores detected by R minus 1.

...

others parameters to be passed on to the randomForest function (see ?randomForest for further information)

Value

• An object of class VSURF, which is a list with the following components:
• varselect.thres
  A vector of indexes of variables selected after "thresholding step", sorted according to their mean VI, in decreasing order.
• varselect.interp
  A vector of indexes of variables selected after "interpretation step".
• varselect.pred
  A vector of indexes of variables selected after "prediction step".
• nums.varselect
  A vector of the 3 numbers of variables selected resp. by "thresholding step", "interpretation step" and "prediction step".
• imp.varselect.thres
  A vector of importances of the varselect.thres variables.
• min.thres
  The minimum predicted value of a pruned CART tree fitted to the curve of the standard deviations of VI.
• ord.imp
  A list containing the order of all variables mean importance. $x contains the mean importances sorted in decreasing order. $ix contains indexes of the variables.
• ord.sd
  A vector of standard deviations of all variables importance. The order is given by ord.imp.
• mean.perf
  Mean OOB error rate, obtained by a random forests build on all variables.
• pred.pruned.tree
  Predictions of the CART tree fitted to the curve of the standard deviations of VI.
• err.interp
  A vector of the mean OOB error rates of the embedded random forests models build during the "interpretation step".
• sd.min
  The standard deviation of OOB error rates associated to the random forests model attaining the minimum mean OOB error rate during the "interpretation step".
• err.pred
  A vector of the mean OOB error rates of the random forests models build during the "prediction step".
• mean.jump
  The mean jump value computed during the "prediction step".
• nmin,nsd,nmj
  Corresponding parameters values.
• overall.time
  Overall computation time.
• comput.times
  A list of the 3 computation times respectively associated with the 3 steps: "thresholding", "interpretation" and "prediction".
• clusterType
  The type of the cluster used to run VSURF.parallel (only if parallel version of VSURF is used).
• ncores
  The number of cores used to run VSURF.parallel (only if parallel version of VSURF is used).
• call
  The original call to VSURF.
• terms
  Terms associated to the formula (only if formula-type call was used).

Details

• First step ("thresholding step"): first,nfor.thresrandom forests are computed using the functionrandomForestwith argumentsimportance=TRUE. Then variables are sorted according to their mean variable importance (VI), in decreasing order. This order is kept all along the procedure. Next, a threshold is computed:min.thres, the minimum predicted value of a pruned CART tree fitted to the curve of the standard deviations of VI. Finally, the actual "thresholding step" is performed: only variables with a mean VI larger thannmin*min.thresare kept.
• Second step ("intepretation step"): the variables selected by the first step are considered.nfor.interpembedded random forests models are grown, starting with the random forest build with only the most important variable and ending with all variables selected in the first step. Then,err.minthe minimum mean out-of-bag (OOB) error of these models and its associated standard deviationsd.minare computed. Finally, the smallest model (and hence its corresponding variables) having a mean OOB error less thanerr.min+nsd*sd.minis selected.
• Third step ("prediction step"): the starting point is the same than in the second step. However, now the variables are added to the model in a stepwise manner.mean.jump, the mean jump value is calculated using variables that have been left out by the second step, and is set as the mean absolute difference between mean OOB errors of one model and its first following model. Hence a variable is included in the model if the mean OOB error decrease is larger thannmj*mean.jump.

VSURF.parallel is able to run VSURF using mutliple cores in parallel (see clusterType and ncores arguments).

References

Genuer, R. and Poggi, J.M. and Tuleau-Malot, C. (2010), Variable selection using random forests, Pattern Recognition Letters 31(14), 2225-2236

See Also

plot.VSURF, summary.VSURF, VSURF.thres, VSURF.interp, VSURF.pred, tune

Examples

data(iris)
iris.vsurf <- VSURF(x=iris[,1:4], y=iris[,5], ntree=100, nfor.thres=20,
                    nfor.interp=10, nfor.pred=10)
iris.vsurf

# A more interesting example with toys data (see \code{\link{toys}})
# (a few minutes to execute)
data(toys)
toys.vsurf <- VSURF(x=toys$x, y=toys$y)
toys.vsurf

# VSURF run on 2 cores in parallel (using a SOCKET cluster):
data(toys)
toys.vsurf.para <- VSURF.parallel(x=toys$x, y=toys$y, ncores=2)