Boruta

0th

Percentile

Feature selection with the Boruta algorithm

Boruta is an all relevant feature selection wrapper algorithm, capable of working with any classification method that output variable importance measure (VIM); by default, Boruta uses Random Forest. The method performs a top-down search for relevant features by comparing original attributes' importance with importance achievable at random, estimated using their permuted copies, and progressively elliminating irrelevant featurs to stabilise that test.

Usage
Boruta(x, ...)
"Boruta"(x, y, pValue = 0.01, mcAdj = TRUE, maxRuns = 100, doTrace = 0, holdHistory = TRUE, getImp = getImpRfZ, ...)
"Boruta"(formula, data = .GlobalEnv, ...)
Arguments
x
data frame of predictors.
...
additional parameters passed to getImp.
y
response vector; factor for classification, numeric vector for regression, Surv object for survival (supports depends on importance adapter capabilities).
pValue
confidence level. Default value should be used.
if set to TRUE, a multiple comparisons adjustment using the Bonferroni method will be applied. Default value should be used; older (1.x and 2.x) versions of Boruta were effectively using FALSE.
maxRuns
maximal number of importance source runs. You may increase it to resolve attributes left Tentative.
doTrace
verbosity level. 0 means no tracing, 1 means reporting decision about each attribute as soon as it is justified, 2 means same as 1, plus reporting each importance source run.
holdHistory
if set to TRUE, the full history of importance is stored and returned as the ImpHistory element of the result. Can be used to decrease a memory footprint of Boruta in case this side data is not used, especially when the number of attributes is huge; yet it disables plotting of such made Boruta objects and the use of the TentativeRoughFix function.
getImp
function used to obtain attribute importance. The default is getImpRfZ, which runs random forest from the ranger package and gathers Z-scores of mean decrease accuracy measure. It should return a numeric vector of a size identical to the number of columns of its first argument, containing importance measure of respective attributes. Any order-preserving transformation of this measure will yield the same result. It is assumed that more important attributes get higher importance. +-Inf are accepted, NaNs and NAs are treated as 0s, with a warning.
formula
alternatively, formula describing model to be analysed.
data
in which to interpret formula.
Details

Boruta iteratively compares importances of attributes with importances of shadow attributes, created by shuffling original ones. Attributes that have significantly worst importance than shadow ones are being consecutively dropped. On the other hand, attributes that are significantly better than shadows are admitted to be Confirmed. Shadows are re-created in each iteration. Algorithm stops when only Confirmed attributes are left, or when it reaches maxRuns importance source runs. If the second scenario occurs, some attributes may be left without a decision. They are claimed Tentative. You may try to extend maxRuns or lower pValue to clarify them, but in some cases their importances do fluctuate too much for Boruta to converge. Instead, you can use TentativeRoughFix function, which will perform other, weaker test to make a final decision, or simply treat them as undecided in further analysis.

Value

An object of class Boruta, which is a list with the following components: , which is a list with the following components:

Note

Version 5.0 and 2.0 change some name conventions and thus may be incompatible with scripts written for earlier Boruta versions. Solutions of most problems of this kind should boil down to change of ZScoreHistory to ImpHistory in script source or Boruta object structure.

References

Miron B. Kursa, Witold R. Rudnicki (2010). Feature Selection with the Boruta Package. Journal of Statistical Software, 36(11), p. 1-13. URL: http://www.jstatsoft.org/v36/i11/

Aliases
• Boruta
• Boruta.default
• Boruta.formula
Examples
set.seed(777);
#Add some nonsense attributes to iris dataset by shuffling original attributes
iris.extended<-data.frame(iris,apply(iris[,-5],2,sample));
names(iris.extended)[6:9]<-paste("Nonsense",1:4,sep="");
#Run Boruta on this data
Boruta(Species~.,data=iris.extended,doTrace=2)->Boruta.iris.extended
#Nonsense attributes should be rejected
print(Boruta.iris.extended);

#Boruta using rFerns' importance
Boruta(Species~.,data=iris.extended,getImp=getImpFerns)->Boruta.ferns.irisE
print(Boruta.ferns.irisE);

## Not run:
# #Boruta on the HouseVotes84 data from mlbench
# #Takes some time, so be patient
# Boruta(Class~.,data=hvo,doTrace=2)->Bor.hvo;
# print(Bor.hvo);
# plot(Bor.hvo);
# plotImpHistory(Bor.hvo);
# ## End(Not run)
## Not run:
# #Boruta on the Ozone data from mlbench
# library(mlbench); data(Ozone);
# library(randomForest);
# na.omit(Ozone)->ozo;
# Boruta(V4~.,data=ozo,doTrace=2)->Bor.ozo;
# cat('Random forest run on all attributes:\n');
# print(randomForest(V4~.,data=ozo));
# cat('Random forest run only on confirmed attributes:\n');
# print(randomForest(ozo[,getSelectedAttributes(Bor.ozo)],ozo\$V4));
# ## End(Not run)
## Not run:
# #Boruta on the Sonar data from mlbench
# library(mlbench); data(Sonar);
# #Takes some time, so be patient
# Boruta(Class~.,data=Sonar,doTrace=2)->Bor.son;
# print(Bor.son);
# #Shows important bands
# plot(Bor.son,sort=FALSE);
# ## End(Not run)

Documentation reproduced from package Boruta, version 5.2.0, License: GPL (>= 2)

Community examples

Looks like there are no examples yet.