step_nzv
creates a specification of a recipe step
that will potentially remove variables that are highly sparse
and unbalanced.
step_nzv(
recipe,
...,
role = NA,
trained = FALSE,
freq_cut = 95/5,
unique_cut = 10,
options = list(freq_cut = 95/5, unique_cut = 10),
removals = NULL,
skip = FALSE,
id = rand_id("nzv")
)# S3 method for step_nzv
tidy(x, ...)
A recipe object. The step will be added to the sequence of operations for this recipe.
One or more selector functions to choose which
variables that will evaluated by the filtering. See
selections()
for more details. For the tidy
method, these are not currently used.
Not used by this step since no new variables are created.
A logical to indicate if the quantities for preprocessing have been estimated.
Numeric parameters for the filtering process. See the Details section below.
A list of options for the filter (see Details below).
A character string that contains the names of
columns that should be removed. These values are not determined
until prep.recipe()
is called.
A logical. Should the step be skipped when the
recipe is baked by bake.recipe()
? While all operations are baked
when prep.recipe()
is run, some operations may not be able to be
conducted on new data (e.g. processing the outcome variable(s)).
Care should be taken when using skip = TRUE
as it may affect
the computations for subsequent operations
A character string that is unique to this step to identify it.
A step_nzv
object.
An updated version of recipe
with the new step
added to the sequence of existing steps (if any). For the
tidy
method, a tibble with columns terms
which
is the columns that will be removed.
This step diagnoses predictors that have one unique value (i.e. are zero variance predictors) or predictors that are have both of the following characteristics:
they have very few unique values relative to the number of samples and
the ratio of the frequency of the most common value to the frequency of the second most common value is large.
For example, an example of near zero variance predictor is one that, for 1000 samples, has two distinct values and 999 of them are a single value.
To be flagged, first the frequency of the most prevalent value
over the second most frequent value (called the "frequency
ratio") must be above freq_cut
. Secondly, the "percent of
unique values," the number of unique values divided by the total
number of samples (times 100), must also be below
unique_cut
.
In the above example, the frequency ratio is 999 and the unique value percentage is 0.0001.
# NOT RUN {
library(modeldata)
data(biomass)
biomass$sparse <- c(1, rep(0, nrow(biomass) - 1))
biomass_tr <- biomass[biomass$dataset == "Training",]
biomass_te <- biomass[biomass$dataset == "Testing",]
rec <- recipe(HHV ~ carbon + hydrogen + oxygen +
nitrogen + sulfur + sparse,
data = biomass_tr)
nzv_filter <- rec %>%
step_nzv(all_predictors())
filter_obj <- prep(nzv_filter, training = biomass_tr)
filtered_te <- bake(filter_obj, biomass_te)
any(names(filtered_te) == "sparse")
tidy(nzv_filter, number = 1)
tidy(filter_obj, number = 1)
# }
Run the code above in your browser using DataLab