Creates a PreprocessorConfig object, which can be used in preprocess.
setup_Preprocessor(
complete_cases = FALSE,
remove_features_thres = NULL,
remove_cases_thres = NULL,
missingness = FALSE,
impute = FALSE,
impute_type = c("missRanger", "micePMM", "meanMode"),
impute_missRanger_params = list(pmm.k = 3L, maxiter = 10L, num.trees = 500L),
impute_numeric = c("mean", "median"),
impute_categorical = "mode",
impute_numeric_values = NULL,
impute_level_values = NULL,
integer2factor = FALSE,
integer2numeric = FALSE,
logical2factor = FALSE,
logical2numeric = FALSE,
numeric2factor = FALSE,
numeric2factor_levels = NULL,
numeric_cut_n = 0L,
numeric_cut_labels = FALSE,
numeric_quant_n = 0L,
numeric_quant_NAonly = FALSE,
unique_len2factor = 0L,
character2factor = FALSE,
factorNA2missing = FALSE,
factorNA2missing_level = "missing",
factor2integer = FALSE,
factor2integer_startat0 = TRUE,
factor2integer_levels = NULL,
scale = FALSE,
center = scale,
scale_centers = NULL,
scale_coefficients = NULL,
remove_constants = FALSE,
remove_constants_skip_missing = TRUE,
remove_features = NULL,
remove_duplicates = FALSE,
one_hot = FALSE,
one_hot_levels = NULL,
add_date_features = FALSE,
date_features = c("weekday", "month", "year"),
add_holidays = FALSE,
holidays = c("USLaborDay", "NewYearsDay", "ChristmasDay"),
exclude = NULL
)PreprocessorConfig object.
Logical: If TRUE, only retain complete cases (no missing data). Not available inside train: a fitted preprocessor is replayed at predict time, where dropping rows would return fewer predictions than rows.
Optional Numeric (0, 1]: Remove features with
missing values in >= to this fraction of cases. 1 removes only fully-missing
features. Not available inside train: each resample would learn its own set, so the folds would train on different features. Name the columns with remove_features instead, or drop them beforehand with preprocess.
Optional Numeric (0, 1]: Remove cases with >= to this fraction of missing features. 1 removes only fully-missing cases. Not available inside train: a fitted preprocessor is replayed at predict time, where dropping rows would return fewer predictions than rows.
Logical: If TRUE, generate new boolean columns for each feature with missing values, indicating which cases were missing data.
Logical: If TRUE, impute missing cases. See impute_type.
Character {"missRanger", "micePMM", "meanMode"}:
Imputation method. "missRanger" fits one random forest per feature with
missing values, kept in the Preprocessor and applied to new data by
apply_preprocessor. "meanMode" imputes the values that impute_numeric
and impute_categorical compute on the training data. "micePMM" imputes by
predictive mean matching with 'mice', which imputes only the data it is
run on: a Preprocessor using it cannot be applied to new data. See the
Imputation section.
List: Parameters passed to
missRanger::missRanger, e.g. "pmm.k", "maxiter", "num.trees". pmm.k
greater than 0 results in predictive mean matching. Reduce num.trees for
faster imputation especially in large datasets. Set pmm.k = 0 to
disable predictive mean matching.
Character {"mean", "median"}: Statistic imputed into
numeric and logical features for impute_type = "meanMode", and into such
features with no missing training values for impute_type = "missRanger".
Integer and bit64::integer64 features take the statistic rounded half to
even, so they remain integer. Logical features take the statistic of their
0/1 values rounded half to even: the more frequent value, FALSE on a tie.
Integer features that code categories belong in factors: see
integer2factor.
Character {"mode"}: Statistic imputed into factor
and character features for impute_type = "meanMode", and into such
features with no missing training values for impute_type = "missRanger".
"mode" is the most frequent value. A tie goes to the first tied level of a
factor and to the first tied value in Unicode code point order of a
character feature.
Optional Named vector: Value imputed into
numeric or logical features, keyed by feature name; a logical feature
takes 1 as TRUE and 0 as FALSE. Covers every such feature for
impute_type = "meanMode" and features with no missing training values
for impute_type = "missRanger". Used in place of the value
impute_numeric computes. A value for an integer feature must be a whole
number, and a value for a logical feature 0 or 1. preprocess stores the
values it uses in the Preprocessor, and apply_preprocessor imputes new
data with them.
Optional Named vector: Value imputed into factor
or character features, keyed by feature name. Covers every such feature
for impute_type = "meanMode" and features with no missing training
values for impute_type = "missRanger". Used in place of the value
impute_categorical computes. preprocess stores the values it uses in the
Preprocessor, and apply_preprocessor imputes new data with them.
Logical: If TRUE, convert all integers to factors. This includes
bit64::integer64 columns.
Logical: If TRUE, convert all integers to numeric
(will only work if integer2factor = FALSE). This includes
bit64::integer64 columns.
Logical: If TRUE, convert all logical variables to factors.
Logical: If TRUE, convert all logical variables to numeric.
Logical: If TRUE, convert all numeric variables to factors.
Optional Character vector: Will be passed to
levels arg of factor() if numeric2factor = TRUE. For advanced/
specific use cases; need to know unique values of numeric vector(s) and given all
numeric vars have same unique values.
Integer [0, Inf): If > 0, convert all numeric variables to factors by
binning using base::cut with breaks equal to this number.
Logical: The labels argument of base::cut.
Integer [0, Inf): If > 0, convert all numeric
variables to factors by binning using base::cut with breaks equal to
this number of quantiles, produced using stats::quantile. 0 disables; 1
is a single break, which bounds no bin, and is rejected.
Logical: If TRUE, only bin numeric variables with missing values.
Integer [0, Inf): Convert all variables with less
than or equal to this number of unique values to factors. 0 and 1 both
disable it: a feature with one unique value is a constant, which
remove_constants covers.
For example, unique_len2factor = 2 converts binary variables encoded as
1, 2 to factors.
Logical: If TRUE, convert all character variables to factors.
Logical: If TRUE, make NA values in factors be of
level factorNA2missing_level. This step runs before imputation, so one
preprocess call can code missing values in categorical variables as a
level and impute numeric variables.
Character: Name of level if
factorNA2missing = TRUE.
Logical: If TRUE, convert all factors to integers.
Logical: If TRUE, start integer coding at 0.
Optional List: Named list of the form
"feature_name" = "levels". Used to code factors identically to the training
data when applying factor2integer to validation or test data using
Preprocessor. Values whose level is absent take the single index above the
known levels.
Logical: If TRUE, scale columns of x.
Logical: If TRUE, center columns of x. If unset, follows scale.
Optional Named vector: Centering values for each feature.
Optional Named vector: Scaling values for each feature.
Logical: If TRUE, remove constant columns.
Logical: If TRUE, skip missing values, before checking if feature is constant.
Optional Character vector: Features to remove.
Logical: If TRUE, remove duplicate cases. Not available inside train: a fitted preprocessor is replayed at predict time, where dropping rows would return fewer predictions than rows.
Logical: If TRUE, convert all factors using one-hot encoding.
Optional List: Named list of the form "feature_name" = "levels". Used when applying
one-hot encoding to validation or test data using Preprocessor.
Logical: If TRUE, extract date features from date columns.
Character {"weekday", "month", "year"} vector: Features to extract from dates.
Logical: If TRUE, extract holidays from date columns.
Character vector: Holidays to flag when add_holidays is
TRUE, named as timeDate holiday functions, e.g. "USLaborDay",
"ChristmasDay". See timeDate::listHolidays() for the valid names. Note
that unprefixed names are not US holidays: "LaborDay" is May 1.
Optional Integer vector: Exclude these columns from preprocessing.
"meanMode" stores one value per feature, computed on the data preprocess()
is given, and apply_preprocessor imputes new data with those values. Each
value has its feature's type: an integer feature receives an integer, a
logical feature a logical, and a factor one of its levels.
"missRanger" fits one random forest per feature with missing values and
keeps the forests, and a copy of the data predict() draws starting values
from, in the Preprocessor, which therefore grows with the data. The
forests are an R object: a config written to JSON carries the method and its
parameters, not the forests. A feature with no missing values in the data
has no forest and takes its impute_numeric or impute_categorical value
when it is missing in new data.
predict() stops on a factor level of an imputed feature that is absent
from the data the forests were fitted on.
"micePMM" imputes the data it is run on and fits no model for new data;
apply_preprocessor refuses a Preprocessor that uses it.
For preprocessing inside train, see the Imputation section of setup_SupervisedPreprocessor.
Steps run in this order, and the order is part of what each one does:
remove_constants runs before the quantile cut because a constant has no
quantiles, factorNA2missing before impute so that missing values in a
categorical feature become a level, and one_hot last
among the encoders so that it sees every factor the earlier steps created.
Feature creation precedes feature transformation: add_date_features and
add_holidays run ahead of the conversions, so the weekday, month and
holiday factors they derive are encoded and the year is scaled exactly as any
column of the input would be. They run after the case and feature filters,
which judge the data as it was given.
keep complete cases only
set aside excluded columns
remove named features
remove constants
remove duplicates
remove cases by missingness threshold
remove features by missingness threshold
add date features
add holidays
integer to factor
logical to factor
numeric to factor
character to factor
features with <= N unique values to factor
integer to numeric
logical to numeric
cut numeric to n bins
cut numeric to n quantiles
factor NA to named level
factor to integer
add missingness column
impute
scale and/or center
one-hot encoding
re-add excluded columns, after the preprocessed ones
EDG
preproc_config <- setup_Preprocessor(factorNA2missing = TRUE)
preproc_config
Run the code above in your browser using DataLab