Preprocess, tune, train, and test supervised learning models using nested resampling in a single call.
train(
x,
dat_validation = NULL,
dat_test = NULL,
weights = NULL,
preprocessor_config = NULL,
decomposition_config = NULL,
hyperparameters = NULL,
tuner_config = NULL,
outer_resampling_config = NULL,
execution_config = setup_FutureExecution(),
question = NULL,
outdir = NULL,
preflight = FALSE,
verbosity = 1L,
...
)Object of class Regression(Supervised), RegressionRes(SupervisedRes),
Classification(Supervised), or ClassificationRes(SupervisedRes).
Tabular data, i.e. data.frame, data.table, or tbl_df (tibble): Training set data.
Tabular data: Validation set data.
Tabular data: Test set data.
Optional vector of case weights.
Optional SupervisedPreprocessorConfig object: Setup using setup_SupervisedPreprocessor.
Optional DecompositionConfig object: Setup using a decomposition
setup_* function.
Hyperparameters object: Setup using one of setup_* functions.
TunerConfig object: Setup using setup_GridSearch.
Optional ResamplerConfig object: Setup using
one of the setup_* resampler functions, e.g. setup_KFold.
This defines the outer resampling method, i.e. the splitting into training and test sets for the
purpose of assessing model performance. If NULL, no outer resampling is performed; dat_test
then provides a single test set for assessing model performance.
ExecutionConfig object: Setup using
setup_SerialExecution, setup_FutureExecution or setup_MiraiExecution.
The setup function selects the backend; each sets the number of workers
per dispatch level, and setup_FutureExecution additionally sets the future
plan.
Optional character string defining the question that the model is trying to answer.
Character, optional: String defining the output directory. The
fitted model is saved there as train_<algorithm>.rds, alongside a run
record, train_<algorithm>.record.json, stating what the run did:
every value resolved, where each came from, and what produced it. See
write_record.
Logical: If TRUE, check the configuration against x with
validate_config before training and stop on any finding of severity
"error". Findings of severity "warning" are reported and training continues.
Defaults to FALSE, except for a SuperConfigTabular, where it defaults to TRUE.
Pass FALSE explicitly to skip it.
Integer: Verbosity level.
Not used.
EDG
Online book & documentation
See docs.rtemis.org/r for detailed documentation.
Preprocessing
There are many different stages at which preprocessing could be applied, when running a
supervised learning pipeline with nested resampling. Some operations are best done before
passing data to train():
Duplicate rows should be removed before resampling, so that duplicates do not end up in different resamples, e.g. one in training and one in test.
Constant columns should be removed before resampling. A column may appear constant in a small resample, even if it is not constant in the full dataset. Removing it inconsistently will throw an error during prediction.
All data-dependent preprocessing steps need to be performed on training data only and applied on validation and test data, e.g. scaling, centering, imputation.
User-defined preprocessing through preprocessor_config is applied on training set data,
the learned parameters are stored in the returned Supervised or SupervisedRes object, and the
preprocessing is applied on validation and test data.
For missing values, prefer an algorithm that accepts them, such as LightGBM, LightRF, Ranger
or CART. For an algorithm that requires complete data, impute with
setup_SupervisedPreprocessor(impute = TRUE), which uses mean and mode imputation by
default. missRanger imputation is fitted on every training set, including every tuning
resample, and multiplies the memory and computation of a run; see the Imputation section of
setup_SupervisedPreprocessor.
Binary Classification
For binary classification, the outcome should be a factor where the 2nd level corresponds to the positive class.
Resampling
An outer resampling method with replacement combined with inner resampling (for tuning) can place duplicated cases from the outer resampling in both the training and test sets of the inner resamples, which underestimates test error.
Reproducibility
For outer resampling, set a seed when defining outer_resampling_config, e.g.
outer_resampling_config = setup_KFold(n_resamples = 10L, seed = 2026L)
For tuning with inner resampling, set a seed when defining tuner_config, e.g.
tuner_config = setup_GridSearch(
resampler_config = setup_KFold(n_resamples = 5L, seed = 2027L)
)
Parallelization
There are three levels of parallelization that may be used during training:
Algorithm training (e.g. a parallelized learner like LightGBM)
Tuning (where the full hyperparameter combination x inner resample cross product is processed in parallel, not just the resamples)
Outer resampling (where multiple outer resamples can be processed in parallel)
train() assigns all workers to exactly one of these levels, so that no two levels
compete for the same cores. The level is chosen by the first of these that applies:
the algorithm parallelizes itself, tuning is needed, outer resampling is set. A run
that needs none of them is sequential.
The workers, backend, and future plan all come from execution_config; see
setup_FutureExecution.
Reproducibility under parallelization
execution_config carries the master seed for the run's computation RNG, from which
one independent substream per outer resample is derived and assigned by fold index.
Results therefore do not depend on the backend, the number of workers, or the order
folds happened to finish in: a parallel run reproduces its sequential counterpart
exactly.
# \donttest{
iris_c_lightRF <- train(
iris,
hyperparameters = setup_LightRF(),
outer_resampling_config = setup_KFold(n_resamples = 3L)
)
# }
Run the code above in your browser using DataLab