The ArchiveAsyncTuning stores all evaluated hyperparameter configurations
and performance scores in a rush::Rush database.
The table ($data) has the following columns:
One column for each hyperparameter of the search space ($search_space).
One (list-)column for the internal_tuned_values
One column for each performance measure ($codomain).
x_domain (list())
Lists of (transformed) hyperparameter values that are passed to the learner.
runtime_learners (numeric(1))
Sum of training and predict times logged in learners per mlr3::ResampleResult / evaluation.
This does not include potential overhead time.
timestamp_xs (POSIXct)
Time stamp when the hyperparameter configuration was sent to a worker.
timestamp_ys (POSIXct)
Time stamp when the evaluation result was written to the archive.
For analyzing the tuning results, it is recommended to pass the ArchiveAsyncTuning to as.data.table().
The returned data table contains the mlr3::ResampleResult for each hyperparameter evaluation.
as.data.table.ArchiveAsyncTuning(x, unnest = "internal_tuned_values", exclude_columns = NULL, measures = NULL)
Returns a tabular view of all evaluated hyperparameter configurations.
ArchiveAsyncTuning -> data.table::data.table()
x (ArchiveAsyncTuning)
unnest (character())
Transforms list columns to separate columns. Set to NULL if no column should be unnested.
exclude_columns (character())
Exclude columns from table. Set to NULL if no column should be excluded.
measures (List of mlr3::Measure)
Score hyperparameter configurations on additional measures.
bbotk::Archive -> bbotk::ArchiveAsync -> ArchiveAsyncTuning
internal_search_space(paradox::ParamSet)
The search space containing those parameters that are internally optimized by the mlr3::Learner.
benchmark_result(mlr3::BenchmarkResult)
Benchmark result.
Inherited methods
bbotk::Archive$format()bbotk::Archive$help()bbotk::ArchiveAsync$best()bbotk::ArchiveAsync$clear()bbotk::ArchiveAsync$data_with_state()bbotk::ArchiveAsync$fail_point()bbotk::ArchiveAsync$fail_points()bbotk::ArchiveAsync$finish_point()bbotk::ArchiveAsync$finish_points()bbotk::ArchiveAsync$nds_selection()bbotk::ArchiveAsync$pop_point()bbotk::ArchiveAsync$push_failed_point()bbotk::ArchiveAsync$push_failed_points()bbotk::ArchiveAsync$push_finished_point()bbotk::ArchiveAsync$push_finished_points()bbotk::ArchiveAsync$push_point()bbotk::ArchiveAsync$push_points()bbotk::ArchiveAsync$push_result()bbotk::ArchiveAsync$push_running_point()bbotk::ArchiveAsync$push_running_points()
ArchiveAsyncTuning$new()Creates a new instance of this R6 class.
ArchiveAsyncTuning$new(
search_space,
codomain,
rush,
internal_search_space = NULL
)search_space(paradox::ParamSet)
Hyperparameter search space.
If NULL (default), the search space is constructed from the paradox::TuneToken of the learner's parameter set (learner$param_set).
When using to_tune() tokens, dependencies for hierarchical search spaces are automatically handled.
codomain(bbotk::Codomain)
Specifies codomain of objective function i.e. a set of performance measures.
Internally created from provided mlr3::Measures.
rush(Rush)
If a rush instance is supplied, the tuning runs without batches.
internal_search_space(paradox::ParamSet or NULL)
The internal search space.
ArchiveAsyncTuning$learner()Retrieve mlr3::Learner of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Learner does not contain a model. Use $learners() to get learners with models.
ArchiveAsyncTuning$learner(i = NULL, uhash = NULL)i(integer(1))
The iteration value to filter for.
uhash(character(1))
The uhash value to filter for.
ArchiveAsyncTuning$learners()Retrieve list of trained mlr3::Learner objects of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
ArchiveAsyncTuning$learners(i = NULL, uhash = NULL)i(integer(1))
The iteration value to filter for.
uhash(character(1))
The uhash value to filter for.
ArchiveAsyncTuning$learner_param_vals()Retrieve param values of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
ArchiveAsyncTuning$learner_param_vals(i = NULL, uhash = NULL)i(integer(1))
The iteration value to filter for.
uhash(character(1))
The uhash value to filter for.
ArchiveAsyncTuning$predictions()Retrieve list of mlr3::Prediction objects of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
ArchiveAsyncTuning$predictions(i = NULL, uhash = NULL)i(integer(1))
The iteration value to filter for.
uhash(character(1))
The uhash value to filter for.
ArchiveAsyncTuning$resample_result()Retrieve mlr3::ResampleResult of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
ArchiveAsyncTuning$resample_result(i = NULL, uhash = NULL)i(integer(1))
The iteration value to filter for.
uhash(character(1))
The uhash value to filter for.
ArchiveAsyncTuning$print()Printer.
ArchiveAsyncTuning$print()ArchiveAsyncTuning$clone()The objects of this class are cloneable with this method.
ArchiveAsyncTuning$clone(deep = FALSE)deepWhether to make a deep clone.
The ArchiveAsyncTuning is a connector to a rush::Rush database.