Container around a data.table::data.table() which stores all evaluated
hyperparameter configurations and performance scores.
The table ($data) has the following columns:
One column for each hyperparameter of the search space ($search_space).
One column for each performance measure ($codomain).
x_domain (list())
Lists of transformed hyperparameter values that are passed to the learner.
runtime_learners (numeric(1))
Sum of training and predict times logged in learners per
mlr3::ResampleResult / evaluation. This does not include potential
overhead time.
timestamp (POSIXct)
Time stamp when the evaluation was logged into the archive.
batch_nr (integer(1))
Hyperparameters are evaluated in batches. Each batch has a unique batch
number.
uhash (character(1))
Connects each hyperparameter configuration to the resampling experiment
stored in the mlr3::BenchmarkResult.
Each row corresponds to a single evaluation of a hyperparameter configuration.
The archive stores additionally a mlr3::BenchmarkResult
($benchmark_result) that records the resampling experiments. Each
experiment corresponds to to a single evaluation of a hyperparameter
configuration. The table ($data) and the benchmark result
($benchmark_result) are linked by the uhash column. If the results are
viewed with as.data.table(), both are joined automatically.
For analyzing the tuning results, it is recommended to pass the archive to
as.data.table(). The returned data table is joined with the benchmark result
which adds the mlr3::ResampleResult for each hyperparameter evaluation.
The archive provides various getters (e.g. $learners()) to ease the access.
All getters extract by position (i) or unique hash (uhash). For a
complete list of all getters see the methods section.
The benchmark result ($benchmark_result) allows to score the hyperparameter
configurations again on a different measure. Alternatively, measures can be
supplied to as.data.table().
The mlr3viz package provides visualizations for tuning results.
as.data.table.ArchiveTuning(x, unnest = "x_domain", exclude_columns = "uhash", measures = NULL)
Returns a tabular view of all evaluated hyperparameter configurations.
ArchiveTuning -> data.table::data.table()
x (ArchiveTuning)
unnest (character())
Transforms list columns to separate columns. Set to NULL if no column
should be unnested.
exclude_columns (character())
Exclude columns from table. Set to NULL if no column should be
excluded.
measures (list of mlr3::Measure)
Score hyperparameter configurations on additional measures.
bbotk::Archive -> ArchiveTuning
benchmark_result(mlr3::BenchmarkResult) Stores benchmark result.
learner()Retrieve mlr3::Learner of the i-th evaluation, by position
or by unique hash uhash. i and uhash are mutually exclusive.
Learner does not contain a model. Use $learners() to get learners with
models.
ArchiveTuning$learner(i = NULL, uhash = NULL)
i(integer(1))
The iteration value to filter for.
uhash(logical(1))
The uhash value to filter for.
learners()Retrieve list of trained mlr3::Learner objects of the i-th evaluation,
by position or by unique hash uhash. i and uhash are mutually
exclusive.
ArchiveTuning$learners(i = NULL, uhash = NULL)
i(integer(1))
The iteration value to filter for.
uhash(logical(1))
The uhash value to filter for.
learner_param_vals()Retrieve param values of the i-th evaluation, by position
or by unique hash uhash. i and uhash are mutually exclusive.
ArchiveTuning$learner_param_vals(i = NULL, uhash = NULL)
i(integer(1))
The iteration value to filter for.
uhash(logical(1))
The uhash value to filter for.
predictions()Retrieve list of mlr3::Prediction objects of the i-th evaluation, by
position or by unique hash uhash. i and uhash are mutually
exclusive.
ArchiveTuning$predictions(i = NULL, uhash = NULL)
i(integer(1))
The iteration value to filter for.
uhash(logical(1))
The uhash value to filter for.
resample_result()Retrieve mlr3::ResampleResult of the i-th evaluation, by position
or by unique hash uhash. i and uhash are mutually exclusive.
ArchiveTuning$resample_result(i = NULL, uhash = NULL)
i(integer(1))
The iteration value to filter for.
uhash(logical(1))
The uhash value to filter for.
print()Printer.
ArchiveTuning$print()
...(ignored).
clone()The objects of this class are cloneable with this method.
ArchiveTuning$clone(deep = FALSE)
deepWhether to make a deep clone.