mlr3hyperband (version 0.4.0)

mlr_optimizers_successive_halving: Hyperparameter Optimization with Successive Halving

Description

OptimizerSuccessiveHalving class that implements the successive halving algorithm (SHA). SHA randomly samples n candidate points and allocates a minimum budget (r_min) to all candidates. The candidates are raced down in stages to a single best candidate by repeatedly increasing the budget by a factor of eta and promoting only the best 1 / eta fraction to the next stage. This means promising points are allocated a higher budget overall and lower performing ones are discarded early on.

#' The budget hyperparameter must be tagged with "budget" in the search space. The minimum budget (r_min) which is allocated in the base stage, is set by the lower bound of the budget parameter. The upper bound defines the maximum budget (r_max) which is allocated to the candidates in the last stage. The number of stages is computed so that each candidate in base stage is allocated the minimum budget and the candidates in the last stage are not evaluated on more than the maximum budget. The following table is the stage layout for eta = 2, r_min = 1 and r_max = 8.

i	n_i	r_i
0	8	1
1	4	2
2	2	4
3	1	8

i is stage number, n_i is the number of configurations and r_i is the budget allocated to a single configuration.

Parameters

n

integer(1) Number of points in base stage.

eta

numeric(1) With every stage, the budget is increased by a factor of eta and only the best 1 / eta points are promoted to the next stage.

sampler

paradox::Sampler Object defining how the samples of the parameter space should be drawn. The default is uniform sampling.

repetitions

integer(1) If 1 (default), optimization is stopped once all stages are evaluated. Otherwise, optimization is stopped after repetitions runs of SHA. The bbotk::Terminator might stop the optimization before all repetitions are executed.

adjust_minimum_budget

logical(1) If TRUE, minimum budget is increased so that the last stage uses the maximum budget defined in the search space.

Archive

The bbotk::Archive holds the following additional columns that are specific to the successive halving algorithm:

• stage (integer(1)) Stage index. Starts counting at 0.

• repetition (integer(1)) Repetition index. Start counting at 1.

Custom Sampler

Hyperband supports custom paradox::Sampler object for initial configurations in each bracket. A custom sampler may look like this (the full example is given in the examples section):

# - beta distribution with alpha = 2 and beta = 5
# - categorical distribution with custom probabilities
sampler = SamplerJointIndep$new(list(
  Sampler1DRfun$new(params[[2]], function(n) rbeta(n, 2, 5)),
  Sampler1DCateg$new(params[[3]], prob = c(0.2, 0.3, 0.5))
))

Progress Bars

$optimize() supports progress bars via the package progressr combined with a Terminator. Simply wrap the function in progressr::with_progress() to enable them. We recommend to use package progress as backend; enable with progressr::handlers("progress").

Logging

Hyperband uses a logger (as implemented in lgr) from package bbotk. Use lgr::get_logger("bbotk") to access and control the logger.

Super class

bbotk::Optimizer -> OptimizerSuccessiveHalving

Methods

Public methods

• OptimizerSuccessiveHalving$new()

• OptimizerSuccessiveHalving$clone()

• bbotk::Optimizer$format()

• bbotk::Optimizer$optimize()

• bbotk::Optimizer$print()

Method new()

Creates a new instance of this R6 class.

Usage

OptimizerSuccessiveHalving$new()

Method clone()

The objects of this class are cloneable with this method.

Usage

OptimizerSuccessiveHalving$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

# NOT RUN {
library(bbotk)
library(data.table)

# set search space
search_space = domain = ps(
  x1 = p_dbl(-5, 10),
  x2 = p_dbl(0, 15),
  fidelity = p_dbl(1e-2, 1, tags = "budget")
)

# objective with modified branin function, see `bbotk::branin()`
objective = ObjectiveRFun$new(
  fun = branin,
  domain = domain,
  codomain = ps(y = p_dbl(tags = "minimize"))
)

# initialize instance and optimizer
instance = OptimInstanceSingleCrit$new(
  objective = objective,
  search_space = search_space,
  terminator = trm("evals", n_evals = 50)
)

optimizer = opt("successive_halving")

# optimize branin function
optimizer$optimize(instance)

# best scoring evaluation
instance$result

# all evaluations
as.data.table(instance$archive)
# }