mlr3hyperband (version 0.4.0)

mlr_optimizers_hyperband: Optimizer Using the Hyperband Algorithm

Description

OptimizerHyperband class that implements hyperband optimization (HB). HB repeatedly calls SHA (OptimizerSuccessiveHalving) with different numbers of starting points. A larger number of starting points corresponds to a smaller budget allocated in the base stage. Each run of SHA within HB is called a bracket. HB considers s_max + 1 brackets with s_max = floor(log(r_max / r_min, eta). The most explorative bracket s = s_max constructs s_max + 1 stages and allocates the minimum budget (r_min) in the base stage. The minimum budget is increased in each bracket by a factor of eta and the number of starting points is computed so that each bracket approximately spends the same budget. Use hyperband_schedule() to get a preview of the bracket layout.

s			3			2			1			0
i		n_i	r_i		n_i	r_i		n_i	r_i		n_i	r_i
0		8	1		6	2		4	4		8	4
1		4	2		3	4		2	8
2		2	4		1	8
3		1	8

s is the bracket number, i is stage number, n_i is the number of configurations and r_i is the budget allocated to a single configuration.

The budget hyperparameter must be tagged with "budget" in the search space. The minimum budget (r_min) which is allocated in the base stage of the most explorative bracket, is set by the lower bound of the budget parameter. The upper bound defines the maximum budget (r_max) which which is allocated to the candidates in the last stages.

Dictionary

This Optimizer can be instantiated via the dictionary mlr_optimizers or with the associated sugar function opt():

mlr_optimizers$get("hyperband")
opt("hyperband")

Parameters

eta

numeric(1) With every stage, the budget is increased by a factor of eta and only the best 1 / eta points are promoted to the next stage. Non-integer values are supported, but eta is not allowed to be less or equal 1.

sampler

paradox::Sampler Object defining how the samples of the parameter space should be drawn in the base stage of each bracket. The default is uniform sampling.

repetitions

integer(1) If 1 (default), optimization is stopped once all brackets are evaluated. Otherwise, optimization is stopped after repetitions runs of hyperband. The bbotk::Terminator might stop the optimization before all repetitions are executed.

Archive

The bbotk::Archive holds the following additional columns that are specific to the hyperband algorithm:

• bracket (integer(1)) The bracket index. Counts down to 0.

• stage (integer(1)) The stages of each bracket. Starts counting at 0.

• repetition (integer(1)) Repetition index. Start counting at 1.

Custom Sampler

Hyperband supports custom paradox::Sampler object for initial configurations in each bracket. A custom sampler may look like this (the full example is given in the examples section):

# - beta distribution with alpha = 2 and beta = 5
# - categorical distribution with custom probabilities
sampler = SamplerJointIndep$new(list(
  Sampler1DRfun$new(params[[2]], function(n) rbeta(n, 2, 5)),
  Sampler1DCateg$new(params[[3]], prob = c(0.2, 0.3, 0.5))
))

Progress Bars

$optimize() supports progress bars via the package progressr combined with a Terminator. Simply wrap the function in progressr::with_progress() to enable them. We recommend to use package progress as backend; enable with progressr::handlers("progress").

Logging

Hyperband uses a logger (as implemented in lgr) from package bbotk. Use lgr::get_logger("bbotk") to access and control the logger.

Super class

bbotk::Optimizer -> OptimizerHyperband

Methods

Public methods

• OptimizerHyperband$new()

• OptimizerHyperband$clone()

• bbotk::Optimizer$format()

• bbotk::Optimizer$optimize()

• bbotk::Optimizer$print()

Method new()

Creates a new instance of this R6 class.

Usage

OptimizerHyperband$new()

Method clone()

The objects of this class are cloneable with this method.

Usage

OptimizerHyperband$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

# NOT RUN {
library(bbotk)
library(data.table)

# set search space
search_space = domain = ps(
  x1 = p_dbl(-5, 10),
  x2 = p_dbl(0, 15),
  fidelity = p_dbl(1e-2, 1, tags = "budget")
)

# objective with modified branin function, see `bbotk::branin()`
objective = ObjectiveRFun$new(
  fun = branin,
  domain = domain,
  codomain = ps(y = p_dbl(tags = "minimize"))
)

# initialize instance and optimizer
instance = OptimInstanceSingleCrit$new(
  objective = objective,
  search_space = search_space,
  terminator = trm("evals", n_evals = 50)
)

optimizer = opt("hyperband")

# optimize branin function
optimizer$optimize(instance)

# best scoring evaluation
instance$result

# all evaluations
as.data.table(instance$archive)
# }