ggmlR (version 0.6.1)

lr_scheduler_cosine: Cosine-annealing learning rate scheduler

Description

Varies the learning rate following a cosine curve from lr_max down to lr_min over T_max steps. Restarts (SGDR-style) if restart = TRUE.

Usage

lr_scheduler_cosine(optimizer, T_max, lr_min = 0, restart = FALSE)

Value

An lr_scheduler_cosine environment

Arguments

optimizer

Optimizer environment.

T_max

Number of steps for one cosine cycle.

lr_min

Minimum learning rate (default 0).

restart

Logical; if TRUE restart after T_max steps.

Examples

# \donttest{
w   <- ag_param(matrix(runif(4), 2, 2))
opt <- optimizer_adam(list(w = w), lr = 0.1)
sch <- lr_scheduler_cosine(opt, T_max = 50L)
for (epoch in 1:50) sch$step()
# }