MiraiExecutionConfigParallel execution through mirai. Takes no plan: mirai has no
equivalent of a future_plan, which is why it is not an argument here.
setup_MiraiExecution(
n_workers = NULL,
n_workers_outer = NULL,
n_workers_tuning = NULL,
n_workers_algorithm = NULL,
device = NULL,
on_error = c("continue", "stop", "stop_outer"),
seed = NULL,
shared_memory = c("auto", "none", "always"),
warm_workers = TRUE
)MiraiExecutionConfig object.
Integer [1, Inf): Number of parallel workers. Left NULL,
it follows the named dispatch levels, or default_n_workers() if none is named.
Optional Integer [1, Inf): Workers for outer resampling, overriding the automatic assignment.
Optional Integer [1, Inf): Workers for tuning, overriding the automatic assignment.
Optional Integer [1, Inf): Threads for a self-parallelizing algorithm, overriding the automatic assignment.
Optional DeviceConfig: Compute device, as a type name --
"cpu", "cuda", "mps" (the Apple silicon GPU) or "opencl" (a LightGBM
GPU build) -- or setup_CUDA to name which GPUs. NULL selects one per
algorithm: "cuda" where the machine has it and the algorithm can use it,
else the CPU. An algorithm that cannot use the requested device runs on the
CPU, and the run's resources line says so; a device the machine lacks is an
error when an algorithm tries to use it. "mps" is never selected
automatically: in rtemis benchmarks on an Apple M5 the CPU was faster for
every network narrower than about 1024 units or trained in batches smaller
than about 2048 cases, often by 2 to 5 times, and "mps" was faster by up to
about 25% only with both and thousands of features. It also slows down across
many fits in one R session, and a seed does not reach dropout on it.
Character {"continue", "stop", "stop_outer"}: Failure policy.
"continue" makes grid cells and unscorable hyperparameter combinations
non-fatal (recorded, warned, and excluded), failing only when nothing is scorable or
the final model fails; "stop" aborts on any error; "stop_outer" tolerates grid-cell
failures but aborts on an outer-fold failure.
Optional Integer [0, Inf): Master seed for the run's computation RNG,
from which one independent substream per parallel task is derived. Left NULL, a
seed is drawn from the current RNG stream and recorded on the returned object, so
every run is reproducible and the seed it used is auditable. Distinct from a
ResamplerConfig seed, which governs how the data is split.
Character {"none", "auto", "always"}: Whether to hand workers
the training data through OS shared memory instead of serializing a copy to each.
"auto" is the default: it shares whenever it can -- workers parallel and on this
machine, mori installed -- and quietly does not when it cannot, falling back to
the ordinary transport. "none" disables it. "always" is a demand: it shares even
when the run is sequential, so a run can be compared against its own shared
counterpart, and raises an error when the request cannot be met, for a caller relying
on sharing to stay inside a memory budget.
Logical: Load rtemis in every worker as the pool is built, rather than leaving each worker to load it on its first task. Either way the result is the same; TRUE records the loading time in the run's worker-pool setup step.
EDG
See setup_FutureExecution for how the three worker levels are assigned and how the run's seed is resolved; both are identical under either backend.
setup_MiraiExecution(n_workers = 2L)
Run the code above in your browser using DataLab