A batchtools local future is an synchronous uniprocess future that will be evaluated in a background R session. A batchtools interactive future is an synchronous uniprocess future that will be evaluated in the current R session (and variables will be assigned to the calling environment rather than to a local one). Both types of futures will block until the futures are resolved.
A batchtools SSH future is an asynchronous multiprocess
future that will be evaluated in a background R session.
We highly recommend using future::multisession
(sic!) futures of the future package instead of
SSH batchtools futures.
BatchtoolsFutureBackend(
workers = 1L,
resources = list(),
finalize = getOption("future.finalize", TRUE),
cluster.functions = NULL,
registry = list(),
conf.file = findConfFile(),
interrupts = TRUE,
delete = getOption("future.batchtools.debug", "on-success"),
...
)BatchtoolsCustomFutureBackend(...)
BatchtoolsInteractiveFutureBackend(fs.latency = 0, ...)
BatchtoolsLocalFutureBackend(fs.latency = 0, ...)
BatchtoolsSSHFutureBackend(workers = availableWorkers(), fs.latency = 65, ...)
A future::FutureBackend object of class BatchtoolsFutureBackend
An object of class BatchtoolsFuture.
An object of class BatchtoolsUniprocessFuture.
An object of class BatchtoolsMulticoreFuture.
(optional) The maximum number of workers the batchtools
backend may use at any time. Interactive and "local" backends can only
process one future at the time (workers = 1), whereas HPC backends,
where futures are resolved via separate jobs on a scheduler, can have
multiple workers. In the latter, the default is workers = NULL, which
will resolve to getOption("future.batchtools.workers", 100).
(optional) A named list passed to the batchtools
job-script template as variable resources. This is based on how
batchtools::submitJobs() works, with the exception for specially
reserved names defined by the future.batchtools package;
resources[["asis"]] is a character vector that are passed as-is to
the job script and are injected as job resource declarations.
resources[["modules"]] is character vector of Linux environment
modules to be loaded.
resources[["startup"]] and resources[["shutdown"]] are character
vectors of shell code to be injected to the job script as-is.
resources[["details"]], if TRUE, results in the job script outputting
job details and job summaries at the beginning and at the end.
All remaining resources named elements are injected as named resource
specification for the scheduler.
If TRUE, a future's batchtools Registry is automatically deleted when the future is garbage collected, otherwise not.
(optional) Assigned as-is to the each future's batchtools Registry.
(optional) A named list of settings applied to each
future's batchtools Registry.
This is a more convenient alternative to using argument conf.file.
(optional) A "batchtools-configuration" R script, which
is sourced when each future's batchtools
Registry is created. Any variables
created by this script is assigned to the registry.
The default file is the one found by batchtools::findConfFile(), if any.
If FALSE, attempts to interrupt futures will not take place on this backend, even if the backend supports it. This is useful when, for instance, it takes a long time to interrupt a future.
Controls if and when the batchtools job registry folder is
deleted.
If "on-success" (default), it is deleted if the future was resolved
successfully and the expression did not produce an error.
If "never", then it is never deleted.
If "always", then it is always deleted.
[numeric(1)]
Expected maximum latency of the file system, in seconds.
Set to a positive number for network file systems like NFS which enables more robust (but also more expensive) mechanisms to
access files and directories.
Usually safe to set to 0 to disable the heuristic, e.g. if you are working on a local file system.
Additional arguments passed to BatchtoolsFutureBackend().
batchtools local futures rely on the batchtools backend set up by
batchtools::makeClusterFunctionsInteractive(external = TRUE)
and batchtools interactive futures on the one set up by
batchtools::makeClusterFunctionsInteractive().
These are supported by all operating systems.
An alternative to batchtools local futures is to use
cluster futures of the future
package with a single local background session, i.e.
plan(cluster, workers = "localhost").
An alternative to batchtools interactive futures is to use
plan(sequential, split = TRUE) futures of the future package.
batchtools SSH futures rely on the batchtools backend set
up by batchtools::makeClusterFunctionsSSH().
The batchtools SSH backend only works on operating systems
supporting the ssh and ps command-line tool, e.g. Linux and macOS.
library(future)
## Create custom cluster functions (here same as "local")
cf <- batchtools::makeClusterFunctionsInteractive(external = TRUE)
print(cf)
str(cf)
# Use custom batchtools backend
plan(future.batchtools::batchtools_custom, cluster.functions = cf)
print(plan())
message("Main process ID: ", Sys.getpid())
f <- future({
data.frame(
hostname = Sys.info()[["nodename"]],
os = Sys.info()[["sysname"]],
cores = unname(parallelly::availableCores()),
pid = Sys.getpid(),
modules = Sys.getenv("LOADEDMODULES")
)
})
info <- value(f)
print(info)
Run the code above in your browser using DataLab