parallelly (version 1.24.0)

availableCores: Get Number of Available Cores on The Current Machine

Description

The current/main R session counts as one, meaning the minimum number of cores available is always at least one.

Usage

availableCores(
  constraints = NULL,
  methods = getOption2("parallelly.availableCores.methods", c("system", "nproc",
    "mc.cores", "_R_CHECK_LIMIT_CORES_", "PBS", "SGE", "Slurm", "LSF", "fallback",
    "custom")),
  na.rm = TRUE,
  logical = getOption2("parallelly.availableCores.logical", TRUE),
  default = c(current = 1L),
  which = c("min", "max", "all")
)

Arguments

constraints

An optional character specifying under what constraints ("purposes") we are requesting the values. For instance, on systems where multicore processing is not supported (i.e. Windows), using constrains = "multicore" will force a single core to be reported.

methods

A character vector specifying how to infer the number of available cores.

na.rm

If TRUE, only non-missing settings are considered/returned.

logical

Passed to detectCores(logical = logical), which, if supported, returns the number of logical CPUs (TRUE) or physical CPUs/cores (FALSE). This argument is only if argument methods includes "system".

default

The default number of cores to return if no non-missing settings are available.

which

A character specifying which settings to return. If "min", the minimum value is returned. If "max", the maximum value is returned (be careful!) If "all", all values are returned.

Value

Return a positive (>= 1) integer. If which = "all", then more than one value may be returned. Together with na.rm = FALSE missing values may also be returned.

Avoid ending up with zero cores

Note that availableCores() may return a single core. Because of this, using something like:

ncores <- availableCores() - 1

may return zero, which is often not intended. Instead, use:

ncores <- max(1, availableCores() - 1)

Advanced usage

It is possible to override the maximum number of cores on the machine as reported by availableCores(methods = "system"). This can be done by first specifying options(parallelly.availableCores.methods = "mc.cores") and then the number of cores to use, e.g. options(mc.cores = 8).

Details

The following settings ("methods") for inferring the number of cores are supported:

• "system" - Query detectCores(logical = logical).

• "nproc" - On Unix, query system command nproc.

• "mc.cores" - If available, returns the value of option mc.cores. Note that mc.cores is defined as the number of additional R processes that can be used in addition to the main R process. This means that with mc.cores = 0 all calculations should be done in the main R process, i.e. we have exactly one core available for our calculations. The mc.cores option defaults to environment variable MC_CORES (and is set accordingly when the parallel package is loaded). The mc.cores option is used by for instance mclapply() of the parallel package.

• "PBS" - Query TORQUE/PBS environment variables PBS_NUM_PPN and NCPUS. Depending on PBS system configuration, these resource parameter may or may not default to one. An example of a job submission that results in this is qsub -l nodes=1:ppn=2, which requests one node with two cores.

• "SGE" - Query Sun/Oracle Grid Engine (SGE) environment variable NSLOTS. An example of a job submission that results in this is qsub -pe smp 2 (or qsub -pe by_node 2), which requests two cores on a single machine.

• "Slurm" - Query Simple Linux Utility for Resource Management (Slurm) environment variable SLURM_CPUS_PER_TASK. This may or may not be set. It can be set when submitting a job, e.g. sbatch --cpus-per-task=2 hello.sh or by adding #SBATCH --cpus-per-task=2 to the hello.sh script. If SLURM_CPUS_PER_TASK is not set, then it will fall back to use SLURM_CPUS_ON_NODE if the job is a single-node job (SLURM_JOB_NUM_NODES is 1), e.g. sbatch --ntasks=2 hello.sh.

• "LSF" - Query Platform Load Sharing Facility (LSF) environment variable LSB_DJOB_NUMPROC. Jobs with multiple (CPU) slots can be submitted on LSF using bsub -n 2 -R "span[hosts=1]" < hello.sh.

• "custom" - If option parallelly.availableCores.custom is set and a function, then this function will be called (without arguments) and it's value will be coerced to an integer, which will be interpreted as a number of available cores. If the value is NA, then it will be ignored.

For any other value of a methods element, the R option with the same name is queried. If that is not set, the system environment variable is queried. If neither is set, a missing value is returned.

See Also

To get the set of available workers regardless of machine, see availableWorkers().

Examples

# NOT RUN {
message(paste("Number of cores available:", availableCores()))

# }
# NOT RUN {
options(mc.cores = 2L)
message(paste("Number of cores available:", availableCores()))
# }
# NOT RUN {
# }
# NOT RUN {
## IMPORTANT: availableCores() may return 1L
options(mc.cores = 1L)
ncores <- max(1, availableCores() - 1)
message(paste("Number of cores to use:", ncores))
# }
# NOT RUN {
# }
# NOT RUN {
## Use 75% of the cores on the system but never more than four
options(parallelly.availableCores.custom = function() {
  ncores <- max(parallel::detectCores(), 1L, na.rm = TRUE)
  min(0.75 * ncores, 4L)
})
message(paste("Number of cores available:", availableCores()))

## What is available minus one core but at least one
options(parallelly.availableCores.custom = function() {
  max(1L, parallelly::availableCores() - 1L)
})
message(paste("Number of cores available:", availableCores()))
# }
# NOT RUN {
# }