parallelStart: Parallelization setup for parallelMap.

Description

Defines the underlying parallelization mode for parallelMap and allows to set a level of parallelization. Only calls to parallelMap with a matching level are parallelized.

Usage

parallelStart(mode, cpus, socket.hosts, level, logging,
    storagedir, bj.resources = list(), show.info, ...)
  parallelStartLocal(show.info)
  parallelStartMulticore(cpus, level, logging, storagedir,
    show.info)
  parallelStartSocket(cpus, socket.hosts, level, logging,
    storagedir, show.info, ...)
  parallelStartMPI(cpus, level, logging, storagedir,
    show.info, ...)
  parallelStartBatchJobs(level, logging, storagedir,
    bj.resources = list(), show.info)

Arguments

mode

[character(1)] Which parallel mode should be used: local, multicore, socket, mpi, BatchJobs. Default is the option parallelMap.default.mode

cpus

[integer(1)] Number of used cpus. For local and BatchJobs mode this argument is ignored. For socket this is the numbers of processes spawned on localhost, if you want processes on multiple machines use socket.hosts. Defa

socket.hosts

[character] Only used in socket mode, otherwise ignored. Names of hosts where parallel processes are spawned. Default is the option parallelMap.default.socket.hosts, if this option exists.

level

[character(1)] You can set this so only calls to parallelMap are parallelized that have the same level specified. Default is the option parallelMap.default.level or, if

logging

[logical(1)] Should slave output be logged to files via sink under the storagedir? Files are named ".log" and put into unique subdirectories named p

storagedir

[character(1)] Existing directory where log files and intermediate objects for BatchsJobs mode are stored. Note that all nodes must have write access to exactly this path. Default is the current working directory.

bj.resources

[list] Resources like walltime for submitting jobs on HPC clusters via BatchJobs. See submitJobs. Defaults are taken from your BatchJobs config file.

show.info

[logical(1)] Verbose output on console? Default is the option parallelMap.default.show.info or, if not set, TRUE.

...

[any] Optional parameters, for socket mode passed to makePSOCKcluster, for mpi mode passed to makeCluster.

Value

Nothing.

Details

The defaults of all settings are taken from your options, which you can also define in your R profile.

For an introductory tutorial and information on the options configuration, please go to the project's github page at https://github.com/berndbischl/parallelMap.

Currently the following modes are supported, which internally dispatch the mapping operation to functions from different parallelization packages:

[object Object],[object Object],[object Object],[object Object],[object Object]

For BatchJobs you need to define a storage directory through the argument storagedir or the option parallelMap.storagedir