pbdMPI (version 0.5-4)

Utility execmpi: Execute MPI code in system

Description

This function basically saves code in a spmd.file and executes MPI via R's system call e.g. system("mpiexec -np 1 Rscript spmd.file").

Usage

execmpi(spmd.code = NULL, spmd.file = NULL,
    mpicmd = NULL, nranks = 1L, rscmd = NULL, verbose = TRUE,
    disable.current.mpi = TRUE, mpiopt = NULL, rsopt = NULL)
runmpi(spmd.code = NULL, spmd.file = NULL,
    mpicmd = NULL, nranks = 1L, rscmd = NULL, verbose = TRUE,
    disable.current.mpi = TRUE, mpiopt = NULL, rsopt = NULL)

Value

Basically, only the PID of the MPI job (in background) will be returned in Linux-alike systems. For Windows, the MPI job is always wait until it is complete.

Arguments

spmd.code

SPMD code to be run via mpicmd and Rscript.

spmd.file

a file contains SPMD code to be run via mpicmd and Rscript.

mpicmd

MPI executable command. If NULL, system default will be searched.

nranks

number of processes to run the SPMD code envoked by mpicmd.

rscmd

Rscript executable command. If NULL, system default will be searched.

verbose

print SPMD code outputs and MPI messages.

disable.current.mpi

force to finalize the current MPI comm if any, for unix-alike system only.

mpiopt

MPI options appended after -np nranks --oversubscribe .

rsopt

Rscript options appended after Rscript .

Author

Wei-Chen Chen and Drew Schmidt.

Details

When the spmd.code is NULL: The code should be already saved in the file named spmd.file for using.

When the spmd.code is not NULL: The spmd.code will be dumped to a temp file (spmd.file) via the call writeLines(spmd.code, conn) where conn <- file(spmd.file, open = "wt"). The file will be closed after the dumping.

When spmd.file is ready (either dumped from spmd.code or provided by the user), the steps below will be followed: If spmd.file = NULL, then a temporary file will be generated and used to dump spmd.code.

For Unix-alike systems, the command cmd <- paste(mpicmd, "-np", nranks, mpiopt, rscmd, rscmd spmd.file, ">", log.file, " 2>&1 & echo \"PID=$!\" &") is executed via system(cmd, intern = TRUE, wait = FALSE, ignore.stdout = TRUE, ignore.stderr = TRUE). The log.file is a temporary file to save the outputs from the spmd.code. The results saved to the log.file will be read back in and cat and return to R.

For OPENMPI, the "--oversubscribe " is added before mpiopt as mpiopt <- paste("--oversubscribe ", mpiopt, sep = "") and is passed to cmd thereon.

For Windows, the cmd will be paste(mpicmd, "-np", nranks, mpiopt, rscmd, rsopt spmd.file) and is executed via system(cmd, intern = TRUE, wait = FALSE, ignore.stdout = TRUE, ignore.stderr = TRUE).

References

Programming with Big Data in R Website: https://pbdr.org/

See Also

pbdCS::pbdRscript().

Examples

### Save code in a file "demo.r" and run with 2 processors by
### SHELL> mpiexec -np 2 Rscript demo.r

spmd.file <- tempfile()
cat("
suppressMessages(library(pbdMPI, quietly = TRUE))
allreduce(2)
finalize()
", file = spmd.file)
pbdMPI::execmpi(spmd.file = spmd.file, nranks = 2L)