This function basically saves code in a spmd.file and executes
MPI via R's system call e.g.
system("mpiexec -np 1 Rscript spmd.file")
.
execmpi(spmd.code = NULL, spmd.file = NULL,
mpicmd = NULL, nranks = 1L, rscmd = NULL, verbose = TRUE,
disable.current.mpi = TRUE, mpiopt = NULL, rsopt = NULL)
runmpi(spmd.code = NULL, spmd.file = NULL,
mpicmd = NULL, nranks = 1L, rscmd = NULL, verbose = TRUE,
disable.current.mpi = TRUE, mpiopt = NULL, rsopt = NULL)
Basically, only the PID of the MPI job (in background) will be returned in Linux-alike systems. For Windows, the MPI job is always wait until it is complete.
SPMD code to be run via mpicmd and Rscript
.
a file contains SPMD code to be run via mpicmd and Rscript
.
MPI executable command. If NULL
, system default will be searched.
number of processes to run the SPMD code envoked by mpicmd.
Rscript
executable command. If NULL
, system default will be searched.
print SPMD code outputs and MPI messages.
force to finalize the current MPI comm if any, for unix-alike system only.
MPI options appended after -np nranks --oversubscribe
.
Rscript
options appended after Rscript
.
Wei-Chen Chen wccsnow@gmail.com and Drew Schmidt.
When the spmd.code
is NULL
: The code should be already
saved in the file named spmd.file
for using.
When the spmd.code
is not NULL
:
The spmd.code
will be dumped to a temp file (spmd.file
) via the
call writeLines(spmd.code, conn)
where
conn <- file(spmd.file, open = "wt")
. The file will be closed after
the dumping.
When spmd.file
is ready (either dumped from spmd.code
or
provided by the user), the steps below will be followed:
If spmd.file = NULL
, then a temporary file will be generated and
used to dump spmd.code
.
For Unix-alike systems, the command
cmd <- paste(mpicmd, "-np", nranks, mpiopt, rscmd, rscmd spmd.file, ">", log.file, " 2>&1 & echo \"PID=$!\" &")
is executed via system(cmd, intern = TRUE, wait = FALSE, ignore.stdout = TRUE, ignore.stderr = TRUE)
. The log.file
is a temporary file to
save the outputs from the spmd.code
. The results saved to the
log.file
will be read back in and cat
and return
to R.
For OPENMPI, the "--oversubscribe " is added before mpiopt
as
mpiopt <- paste("--oversubscribe ", mpiopt, sep = "")
and is passed to cmd
thereon.
For Windows, the cmd
will be
paste(mpicmd, "-np", nranks, mpiopt, rscmd, rsopt spmd.file)
and is executed via
system(cmd, intern = TRUE, wait = FALSE, ignore.stdout = TRUE, ignore.stderr = TRUE)
.
Programming with Big Data in R Website: https://pbdr.org/
pbdCS::pbdRscript()
.
### Save code in a file "demo.r" and run with 2 processors by
### SHELL> mpiexec -np 2 Rscript demo.r
spmd.file <- tempfile()
cat("
suppressMessages(library(pbdMPI, quietly = TRUE))
allreduce(2)
finalize()
", file = spmd.file)
pbdMPI::execmpi(spmd.file = spmd.file, nranks = 2L)
Run the code above in your browser using DataLab