pbdMPI (version 0.5-3)

Task Pull: Functions for Task Pull Parallelism

Description

These functions are designed for SPMD but assume that rank 0 is a manager and the rest are workers.

Usage

task.pull(jids, FUN, ..., rank.manager = .pbd_env$SPMD.CT$rank.root,
          comm = .pbd_env$SPMD.CT$comm, bcast = .pbd_env$SPMD.TP$bcast,
          barrier = .pbd_env$SPMD.TP$barrier,
          try = .pbd_env$SPMD.TP$try,
          try.silent = .pbd_env$SPMD.TP$try.silent)
task.pull.workers(FUN = function(jid, ...){ return(jid) }, ...,
                  rank.manager = .pbd_env$SPMD.CT$rank.root,
                  comm = .pbd_env$SPMD.CT$comm,
                  try = .pbd_env$SPMD.TP$try,
                  try.silent = .pbd_env$SPMD.TP$try.silent)
task.pull.manager(jids, rank.manager = .pbd_env$SPMD.CT$rank.root,
                 comm = .pbd_env$SPMD.CT$comm)

Value

A list with length comm.size() - 1

will be returned to the manager and NULL to the workers. Each element of the list contains the returns ret of their FUN

results.

Arguments

jids

all job ids (a vector of positive integers).

FUN

a function to be evaluated by workers.

...

extra parameters for FUN.

rank.manager

rank of the manager from where jid is sent.

comm

a communicator number.

bcast

if bcast to all ranks.

barrier

if barrier for all ranks.

try

wheter to use try() to avoid crashes. CAUTION: try = FALSE is not safe and can crash all MPI/R jobs.

try.silent

turn off error messages from try().

Author

Wei-Chen Chen , George Ostrouchov, Drew Schmidt, Pragneshkumar Patel, and Hao Yu.

Details

All of these functions are designed to emulate a manager/workers paradigm in an SPMD environment. If your chunk workloads are known and similar, consider a direct SPMD solution.

FUN is a user defined function which has jid as its first argument and other variables are given in ....

The manager will be queried by workers whenever a worker finishes a job to see if more jobs are available.

References

Programming with Big Data in R Website: https://pbdr.org/

See Also

get.jid().

Examples

if (FALSE) {
### Under command mode, run the demo with 2 processors by
### (Use Rscript.exe for windows system)
# mpiexec -np 2 Rscript -e "demo(task_pull,'pbdMPI',ask=F,echo=F)"
### Or
# execmpi("demo(task_pull,'pbdMPI',ask=F,echo=F)", nranks = 2L)
}