Task Pull: Functions for Task Pull Parallelism

Description

These functions are designed in SPMD but assuming rank 0 is a master and rests are workers.

Usage

task.pull(jids, FUN, ..., rank.master = .pbd_env$SPMD.CT$rank.root,
          comm = .pbd_env$SPMD.CT$comm, bcast = .pbd_env$SPMD.TP$bcast,
          barrier = .pbd_env$SPMD.TP$barrier,
          try = .pbd_env$SPMD.TP$try,
          try.silent = .pbd_env$SPMD.TP$try.silent)
task.pull.workers(FUN = function(jid, ...){ return(jid) }, ...,
                  rank.master = .pbd_env$SPMD.CT$rank.root,
                  comm = .pbd_env$SPMD.CT$comm,
                  try = .pbd_env$SPMD.TP$try,
                  try.silent = .pbd_env$SPMD.TP$try.silent)
task.pull.master(jids, rank.master = .pbd_env$SPMD.CT$rank.root,
                 comm = .pbd_env$SPMD.CT$comm)

Arguments

jids

all job ids (a vector of positive integers).

FUN

a function to be evaluated in workers.

...

extra variables for FUN.

rank.master

a rank of master where jid sent from.

comm

a communicator number.

bcast

if bcast to all ranks.

barrier

if barrier for all ranks.

try

if use try() to avoid breaks. CAUTION: try = FALSE is not safe and can stop all MPI/R jobs.

try.silent

if turn off the error message from try().

Value

A list with length comm.size() - 1 will be returned for mater, but NULL for workers. Each element of the list contains returns ret of the FUN call.

Details

All of these functions are for SPMD, NOT for master/workers.

FUN is a user defined function which has jid as the first argument and other variables are given in ....

The jids will be asked by workers when jobs are available and workers are no job in hand.

References

Programming with Big Data in R Website: http://r-pbd.org/

Examples

Run this code

# NOT RUN {
### Under command mode, run the demo with 4 processors by
### (Use Rscript.exe for windows system)
# mpiexec -np 4 Rscript -e "demo(task_pull,'pbdMPI',ask=F,echo=F)"
### Or
# execmpi("demo(task_pull,'pbdMPI',ask=F,echo=F)", nranks = 4L)
# }

Run the code above in your browser using DataLab

Last chance! 50% off unlimited learning