multidplyr (version 0.0.0.9000)

cluster_map: Call a function on each node of a cluster

Description

`cluster_map()` passes a function and arguments; `cluster_call()` quotes the input and then re-calls. Jobs are submitted to workers in parallel, and then we wait until they're complete.

Usage

cluster_map(.x, .f, ...)
cluster_call(x, code)

Arguments

.f

Function to call. Must be a function, string, or formula. * If a **function**, it will be copied to each worke. Must be self-contained because its environment will be set to the global environment prior to being distributed to the workers.

* If a **string**, the function will not be copied, and a function with that name will be called on each worker. Can use `::`.

* If a **formula**, e.g. `~ .x + 2`, it is converted to a function.

...

Arguments to .f. Eagerly evaluated before distribution.

x, .x

A cluster

code

An expression to execute on each worker.

Value

A list, with one element for each worker.

Examples

# NOT RUN {
cl <- new_cluster(2)

# This function will be copied to each node
f <- function() 1 + 1
cl %>% cluster_map(f)

# The function will be called on each node
cl %>% cluster_map("Sys.getpid")

# cluster_call() provides a slightly simpler sytanx where you just
# provide an expression to be executed on each worke
cl %>% cluster_call(1 + 1)
cl %>% cluster_call(Sys.getpid())

invisible(cl %>% cluster_call(x <- runif(1)))
cl %>% cluster_call(x)
# }