doRedis (version 2.0.0)

setReduce: Set two-level distributed reduction

Description

Instruct doRedis to perform either the .combine reduction function or another specified function per task on each worker before returning results, cf. foreach. Combined results are then processed through the specified function fun for two levels of reduction functions. This option only applies when the chunkSize option is greater than one, and automatically sets .multicombine=FALSE.

Usage

setReduce(fun = NULL)

Arguments

fun

a function of two arguments, set to NULL to disable combining, or leave missing to implicitly set the gather function formally identical to the .combine function but with an empty environment.

Value

fun is invisibly returned, or TRUE is returned when fun is missing (in which case the .combine function is used).

Details

This approach can improve performance when the .combine function is expensive to compute, and when function emits significantly less data than it consumes. The same effect is usually achievable by simply adding the reduction function to the end of the foreach loop expression (but must be decided prior to run time).

See Also

foreach, setChunkSize

Examples

# NOT RUN {
setChunkSize(3)
foreach(j=1:10, .combine=c, .options.redis=list(reduce=list)) %dopar% j
# }
# NOT RUN {
# }