sparklyr (version 1.8.5)

registerDoSpark: Register a Parallel Backend

Description

Registers a parallel backend using the foreach package.

Usage

registerDoSpark(spark_conn, parallelism = NULL, ...)

Value

None

Arguments

spark_conn

Spark connection to use

parallelism

Level of parallelism to use for task execution (if unspecified, then it will take the value of `SparkContext.defaultParallelism()` which by default is the number of cores available to the `sparklyr` application)

...

additional options for sparklyr parallel backend (currently only the only valid option is `nocompile`)

Examples

if (FALSE) {

sc <- spark_connect(master = "local")
registerDoSpark(sc, nocompile = FALSE)
}