spark_connect
From sparklyr v0.3.0
by Javier Luraschi
Connect to Spark
Connect to Spark
Usage
spark_connect(master, spark_home = Sys.getenv("SPARK_HOME"), method = c("shell"), app_name = "sparklyr", version = NULL, hadoop_version = NULL, config = spark_config(), extensions = sparklyr::registered_extensions())Arguments
- master
- Spark cluster url to connect to. Use
"local"to connect to a local instance of Spark installed viaspark_install. - spark_home
- Spark home directory (defaults to SPARK_HOME environment variable).
If
SPARK_HOMEis defined it will be always be used unless theversionparamater is specified to force the use of a locally installed version. - method
- of connecting to spark (currently only "shell" is supported, additional methods may come later)
- app_name
- Application name to be used while running in the Spark cluster
- version
- Version of Spark (only applicable for local master)
- hadoop_version
- Version of Hadoop (only applicable for local master)
- config
- Configuration for connection (see
spark_config for details). - extensions
- Extension packages to enable for this connection. By default will
enable all packages that previously called
sparklyr::register_extension.
Value
-
Connection to Spark local instance or remote cluster
See Also
Other Spark connections: spark_disconnect
Community examples
Looks like there are no examples yet.