Connect to Spark
spark_connect(master, spark_home = Sys.getenv("SPARK_HOME"),
method = c("shell"), app_name = "sparklyr", version = NULL,
hadoop_version = NULL, config = spark_config(),
extensions = sparklyr::registered_extensions())
Spark cluster url to connect to. Use "local"
to connect to a local
instance of Spark installed via spark_install
.
Spark home directory (defaults to SPARK_HOME environment variable).
If SPARK_HOME
is defined it will be always be used unless the version
paramater is specified to force the use of a locally installed version.
of connecting to spark (currently only "shell" is supported, additional methods may come later)
Application name to be used while running in the Spark cluster
Version of Spark (only applicable for local master)
Version of Hadoop (only applicable for local master)
Configuration for connection (see spark_config for details
).
Extension packages to enable for this connection. By default will
enable all packages that previously called sparklyr::register_extension
.
Connection to Spark local instance or remote cluster
Other Spark connections: spark_disconnect