spark_connect
From sparklyr v0.2.26
by Javier Luraschi
Connect to Spark
Connect to Spark
Check if a Spark connection is open
Usage
spark_connect(master, spark_home = Sys.getenv("SPARK_HOME"), app_name = "sparklyr", version = NULL, hadoop_version = NULL, config = spark_config(), extensions = sparkapi::registered_extensions())
spark_connection_is_open(sc)Arguments
- master
- Spark cluster url to connect to. Use
"local"to connect to a local instance of Spark installed viaspark_install. - spark_home
- Spark home directory (defaults to SPARK_HOME environment variable).
If
SPARK_HOMEis defined it will be always be used unless theversionparamater is specified to force the use of a locally installed version. - app_name
- Application name to be used while running in the Spark cluster
- version
- Version of the Spark (only applicable for local master)
- hadoop_version
- Version of Hadoop (only applicable for local master)
- config
- Configuration for connection (see
spark_config for details). - extensions
- Extension packages to enable for this connection. By default will
enable all packages that previously called
sparkapi::register_extension. - sc
spark_connection
Value
-
Connection to Spark local instance or remote cluster
See Also
Other Spark connections: spark_disconnect
Community examples
Looks like there are no examples yet.