spark_connect

0th

Percentile

Connect to Spark

Connect to Spark

Usage
spark_connect(master, spark_home = Sys.getenv("SPARK_HOME"),
  method = c("shell"), app_name = "sparklyr", version = NULL,
  hadoop_version = NULL, config = spark_config(),
  extensions = sparklyr::registered_extensions())
Arguments
master

Spark cluster url to connect to. Use "local" to connect to a local instance of Spark installed via spark_install.

spark_home

Spark home directory (defaults to SPARK_HOME environment variable). If SPARK_HOME is defined it will be always be used unless the version paramater is specified to force the use of a locally installed version.

method

of connecting to spark (currently only "shell" is supported, additional methods may come later)

app_name

Application name to be used while running in the Spark cluster

version

Version of Spark (only applicable for local master)

hadoop_version

Version of Hadoop (only applicable for local master)

config

Configuration for connection (see spark_config for details).

extensions

Extension packages to enable for this connection. By default will enable all packages that previously called sparklyr::register_extension.

Value

Connection to Spark local instance or remote cluster

See Also

Other Spark connections: spark_disconnect

Aliases
  • spark_connect
Documentation reproduced from package sparklyr, version 0.4, License: Apache License 2.0 | file LICENSE

Community examples

Looks like there are no examples yet.