Convenience function to initialize a Kubernetes configuration instead
of spark_config()
, exposes common properties to set in Kubernetes
clusters.
spark_config_kubernetes(master, image = "spark:sparklyr",
driver = random_string("sparklyr-"), account = "spark",
jar = "local:///opt/sparklyr/sparklyr-2.3-2.11.jar", forward = TRUE,
...)
Kubernetes url to connect to, found by running kubectl cluster-info
.
Container image to use to launch Spark and sparklyr. Also known
as spark.kubernetes.container.image
.
Name of the driver pod. If not set, the driver pod name is set
to "sparklyr" suffixed by id to avoid name conflicts. Also known as
spark.kubernetes.driver.pod.name
.
Service account that is used when running the driver pod. The driver
pod uses this service account when requesting executor pods from the API
server. Also known as spark.kubernetes.authenticate.driver.serviceAccountName
.
Path to the sparklyr jars; either, a local path inside the container
image with the sparklyr jars copied when the image was created or, a path
accesible by the container where the sparklyr jars were copied. You can find
a path to the sparklyr jars by running system.file("java/", package = "sparklyr")
.
Should ports used in sparklyr be forwarded automatically through Kubernetes?
Default to TRUE
which runs kubectl port-forward
.
Additional parameters, currently not in use.