Spark Python Task
spark_python_task(python_file, parameters = list())
The URI of the Python file to be executed. DBFS and S3 paths are supported.
List. Command line parameters passed to the Python file. Use Task parameter variables to set parameters containing information about job runs.
Other Task Objects:
email_notifications()
,
libraries()
,
new_cluster()
,
notebook_task()
,
pipeline_task()
,
python_wheel_task()
,
spark_jar_task()
,
spark_submit_task()