S3 Storage Info
s3_storage_info(
destination,
region = NULL,
endpoint = NULL,
enable_encryption = FALSE,
encryption_type = c("sse-s3", "sse-kms"),
kms_key = NULL,
canned_acl = NULL
)S3 destination. For example: s3://my-bucket/some-prefix.
You must configure the cluster with an instance profile and the instance
profile must have write access to the destination. You cannot use AWS
keys.
S3 region. For example: us-west-2. Either region or endpoint
must be set. If both are set, endpoint is used.
S3 endpoint. For example:
https://s3-us-west-2.amazonaws.com. Either region or endpoint must be set.
If both are set, endpoint is used.
Boolean (Default: FALSE). If TRUE Enable server
side encryption.
Encryption type, it could be sse-s3 or sse-kms. It
is used only when encryption is enabled and the default type is sse-s3.
KMS key used if encryption is enabled and encryption type is
set to sse-kms.
Set canned access control list. For example:
bucket-owner-full-control. If canned_acl is set, the cluster instance
profile must have s3:PutObjectAcl permission on the destination bucket and
prefix. The full list of possible canned ACLs can be found in
docs.
By default only the object owner gets full control. If you are using cross
account role for writing data, you may want to set
bucket-owner-full-control to make bucket owner able to read the logs.
cluster_log_conf(), init_script_info()
Other Cluster Log Configuration Objects:
cluster_log_conf(),
dbfs_storage_info()
Other Init Script Info Objects:
dbfs_storage_info(),
file_storage_info()