botor (version 0.2.0)

s3_read: Download and read a file from S3, then clean up

Description

Download and read a file from S3, then clean up

Usage

s3_read(uri, fun, ..., extract = c("none", "gzip", "bzip2", "xz"))

Arguments

uri

string, URI of an S3 object, should start with s3://, then bucket name and object key

fun

R function to read the file, eg fromJSON, stream_in, fread or readRDS

...

optional params passed to fun

extract

optionally extract/decompress the file after downloading from S3 but before passing to fun

Value

R object

Examples

# NOT RUN {
s3_read('s3://botor/example-data/mtcars.csv', read.csv)
s3_read('s3://botor/example-data/mtcars.csv', data.table::fread)
s3_read('s3://botor/example-data/mtcars.csv2', read.csv2)
s3_read('s3://botor/example-data/mtcars.RDS', readRDS)
s3_read('s3://botor/example-data/mtcars.json', jsonlite::fromJSON)
s3_read('s3://botor/example-data/mtcars.jsonl', jsonlite::stream_in)

## read compressed data
s3_read('s3://botor/example-data/mtcars.csv.gz', read.csv, extract = 'gzip')
s3_read('s3://botor/example-data/mtcars.csv.gz', data.table::fread, extract = 'gzip')
s3_read('s3://botor/example-data/mtcars.csv.bz2', read.csv, extract = 'bzip2')
s3_read('s3://botor/example-data/mtcars.csv.xz', read.csv, extract = 'xz')
# }