sooty (version 0.1.0)

sooty_files: Obtain object storage catalogues as a dataframe of file/object identifiers.

Description

The object (file) catalogue of available sources is stored in Parquet format on Pawsey object storage. This function retrieves the curated catalogue, or the raw catalogue.

Usage

sooty_files(curated = TRUE)

Value

a data frame, see details

Arguments

curated

logical TRUE by default, set to FALSE to return raw object catalogue

Details

In the curated case, the returned data frame has columns 'date', 'source' which are the main useful fields, these describe the date of the data in the file, and its full URI (Uniform Resource Identifier) source on S3 object storage. There are also fields 'Bucket', 'Key', and 'protocol' from which 'source' is constructed.

The original publisher URI can be reconstructed by replacing the value of 'protocol' in 'source' with 'https://'.

The public object URI can be reconstructed by replacing the value of 'protocol' in 'source' with 'https://projects.pawsey.org.au'.

Examples

if (interactive()) {
  sooty_files(FALSE)
}

sooty_files()