ohvbd (version 1.0.0)

fetch_glean_vt_chunked: Get and parse multiple VecTraits datasets by ID in chunks

Description

Retrieve and parse VecTraits datasets specified by their dataset IDs in batches.

This is not usually necessary (generally you just need fetch_vt()) but allows one to release data that is not in use from memory. If you would like more control on extraction or parsing then it is best to wrap fetch_vt() and glean_vt() in your own chunker instead.

Usage

fetch_glean_vt_chunked(
  ids,
  chunksize = 20,
  cols = NULL,
  returnunique = FALSE,
  rate = 5,
  connections = 2,
  basereq = vb_basereq()
)

Value

An ohvbd.data.frame containing the requested data.

Arguments

ids

a numeric vector of IDs (preferably in an ohvbd.ids object) indicating the particular datasets to download.

chunksize

an integer defining the size of chunks to retrieve in one iteration.

cols

a character vector of columns to extract from the dataset.

returnunique

whether to return only the unique rows within each dataset according to the filtered columns.

rate

maximum number of calls to the API per second.

connections

number of simultaneous connections to the server at once. Maximum 8. Do not enable unless you really need to as this hits the server significantly harder than usual.

basereq

an httr2 request object, as generated by vb_basereq(). If NA, uses the default request.

Author

Francis Windram

Examples

if (FALSE) { # interactive()
fetch_glean_vt_chunked(c(54,55,56), chunksize = 2, rate=5)
}