disk.frame (version 0.1.0)

recommend_nchunks: Recommend number of chunks based on input size

Description

Computes the recommended number of chunks to break a data.frame into. It can accept filesizes in bytes (as integer) or a data.frame

Usage

recommend_nchunks(df, type = "csv",
  minchunks = parallel::detectCores(logical = FALSE), conservatism = 2)

Arguments

df

a disk.frame or the file size in bytes of a CSV file holding the data

type

only = "csv" is supported. It indicates the file type corresponding to file size `df`

minchunks

the minimum number of chunks. Defaults to the number of CPU cores (without hyper-threading)

conservatism

a multiplier to the recommended number of chunks. The more chunks the smaller the chunk size and more likely that each chunk can fit into RAM

Examples

# NOT RUN {
# recommend nchunks based on data.frame
recommend_nchunks(cars)

# recommend nchunks based on file size ONLY CSV is implemented at the moment
recommend_nchunks(1024^3)
# }