ngstk (version 0.2.3)

batch_file: Process the input file a batch of one batch

Description

Process the input file a batch of one batch

Usage

batch_file(filename = "", batch_lines = 1e+07, handler = NULL,
  param_names = c("x", "i"), extra_fread_params = list(sep = "\n",
  header = FALSE, return_1L = TRUE), extra_params = list(),
  start_index = 1)

Arguments

filename

Filename need to process

batch_lines

Batch lines to process the data, default 10000000

handler

The function to process the data

param_names

Hander function required parameter names

extra_fread_params

Extra fread parameters in read data step, default is list(sep = '\n', header = TRUE, return_1L = TRUE), return_1L to get x[[1L]]

extra_params

Extra paramemters pass to handler function

start_index

default is 1, control the skip rows, n = (i-1) * batch_lines

Examples

# NOT RUN {
dat <- data.frame(a=1:100, b=1:100)
filename <- tempfile()
write.table(dat, filename, sep = '\t', row.names = FALSE, quote = FALSE)
handler_fun <- function(x, i = 1) {
  return(x[i])
}
batch_file(filename, 10, handler_fun)
# }