Learn R Programming

searchAnalyzeR (version 0.1.0)

chunk_process: Process Large Dataset in Chunks

Description

Generic function to process a large dataset in manageable chunks to reduce memory usage.

Usage

chunk_process(data, chunk_size = 10000, fn, combine_fn = rbind, ...)

Value

Combined results after processing all chunks

Arguments

data

Large data frame to process

chunk_size

Number of rows per chunk

fn

Function to apply to each chunk

combine_fn

Function to combine results from chunks

...

Additional arguments passed to fn