Learn R Programming

Streamer (version 1.18.0)

Enabling stream processing of large files

Description

Large data files can be difficult to work with in R, where data generally resides in memory. This package encourages a style of programming where data is 'streamed' from disk into R via a `producer' and through a series of `consumers' that, typically reduce the original data to a manageable size. The package provides useful Producer and Consumer stream components for operations such as data input, sampling, indexing, and transformation; see package?Streamer for details.

Copy Link

Version

Version

1.18.0

License

Artistic-2.0

Maintainer

Martin Morgan

Last Published

February 15th, 2017

Functions in Streamer (1.18.0)

Seq

Producer class to generate (numeric) sequences
Function*

Classes for user-defined Producers and Consumers
ConnectionProducer

Producer classes to read file connections
reset

Function to reset a Stream, Producer, or Consumer
Consumer

Class defining methods for all consumers
Producer

Class defining methods for all Producers
Stream

Class to represent a Producer and zero or more Consumers
Reducer

Consumer class to combine successive records
Utility

Consumer classes with simple functionality, e.g., RawToChar, Rev
Streamer-package

Package to enable stream (iterative) processing of large data
yield

Function to yield one task from a Stream or Producer
status

Function to report current status of a stream
ParallelParam

Classes to configure parallel evaluation
DAGTeam

Consumer classes for directed acyclic graph evaluation
Downsample

Consumer class to down-sample data
RawInput

Class "RawInput"
Team

Consumer classes for parallel evaluation