Learn R Programming

lazyarray (version 1.1.0)

Persistent Large Data Array with Lazy-Loading on Demand

Description

Multi-threaded serialization of compressed array that fully utilizes modern solid state drives. It allows to store and load extremely large data on demand within seconds without occupying too much memories. With data stored on hard drive, a lazy-array data can be loaded, shared across multiple R sessions. For arrays with partition mode on, multiple R sessions can write to a same array simultaneously along the last dimension (partition). The internal storage format is provided by 'fstcore' package geared by 'LZ4' and 'ZSTD' compressors.

Copy Link

Version

Install

install.packages('lazyarray')

Monthly Downloads

171

Version

1.1.0

License

AGPL-3

Issues

Pull Requests

Stars

Forks

Maintainer

Zhengjia Wang

Last Published

July 18th, 2020

Functions in lazyarray (1.1.0)

set_lazy_threads

Set Number of Threads for Lazy Arrays
load_lazyarray

Load Lazy Array from Given Path
ClassLazyArray

Internal Class definition for LazyArray
auto_clear_lazyarray

Automatically remove array data
create_lazyarray

Create a lazy-array with given format and dimension
lazyarray

Create or load 'lazyarray' instance