Learn R Programming

⚠️There's a newer version (0.8.6.1) of this package.Take me there.

datadr: Divide and Recombine in R

datadr is an R package that leverages RHIPE to provide a simple interface to division and recombination (D&R) methods for large complex data.

To get started, see the package documentation and function reference located here.

Visualization tools based on D&R can be found here.

Installation

# from CRAN:
install.packages("datadr")

# from github:
devtools::install_github("delta-rho/datadr")

License

This software is currently under the BSD license. Please read the license document.

Acknowledgement

datadr development is sponsored by:

U.S. Department of Defense Advanced Research Projects Agency, XDATA program
U.S. Department of Homeland Security, Science and Technology Directorate, Homeland Security Advanced Research Projects Agency (HSARPA)
Pacific Northwest National Laboratory, operated by Battelle for the U.S. Department of Energy, LDRD Program, Signature Discovery and Future Power Grid Initiatives

Copy Link

Version

Install

install.packages('datadr')

Monthly Downloads

30

Version

0.8.6

License

BSD_3_clause + file LICENSE

Maintainer

Ryan Hafen

Last Published

October 2nd, 2016

Functions in datadr (0.8.6)

"Census Income" Dataset

Bag of Little Bootstraps Transformation Method

Filter a 'ddo' or 'ddf' Object

LM Transformation Method

Apply a function to all key-value pairs of a ddo/ddf object

readHDFStextFile

Experimental HDFS text reader helper function

readTextFileByChunk

Experimental sequential text reader helper function

Apply transformation function(s)

"rbind" Recombination

Mean Coefficient Recombination

Add Key-Value Pairs to a Data Connection

Add a Transformation Function to a Distributed Data Object

Instantiate a Distributed Data Frame ('ddf')

ddo-ddf-accessors

Accessor Functions

HexBin Aggregation for Distributed Data Frames

Get names of the conditioning variable cuts

Join Data Sources by Key

Connect to Data Source on HDFS

"DDF" Recombination

"Collect" Recombination

Mean Recombination

"DDO" Recombination

Accessor methods for 'ddf' objects

Sample Quantiles for 'ddf' Objects

Persist a Transformed 'ddo' or 'ddf' Object

Print a key-value pair

Digest File Hash Function

divide-internals

Functions used in divide()

Take a Sample of Key-Value Pairs Take a sample of key-value Pairs

Connect to Data Source on Local Disk

Specify a Collection of Key-Value Pairs

Remove Key-Value Pairs from a Data Connection

Print value of a key-value pair

Convert dplyr grouped_df to ddf

updateAttributes

Update Attributes of a 'ddo' or 'ddf' Object

Divide a Distributed Data Object

Division-Agnostic Aggregation

Turn 'ddo' / 'ddf' Object into a list

as.data.frame.ddf

Turn 'ddf' Object into Data Frame

Conditioning Variable Division

Convert 'ddo' / 'ddf' Objects

Get Global Variables and Package Dependencies

GLM Transformation Method

Apply Function to Key-Value Pair

Specify a Key-Value Pair

Specify Control Parameters for RHIPE Job

Random Replicate Division

Construct Between Subset Variable (BSV)

Character File Hash Function

Print a "ddo" or "ddf" Object

setupTransformEnv

Set up transformation environment

Extract "Split" Variable(s)

Subsetting Distributed Data Frames

"Flatten" a ddf Subset

localDiskControl

Specify Control Parameters for MapReduce on a Local Disk Connection

makeExtractable

Take a ddo/ddf HDFS data object and turn it into a mapfile

ddo-ddf-attributes

Managing attributes of 'ddo' or 'ddf' objects

Instantiate a Distributed Data Object ('ddo')

Execute a MapReduce Job

mr-summary-stats

Functions to Compute Summary Statistics in MapReduce