Learn R Programming

⚠️There's a newer version (0.8.6.1) of this package.Take me there.

datadr: Divide and Recombine in R

datadr is an R package that leverages RHIPE to provide a simple interface to division and recombination (D&R) methods for large complex data.

To get started, see the package documentation and function reference located here.

Visualization tools based on D&R can be found here.

Installation

options(repos = c(tessera = "http://packages.tessera.io", getOption("repos")))
install.packages("datadr")

Alternatively, you can install directly from github:

devtools::install_github("tesseradata/datadr")

License

This software is currently under the BSD license. Please read the license document.

Acknowledgement

datadr development is sponsored by:

U.S. Department of Defense Advanced Research Projects Agency, XDATA program
U.S. Department of Homeland Security, Science and Technology Directorate, Homeland Security Advanced Research Projects Agency (HSARPA)
Pacific Northwest National Laboratory, operated by Battelle for the U.S. Department of Energy, LDRD Program, Signature Discovery and Future Power Grid Initiatives

Copy Link

Version

Install

install.packages('datadr')

Monthly Downloads

11

Version

0.8.5

License

BSD_3_clause + file LICENSE

Maintainer

Ryan Hafen

Last Published

March 16th, 2016

Functions in datadr (0.8.5)

Get names of the conditioning variable cuts

"DDF" Recombination

Add Key-Value Pairs to a Data Connection

Persist a Transformed 'ddo' or 'ddf' Object

Sample Quantiles for 'ddf' Objects

"Census Income" Dataset

Divide a Distributed Data Object

as.data.frame.ddf

Turn 'ddf' Object into Data Frame

Construct Between Subset Variable (BSV)

Digest File Hash Function

"Flatten" a ddf Subset

ddo-ddf-accessors

Accessor Functions

"rbind" Recombination

Take a Sample of Key-Value Pairs Take a sample of key-value Pairs

Get Global Variables and Package Dependencies

Division-Agnostic Aggregation

Print a key-value pair

mr-summary-stats

Functions to Compute Summary Statistics in MapReduce

Filter a 'ddo' or 'ddf' Object

Remove Key-Value Pairs from a Data Connection

Connect to Data Source on HDFS

Mean Coefficient Recombination

Join Data Sources by Key

Character File Hash Function

Conditioning Variable Division

Bag of Little Bootstraps Transformation Method

Instantiate a Distributed Data Frame ('ddf')

Apply a function to all key-value pairs of a ddo/ddf object

Extract "Split" Variable(s)

Execute a MapReduce Job

Subsetting Distributed Data Frames

Convert 'ddo' / 'ddf' Objects

Connect to Data Source on Local Disk

"DDO" Recombination

Apply transformation function(s)

Instantiate a Distributed Data Object ('ddo')

LM Transformation Method

Accessor methods for 'ddf' objects

Specify Control Parameters for RHIPE Job

readTextFileByChunk

Experimental sequential text reader helper function

Convert dplyr grouped_df to ddf

Add a Transformation Function to a Distributed Data Object

Print a "ddo" or "ddf" Object

ddo-ddf-attributes

Managing attributes of 'ddo' or 'ddf' objects

divide-internals

Functions used in divide()

readHDFStextFile

Experimental HDFS text reader helper function

Random Replicate Division

HexBin Aggregation for Distributed Data Frames

"Collect" Recombination

makeExtractable

Take a ddo/ddf HDFS data object and turn it into a mapfile

updateAttributes

Update Attributes of a 'ddo' or 'ddf' Object

Apply Function to Key-Value Pair

GLM Transformation Method

Specify a Collection of Key-Value Pairs

Mean Recombination

localDiskControl

Specify Control Parameters for MapReduce on a Local Disk Connection

setupTransformEnv

Set up transformation environment

Print value of a key-value pair

Turn 'ddo' / 'ddf' Object into a list

Specify a Key-Value Pair