read_df_blocks

read multiple blocks of data frames from file

Allows the user to carry out GLM on very large
data sets. Data can be created using the data_frame() function and appended
to the object with object$append(data); data_frame and data_matrix objects
are available that allow the user to store large data on disk. The data is
stored as doubles in binary format and any character columns are transformed
to factors and then stored as numeric (binary) data while a look-up table is
stored in a separate .meta_data file in the same folder. The data is stored in
blocks and GLM regression algorithm is modified and carries out a MapReduce-
like algorithm to fit the model. The functions bglm(), and summary()
and bglm_predict() are available for creating and post-processing of models.
The library requires Armadillo installed on your system. It may not
function on windows since multi-core processing is done using mclapply()
which forks R on Unix/Linux type operating systems.

Chibisi ChimaOkereke

bigReg

Generalized Linear Models (GLM) for Large Data Sets

read_df_blocks function

<dl><dt>size</dt>
<dd>number of elements in each block</dd>
<dt>filePaths</dt>
<dd>path to where the blocks are stored</dd>
<dt>df</dt>
<dd>an empty list having the same number of elements as columns in the table</dd>
<dt>ncols</dt>
<dd>number of columns in the dataframe block</dd>
<dt>factors</dt>
<dd>list containing factors</dd>
<dt>factor_indices</dt>
<dd>numeric vector containing the indicies that denote the factors</dd></dl>

Arguments

read multiple blocks of data frames from file — read_df_blocks

<dl>

<dt>size</dt>
<dd>number of elements in each block</dd>


<dt>filePaths</dt>
<dd>path to where the blocks are stored</dd>


<dt>df</dt>
<dd>an empty list having the same number of elements as columns in the table</dd>


<dt>ncols</dt>
<dd>number of columns in the dataframe block</dd>


<dt>factors</dt>
<dd>list containing factors</dd>


<dt>factor_indices</dt>
<dd>numeric vector containing the indicies that denote the factors</dd>

</dl>

read_df_blocks: read multiple blocks of data frames from file

Description

Usage

Arguments