pbdDMAT v0.5-1

0

Monthly downloads

0th

Percentile

'pbdR' Distributed Matrix Methods

A set of classes for managing distributed matrices, and a collection of methods for computing linear algebra and statistics. Computation is handled mostly by routines from the 'pbdBASE' package, which itself relies on the 'ScaLAPACK' and 'PBLAS' numerical libraries for distributed computing.

Readme

pbdDMAT

pbdDMAT is an R package for distributed matrix algebra and statistics computations over MPI.

With few exceptions (ff, bigalgebra, etc.), R does computations in memory. If the memory of a matrix is too large for a single node, then distributing the ownership of the matrix across multiple nodes is an effective strategy in working with such large data.

The pbdDMAT package contains numerous routines to help with the distribution and management of data, as well as functions for summarizing, inspecting, and analyzing distributed matrices.

Often the syntax is identical to serial R, only instead of calling cov(x) on a matrix x, you would call it on a distributed matrix x. This is possible by extensive use of R's S3 and S4 methods.

Much of the numerical linear algebra is powered by the ScaLAPACK library, which is the distributed analogue of LAPACK, used extensively by R.

Installation

pbdDMAT requires

  • A system installation of MPI
  • R version 3.0.0 or higher
  • The pbdMPI and pbdBASE packages, as well as their dependencies.

Assuming you meet the system dependencies, you can install the stable version from CRAN using the usual install.packages():

install.package("pbdDMAT")

The development version is maintained on GitHub:

remotes::install_github("RBigData/pbdDMAT")

See the vignette for installation troubleshooting.

Usage

# load the package
library(pbdDMAT)

# initialize the specialized MPI communicators
init.grid()

# create a 100x100 distributed matrix object
dx <- ddmatrix(1:100, 10)

# print
dx
print(dx, all=TRUE)


# shut down the communicators and exit
finalize()

Save this program as pbd_example.r and run it via:

mpirun -np 2 Rscript pbd_example.r

Numerous other examples can be found in both the pbdDMAT vignette, as well as the pbdDEMO package and its corresponding vignette.

Authors

pbdDMAT is authored and maintained by the pbdR core team:

  • Drew Schmidt
  • Wei-Chen Chen
  • George Ostrouchov
  • Pragneshkumar Patel

With additional contributions from:

  • Lamy de la Chapelle Sebastien
  • The R Core team (some wrapper code taken from the base and stats packages)
  • ZhaoKang Wang (fixes/improvements to apply())
  • Michael Lawrence (fix for as.vector())

Functions in pbdDMAT

Name Description
binds Row and Column binds for Distributed Matrices
ddmatrix-lu LU Factorization
ddmatrix-chol Cholesky Factorization
ddmatrix-prcomp Principal Components Analysis
ddmatrix-eigen eigen
isdot Type Checks, Including NA, NaN, etc.
getLocal getLocal
headsortails Head and Tail of a Distributed Matrix
qr QR Decomposition Methods
redistribute Distribute/Redistribute matrices across the process grid
sd Covariance and Correlation
sparsity Sparsity of Matrix Objects
as.ddmatrix Non-Distributed object to Distributed Object Converters
extract Extract or Replace Parts of a Distributed Matrix
as.matrix Distributed object to Matrix Converters
ddmatrix-print Printing a Distributed Matrix
ddmatrix-summary Distributed Matrix Summary
ddmatrix-apply Apply Family of Functions
Accessors Accessor Functions for Distributed Matrix Slots
expm Matrix Exponentiation
sweep Sweep
ddmatrix-scale Scale
companion Generate Companion Matrices
ddmatrix-class Class ddmatrix
lm.fit Fitter for Linear Models
pbdDMAT Control Some default parameters for pbdDMAT.
transpose Distributed Matrix Transpose
arithmetic Arithmetic Operators
diag-constructors Distributed Matrix Diagonals
covariance Covariance and Correlation
eigen2 eigen2
as.rowcyclic Distribute/Redistribute matrices across the process grid
as.vector Distributed object to Vector Converters
condnums Compute or estimate the Condition Number of a Distributed Matrix
ddmatrix-solve Solve
ddmatrix-constructors Distributed Matrix Creation
insert Directly Insert Into Distributed Matrix Submatrix Slot
math Miscellaneous Mathematical Functions
matmult Matrix Multiplication
reductions Arithmetic Reductions: Sums, Means, and Prods
ddmatrix-sumstats Basic Summary Statistics
pbdDMAT-package Distributed Matrix Methods
na Handle Missing Values in Distributed Matrices
rounding Rounding of Numbers
ddmatrix-svd Singular Value Decomposition
isSymmetric isSymmetric
Comparators Logical Comparisons
chol2inv Inverse from Choleski (or QR) Decomposition
Hilbert Generate Hilbert Matrices
ddmatrix-norm Norm
No Results!

Vignettes of pbdDMAT

Name
include/00-acknowledgement.tex
include/00-copyright.tex
include/00-zzz-abstract.tex
include/01-introduction.tex
include/02-classesnmeths.tex
include/03-users.tex
include/04-examples.tex
include/05-advanced.tex
include/06-adveg.tex
include/07-developers.tex
include/pbdDMAT.bib
include/preamble.tex
include/titlepage.tex
build_pdf.sh
pbdDMAT-guide.Rnw
No Results!

Last month downloads

Details

License GPL (>= 2)
SystemRequirements OpenMPI (>= 1.5.4) on Solaris, Linux, Mac, and FreeBSD. MS-MPI (Microsoft HPC Pack 2012) or MPICH2 (>= 1.4.1p1) on Windows.
LazyLoad yes
LazyData yes
ByteCompile yes
URL https://pbdr.org/
BugReports http://group.pbdr.org/
MailingList Please send questions and comments regarding pbdR to RBigData@gmail.com
RoxygenNote 6.1.1
NeedsCompilation yes
Packaged 2019-03-14 23:31:20 UTC; mschmid3
Repository CRAN
Date/Publication 2019-03-17 19:00:03 UTC

Include our badge in your README

[![Rdoc](http://www.rdocumentation.org/badges/version/pbdDMAT)](http://www.rdocumentation.org/packages/pbdDMAT)