distcomp v1.0-1


Monthly downloads



Computations over Distributed Data without Aggregation

Implementing algorithms and fitting models when sites (possibly remote) share computation summaries rather than actual data over HTTP with a master R process (using 'opencpu', for example). A stratified Cox model and a singular value decomposition are provided. The former makes direct use of code from the R 'survival' package. (That is, the underlying Cox model code is derived from that in the R 'survival' package.) Sites may provide data via several means: CSV files, Redcap API, etc. An extensible design allows for new methods to be added in the future. Web applications are provided (via 'shiny') for the implemented methods to help in designing and deploying the computations.



Install this package the usual way in R or via:


Then, you will find a document that describes several examples installed under the R library tree. For example:

list.files(system.file("doc", package = "distcomp"))
list.files(system.file("doc_src", package = "distcomp"))

The examples described in the Journal of Statistical Software paper are available as follows:

list.files(system.file("ex", package = "distcomp"))

Use of this package requires some configuration. In particular, to run the examples on a local machine where a single opencpu server will be emulating several sites, a suitable R profile needs to be set up. That profile will be something along the lines of

{r, eval=FALSE} library(distcomp) distcompSetup(workspace = "full_path_to_workspace_directory", ssl_verifyhost = 0L, ssl_verifypeer = 0L) where the workspace is a directory that the opencpu server can serialize objects to. On Unix or Mac, the above can be inserted into an .Rprofile file, but on Windows, we find that the Rprofile.site file needs to contain the above lines.

The effect of this is that every R process (including the opencpu process) has access to the distcomp library and the workspace.

OSX Yosemite Issues (10.10.4 and below)

In some earlier versions of Yosemite (MacOS 10.10.2 for example) there were issues with the opencpu package. We suspected it had more to do with Yosemite than opencpu or httpuv---see the number of DNS resolver issues people are having with the move to discoveryd via a google search.

Furthermore, even if opencpu came up, which it did sometimes after many tries, you were better off using the ip address in the url in place of localhost due to those DNS problems.

We no longer see this problem in recent versions of Yosemite (10.10.5+).

Functions in distcomp

Name Description
createInstanceObject Given the definition identifier of an object, instantiate and store object in workspace
dccoxph Functions copied and modified from survival package
SVDMaster Create a master object to control worker objects generated by
SVDWorker Create a worker object for use as a worker with master objects generated by
availableComputations Return the currently available (implemented) computations
availableDataSources Return currently implemented data sources
CoxMaster Create a master object to control worker objects generated by
CoxWorker Create a worker object for use as a worker with master objects generated by
defineNewComputation Define a new computation
destroyInstanceObject Destroy an instance object given its identifier
getConfig Return the workspace and configuration setup values
distcompSetup Setup a workspace and configuration for a distributed computation
executeMethod Given the id of a serialized object, invoke a method on the object with arguments
setupMaster Setup a computation master
.makeOpencpuURL Make an appropriate opencpu URL for a specified function and url prefix for the
distcomp Distributed Computing with R
saveNewComputation Save a computation instance, given the computation definition, associated data and
setComputationInfo Set a name to a value in a global variable
generateId Generate an identifier for an object
getComputationInfo Get the value of a variable from the global store
makeMaster Make a master object given a definition
makeWorker Make a worker object given a definition and data
uploadNewComputation Upload a new computation and data to an opencpu server
writeCode Write the code necessary to run a master process
setupWorker Setup a worker site
makeDefinition Make a computation definition given the computation type
resetComputationInfo Clear the contents of the global store
runDistcompApp Run a specified distcomp web application
No Results!

Last month downloads


URL http://dx.doi.org/10.18637/jss.v077.i13
Copyright inst/COPYRIGHTS
License LGPL (>= 2)
RoxygenNote 6.0.1
NeedsCompilation yes
Packaged 2017-05-16 18:43:31 UTC; naras
Repository CRAN
Date/Publication 2017-05-16 19:08:02 UTC

Include our badge in your README