secr (version 2.9.0)

Parallel: Multi-core Processing

Description

From version 2.4.0 secr makes limited use of multiple cores (CPUs) through the package parallel. Only the few secr functions listed below make any use of parallel processing. Increased speed can be expected with sim.secr (e.g., x3 with 4 cores), but gains in secr.fit are much smaller and may be negative. The function par.secr.fit added in 2.7.1 is useful if you have a set of different models you wish to fit: if you have as many cores as models then all will complete in about the time required to run the longest. llll{ Function Unit Benefit Notes secr.fit session likelihood small-moderate multi-session models only score.test model moderate multi-model comparisons only derived session moderate SE by parameter if one session mask.check spacing x buffer moderate-large no file output, suppresses messages sim.secr replicate large all models, suppresses messages ip.secr replicate large LLsurface.secr parameter combination large par.secr.fit model large par.derived fitted model large par.region.N fitted model large } `Unit' refers to the unit of work sent to each worker process. As a guide, a `large' benefit means >60% reduction in process time with 4 CPUs. parallel offers several different mechanisms, bringing together the functionality of multicore and snow. The mechanism used by secr is the simplest available, and is expected to work across all operating systems. Technically, it relies on Rscript and communication between the master and worker processes is via sockets. As stated in the parallel documentation "Users of Windows and Mac OS X may expect pop-up dialog boxes from the firewall asking if an R process should accept incoming connections". To use multiple cores, install parallel from CRAN and set ncores > 1 in the function call. Use detectCores() to get an idea of how many cores are available on your machine; this may (in Windows) include virtual cores over and above the number of physical cores. See RShowDoc("parallel", package = "parallel") in core R for explanation. You may possibly get warnings from R about closing unused connections. These can safely be ignored. In sim.secr, new datasets are generated in the master process, so there is no need to manage the random number streams in the worker processes. In secr.fit the output component `proctime' misrepresents the elapsed processing time when multiple cores are used. Worker processes are created in secr.fit with makeCluster and the options methods = FALSE, useXDR = FALSE. This has been tested only on Windows systems. The code used internally by secr is quite simple and could be adapted as a wrapper for user-defined simulations. See Examples.

Examples

# R version 2.15.2 (2012-10-26)
# Platform: x86_64-w64-mingw32/x64 (64-bit)
# quad-core i7 CPU

library(parallel)
detectCores()
# [1] 8

# ovenCH is a 5-session dataset
# multiple cores help a little here

system.time(f5 <- secr.fit(ovenCH, buffer = 400, trace = FALSE, ncores = 1))
#   user  system elapsed 
#  61.21    0.95   62.28 
system.time(f5 <- secr.fit(ovenCH, buffer = 400, trace = FALSE, ncores = 5))
#   user  system elapsed 
#   8.51    8.66   35.81 

# however, there is substantial benefit when simulating

system.time(s1 <- sim.secr(secrdemo.0, nsim = 20))
#   user  system elapsed 
# 789.90    4.41  795.07 
system.time(s4 <- sim.secr(secrdemo.0, nsim = 20, ncores = 4))
#   user  system elapsed 
#  26.91    0.34  276.15 

system.time(ip.secr (captdata, ncores = 1))
#   user  system elapsed 
# 149.93    0.01  150.72 
system.time(ip.secr (captdata, ncores = 6))
#  user  system elapsed 
#   0.47    0.19   41.06 

system.time(score.test (secrdemo.0, g0 ~ b, g0~t, g0 ~ bk, ncores = 1))
#   user  system elapsed 
# 130.73    0.45  131.34 
system.time(score.test (secrdemo.0, g0 ~ b, g0~t, g0 ~ bk, ncores = 4))
#   user  system elapsed 
#   0.04    0.01  109.69

system.time(derived(ovenbird.model.D, ncores=1))
#   user  system elapsed 
#   7.99    0.02    8.00 
system.time(derived(ovenbird.model.D, ncores=5))
#   user  system elapsed 
#   0.05    0.04    4.06 

system.time( LLsurface.secr(secrdemo.0, ncores=1))
#   user  system elapsed 
#  40.97    0.64   41.66 
system.time( LLsurface.secr(secrdemo.0, ncores=4))
#   user  system elapsed 
#   0.05    0.06   13.82 
system.time( LLsurface.secr(secrdemo.0, ncores=8))
#   user  system elapsed 
#   0.03    0.11   11.14 

## the code used in LLsurface.secr() looks like this

if (ncores > 1) {
    require(parallel)
    clust <- makeCluster(ncores)
    # load 'secr' in each worker process
    clusterEvalQ(clust, library(secr))
    # send data to each worker process from master process
    clusterExport(clust, c("object", "details"), environment())
    # run function LL for each row of matrix 'grid'
    # LL accepts one argument, a vector of parameter values
    # (can also use parLapply, parSapply etc.)
    temp <- parRapply (clust, grid, LL)
    stopCluster(clust)
}