startJuliaServer: Start a Julia server that may serve multiple clients (R processes)

Description

Starting a Julia server allows that different R processes may connect to the the same Julia server and share a single session. This can be useful for saving start-up/precompilation time when starting additional processes or when sharing global variables between processes. A usage scenario for this is a web server where reloading Julia for each new session should be avoided. For the standard way of starting Julia, this function is not needed. It is also not needed if child processes should use separate Julia sessions.

Usage

startJuliaServer(port = 11980)

Value

the port number (invisibly)

Arguments

port: a hint for the port that is used by the server. If it is not available, a different port is used. The final port is returned (invisibly).

Details

The functions communicates the server address via setting the JULIACONNECTOR_SERVER environment variable. A possible value for the variable is "localhost:11980". The JULIACONNECTOR_SERVER variable is communicated automatically via the system environment to child processes that are started after this function has been called. The child processes will then connect to the same Julia server if the variable is set. The variable can also be set explicitly in child processes before connecting to Julia to connect to a running server. Unsetting the variable will result in a normal Julia start-up in the first call to Julia, using a single-client Julia session.

For security reasons, the Julia server accepts only connections from localhost.

For using Julia with multiple clients, it can be good to advise Julia to use multiple threads via setting the JULIA_NUM_THREADS environment variable before starting Julia.

Examples

Run this code

if (juliaSetupOk() && Sys.getenv("NOT_CRAN") == "true") {
   library(JuliaConnectoR)
   library(future)

   # 1) Start a shared Julia TCP server in the main R session, which may use all CPU threads
   Sys.setenv("JULIA_NUM_THREADS" = parallel::detectCores())
   stopJulia() # stop current Julia connection before starting server
   startJuliaServer()

   # 2) Create R workers
   cl <- parallelly::makeClusterPSOCK(workers = 2)

   # 3) Warm-up R workers, load Julia and execute first Julia function
   # (In practice, you could, e.g., trigger Julia pre-compilation here.)
   parallel::clusterEvalQ(cl, {
      library(JuliaConnectoR)
      JuliaConnectoR::juliaEval("1")  # open Julia TCP connection on each worker
   })

   ## 4) Re-Use those warmed workers for use via futures
   plan(cluster, workers = cl)

   ## 5) Create a global variable for demonstration purposes (and stay thread safe)
   juliaEval("global const counter = Threads.Atomic{Int}(0)")

   incrementCounter <- function() {
      JuliaConnectoR::juliaEval("Threads.atomic_add!(counter, 1)")
   }

   ## 6) Execute futures via worker-pool
   f1 <- future({incrementCounter()})
   f2 <- future({incrementCounter()})
   value(f1)
   value(f2)

   # Reading the value should demonstrate that the
   # global variable is shared across workers.
   juliaEval("counter[]")

   ## 7) Cleanup
   plan(sequential)
   parallel::stopCluster(cl)
   stopJulia()

}
# \dontshow{
JuliaConnectoR:::stopJulia()
# }