Learn R Programming

batchtools (version 0.9.0)

makeClusterFunctionsDocker: ClusterFunctions for Docker

Description

Cluster functions for Docker/Docker Swarm.

The submitJob function executes docker [docker.args] run --detach=true [image.args] [resources] [image] [cmd]. Arguments docker.args, image.args and image can be set on construction. The resources part takes the named resources ncpus and memory from submitJobs and maps them to the arguments --cpu-shares and --memory (in Megabytes). To reliably identify jobs in the swarm, jobs are labeled with “batchtools=[job.hash]” and named using the current login name and the job hash.

listJobsRunning uses docker [docker.args] ps --format={{.ID}} to filter for running jobs.

killJobs uses docker [docker.args] kill [batch.id] to filter for running jobs.

These cluster functions use a Hook to remove finished jobs before a new submit and every time the Registry is synchronized (using syncRegistry). This is currently required because docker does not remove exited containers automatically. Use docker ps -a --filter 'label=batchtools' --filter 'status=exited' to identify and remove terminated containers manually (or via a cron job).

Usage

makeClusterFunctionsDocker(image, docker.args = character(0L),
  image.args = character(0L))

Arguments

image
[character(1)] Name of the docker image to run.
docker.args
[character] Additional arguments passed to “docker” *before* the command (“run”, “ps” or “kill”) to execute (e.g., the docker host).
image.args
[character] Additional arguments passed to “docker run” (e.g., to define mounts or environment variables).

Value

[ClusterFunctions].

See Also

Other ClusterFunctions: makeClusterFunctionsInteractive, makeClusterFunctionsLSF, makeClusterFunctionsMulticore, makeClusterFunctionsOpenLava, makeClusterFunctionsSGE, makeClusterFunctionsSSH, makeClusterFunctionsSlurm, makeClusterFunctionsSocket, makeClusterFunctionsTorque, makeClusterFunctions