clustermq (version 0.8.95)

Q_rows: Queue function calls defined by rows in a data.frame

Description

Queue function calls defined by rows in a data.frame

Usage

Q_rows(
  df,
  fun,
  const = list(),
  export = list(),
  pkgs = c(),
  seed = 128965,
  memory = NULL,
  template = list(),
  n_jobs = NULL,
  job_size = NULL,
  rettype = "list",
  fail_on_error = TRUE,
  workers = NULL,
  log_worker = FALSE,
  chunk_size = NA,
  timeout = Inf,
  max_calls_worker = Inf,
  verbose = TRUE
)

Arguments

df

data.frame with iterated arguments

fun

A function to call

const

A list of constant arguments passed to each function call

export

List of objects to be exported to the worker

pkgs

Character vector of packages to load on the worker

seed

A seed to set for each function call

memory

Short for template=list(memory=value)

template

A named list of values to fill in template

n_jobs

The number of LSF jobs to submit; upper limit of jobs if job_size is given as well

job_size

The number of function calls per job

rettype

Return type of function call (vector type or 'list')

fail_on_error

If an error occurs on the workers, continue or fail?

workers

Optional instance of QSys representing a worker pool

log_worker

Write a log file for each worker

chunk_size

Number of function calls to chunk together defaults to 100 chunks per worker or max. 10 kb per chunk

timeout

Maximum time in seconds to wait for worker (default: Inf)

max_calls_worker

Maxmimum number of function calls that will be sent to one worker

verbose

Print status messages and progress bar (default: TRUE)

Examples

# NOT RUN {
# Run a simple multiplication for data frame columns x and y on a worker node
fx = function (x, y) x * y
df = data.frame(x = 5, y = 10)
Q_rows(df, fx, job_size = 1)
# [1] 50

# Q_rows also matches the names of a data frame with the function arguments
fx = function (x, y) x - y
df = data.frame(y = 5, x = 10)
Q_rows(df, fx, job_size = 1)
# [1] 5
# }