await: Call a function repeatedly until a status is reached.

Description

await repeatedly calls a Civis API endpoint such as scripts_get_sql_runs that monitors the status of a script, job, import, or model. It blocks until the function returns a result with a successful or error status. If the script, job, import or model results in an error state, await throws an error with useful debugging information.

await_all is a vectorized version of await. It repeatedly calls a Civis API endpoint for all values of a vector, e.g. a vector of script, job, import, run, or model ids. It blocks until all calls have returned a result with a given status, and silently captures jobs that return errors.

Usage

await(
  f,
  ...,
  .status_key = "state",
  .success_states = c("succeeded", "success"),
  .error_states = c("failed", "cancelled"),
  .timeout = NULL,
  .interval = getOption("civis.default_polling_interval"),
  .verbose = FALSE
)
await_all(
  f,
  .x,
  .y = NULL,
  ...,
  .status_key = "state",
  .success_states = c("succeeded", "success"),
  .error_states = c("failed", "cancelled"),
  .timeout = NULL,
  .interval = NULL,
  .verbose = FALSE
)

Arguments

f: function to be called repeatedly until a status is reached.
...: arguments to f
.status_key: The name of the element of the list returned by f containing the status. For most Civis API endpoints, this is the default, "state".
.success_states: list of states indicating remote work has completed successfully. For most Civis API endpoints, this set of states is the default, c("succeeded", "success").
.error_states: list of states indicating remote work is in an error state. For most Civis API endpoints, this set of states is the default, c("failed", "cancelled").
.timeout: Number of seconds after which to timeout.
.interval: The interval for retries (in seconds). If NULL (default), use exponentially increasing intervals with jitter (see 'Details')
.verbose: Print the status of f at a given retry with the retry time (default FALSE)
.x: a vector of values to be passed to f
.y: a vector of values to be passed to f (default NULL)

Functions

await_all(): Call a function repeatedly for all values of a vector until all have reached a completed status

Details

await and await_all can wrap Civis API endpoints in generated_client.R. The default values for .status_key, .success_states, and .error_states are suitable for most endpoints. The final status of f can be obtained using get_status.

If an error state is reached, await throws a civis_await_error. await_all silently captures and returns a civis_await_error for any job reaching an error state as an element in the list of results.

If .timeout is specified and the job fails to reach a success state within the time limit, await throws a civis_timeout_error. Likewise, await_all throws a civis_timeout_error if all jobs fail to reach a success state within the time limit.

These errors can be caught using try or tryCatch. Useful debugging information can be returned using get_error and fetch_logs.

The set of possible states for jobs on Civis platform are: "succeeded", "success", "failed", "queued", "running", and "cancelled".

Unless .interval is specified, retries are attempted with exponentially increasing intervals using .25 * (1.2^i)) + runif(1, 0, .2), where i is the index of the current retry. Approximate intervals for a given number of retries are as follows:

1-5: .5s
6-10: 1-5s
11-19: 5-10s
20-29: 10s - 1m

The polling interval can be set to a fixed value globally with options("civis.default_polling_interval" = INTERVAL_IN_SECONDS).

Examples

Run this code

if (FALSE) {

   # Executing a query
   q_id <- queries_post(db_id, query, n_rows, cred_id)[["id"]]
   r <- await(queries_get, id = q_id)
   get_status(r)

   r <- tryCatch(await(queries_get, id = q_id), error = function(e) e)
   get_error(r)

   r <- try(await(queries_get, id = q_id))
   get_error(r)

   jobs <- c(1234, 5678)
   runs <- c(1234, 5678)
   rs <- await_all(scripts_get_r_runs, .x = jobs, .y = runs)
}