Future promise work queue
Future promise work queue
The global loop is used by default as the internal WorkQueue "delayed check" uses a single delay check for the whole queue, rather than having each item in the queue attempt to process.
This behavior might change in the future, but we are not exactly sure how at this point.
If a private later loop wants to become synchronous by running until all jobs are completed but is waiting on a future_promise(), the private loop will not complete unless the global loop is allowed to move forward.
However, it is possible to use a private loop inside a user-defined WorkQueue may work which can be provided directly to future_promise(queue=custom_queue). Having a concrete example (or need) will help us understand the problem better. If you have an example, please reach out .
new()Create a new WorkQueue
WorkQueue$new(
can_proceed = future_worker_is_free,
queue = fastmap::fastqueue(),
loop = later::global_loop()
)can_proceedFunction that should return a logical value. If TRUE is returned, then the next scheduled work will be executed. By default, this function checks if future::nbrOfFreeWorkers() > 0
queueQueue object to use to store the scheduled work. By default, this is a "First In, First Out" queue using fastmap::fastqueue(). If using your own queue, it should have the methods $add(x), $remove(), $size().
looplater loop to use for calculating the next delayed check. Defaults to later::global_loop().
Schedule work
schedule_work()WorkQueue$schedule_work(fn)fnfunction to execute when can_proceed() returns TRUE.
clone()The objects of this class are cloneable with this method.
WorkQueue$clone(deep = FALSE)deepWhether to make a deep clone.
An R6 class to help with scheduling work to be completed. WorkQueue will only execute work if the can_proceed() returns TRUE. For the use case of future, can_proceed() defaults to future::nbrOfFreeWorkers() > 0 which will not allow for work to be executed if a future worker is not available.
WorkQueue will constantly try to start new work once prior work item finishes. However, if can_proceed() returns FALSE (no future workers are available) and there is more work to be done, then work is attempted later a random amount of time later using exponential backoff. The exponential backoff will cap out at 10 seconds to prevent unnecessarily large wait times.
Each time WorkQueue tries to start more work, it will repeat until can_proceed() returns FALSE or there is no more work in the queue.
future_promise_queue() which returns a WorkQueue which is cached per R session.