Parallelise multiple file transfers in the background
init_pool(max_concurrent_transfers = 10, restart = FALSE, ...)delete_pool()
The maximum number of concurrent file transfers to support, which translates into the number of background R processes to create. Each concurrent transfer requires a separate R process, so limit this is you are low on memory.
For init_pool, whether to terminate an already running pool first.
Other arguments passed on to parallel::makeCluster.
AzureStor can parallelise file transfers by utilizing a pool of R processes in the background. This often leads to significant speedups when transferring multiple small files. The pool is created by calling init_pool, or automatically the first time that a multiple file transfer is begun. It remains persistent for the session or until terminated by delete_pool.
If init_pool is called and the current pool is smaller than max_concurrent_transfers, it is resized.