Learn R Programming

taskqueue (version 0.2.0)

Task Queue for Parallel Computing Based on PostgreSQL

Description

Implements a task queue system for asynchronous parallel computing using 'PostgreSQL' as a backend. Designed for embarrassingly parallel problems where tasks do not communicate with each other. Dynamically distributes tasks to workers, handles uneven load balancing, and allows new workers to join at any time. Particularly useful for running large numbers of independent tasks on high-performance computing (HPC) clusters with 'SLURM' job schedulers.

Copy Link

Version

Install

install.packages('taskqueue')

Version

0.2.0

License

MIT + file LICENSE

Issues

0

Pull Requests

0

Stars

1

Forks

0

Maintainer

Bangyou Zheng

Last Published

December 3rd, 2025

Functions in taskqueue (0.2.0)

task_get

Get Detailed Task Information
resource_get

Get Information for a Specific Resource
task_reset

Reset Task Status to Idle
resource_list

List All Computing Resources
project_list

List All Projects
shiny_app

Launch Shiny App for Monitoring Projects
tq_apply

Apply a Function with Task Queue (Simplified Workflow)
worker

Execute Tasks as a Worker
worker_slurm

Create a worker on slurm cluster
project_stop

Stop a Project
project_resource_add

Assign a Resource to a Project
project_start

Start a Project
resource_add

Add a New Computing Resource
taskqueue_reset

Reset taskqueue Options to Defaults
table_exist

Check whether a table is existed
project_status

Display Project Status
db_init

Initialize PostgreSQL Database for taskqueue
project_reset

Reset a Project
project_resource_add_jobs

Manage SLURM Job List for Project Resource
task_add

Add Tasks to a Project
task_status

Get Task Status Summary
taskqueue_options

Set or Get taskqueue Options
task_clean

Remove All Tasks from a Project
db_disconnect

Disconnect from PostgreSQL Database
.check_linux_absolute_path

Check absolute path for Linux
.check_windows_absolute_path

Check absolute path for Windows
project_resource_get

Get resources of a project
project_delete

Delete a Project
project_resource_log_delete

Delete Log Files for a Project Resource
project_get

Get Project Information
db_clean

Clean All Tables and Definitions from Database
db_connect

Connect to PostgreSQL Database
is_db_connect

Test Database Connection
db_sql

A Wrapper function for DBI interface
.check_absolute_path

Check absolute path for system
project_add

Create a New Project