Learn R Programming

multiRL (version 0.4.5)

Reinforcement Learning Tools for Multi-Armed Bandit

Description

A flexible general-purpose toolbox for implementing Rescorla-Wagner models in multi-armed bandit tasks. As the successor and functional extension of the 'binaryRL' package, 'multiRL' modularizes the Markov Decision Process (MDP) into six core components. This framework enables users to construct custom models via intuitive if-else syntax and define latent learning rules for agents. For parameter estimation, it provides both likelihood-based inference (MLE and MAP) and simulation-based inference (ABC and RNN), with full support for parallel processing across subjects. The workflow is highly standardized, featuring four main functions that strictly follow the four-step protocol (and ten rules) proposed by Wilson & Collins (2019) . Beyond the three built-in models (TD, RSTD, and Utility), users can easily derive new variants by declaring which variables are treated as free parameters.

Copy Link

Version

Install

install.packages('multiRL')

Monthly Downloads

149

Version

0.4.5

License

GPL-3

Maintainer

YuKi

Last Published

June 9th, 2026

Functions in multiRL (0.4.5)

engine_RNN

The Engine of Recurrent Neural Network (RNN)
estimate_1_MLE

Estimation Method: Maximum Likelihood Estimation (MLE)
estimate_2_RNN

Estimation Method: Recurrent Neural Network (RNN)
func_zeta

Function: Decay Rate
estimate_2_SBI

Simulated-Based Inference (SBI)
func_gamma

Function: Utility
func_delta

Function: Bias
func_alpha

Function: Learning Rate
func_epsilon

Function: Exploration or Exploitation
funcs

Core Functions
func_beta

Function: Probability
estimation_methods

Estimate Methods
fit_p

Step 3: Optimizing parameters to fit real data
process_4_output_cpp

multiRL.output
process_1_input

multiRL.input
priors

Density and Random Function
process_2_behrule

multiRL.behrule
process_3_record

multiRL.record
plot.multiRL.replay

plot.multiRL.replay
layer

Layers and Loss Functions (RNN)
policy

Policy of Agent
multiRL-package

multiRL: Reinforcement Learning Tools for Multi-Armed Bandit
params

Model Parameters
run_m

Step 1: Building reinforcement learning model
rpl_e

Step 4: Replaying the experiment with optimal parameters
reduction

Dimension Reduction Methods (ABC)
process_5_metric

multiRL.metric
process_4_output_r

multiRL.output
summary,multiRL.model-method

summary
rcv_d

Step 2: Generating fake data for parameter and model recovery
system

Cognitive Processing System
settings

Settings of Model
TAB

Group 2 from Mason et al. (2024)
WMT

Data from Collins and Frank (2012)
TD

Temporal Differences Model
colnames

Column Names
MAB

Simulated Multi-Arm Bandit Dataset
control

Controls of Estimation Methods
algorithm

Algorithm Packages (MLE, MAP)
RSTD

Risk Sensitive Model
behrule

Behavior Rules
Utility

Utility Model
engine_ABC

The Engine of Approximate Bayesian Computation (ABC)
estimate_1_MAP

Estimation Method: Maximum A Posteriori (MAP)
data

Dataset Structure
estimate_2_ABC

Estimation Method: Approximate Bayesian Computation (ABC)
estimate_1_LBI

Likelihood-Based Inference (LBI)
estimate

Estimate Methods
estimate_0_ENV

Tool for Generating an Environment for Models