Learn R Programming

multiRL (version 0.3.7)

Reinforcement Learning Tools for Multi-Armed Bandit

Description

A flexible general-purpose toolbox for implementing Rescorla-Wagner models in multi-armed bandit tasks. As the successor and functional extension of the 'binaryRL' package, 'multiRL' modularizes the Markov Decision Process (MDP) into six core components. This framework enables users to construct custom models via intuitive if-else syntax and define latent learning rules for agents. For parameter estimation, it provides both likelihood-based inference (MLE and MAP) and simulation-based inference (ABC and RNN), with full support for parallel processing across subjects. The workflow is highly standardized, featuring four main functions that strictly follow the four-step protocol (and ten rules) proposed by Wilson & Collins (2019) . Beyond the three built-in models (TD, RSTD, and Utility), users can easily derive new variants by declaring which variables are treated as free parameters.

Copy Link

Version

Install

install.packages('multiRL')

Monthly Downloads

261

Version

0.3.7

License

GPL-3

Maintainer

YuKi

Last Published

March 31st, 2026

Functions in multiRL (0.3.7)

estimate_1_LBI

Likelihood-Based Inference (LBI)
estimate_2_RNN

Estimation Method: Recurrent Neural Network (RNN)
estimate_1_MAP

Estimation Method: Maximum A Posteriori (MAP)
engine_ABC

The Engine of Approximate Bayesian Computation (ABC)
estimate_1_MLE

Estimation Method: Maximum Likelihood Estimation (MLE)
engine_RNN

The Engine of Recurrent Neural Network (RNN)
estimate_0_ENV

Tool for Generating an Environment for Models
estimate_2_SBI

Simulated-Based Inference (SBI)
estimate

Estimate Methods
estimate_2_ABC

Estimation Method: Approximate Bayesian Computation (ABC)
funcs

Core Functions
func_gamma

Function: Utility
estimation_methods

Estimate Methods
func_delta

Function: Bias
func_epsilon

Function: Exploration or Exploitation
func_zeta

Function: Decay Rate
layer

Layers and Loss Functions (RNN)
func_alpha

Function: Learning Rate
func_beta

Function: Probability
fit_p

Step 3: Optimizing parameters to fit real data
process_4_output_cpp

multiRL.output
multiRL-package

multiRL: Reinforcement Learning Tools for Multi-Armed Bandit
process_1_input

multiRL.input
priors

Density and Random Function
process_2_behrule

multiRL.behrule
process_4_output_r

multiRL.output
policy

Policy of Agent
process_3_record

multiRL.record
plot.multiRL.replay

plot.multiRL.replay
params

Model Parameters
summary,multiRL.model-method

summary
rpl_e

Step 4: Replaying the experiment with optimal parameters
system

Cognitive Processing System
reduction

Dimension Reduction Methods (ABC)
rcv_d

Step 2: Generating fake data for parameter and model recovery
process_5_metric

multiRL.metric
run_m

Step 1: Building reinforcement learning model
settings

Settings of Model
MAB

Simulated Multi-Arm Bandit Dataset
data

Dataset Structure
algorithm

Algorithm Packages (MLE, MAP)
TAB

Group 2 from Mason et al. (2024)
RSTD

Risk Sensitive Model
control

Controls of Estimation Methods
TD

Temporal Differences Model
colnames

Column Names
Utility

Utility Model
behrule

Behavior Rules