func_delta

$$
 \text{Bias} = \delta \cdot \sqrt{\frac{\log(N + e)}{N + 10^{-10}}}
 $$

A flexible general-purpose toolbox for implementing Rescorla-Wagner models
in multi-armed bandit tasks.
As the successor and functional extension of the 'binaryRL' package,
'multiRL' modularizes the Markov Decision Process (MDP) into six core
components. This framework enables users to construct custom models via
intuitive if-else syntax and define latent learning rules for agents.
For parameter estimation, it provides both likelihood-based
inference (MLE and MAP) and simulation-based inference (ABC and
RNN), with full support for parallel processing across subjects.
The workflow is highly standardized, featuring four main functions
that strictly follow the four-step protocol (and ten rules)
proposed by Wilson & Collins (2019) <doi:10.7554/eLife.49547>.
Beyond the three built-in models (TD, RSTD, and Utility), users
can easily derive new variants by declaring which variables are
treated as free parameters.

YuKi 

multiRL

Reinforcement Learning Tools for Multi-Armed Bandit

Xinyu 

func_delta function

<dl><dt>count</dt>
<dd>How many times this action has been executed</dd>
<dt>params</dt>
<dd>Parameters used by the model’s internal functions,
see params</dd>
<dt>...</dt>
<dd>It currently contains the following information; additional information 
 may be added in future package versions.<ul>
<li>idinfo:<ul>
<li>subid</li>
<li>block</li>
<li>trial</li>
</ul></li>
<li>exinfo: 
 contains information whose column names are specified by the user.<ul>
<li>Frame</li>
<li>RT</li>
<li>NetWorth</li>
<li>...</li>
</ul></li>
<li>behave: 
 includes the following:<ul>
<li>action: 
 the behavior performed by the human in the given trial.</li>
<li>latent: 
 the object updated by the agent in the given trial.</li>
<li>simulation: 
 the actual behavior performed by the agent.</li>
</ul></li>
</ul></dd></dl>

Arguments

<pre><code>func_delta &lt;- function(
 count,
 params,
 ...
){ list2env(list(...), envir = environment())
 
 # If you need extra information(...)
 # Column names may be lost(C++), indexes are recommended
 # e.g.
 # Trial &lt;- idinfo[3]
 # Frame &lt;- exinfo[1]
 # Action &lt;- behave[1]
 
 delta &lt;- params[["delta"]]
 
 bias &lt;- delta * sqrt(log(count + exp(1)) / (count + 1e-10))
 
 return(bias)
}
</code></pre>

Body

Function: Upper-Confidence-Bound — func_delta

<dl>

<dt>count</dt>
<dd>How many times this action has been executed</dd>


<dt>params</dt>
<dd>Parameters used by the model’s internal functions,
see params</dd>


<dt>...</dt>
<dd>It currently contains the following information; additional information 
 may be added in future package versions.<ul>
<li>idinfo:<ul>
<li>subid</li>
<li>block</li>
<li>trial</li>
</ul></li>
<li>exinfo: 
 contains information whose column names are specified by the user.<ul>
<li>Frame</li>
<li>RT</li>
<li>NetWorth</li>
<li>...</li>
</ul></li>
<li>behave: 
 includes the following:<ul>
<li>action: 
 the behavior performed by the human in the given trial.</li>
<li>latent: 
 the object updated by the agent in the given trial.</li>
<li>simulation: 
 the actual behavior performed by the agent.</li>
</ul></li>
</ul></dd>

</dl>

<pre><code>func_delta &lt;- function(
 count,
 params,
 ...
){

 list2env(list(...), envir = environment())
 
 # If you need extra information(...)
 # Column names may be lost(C++), indexes are recommended
 # e.g.
 # Trial &lt;- idinfo[3]
 # Frame &lt;- exinfo[1]
 # Action &lt;- behave[1]
 
 delta &lt;- params[["delta"]]
 
 bias &lt;- delta * sqrt(log(count + exp(1)) / (count + 1e-10))
 
 return(bias)
}
</code></pre>

func_delta: Function: Upper-Confidence-Bound

Description

Usage

Value

Arguments

Body