fit_p: Step 3: Optimizing parameters to fit real data

Description

This function optimizes free parameters of reinforcement learning models built with the `run_m` function. After constructing a reinforcement learning model (a function with only ONE argument, `params`), the `fit_p` function searches for the optimal values of these free parameters.

The function provides several optimization algorithms:

1. L-BFGS-B (from `stats::optim`);
2. Simulated Annealing (`GenSA`);
3. Genetic Algorithm (`GA`);
4. Differential Evolution (`DEoptim`);
5. Particle Swarm Optimization (`pso`);
6. Bayesian Optimization (`mlrMBO`);
7. Covariance Matrix Adapting Evolutionary Strategy (`cmaes`);
8. Nonlinear Optimization (`nloptr`)

For more information, please refer to the GitHub repository: https://github.com/yuki-961004/binaryRL

Usage

fit_p(
  data,
  id = NULL,
  n_trials = NULL,
  fit_model = list(TD, RSTD, Utility),
  funcs = NULL,
  model_name = c("TD", "RSTD", "Utility"),
  lower = list(c(0, 0), c(0, 0, 0), c(0, 0, 0)),
  upper = list(c(1, 1), c(1, 1, 1), c(1, 1, 1)),
  initial_params = NA,
  initial_size = 50,
  iteration = 10,
  seed = 123,
  nc = 1,
  algorithm
)

Value

The optimal parameters found by the algorithm for each subject, along with the model fit calculated using these parameters. This is returned as an object of class binaryRL containing results for all subjects with all models.

Arguments

data

[data.frame] raw data. This data should include the following mandatory columns:

"sub"
"time_line" (e.g., "Block", "Trial")
"L_choice"
"R_choice"
"L_reward"
"R_reward"
"sub_choose"

id

[vector] which subject is going to be analyzed. is being analyzed. The value should correspond to an entry in the "sub" column, which must contain the subject IDs. e.g., `id = unique(data$Subject)`

n_trials

[integer] number of total trials

fit_model

[list] A collection of functions applied to fit models to the data.

funcs

[vector] A character vector containing the names of all user-defined functions required for the computation.

model_name

[list] the name of fit modals

lower

[list] The lower bounds for model fit models

upper

[list] The upper bounds for model fit models

initial_params

[vector] Initial values for the free parameters. These need to be set only when using L-BFGS-B. Other algorithms automatically generate initial values. for `L-BFGS-B`, `GenSA`, set `initial = c(0, 0, ...)`

initial_size

[integer] Initial values for the free parameters. These need to be set only when using L-BFGS-B. Other algorithms automatically generate initial values. for `Bayesian`, `GA`, set `initial = 50`

iteration

[integer] the number of iteration

seed

[integer] random seed. This ensures that the results are reproducible and remain the same each time the function is run. default: `seed = 123`

nc

[integer] Number of CPU cores to use for parallel computation.

algorithm

[character] Choose an algorithm package from `L-BFGS-B`, `GenSA`, `GA`, `DEoptim`, `PSO`, `Bayesian`, `CMA-ES`. In addition, any algorithm from the `nloptr` package is also supported. If your chosen `nloptr` algorithm requires a local search, you need to input a character vector. The first element represents the algorithm used for global search, and the second element represents the algorithm used for local search.