whatifbandit (version 0.3.0)

summary.multiple.mab: Summary Generic For "multiple.mab" Class

Description

Summarizes results of multiple Multi-Arm Bandit Trials. Provides empirically estimated and normally approximated confidence intervals on AIPW estimates for probability of success, the number of times each arm was the chosen as the best treatment across all simulations, and the average for how many units were assigned to each treatment across all the simulations.

Usage

# S3 method for multiple.mab
summary(object, level = 0.95, ...)

Value

A tibble containing summary information from the repeated trials with the columns:

• Treatment_Arm: Contains the treatment condition.

• average_probability_of_success: The average of the AIPW estimates for the probability of success for each treatment across the trials.

• SE_avg: The standard error for the AIPW estimates, calculated as the square root of the average of the variances.

• SE_empirical: The standard error estimated empirically as the standard deviation of the all the calculated AIPW estimates for probability of success.

• lower_normal: The lower bound on the normal confidence interval for the estimated_probability_of_success. Default is 95%.

• upper_normal: The upper bound on the normal confidence interval for the estimated_probability_of_success. Default is 95%.

• lower_empirical: The lower bound on the empirical confidence interval for the estimated_probability_of_success. Calculated using the observed distribution of AIPW estimated probabilities of success. Default is 95%.

• upper_empirical: The upper bound on the empirical confidence interval for the estimated_probability_of_success. Calculated using the observed distribution of AIPW estimated probabilities of success. Default is 95%.

• times_best: The number of times each treatment arm was selected as the best for an individual simulation.

• average_num_assigned: The average number of observations assigned to each treatment under the simulated trials.

• sd_num_assigned: The standard deviation for the number of observations assigned to each treatment under the simulated trials.

• level: The confidence level for the confidence interval, default is 95%.

Arguments

object

A multiple.mab object created by multiple_mab_simulation.

level

Numeric value of length 1; indicates confidence interval Width (i.e 0.90, 0.95, 0.99). Defaults to 0.95.

...

Additional arguments.

Details

The empirically estimated variances and confidence intervals, use the variance measured directly in the AIPW estimates for each treatment over all the simulations. The normal confidence intervals are estimated using an average of the measured variances across the simulations.

The best arm at the end of each trial is chosen by the highest UCB1 value or Thompson sampling probability. These values indicate which treatment would be chosen next, or have the highest probability of being chosen next, therefore representing the current best treatment.

Additionally, an average and standard deviation for the number of units assigned to each treatment across all the simulations is provided.

... is provided to be compatible with summary(), the function does not have any additional arguments.

Examples

# Objects returned by `multiple_mab_simulation()` have a `multiple.mab` class.
# This class has a summary generic that can produce quick results of the trials
data(tanf)
tanf <- tanf[1:100, ]
# Simulating a few trials
seeds <- sample.int(10000, 5)
x <- multiple_mab_simulation(
  data = tanf,
  assignment_method = "Batch",
  period_length = 20,
  whole_experiment = TRUE,
  blocking = FALSE,
  perfect_assignment = TRUE,
  algorithm = "Thompson",
  prior_periods = "All",
  control_augment = 0,
  data_cols = c(
    condition_col = "condition",
    id_col = "ic_case_id",
    success_col = "success"
  ),
  verbose = FALSE,
  times = 5,
  seeds = seeds,
  keep_data = FALSE
)

# Creating summary table
## Defaults to 95% confidence interval
summary(x) |> print(width = Inf)

## 70% confidence level
summary(x, level = 0.7) |> print(width = Inf)