whatifbandit (version 0.3.0)

summary.mab: Summary Generic For "mab" Class

Description

Summarizes the Results of a Single Multi-Arm Bandit Trial. Provides confidence intervals around the AIPW estimates, final calculations of the Thompson sampling probabilities or UCB1 values, and the number of observations assigned for each arm.

Usage

# S3 method for mab
summary(object, level = 0.95, ...)

Value

A tibble containing summary information from the trial with the columns:

• Treatment_Arm: Contains the treatment condition.

• Probability_Of_Best_Arm/UCB1_Value: Final Thompson sampling probabilities or UCB1 values for each treatment.

• estimated_probability_of_success: The AIPW estimates for the probability of success for each treatment.

• SE: The standard error for the AIPW estimates.

• lower_bound: The lower bound on the normal confidence interval for the estimated_probability_of_success. Default is 95%.

• upper_bound: The upper bound on the normal confidence interval for the estimated_probability_of_success. Default is 95%.

• num_assigned: The number of observations assigned to each treatment under the simulated trial.

• level: The confidence level for the confidence interval, default is 95%.

• periods: The total number of periods of the simulation.

Arguments

object

A mab class object created by single_mab_simulation().

level

Numeric value of length 1; indicates confidence interval Width (i.e 0.90, 0.95, 0.99). Defaults to 0.95.

...

Additional arguments.

Details

The confidence intervals applied follow a standard normal distribution because it is assumed the AIPW estimators are asymptotically normal as shown in Hadad et al. (2021).

... is provided to be compatible with summary(), the function does not have any additional arguments.

All of the data provided to create a table like this is present in the object created by single_mab_simulation() but this provides a simple shortcut, which is useful when testing many different simulations.

References

Hadad, Vitor, David A. Hirshberg, Ruohan Zhan, Stefan Wager, and Susan Athey. 2021. "Confidence Intervals for Policy Evaluation in Adaptive Experiments." Proceedings of the National Academy of Sciences of the United States of America 118 (15): e2014602118. tools:::Rd_expr_doi("10.1073/pnas.2014602118").

Examples

# Objects returned by `single_mab_simulation()` have a `mab` class.
# This class has a summary generic that can produce quick results of the trial.

# Loading Data and running a quick simulation
data(tanf)
x <- single_mab_simulation(
  data = tanf,
  algorithm = "Thompson",
  assignment_method = "Batch",
  period_length = 600,
  whole_experiment = TRUE,
  perfect_assignment = TRUE,
  blocking = FALSE,
  prior_periods = "All",
  data_cols = c(
    condition_col = "condition",
    id_col = "ic_case_id",
    success_col = "success"
  )
)

# Creating summary table
## Defaults to 95% confidence interval
summary(x) |> print(width = Inf)

## 70% confidence level
summary(x, level = 0.7) |> print(width = Inf)