LLMR (version 0.4.2)

build_factorial_experiments: Build Factorial Experiment Design

Description

Creates a tibble of experiments for factorial designs where you want to test all combinations of configs, messages, and repetitions with automatic metadata.

Usage

build_factorial_experiments(
  configs,
  messages_list,
  repetitions = 1,
  config_labels = NULL,
  message_labels = NULL
)

Value

A tibble with columns: config (list-column), messages (list-column), config_label, message_label, and repetition. Ready for use with call_llm_par().

Arguments

configs

List of llm_config objects to test.

messages_list

List of message lists to test (each element is a message list for one condition).

repetitions

Integer. Number of repetitions per combination. Default is 1.

config_labels

Character vector of labels for configs. If NULL, uses "provider_model".

message_labels

Character vector of labels for message sets. If NULL, uses "messages_1", etc.

Examples

if (FALSE) {
  # Factorial design: 3 configs x 2 message conditions x 10 reps = 60 experiments
  configs <- list(gpt4_config, claude_config, llama_config)
  messages_list <- list(control_messages, treatment_messages)

  experiments <- build_factorial_experiments(
    configs = configs,
    messages_list = messages_list,
    repetitions = 10,
    config_labels = c("gpt4", "claude", "llama"),
    message_labels = c("control", "treatment")
  )

  # Use with call_llm_par
  results <- call_llm_par(experiments, progress = TRUE)
}