LLMR (version 0.6.0)

build_factorial_experiments: Build Factorial Experiment Design

Description

Creates a tibble of experiments for factorial designs where you want to test all combinations of configs, messages, and repetitions with automatic metadata.

Usage

build_factorial_experiments(
  configs,
  user_prompts,
  system_prompts = NULL,
  repetitions = 1,
  config_labels = NULL,
  user_prompt_labels = NULL,
  system_prompt_labels = NULL
)

Value

A tibble with columns: config (list-column), messages (list-column), config_label, message_label, and repetition. Ready for use with call_llm_par().

Arguments

configs

List of llm_config objects to test.

user_prompts

Character vector (or list) of user‑turn prompts.

system_prompts

Optional character vector of system messages (recycled against user prompts). Missing/NA values are ignored; messages are user-only.

repetitions

Integer. Number of repetitions per combination. Default is 1.

config_labels

Character vector of labels for configs. If NULL, uses "provider_model".

user_prompt_labels

Optional labels for the user prompts.

system_prompt_labels

Optional labels for the system prompts.

Examples

if (FALSE) {
  # Factorial design: 3 configs x 2 message conditions x 10 reps = 60 experiments
  configs <- list(gpt4_config, claude_config, llama_config)
  messages <- list("Control prompt", "Treatment prompt")

  experiments <- build_factorial_experiments(
    configs = configs,
    messages = messages,
    repetitions = 10,
    config_labels = c("gpt4", "claude", "llama"),
    message_labels = c("control", "treatment")
  )

  # Use with call_llm_par
  results <- call_llm_par(experiments, progress = TRUE)
}