LLMR (version 0.4.2)

call_llm_compare: Mode 3: Model Comparison - Multiple Configs, Fixed Message

Description

Compares different configurations (models, providers, settings) using the same message. Perfect for benchmarking across different models or providers. This function requires setting up the parallel environment using `setup_llm_parallel`.

Usage

call_llm_compare(configs_list, messages, ...)

Value

A tibble with columns: config_index (metadata), provider, model, all varying model parameters, response_text, raw_response_json, success, error_message.

Arguments

configs_list

A list of llm_config objects to compare.

messages

List of message objects (same for all configs).

...

Additional arguments passed to `call_llm_par` (e.g., tries, verbose, progress).

Examples

if (FALSE) {
  # Compare different models
  config1 <- llm_config(provider = "openai", model = "gpt-4o-mini",
                        api_key = Sys.getenv("OPENAI_API_KEY"))
  config2 <- llm_config(provider = "openai", model = "gpt-3.5-turbo",
                        api_key = Sys.getenv("OPENAI_API_KEY"))

  configs_list <- list(config1, config2)
  messages <- list(list(role = "user", content = "Explain quantum computing"))

  setup_llm_parallel(workers = 4, verbose = TRUE)
  results <- call_llm_compare(configs_list, messages)
  reset_llm_parallel(verbose = TRUE)
}