NaileR (version 1.2.2)

dist_ref_llm: LLM response consistency

Description

Compute distances between an LLM response of interest and some other responses to the same prompt.

Usage

dist_ref_llm(ppt, ref, n)

Value

A list containing:

• a list with the newly-generated prompts;

• a vector of distances to the reference response.

Arguments

ppt

an LLM prompt.

ref

the reference response.

n

the number of new responses to be generated.

Examples

if (FALSE) {
# Processing time is often longer than ten seconds
# because the function uses a large language model.

data(iris)

intro_iris <- "A study measured various parts of iris flowers
from 3 different species: setosa, versicolor and virginica.
I will give you the results from this study.
You will have to identify what sets these flowers apart."
intro_iris <- gsub('\n', ' ', intro_iris) |>
stringr::str_squish()

req_iris <- "Please explain what makes each species distinct.
Also, tell me which species has the biggest flowers,
and which species has the smallest."
req_iris <- gsub('\n', ' ', req_iris) |>
stringr::str_squish()

res_iris <- nail_catdes(iris, num.var = 5,
introduction = intro_iris, request = req_iris)

dist_ref_llm(res_iris$prompt, res_iris$response, n = 5)
}