trail_icr

internal

Tools for AI-assisted qualitative data coding using large language
models ('LLMs') via the 'ellmer' package, supporting providers including
'OpenAI', 'Anthropic', 'Google', 'Azure', and local models via 'Ollama'.
Provides a 'codebook'-based workflow for defining coding instructions and
applying them to texts, images, and other data. Includes built-in 'codebooks'
for common applications such as sentiment analysis and policy coding, and
functions for creating custom 'codebooks' for specific research questions.
Supports systematic replication across models and settings, computing
inter-coder reliability statistics including Krippendorff's alpha
(Krippendorff 2019, <doi:10.4135/9781071878781>) and Fleiss' kappa
(Fleiss 1971, <doi:10.1037/h0031619>), as well as gold-standard validation
metrics including accuracy, precision, recall, and F1 scores following
Sokolova and Lapalme (2009, <doi:10.1016/j.ipm.2009.03.002>). Provides audit
trail functionality for documenting coding workflows following Lincoln and
Guba's (1985, ISBN:0803924313) framework for establishing trustworthiness
in qualitative research.

Seraphine F. Maerz

quallmer

Qualitative Analysis with Large Language Models

Kenneth Benoit

trail_icr function

<dl><dt>x</dt>
<dd>A <code>trail_compare</code> object or a list of <code>trail_record</code>
objects.</dd>
<dt>id_col</dt>
<dd>Character scalar. Name of the unit identifier column in
the resulting wide data (defaults to "id").</dd>
<dt>label_col</dt>
<dd>Character scalar. Name of the label column in each
record's annotations (defaults to "label").</dd>
<dt>min_coders</dt>
<dd>Integer. Minimum number of non-missing coders per
unit required for inclusion.</dd>
<dt>icr_fun</dt>
<dd>Function used to compute inter-rater reliability.
Defaults to <code>validate()</code>, which is expected to accept
<code>data</code>, <code>id</code>, <code>coder_cols</code>, <code>min_coders</code>,
and <code>mode = "icr"</code>. It should also understand
<code>output = "list"</code> to return a named list of statistics.</dd>
<dt>...</dt>
<dd>Additional arguments passed on to <code>icr_fun</code>.</dd></dl>

Arguments

Compute inter-rater reliability across Trail settings (deprecated) — trail_icr

<dl>

<dt>x</dt>
<dd>A <code>trail_compare</code> object or a list of <code>trail_record</code>
objects.</dd>


<dt>id_col</dt>
<dd>Character scalar. Name of the unit identifier column in
the resulting wide data (defaults to "id").</dd>


<dt>label_col</dt>
<dd>Character scalar. Name of the label column in each
record's annotations (defaults to "label").</dd>


<dt>min_coders</dt>
<dd>Integer. Minimum number of non-missing coders per
unit required for inclusion.</dd>


<dt>icr_fun</dt>
<dd>Function used to compute inter-rater reliability.
Defaults to <code>validate()</code>, which is expected to accept
<code>data</code>, <code>id</code>, <code>coder_cols</code>, <code>min_coders</code>,
and <code>mode = "icr"</code>. It should also understand
<code>output = "list"</code> to return a named list of statistics.</dd>


<dt>...</dt>
<dd>Additional arguments passed on to <code>icr_fun</code>.</dd>

</dl>

trail_icr: Compute inter-rater reliability across Trail settings (deprecated)

Description

Usage

Value

Arguments

Details

See Also