ICC_models

Interrater agreement of log probabilities (treated as "ratings"/rows) among BERT language models (treated as "raters"/columns), with both row and column as ("two-way") random effects.

The Fill-Mask Association Test ('FMAT')
<doi:10.1037/pspa0000396>
is an integrative and probability-based method using
Masked Language Models to measure conceptual associations
(e.g., attitudes, biases, stereotypes, social norms, cultural values)
as propositions in natural language.
Supported language models include 'BERT'
<doi:10.48550/arXiv.1810.04805> and its variants available at 'Hugging Face'
<https://huggingface.co/models?pipeline_tag=fill-mask>.
Methodological references and installation guidance are provided at
<https://psychbruce.github.io/FMAT/>.

Han-Wu-Shuang Bao

FMAT

The Fill-Mask Association Test

ICC_models function

<dl><dt>data</dt>
<dd>Raw data returned from <code>FMAT_run()</code>.</dd>
<dt>type</dt>
<dd>Interrater <code>"agreement"</code> (default) or <code>"consistency"</code>.</dd>
<dt>unit</dt>
<dd>Reliability of <code>"average"</code> scores (default) or <code>"single"</code> scores.</dd></dl>

Arguments

Intraclass correlation coefficient (ICC) of BERT models. — ICC_models

<dl>

<dt>data</dt>
<dd>Raw data returned from <code>FMAT_run()</code>.</dd>


<dt>type</dt>
<dd>Interrater <code>"agreement"</code> (default) or <code>"consistency"</code>.</dd>


<dt>unit</dt>
<dd>Reliability of <code>"average"</code> scores (default) or <code>"single"</code> scores.</dd>

</dl>

ICC_models: Intraclass correlation coefficient (ICC) of BERT models.

Description

Usage

Value

Arguments