krippalpha

From icr v0.5.7 by Alexander Staudt

Krippendorff's alpha

krippalpha computes Krippendorff's reliability coefficient alpha.

Usage

krippalpha(data, metric = "nominal", bootstrap = FALSE,
  bootnp = FALSE, nboot = 20000, nnp = 1000, cores = 1,
  seed = rep(12345, 6))

Arguments

data: a matrix or data frame (coercible to a matrix) of reliability data. Data of type character are converted to numeric via as.factor().
metric: metric difference function to be applied to disagreements. Supports nominal, ordinal, interval, and ratio. Defaults to nominal.
bootstrap: logical indicating whether uncertainty estimates should be obtained using the bootstrap algorithm defined by Krippendorff. Defaults to FALSE.
bootnp: logical indicating whether non-parametric bootstrap uncertainty estimates should be computed. Defaults to FALSE.
nboot: number of bootstraps used in Krippendorff's algorithm. Defaults to 20000.
nnp: number of non-parametric bootstraps. Defaults to 1000.
cores: number of cores across which bootstrap-computations are distributed. Defaults to 1. If more cores are specified than available, the number will be set to the maximum number of available cores.
seed: numeric vector of length 6 for the internal L'Ecuyer-CMRG random number generator (see details). Defaults to c(12345, 12345, 12345, 12345, 12345, 12345).

Details

krippalpha takes the seed vector to seed the internal random number generator of both bootstrap-routines. It does not advance R's RNG state.

Value

Returns a list of type icr with following elements:

alpha

value of inter-coder reliability coefficient

metric

integer representation of metric used to compute alpha: 1 nominal, 2 ordinal, 3 interval, 4 ratio

n_coders

number of coders

n_units

number of units to be coded

n_values

number of unique values in reliability data

coincidence_matrix

matrix containing coincidences within coder-value pairs

delta_matrix

matrix of metric differences depending on method

D_e

expected disagreement

D_o

observed disagreement

bootstrap

TRUE if Krippendorff bootstrapping algorithm was run, FALSE otherwise

nboot

number of bootstraps

bootnp

TRUE if nonparametric bootstrap was run, FALSE otherwise

nnp

number of non-parametric bootstraps

bootstraps

vector of bootstrapped values of alpha (Krippendorff's algorithm)

bootstrapsNP

vector of non-parametrically bootstrapped values of alpha

Note

krippalpha's bootstrap-routines use L'Ecuyer's CMRG random number generator (see L'Ecyuer et al. 2002) to create random numbers suitable for parallel computations. The routines interface to L'Ecuyer's C++ code, which can be found at https://pubsonline.informs.org/doi/abs/10.1287/opre.50.6.1073.358

References

Krippendorff, K. (2004) Content Analysis: An Introduction to Its Methodology. Beverly Hills: Sage.

Krippendorff, K. (2011) Computing Krippendorff's Alpha Reliability. Departmental Papers (ASC) 43. http://repository.upenn.edu/asc_papers/43.

Krippendorff, K. (2016) Bootstrapping Distributions for Krippendorff's Alpha. http://web.asc.upenn.edu/usr/krippendorff/boot.c-Alpha.pdf.

L'Ecuyer, P. (1999) Good Parameter Sets for Combined Multiple Recursive Random Number Generators. Operations Research, 47 (1), 159--164. https://pubsonline.informs.org/doi/10.1287/opre.47.1.159.

L'Ecuyer, P., Simard, R, Chen, E. J., and Kelton, W. D. (2002) An Objected-Oriented Random-Number Package with Many Long Streams and Substreams. Operations Research, 50 (6), 1073--1075. http://www.iro.umontreal.ca/~lecuyer/myftp/streams00/c++/streams4.pdf.

Examples

# NOT RUN {
data(codings)
krippalpha(codings)

krippalpha(codings, metric = "nominal", bootstrap = TRUE, bootnp = TRUE)

# }

Documentation reproduced from package icr, version 0.5.7, License: GPL (>= 2)

Community examples

Looks like there are no examples yet.