krippalpha
Krippendorff's alpha
krippalpha
computes Krippendorff's reliability coefficient alpha.
Usage
krippalpha(data, metric = "nominal", bootstrap = FALSE,
bootnp = FALSE, nboot = 20000, nnp = 1000, cores = 1,
seed = rep(12345, 6))
Arguments
- data
a matrix or data frame (coercible to a matrix) of reliability data. Data of type
character
are converted tonumeric
viaas.factor()
.- metric
metric difference function to be applied to disagreements. Supports
nominal
,ordinal
,interval
, andratio
. Defaults tonominal
.- bootstrap
logical indicating whether uncertainty estimates should be obtained using the bootstrap algorithm defined by Krippendorff. Defaults to
FALSE
.- bootnp
logical indicating whether non-parametric bootstrap uncertainty estimates should be computed. Defaults to
FALSE
.- nboot
number of bootstraps used in Krippendorff's algorithm. Defaults to
20000
.- nnp
number of non-parametric bootstraps. Defaults to
1000
.- cores
number of cores across which bootstrap-computations are distributed. Defaults to 1. If more cores are specified than available, the number will be set to the maximum number of available cores.
- seed
numeric vector of length 6 for the internal L'Ecuyer-CMRG random number generator (see details). Defaults to
c(12345, 12345, 12345, 12345, 12345, 12345)
.
Details
krippalpha
takes the seed vector to seed the internal random number generator of both bootstrap-routines. It does not advance R's RNG state.
Value
Returns a list of type icr
with following elements:
value of inter-coder reliability coefficient
integer representation of metric used to compute alpha: 1 nominal, 2 ordinal, 3 interval, 4 ratio
number of coders
number of units to be coded
number of unique values in reliability data
matrix containing coincidences within coder-value pairs
matrix of metric differences depending on method
expected disagreement
observed disagreement
TRUE
if Krippendorff bootstrapping algorithm was run, FALSE
otherwise
number of bootstraps
TRUE
if nonparametric bootstrap was run, FALSE
otherwise
number of non-parametric bootstraps
vector of bootstrapped values of alpha (Krippendorff's algorithm)
vector of non-parametrically bootstrapped values of alpha
Note
krippalpha
's bootstrap-routines use L'Ecuyer's CMRG random number generator (see L'Ecyuer et al. 2002) to create random numbers suitable for parallel computations. The routines interface to L'Ecuyer's C++ code, which can be found at https://pubsonline.informs.org/doi/abs/10.1287/opre.50.6.1073.358
References
Krippendorff, K. (2004) Content Analysis: An Introduction to Its Methodology. Beverly Hills: Sage.
Krippendorff, K. (2011) Computing Krippendorff's Alpha Reliability. Departmental Papers (ASC) 43. http://repository.upenn.edu/asc_papers/43.
Krippendorff, K. (2016) Bootstrapping Distributions for Krippendorff's Alpha. http://web.asc.upenn.edu/usr/krippendorff/boot.c-Alpha.pdf.
L'Ecuyer, P. (1999) Good Parameter Sets for Combined Multiple Recursive Random Number Generators. Operations Research, 47 (1), 159--164. https://pubsonline.informs.org/doi/10.1287/opre.47.1.159.
L'Ecuyer, P., Simard, R, Chen, E. J., and Kelton, W. D. (2002) An Objected-Oriented Random-Number Package with Many Long Streams and Substreams. Operations Research, 50 (6), 1073--1075. http://www.iro.umontreal.ca/~lecuyer/myftp/streams00/c++/streams4.pdf.
Examples
# NOT RUN {
data(codings)
krippalpha(codings)
krippalpha(codings, metric = "nominal", bootstrap = TRUE, bootnp = TRUE)
# }