Computes unweighted Cohen's kappa for either a pair of nominal rating vectors or all pairwise combinations of nominal columns in a matrix or data frame.
cohen_kappa(
data,
y = NULL,
na_method = c("error", "pairwise", "complete"),
ci = FALSE,
p_value = FALSE,
conf_level = 0.95,
n_threads = getOption("matrixCorr.threads", 1L),
output = c("matrix", "sparse", "edge_list"),
threshold = 0,
diag = TRUE,
...
)# S3 method for cohen_kappa
summary(object, digits = 4, ci_digits = 3, p_digits = 4, ...)
# S3 method for summary.cohen_kappa
print(
x,
digits = NULL,
n = NULL,
topn = NULL,
max_vars = NULL,
width = NULL,
show_ci = NULL,
...
)
# S3 method for cohen_kappa
print(
x,
digits = 4,
n = NULL,
topn = NULL,
max_vars = NULL,
width = NULL,
show_ci = NULL,
...
)
# S3 method for cohen_kappa
plot(
x,
title = NULL,
low_color = "indianred1",
high_color = "steelblue1",
mid_color = "white",
value_text_size = 4,
ci_text_size = 3,
show_value = TRUE,
...
)
If y is supplied, a scalar S3 object of class "cohen_kappa"
backed by a numeric value, with attributes diagnostics, and
optionally ci, inference, and conf.level. Otherwise a
symmetric matrix-style result with estimator class cohen_kappa.
In matrix mode, a matrix or data frame whose rows are
observational units and whose columns are raters or classifiers. Supported
column types are factor, ordered factor, character, logical, integer, and
numeric, all treated as nominal categories here. If the ratings are truly
ordinal and disagreements should be weighted by distance, use
weighted_kappa() instead. In two-vector mode, the first nominal rating
vector.
Optional second nominal rating vector. When supplied, the function
returns a single Cohen's kappa estimate for data and y.
Character scalar controlling missing-data handling.
"error" rejects missing values. "complete" applies listwise
deletion across retained columns before computing the matrix. In matrix
mode, "pairwise" uses pair-specific complete observations.
Logical; if TRUE, attach large-sample delta-method
confidence intervals.
Logical; if TRUE, attach large-sample delta-method
Wald test statistics and p-values.
Confidence level used when ci = TRUE. Default is
0.95.
Integer \(\ge 1\). Number of OpenMP threads used by the
matrix C++ backend. Defaults to
getOption("matrixCorr.threads", 1L).
Output representation for matrix mode.
"matrix" (default): full dense matrix.
"sparse": sparse matrix from Matrix containing only
retained entries after thresholding.
"edge_list": long-form data frame representation.
Non-negative absolute-value filter for non-matrix outputs:
retain entries with abs(value) >= threshold. Must be 0 when
output = "matrix".
Logical; whether to include diagonal entries in sparse and edge-list outputs.
Additional theme arguments.
A scalar or matrix-style cohen_kappa object.
Integer; number of decimal places for displayed values.
Integer; number of decimal places for confidence limits.
Integer; number of decimal places for p-values.
A scalar or matrix-style cohen_kappa object.
Optional preview row threshold.
Optional number of leading/trailing rows to show when truncated.
Optional maximum number of visible columns.
Optional display width.
One of "yes" or "no".
Optional plot title.
Fill/color used for negative agreement.
Fill/color used for positive agreement.
Unused placeholder for API consistency with matrix heatmaps.
Text size for the estimate label.
Text size for the CI label.
Logical; whether to print the estimate and CI labels.
Thiago de Paula Oliveira
Cohen's kappa is an agreement coefficient for two raters assigning the same units to mutually exclusive nominal categories. For contingency-table cell proportions \(p_{ij}\), $$ \kappa = \frac{p_o - p_e}{1 - p_e}, $$ where \(p_o = \sum_i p_{ii}\) is the observed agreement and \(p_e = \sum_i p_{i+} p_{+i}\) is the chance agreement implied by the marginal category proportions.
This implementation is strictly the original unweighted nominal-scale Cohen's
kappa. If the categories are ordinal and near disagreements should count as
less severe than distant disagreements, use weighted_kappa() instead.
In matrix mode, columns are treated as raters/classifiers and rows as shared observational units. All pairwise Cohen's kappas between columns are computed. Category labels are encoded in R before dispatch to C++, using a common label mapping across all columns so that identical labels in different columns correspond to the same category code.
Missing values are encoded as NA_integer_ before entering the C++
backend. With na_method = "error", missing values are rejected before
computation. With na_method = "complete", listwise deletion is
applied across retained columns. With na_method = "pairwise", each
pair uses its own complete observations. Pairwise complete counts are stored
in attr(x, "diagnostics")$n_complete.
Confidence intervals and standard errors. The implementation uses the exact large-sample formula. Let \(p_{ab}\) be the empirical cell proportions, \(q_a = \sum_b p_{ab}\) the row margins, \(r_b = \sum_a p_{ab}\) the column margins, and \(D = 1 - p_e\). For each cell \((a,b)\), define the influence contribution $$ g_{ab} \;=\; \frac{\mathbf{1}(a=b)\,D + (p_o - 1)\,(r_a + q_b)}{D^2}. $$ The variance estimator used by the code is $$ \widehat{\mathrm{Var}}(\hat\kappa) \;=\; \frac{1}{n}\left( \sum_{a,b} p_{ab} g_{ab}^2 - \Big(\sum_{a,b} p_{ab} g_{ab}\Big)^2 \right), $$ with \(n\) the number of complete paired ratings and \(\widehat{\mathrm{se}}(\hat\kappa)=\sqrt{\widehat{\mathrm{Var}}(\hat\kappa)}\). The confidence interval is the Wald interval $$ \hat\kappa \pm z_{1-\alpha/2}\,\widehat{\mathrm{se}}(\hat\kappa), $$ truncated to \([-1, 1]\) in the returned result.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37-46. tools:::Rd_expr_doi("10.1177/001316446002000104")
weighted_kappa() for two-rater ordered-category agreement with
distance-sensitive disagreement weights; multirater_kappa() for
panel-level nominal agreement among three or more raters.
x <- factor(c("A", "A", "B", "B", "A", "B"))
y <- factor(c("A", "B", "B", "B", "A", "A"))
cohen_kappa(x, y)
raters <- data.frame(
r1 = factor(c("low", "low", "high", "high", "mid")),
r2 = factor(c("low", "mid", "high", "high", "mid")),
r3 = c("low", "low", "high", "mid", "mid")
)
ck <- cohen_kappa(raters)
print(ck)
summary(ck)
estimate(ck)
tidy(ck)
plot(ck)
Run the code above in your browser using DataLab