clevr (version 0.1.2)

accuracy_pairs: Accuracy of Linked Pairs

Description

Computes the accuracy of a set of predicted coreferent (linked) pairs given a set of ground truth coreferent pairs.

Usage

accuracy_pairs(true_pairs, pred_pairs, num_pairs, ordered = FALSE)

Arguments

true_pairs

set of true coreferent pairs stored in a matrix or data.frame, where rows index pairs and columns index the ids of the constituents. Any pairs not included are assumed to be non-coreferent. Duplicate pairs (including equivalent pairs with reversed ids) are automatically removed.

pred_pairs

set of predicted coreferent pairs, following the same specification as true_pairs.

num_pairs

the total number of coreferent and non-coreferent pairs, excluding equivalent pairs with reversed ids.

ordered

whether to treat the element pairs as ordered---i.e. whether pair \((x, y)\) is distinct from pair \((y, x)\) for \(x \neq y\). Defaults to FALSE, which is appropriate for clustering, undirected link prediction, record linkage etc.

Details

The accuracy is defined as: $$\frac{|T \cap P| + |T' \cap P'|}{N}$$ where:

• \(T\) is the set of true coreferent pairs,

• \(P\) is the set of predicted coreferent pairs,

• \(T'\) is the set of true non-coreferent pairs,

• \(P'\) is the set of predicted non-coreferent pairs, and

• \(N\) is the total number of coreferent and non-coreferent pairs.

Examples

true_pairs <- rbind(c(1,2), c(2,3), c(1,3)) # ground truth is 3-clique
pred_pairs <- rbind(c(1,2), c(2,3))         # prediction misses one edge
num_pairs <- 3                              # assuming 3 elements
accuracy_pairs(true_pairs, pred_pairs, num_pairs)