arete (version 0.1)

file_comparison: Compare the contents of two WebAnno tsv files.

Description

Detect differences between two WebAnno files of the same text for annotation monitoring.

Usage

file_comparison(
  input,
  method = "kripp",
  null_category = TRUE,
  log = NULL,
  verbose = TRUE
)

Value

list. Later should be a dataframe with differences per block in the text.

Arguments

input

list of character or WebAnnoTSV. The contents of WebAnno TSV v3.3 files as created by webanno_open or a set of paths leading to them.

method

character. A choice of "absolute" and "kripp".

null_category

logical. In cases where one annotator labels something and the remaining do not, should it be assigned a category or be set as NA?

log

character. Optional path to save the

verbose

boolean. Print the output of the function at the end.

Details

Right now, finds out the total sum of differences between all aspects of a given text. Method kripp calculates the Krippendorf Alpha, "a reliability coefficient developed to measure the agreement among observers, coders, judges, raters, or measuring instruments drawing distinctions among typically unstructured phenomena or assign computable values to them. alpha emerged in content analysis but is widely applicable wherever two or more methods of generating data are applied to the same set of objects, units of analysis, or items and the question is how much the resulting data can be trusted to represent something real" (Krippendorf, 2011).

References

Klaus Krippendorff (2011). Computing Krippendorff’s Alpha-Reliability. Departmental Papers (ASC). University of Pennsylvania. https://repository.upenn.edu/handle/20.500.14332/2089

Examples

example = arete_data("annotations")

file_comparison(example)