quallmer (version 0.4.0)

quallmer-package: quallmer: Qualitative Analysis with Large Language Models

Description

Tools for AI-assisted qualitative data coding using large language models ('LLMs') via the 'ellmer' package, supporting providers including 'OpenAI', 'Anthropic', 'Google', 'Azure', and local models via 'Ollama'. Provides a 'codebook'-based workflow for defining coding instructions and applying them to texts, images, and other data. Includes built-in 'codebooks' for common applications such as sentiment analysis and policy coding, and functions for creating custom 'codebooks' for specific research questions. Supports systematic replication across models and settings, computing inter-coder reliability statistics including Krippendorff's alpha (Krippendorff 2019, tools:::Rd_expr_doi("10.4135/9781071878781")) and Fleiss' kappa (Fleiss 1971, tools:::Rd_expr_doi("10.1037/h0031619")), as well as gold-standard validation metrics including accuracy, precision, recall, and F1 scores following Sokolova and Lapalme (2009, tools:::Rd_expr_doi("10.1016/j.ipm.2009.03.002")). Provides audit trail functionality for documenting coding workflows following Lincoln and Guba's (1985, ISBN:0803924313) framework for establishing trustworthiness in qualitative research.

Author

Maintainer: Seraphine F. Maerz (ORCID)

Authors:

• Kenneth Benoit (ORCID)

References

Krippendorff, K. (2019). Content Analysis: An Introduction to Its Methodology. 4th ed. Thousand Oaks, CA: SAGE. tools:::Rd_expr_doi("10.4135/9781071878781")

Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378–382. tools:::Rd_expr_doi("10.1037/h0031619")

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. tools:::Rd_expr_doi("10.1177/001316446002000104")

Sokolova, M., & Lapalme, G. (2009). A systematic analysis of performance measures for classification tasks. Information Processing & Management, 45(4), 427–437. tools:::Rd_expr_doi("10.1016/j.ipm.2009.03.002")

Wickham H, Cheng J, Jacobs A, Aden-Buie G, Schloerke B (2025). ellmer: Chat with Large Language Models. R package. https://github.com/tidyverse/ellmer

See Also

Useful links:

• https://quallmer.github.io/quallmer/