Learn R Programming

validateIt: An R Package for Topic and Label Validation

Authors: Luwei Ying, Jacob Montgomery and Brandon Stewart

Please email all comments/questions to luwei.ying [AT] wustl.edu

Installation Instructions

The package is currently not available on CRAN. You can install the most recent development version using the devtools package. First you have to install devtools using the following code. Note that you only have to do this once

if(!require(devtools)) install.packages("devtools")

Then you'll need to install the dependency, "pyMTurkR", using the function install_github. If you need to interact with Mturk, be sure to follow the detailed instructions here: github.com/cloudyr/pyMTurkR

devtools::install_github("cloudyr/pyMTurkR")

Now you can install our package

devtools::install_github("Luwei-Ying/validateIt", dependencies=TRUE)

Note that this will install all the packages suggested and required to run our package. It may take a few minutes the first time, but this only needs to be done on the first use. In the future you can update to the most recent development version using the same code.

Copy Link

Version

Install

install.packages('validateIt')

Monthly Downloads

163

Version

1.2.1

License

GPL (>= 2)

Maintainer

Luwei Ying

Last Published

May 16th, 2023

Functions in validateIt (1.2.1)

heldouttest

An Example Heldout Test Set
checkAgree

Check Agreement Rate between Identical Trails
plotResults

Plot results
record

Reform tasks to facilitate sending to Mturk
getResults

Get results from Mturk
R4WSItasktest

Example R4WSI0 Tasks
Topic_Model_Validation_Overview

Topic_Model_Validation Repository Overview
goldR4WSItest

Example Gold-Standard R4WSI0 Tasks
recordtest

Example Local Record of the R4WSI Tasks
masstest

An Example of the Combined Mass for Words with the Same Roots
resultstest

Example Results Retrieved from Mturk
modtest

An Example Topic Model
validateTopic

Create validation tasks for topic model selection
evalResults

Evaluate results
keypostedtest

Example Answer Keys
sendTasks

Send prepared task to Mturk and record the API-returned HIT ids.
pickLabel

Pick the optimal label from candidate labels
stmPreptest

An Example Object of Prepared Documents
tidyeval

Tidy eval helpers
mixGold

Mix the gold-standard tasks with the tasks need to be validated
validateLabel

Create validation tasks for labels assigned to the topics in the topic model of choice.
combMass

Combine the mass of words with the same root
allR4WSItasktest

Example R4WSI Tasks with Regular and Gold-Standard Tasks