validateTopic

Create validation tasks for topic model selection

By creating crowd-sourcing tasks that can be easily posted and results retrieved using Amazon's Mechanical Turk (MTurk) API, researchers can use this solution to validate the quality of topics obtained from unsupervised or semi-supervised learning methods, and the relevance of topic labels assigned. This helps ensure that the topic modeling results are accurate and useful for research purposes. See Ying and others (2022) <doi:10.1101/2023.05.02.538599>. For more information, please visit <https://github.com/Triads-Developer/Topic_Model_Validation>.

Luwei Ying

validateIt

Validating Topic Coherence and Topic Labels

Jacob Montgomery

Brandon Stewart

validateTopic function

<dl><dt>type</dt>
<dd>Task structures to be specified. Must be one of "WI" (word intrusion),
"T8WSI" (top 8 word set intrusion), and "R4WSI" (random 4 word set intrusion).</dd>
<dt>n</dt>
<dd>The number of desired tasks</dd>
<dt>text</dt>
<dd>The pool of documents to be shown to the Mturk workers</dd>
<dt>vocab</dt>
<dd>A character vector specifying the words in the corpus. Usually, it
can be found in topic model output.</dd>
<dt>beta</dt>
<dd>A matrix of word probabilities for each topic. Each row represents a
topic and each column represents a word. Note this should not be in the logged form.</dd>
<dt>theta</dt>
<dd>A matrix of topic proportions. Each row represents a document and each
clums represents a topic. Must be specified if task = "T8WSI" or "R4WSI".</dd>
<dt>thres</dt>
<dd>the threshold to draw words from, default to top 50 words.</dd></dl>

Arguments

Create validation tasks for topic model selection — validateTopic

<dl>

<dt>type</dt>
<dd>Task structures to be specified. Must be one of "WI" (word intrusion),
"T8WSI" (top 8 word set intrusion), and "R4WSI" (random 4 word set intrusion).</dd>


<dt>n</dt>
<dd>The number of desired tasks</dd>


<dt>text</dt>
<dd>The pool of documents to be shown to the Mturk workers</dd>


<dt>vocab</dt>
<dd>A character vector specifying the words in the corpus. Usually, it
can be found in topic model output.</dd>


<dt>beta</dt>
<dd>A matrix of word probabilities for each topic. Each row represents a
topic and each column represents a word. Note this should not be in the logged form.</dd>


<dt>theta</dt>
<dd>A matrix of topic proportions. Each row represents a document and each
clums represents a topic. Must be specified if task = "T8WSI" or "R4WSI".</dd>


<dt>thres</dt>
<dd>the threshold to draw words from, default to top 50 words.</dd>

</dl>

validateTopic: Create validation tasks for topic model selection

Description

Usage

Value

Arguments

Details