RcmdrPlugin.temis (version 0.6.1)

specificTerms: List terms specific of a document or level

Description

List terms most associated (positively or negatively) with each document or each of a variable's levels.

Usage

specificTerms(dtm, variable, p = 0.1, n.max = 25, sparsity = 0.95, min.occ = 2)

Arguments

dtm

a document-term matrix.

variable

a vector whose length is the number of rows of dtm, or NULL to report specific terms by document.

p

the maximum probability up to which terms should be reported.

n.max

the maximum number of terms to report for each level.

sparsity

Optional sparsity threshold (between 0 and 1) below which terms should be skipped. See removeSparseTerms from tm.

min.occ

the minimum number of occurrences in the whole dtm below which terms should be skipped.

Value

• A list of matrices, one for each level of the variable, with seven columns:
• % Term/Level
  the percent of the term's occurrences in all terms occurrences in the level.
• % Level/Term
  the percent of the term's occurrences that appear in the level (rather than in other levels).
• Global %
  the percent of the term's occurrences in all terms occurrences in the corpus.
• Level
  the number of occurrences of the term in the level (internal).
• Global
  the number of occurrences of the term in the corpus.
• t value
  the quantile of a normal distribution corresponding the probability Prob..
• Prob.
  the probability of observing such an extreme (high or low) number of occurrences of the term in the level, under an hypergeometric distribution.

Details

Specific terms reported here are those whose observed frequency in the document or level has the lowest probability under an hypergeometric distribution, based on their global frequencies in the corpus and on the number of occurrences of all terms in the document or variable level considered. The positive or negative character of the association is visible from the sign of the t value, or by comparing the value of the % Term/Level column with that of the Global % column.

All terms with a probability below p are reported, up to n.max terms for each category.

See Also

frequentTerms, DocumentTermMatrix, removeSparseTerms