quanteda (version 0.9.2-0)

docfreq: compute the (weighted) document frequency of a feature

Description

For a dfm-class object, returns a (weighted) document frequency for each term. The default is a simple count of the number of documents in which a feature occurs more than a given frequency threshold. (The default threshold is zero, meaning that any feature occuring at least once in a document will be counted.)

Usage

docfreq(x, scheme = c("count", "inverse", "inversemax", "inverseprob",
  "unary"), smoothing = 0, k = 0, base = 10, threshold = 0,
  USE.NAMES = TRUE)
## S3 method for class 'dfm':
docfreq(x, scheme = c("count", "inverse", "inversemax",
  "inverseprob", "unary"), smoothing = 0, k = 0, base = 10,
  threshold = 0, USE.NAMES = TRUE)
df(x, ...)
## S3 method for class 'dfm':
df(x, ...)

Arguments

x

a dfm-class document-feature matrix

scheme

type of document frequency weighting

smoothing

added to the quotient before taking the logarithm

k

added to the denominator in the "inverse" weighting types, to prevent a zero document count for a term

base

the base with respect to which logarithms in the inverse document frequency weightings are computed; default is 10 (see Manning, Raghavan, and Schutze 2008, p123).

threshold

numeric value of the threshold above which a feature will considered in the computation of document frequency. The default is 0, meaning that a feature's document frequency will be the number of documents in which it occurs greater than zero

USE.NAMES

logical; if TRUE attach feature labels as names of the resulting numeric vector

...

not used

Value

• a numeric vector of document frequencies for each feature

References

Manning, C. D., Raghavan, P., & Schutze, H. (2008). Introduction to Information Retrieval. Cambridge University Press.

Examples

mydfm <- dfm(inaugTexts[1:2], verbose = FALSE)
docfreq(mydfm[, 1:20])

#' # replication of worked example from
# https://en.wikipedia.org/wiki/Tf-idf#Example_of_tf.E2.80.93idf
(wikiDfm <- new("dfmSparse", 
                Matrix::Matrix(c(1,1,2,1,0,0, 1,1,0,0,2,3),
                   byrow = TRUE, nrow = 2,  
                   dimnames = list(docs = c("document1", "document2"), 
                     features = c("this", "is", "a", "sample", "another",
                                  "example")), sparse = TRUE)))
docfreq(wikiDfm)
df(wikiDfm)
docfreq(wikiDfm, scheme = "inverse")
docfreq(wikiDfm, scheme = "inverse", k = 1, smoothing = 1)
docfreq(wikiDfm, scheme = "unary")
docfreq(wikiDfm, scheme = "inversemax")
docfreq(wikiDfm, scheme = "inverseprob")