perplexity

Compute the perplexity of a trained word2vec model with data.

internal

Create dense vector representation of words and documents using 'quanteda'. Currently implements Word2vec (Mikolov et al., 2013) <doi:10.48550/arXiv.1310.4546> and Latent Semantic Analysis (Deerwester et al., 1990) <doi:10.1002/(SICI)1097-4571(199009)41:6%3C391::AID-ASI1%3E3.0.CO;2-9>.

Kohei Watanabe

wordvector

Word and Document Vector Models

Jan Wijffels

BNOSAC 

Max Fomichev

perplexity function

<dl><dt>x</dt>
<dd>a trained <code>textmodel_wordvector</code> object.</dd>
<dt>targets</dt>
<dd>words for which probabilities are computed.</dd>
<dt>data</dt>
<dd>a quanteda::tokens or quanteda::dfm; the probabilities of words are
tested against occurrences of words in it.</dd></dl>

Arguments

Compute perplexity of a model — perplexity

<dl>

<dt>x</dt>
<dd>a trained <code>textmodel_wordvector</code> object.</dd>


<dt>targets</dt>
<dd>words for which probabilities are computed.</dd>


<dt>data</dt>
<dd>a quanteda::tokens or quanteda::dfm; the probabilities of words are
tested against occurrences of words in it.</dd>

</dl>

perplexity: Compute perplexity of a model

Description

Usage

Arguments