TokenStream-class

Class for token stream operations.

Library for corpus analysis using the Corpus Workbench as an
efficient back end for indexing and querying large corpora. The package offers
functionality to flexibly create partitions and to carry out basic statistical
operations (count, co-occurrences etc.). The original full text of documents
can be reconstructed and inspected at any time. Beyond that, the package is
intended to serve as an interface to packages implementing advanced statistical
procedures. Respective data structures (document term matrices, term co-
occurrence matrices etc.) can be created based on the indexed corpora.

Andreas Blaette

polmineR

Toolkit for Corpus Analysis

TokenStream-class function

<dl class="dl-horizontal">
<dt><code>tokenStream</code></dt><dd>a data.table that will include a column "token"</dd></dl><dt><code>corpus</code></dt><dd>the CWB corpus (character vector length 1)</dd><dt><code>encoding</code></dt><dd>the encoding of the CWB corpus (character vector length 1)</dd>

TokenStream-class: Class for token stream operations.

Description

Arguments

Slots