reexports

%&gt;%

.data

%||%

%fin%

morphemepiece_vocab

morphemepiece_lookup

These objects are imported from other packages. Follow the links
below to see their documentation.<dl class="dl-horizontal">
 <dt>fastmatch</dt><dd><code><a rd-options="fastmatch:fmatch" href="/link/%25fin%25?package=morphemepiece&version=1.2.3&to=fastmatch%3Afmatch" data-mini-rdoc="fastmatch:fmatch::%fin%">%fin%</a></code></dd><dt>magrittr</dt><dd><code><a rd-options="magrittr:pipe" href="/link/%25%26gt%3B%25?package=morphemepiece&version=1.2.3&to=magrittr%3Apipe" data-mini-rdoc="magrittr:pipe::%&gt;%">%&gt;%</a></code></dd><dt>morphemepiece.data</dt><dd><code><a rd-options="morphemepiece.data" href="/link/morphemepiece_lookup?package=morphemepiece&version=1.2.3&to=morphemepiece.data" data-mini-rdoc="morphemepiece.data::morphemepiece_lookup">morphemepiece_lookup</a></code>, <code><a rd-options="morphemepiece.data" href="/link/morphemepiece_vocab?package=morphemepiece&version=1.2.3&to=morphemepiece.data" data-mini-rdoc="morphemepiece.data::morphemepiece_vocab">morphemepiece_vocab</a></code></dd><dt>rlang</dt><dd><code><a rd-options="rlang:op-null-default" href="/link/%25%7C%7C%25?package=morphemepiece&version=1.2.3&to=rlang%3Aop-null-default" data-mini-rdoc="rlang:op-null-default::%||%">%||%</a></code>, <code><a rd-options="rlang:dot-data" href="/link/.data?package=morphemepiece&version=1.2.3&to=rlang%3Adot-data" data-mini-rdoc="rlang:dot-data::.data">.data</a></code></dd>
</dl>

internal

Tokenize text into morphemes. The morphemepiece algorithm uses a
lookup table to determine the morpheme breakdown of words, and falls back on a
modified wordpiece tokenization algorithm for words not found in the lookup
table.

reexports: Objects exported from other packages

Description

Arguments