reexports

%||%

%fin%

wordpiece_vocab

These objects are imported from other packages. Follow the links
below to see their documentation.<dl class="dl-horizontal">
 <dt>fastmatch</dt><dd><code><a rd-options="fastmatch:fmatch" href="/link/%25fin%25?package=wordpiece&version=2.1.3&to=fastmatch%3Afmatch" data-mini-rdoc="fastmatch:fmatch::%fin%">%fin%</a></code></dd><dt>rlang</dt><dd><code><a rd-options="rlang:op-null-default" href="/link/%25%7C%7C%25?package=wordpiece&version=2.1.3&to=rlang%3Aop-null-default" data-mini-rdoc="rlang:op-null-default::%||%">%||%</a></code></dd><dt>wordpiece.data</dt><dd><code><a rd-options="wordpiece.data" href="/link/wordpiece_vocab?package=wordpiece&version=2.1.3&to=wordpiece.data" data-mini-rdoc="wordpiece.data::wordpiece_vocab">wordpiece_vocab</a></code></dd>
</dl>

internal

Apply 'Wordpiece' (<arXiv:1609.08144>) tokenization to input text,
given an appropriate vocabulary. The 'BERT' (<arXiv:1810.04805>) tokenization
conventions are used by default.

Jonathan Bratt

wordpiece

R Implementation of Wordpiece Tokenization

Jon Harmon

Bedford Freeman & Worth Pub Grp LLC DBA Macmillan Learning 

reexports function

Objects exported from other packages — reexports

Objects exported from other packages

reexports: Objects exported from other packages

Description

Arguments