.validate_morphemepiece_vocabulary

<p>morphemepiece_vocabulary object to validate</p>

vocab

<p>Validator for Objects of Class morphemepiece_vocabulary</p>

internal

Tokenize text into morphemes. The morphemepiece algorithm uses a
lookup table to determine the morpheme breakdown of words, and falls back on a
modified wordpiece tokenization algorithm for words not found in the lookup
table.

Jonathan Bratt

.validate_morphemepiece_vocabulary: Validator for Objects of Class morphemepiece_vocabulary

Description

Usage

Arguments

Value