RBERT (version 0.1.11)

clean_text: Perform invalid character removal and whitespace cleanup on text.

Description

(R implementation of BasicTokenizer._clean_text from BERT: tokenization.py.)

Usage

clean_text(text)

Arguments

text

A character scalar.

Value

Cleaned up text.