RBERT (version 0.1.11)

whitespace_tokenize: Run basic whitespace cleaning and splitting on a piece of text.

Description

Run basic whitespace cleaning and splitting on a piece of text.

Usage

whitespace_tokenize(text)

Arguments

text

Character scalar to tokenize.

Value

Character vector of tokens.

Examples

# NOT RUN {
whitespace_tokenize(text = " some\ttext \n with  whitespace ")
# }