Rdocumentation
powered by
Learn R Programming
syuzhet (version 1.0.0)
get_tokens: Word Tokenization
Description
Parses a string into a vector of word tokens.
Usage
get_tokens(text_of_file, pattern = "\\W")
Arguments
text_of_file
A Text String
pattern
A regular expression for token breaking
Value
A Character Vector of Words