quanteda (version 4.5.0)

tokenize_dictionary_values: Internal function to tokenize dictionary values

Description

Internal function to tokenize dictionary values

Usage

tokenize_dictionary_values(dict, separator, ...)

Arguments

dict

list of object

Examples

dict <- dictionary(list(ASIA = list("IN" = "\u5370\u5ea6",
                                    "ID" = "\u5370\u5ea6\u5c3c\u897f\u4e9a")))
quanteda:::tokenize_dictionary_values(dict, " ")