textpress (version 1.1.0)

nlp_cast_tokens: Convert Token List to Data Frame

Description

This function converts a list of tokens into a data frame, extracting and separating document and sentence identifiers if needed.

Usage

nlp_cast_tokens(tok)

Value

A data frame with columns for token name and token.

Arguments

tok

A list where each element contains tokens corresponding to a document or a sentence.

Examples

tok <- list(
  tokens = list(
    "1.1" = c("Hello", "world", "."),
    "1.2" = c("This", "is", "an", "example", "."),
    "2.1" = c("This", "is", "a", "party", "!")
  )
)
dtm <- nlp_cast_tokens(tok)