{"Package":"textrecipes","Title":"Extra 'Recipes' for Text Processing","Version":"0.3.0","Authors@R":"\nperson(given = \"Emil\",\nfamily = \"Hvitfeldt\",\nrole = c(\"aut\", \"cre\"),\nemail = \"emilhhvitfeldt@gmail.com\",\ncomment = c(ORCID = \"0000-0002-0679-1945\"))","Description":"Converting text to numerical features requires\nspecifically created procedures, which are implemented as steps\naccording to the 'recipes' package. These steps allows for\ntokenization, filtering, counting (tf and tfidf) and feature hashing.","License":"MIT + file LICENSE","URL":"https://github.com/tidymodels/textrecipes,\nhttps://textrecipes.tidymodels.org","BugReports":"https://github.com/tidymodels/textrecipes/issues","Depends":"R (>= 2.10), recipes (>= 0.1.4)","Imports":"dplyr, generics, magrittr, Matrix, purrr, Rcpp, rlang,\nSnowballC, stringr, tibble, tidyr, tokenizers, vctrs","Suggests":"covr, knitr, modeldata, rmarkdown, spacyr, stopwords,\ntestthat (>= 2.1.0), text2vec, textfeatures (>= 0.3.3), stringi","LinkingTo":"Rcpp","VignetteBuilder":"knitr","Encoding":"UTF-8","LazyData":"true","RoxygenNote":"7.1.1","SystemRequirements":"GNU make, C++11","NeedsCompilation":"yes","Packaged":"2020-07-08 20:52:27 UTC; emilhvitfeldthansen","Author":"Emil Hvitfeldt [aut, cre] ()","Maintainer":"Emil Hvitfeldt ","Repository":"CRAN","Date/Publication":"2020-07-08 21:50:02 UTC","repoType":"cran","tarballUrl":"ftp://cran.r-project.org/pub/R/src/contrib/textrecipes_0.3.0.tar.gz","jsonAuthors":[{"name":"Emil Hvitfeldt","email":"emilhhvitfeldt@gmail.com","maintainer":true}],"readme":"\n\n\n# textrecipes

\n\n\n\n[![R build\nstatus](https://github.com/tidymodels/textrecipes/workflows/R-CMD-check/badge.svg)](https://github.com/tidymodels/textrecipes/actions)\n[![Codecov test\ncoverage](https://codecov.io/gh/tidymodels/textrecipes/branch/master/graph/badge.svg)](https://codecov.io/github/tidymodels/textrecipes?branch=master)\n[![CRAN\nstatus](http://www.r-pkg.org/badges/version/textrecipes)](https://CRAN.R-project.org/package=textrecipes)\n[![Downloads](http://cranlogs.r-pkg.org/badges/textrecipes)](https://CRAN.R-project.org/package=textrecipes)\n[![Lifecycle:\nmaturing](https://img.shields.io/badge/lifecycle-maturing-blue.svg)](https://www.tidyverse.org/lifecycle/#maturing)\n\n\n## Introduction\n\n**textrecipes** contain extra steps for the\n[`recipes`](https://CRAN.R-project.org/package=recipes) package for\npreprocessing text data.\n\n## Installation\n\nYou can install the released version of textrecipes from\n[CRAN](https://CRAN.R-project.org) with:\n\n``` r\ninstall.packages(\"textrecipes\")\n```\n\nInstall the development version from GitHub with:\n\n``` r\nrequire(\"devtools\")\ninstall_github(\"tidymodels/textrecipes\")\n```\n\n## Example\n\nIn the following example we will go through the steps needed, to convert\na character variable to the TF-IDF of its tokenized words after removing\nstopwords, and, limiting ourself to only the 100 most used words. The\npreprocessing will be conducted on the variable `essay0` and `essay1`.\n\n``` r\nlibrary(recipes)\nlibrary(textrecipes)\nlibrary(modeldata)\n\ndata(okc_text)\n\nokc_rec <- recipe(~ essay0 + essay1, data = okc_text) %>%\n step_tokenize(essay0, essay1) %>% # Tokenizes to words by default\n step_stopwords(essay0, essay1) %>% # Uses the english snowball list by default\n step_tokenfilter(essay0, essay1, max_tokens = 100) %>%\n step_tfidf(essay0, essay1)\n \nokc_obj <- okc_rec %>%\n prep()\n \nstr(bake(okc_obj, okc_text), list.len = 15)\n#> tibble [750 × 200] (S3: tbl_df/tbl/data.frame)\n#> $ tfidf_essay0_also : num [1:750] 0 0 0.0252 0.2232 0 ...\n#> $ tfidf_essay0_always : num [1:750] 0 0 0 0 0 ...\n#> $ tfidf_essay0_amp : num [1:750] 0.47 0.583 0 0 0 ...\n#> $ tfidf_essay0_anything : num [1:750] 0 0 0.113 0 0 ...\n#> $ tfidf_essay0_area : num [1:750] 0 0 0 0 0 ...\n#> $ tfidf_essay0_around : num [1:750] 0 0 0.0348 0 0 ...\n#> $ tfidf_essay0_art : num [1:750] 0 0 0 0 0 ...\n#> $ tfidf_essay0_back : num [1:750] 0 0 0 0 0 ...\n#> $ tfidf_essay0_bay : num [1:750] 0 0 0 0 0 ...\n#> $ tfidf_essay0_believe : num [1:750] 0 0 0 0 0.314 ...\n#> $ tfidf_essay0_big : num [1:750] 0.0781 0 0 0 0 ...\n#> $ tfidf_essay0_bit : num [1:750] 0 0 0 0 0 0 0 0 0 0 ...\n#> $ tfidf_essay0_br : num [1:750] 0.121 0.565 0.121 0 0 ...\n#> $ tfidf_essay0_can : num [1:750] 0.0488 0 0.0244 0 0 ...\n#> $ tfidf_essay0_city : num [1:750] 0 0 0 0 0 0 0 0 0 0 ...\n#> [list output truncated]\n```\n\n## Type chart\n\n**textrecipes** includes a little departure in design from **recipes**,\nin the sense that it allows for some input and output to be in the form\nof list columns. To avoid confusion, here is a table of steps with their\nexpected input and output respectively. Notice how you need to end with\nnumeric for future analysis to work.\n\n| Step | Input | Output |\n| --------------------------- | ------------- | ------------- |\n| `step_tokenize()` | character | `tokenlist()` |\n| `step_untokenize()` | `tokenlist()` | character |\n| `step_lemma()` | `tokenlist()` | `tokenlist()` |\n| `step_stem()` | `tokenlist()` | `tokenlist()` |\n| `step_stopwords()` | `tokenlist()` | `tokenlist()` |\n| `step_pos_filter()` | `tokenlist()` | `tokenlist()` |\n| `step_ngram()` | `tokenlist()` | `tokenlist()` |\n| `step_tokenfilter()` | `tokenlist()` | `tokenlist()` |\n| `step_tokenmerge()` | `tokenlist()` | `tokenlist()` |\n| `step_tfidf()` | `tokenlist()` | numeric |\n| `step_tf()` | `tokenlist()` | numeric |\n| `step_texthash()` | `tokenlist()` | numeric |\n| `step_word_embeddings()` | `tokenlist()` | numeric |\n| `step_textfeature()` | character | numeric |\n| `step_sequence_onehot()` | character | numeric |\n| `step_lda()` | character | numeric |\n| `step_text_normalization()` | character | character |\n\nThis means that valid sequences includes\n\n``` r\nrecipe(~ ., data = data) %>%\n step_tokenize(text) %>%\n step_stem(text) %>%\n step_stopwords(text) %>%\n step_topwords(text) %>%\n step_tf(text)\n\n# or\n\nrecipe(~ ., data = data) %>%\n step_tokenize(text) %>%\n step_stem(text) %>%\n step_tfidf(text)\n```\n\n## Contributing\n\nThis project is released with a [Contributor Code of\nConduct](https://contributor-covenant.org/version/2/0/CODE_OF_CONDUCT.html).\nBy contributing to this project, you agree to abide by its terms.\n\n - For questions and discussions about tidymodels packages, modeling,\n and machine learning, please [post on RStudio\n Community](https://rstd.io/tidymodels-community).\n\n - If you think you have encountered a bug, please [submit an\n issue](https://github.com/tidymodels/textrecipes/issues).\n\n - Either way, learn how to create and share a\n [reprex](https://rstd.io/reprex) (a minimal, reproducible example),\n to clearly communicate about your code.\n\n - Check out further details on [contributing guidelines for tidymodels\n packages](https://www.tidymodels.org/contribute/) and [how to get\n help](https://www.tidymodels.org/help/).","jobInfo":{"package":"textrecipes","version":"0.3.0","parsingStatus":"success","parserVersion":1,"parsedAt":"2020-11-12T20:29:59+0000"}}

Description

Usage

Arguments