{"Package":"kgrams","Title":"Classical k-gram Language Models","Version":"0.1.2","Authors@R":"\nperson(given = \"Valerio\",\nfamily = \"Gherardi\",\nrole = c(\"aut\", \"cre\"),\nemail = \"vgherard840@gmail.com\",\ncomment = c(ORCID = \"0000-0002-8215-3013\"))","Description":"\nTools for training and evaluating k-gram language models in R,\nsupporting several probability smoothing techniques,\nperplexity computations, random text generation and more.","License":"GPL (>= 3)","Encoding":"UTF-8","LazyData":"true","RoxygenNote":"7.1.2","SystemRequirements":"C++11","LinkingTo":"Rcpp, RcppProgress","Imports":"Rcpp, rlang, methods, utils, RcppProgress (>= 0.1), Rdpack","Depends":"R (>= 3.5)","Suggests":"testthat (>= 3.0.0), covr, knitr, rmarkdown","Config/testthat/edition":"3","RdMacros":"Rdpack","VignetteBuilder":"knitr","URL":"https://vgherard.github.io/kgrams/,\nhttps://github.com/vgherard/kgrams","BugReports":"https://github.com/vgherard/kgrams/issues","NeedsCompilation":"yes","Packaged":"2021-11-15 17:17:57 UTC; valerio","Author":"Valerio Gherardi [aut, cre] ()","Maintainer":"Valerio Gherardi ","Repository":"CRAN","Date/Publication":"2021-11-16 08:40:02 UTC","repoType":"cran","tarballUrl":"ftp://cran.r-project.org/pub/R/src/contrib/kgrams_0.1.2.tar.gz","jsonAuthors":[{"name":"Valerio Gherardi","email":"vgherard840@gmail.com","maintainer":true}],"readme":"\n\n\n# kgrams\n\n\n\n[![Project Status: Active – The project has reached a stable, usable\nstate and is being actively\ndeveloped.](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active)\n[![R-CMD-check](https://github.com/vgherard/kgrams/workflows/R-CMD-check/badge.svg)](https://github.com/vgherard/kgrams/actions)\n[![Codecov test\ncoverage](https://codecov.io/gh/vgherard/kgrams/branch/main/graph/badge.svg)](https://app.codecov.io/gh/vgherard/kgrams?branch=main)\n[![CRAN\nstatus](https://www.r-pkg.org/badges/version/kgrams)](https://CRAN.R-project.org/package=kgrams)\n[![R-universe\nstatus](https://vgherard.r-universe.dev/badges/kgrams)](https://vgherard.r-universe.dev/)\n[![Website](https://img.shields.io/badge/Website-here-blue)](https://vgherard.github.io/kgrams/)\n[![Tweet](https://img.shields.io/twitter/url/http/shields.io.svg?style=social)](https://twitter.com/intent/tweet?text=%7Bkgrams%7D:%20Classical%20k-gram%20Language%20Models&url=https://github.com/vgherard/kgrams&via=ValerioGherardi&hashtags=rstats,MachineLearning,NaturalLanguageProcessing)\n\n\n[`kgrams`](https://vgherard.github.io/kgrams/) provides tools for\ntraining and evaluating *k*-gram language models, including several\nprobability smoothing methods, perplexity computations, random text\ngeneration and more. It is based on an C++ back-end which makes `kgrams`\nfast, coupled with an accessible R API which aims at streamlining the\nprocess of model building, and can be suitable for small- and\nmedium-sized NLP experiments, baseline model building, and for\npedagogical purposes.\n\n## For beginners\n\nIf you have no idea about what *k*-gram models are *and* didn’t get here\nby accident, you can check out my hands-on [tutorial post on *k*-gram\nlanguage\nmodels](https://datascienceplus.com/an-introduction-to-k-gram-language-models-in-r/)\nusing R at [DataScience+](https://datascienceplus.com/).\n\n## Installation\n\n#### Released version\n\nYou can install the latest release of `kgrams` from\n[CRAN](https://CRAN.R-project.org/package=kgrams) with:\n\n``` r\ninstall.packages(\"kgrams\")\n```\n\n#### Development version\n\nYou can install the development version from [my\nR-universe](https://vgherard.r-universe.dev/) with:\n\n``` r\ninstall.packages(\"kgrams\", repos = \"https://vgherard.r-universe.dev/\")\n```\n\n## Example\n\nThis example shows how to train a modified Kneser-Ney 4-gram model on\nShakespeare’s play “Much Ado About Nothing” using `kgrams`.\n\n``` r\nlibrary(kgrams)\n# Get k-gram frequency counts from text, for k = 1:4\nfreqs <- kgram_freqs(kgrams::much_ado, N = 4)\n# Build modified Kneser-Ney 4-gram model, with discount parameters D1, D2, D3.\nmkn <- language_model(freqs, smoother = \"mkn\", D1 = 0.25, D2 = 0.5, D3 = 0.75)\n```\n\nWe can now use this `language_model` to compute sentence and word\ncontinuation probabilities:\n\n``` r\n# Compute sentence probabilities\nprobability(c(\"did he break out into tears ?\",\n \"we are predicting sentence probabilities .\"\n ), \n model = mkn\n )\n#> [1] 2.466856e-04 1.184963e-20\n# Compute word continuation probabilities\nprobability(c(\"tears\", \"pieces\") %|% \"did he break out into\", model = mkn)\n#> [1] 9.389238e-01 3.834498e-07\n```\n\nHere are some sentences sampled from the language model’s distribution\nat temperatures `t = c(1, 0.1, 10)`:\n\n``` r\n# Sample sentences from the language model at different temperatures\nset.seed(840)\nsample_sentences(model = mkn, n = 3, max_length = 10, t = 1)\n#> [1] \"i have studied eight or nine truly by your office [...] (truncated output)\"\n#> [2] \"ere you go : \" \n#> [3] \"don pedro welcome signior : \"\nsample_sentences(model = mkn, n = 3, max_length = 10, t = 0.1)\n#> [1] \"i will not be sworn but love may transform me [...] (truncated output)\" \n#> [2] \"i will not fail . \" \n#> [3] \"i will go to benedick and counsel him to fight [...] (truncated output)\"\nsample_sentences(model = mkn, n = 3, max_length = 10, t = 10)\n#> [1] \"july cham's incite start ancientry effect torture tore pains endings [...] (truncated output)\" \n#> [2] \"lastly gallants happiness publish margaret what by spots commodity wake [...] (truncated output)\"\n#> [3] \"born all's 'fool' nest praise hurt messina build afar dancing [...] (truncated output)\"\n```\n\n## Getting Help\n\nFor further help, you can consult the reference page of the `kgrams`\n[website](https://vgherard.github.io/kgrams/) or [open an\nissue](https://github.com/vgherard/kgrams/issues) on the GitHub\nrepository of `kgrams`. A vignette is available on the website,\nillustrating the process of building language models in-depth.","jobInfo":{"package":"kgrams","version":"0.1.2","parsingStatus":"success","parserVersion":1,"parsedAt":"2021-11-16T10:29:48+0000"}}

Description

Arguments

See Also