⚠️There's a newer version (2.3) of this package. Take me there.

stopwords: the R package

R package providing “one-stop shopping” (or should that be “one-shop stopping”?) for stopword lists in R, for multiple languages and sources. No longer should text analysis or NLP packages bake in their own stopword lists or functions, since this package can accommodate them all, and is easily extended.

Created by David Muhr, and extended in cooperation with Kenneth Benoit and Kohei Watanabe.

Installation

# from CRAN
install.packages("stopwords")

# Or get the development version from GitHub:
# install.packages("devtools")
devtools::install_github("quanteda/stopwords")

Usage

head(stopwords::stopwords("de", source = "snowball"), 20)
##  [1] "aber"    "alle"    "allem"   "allen"   "aller"   "alles"   "als"    
##  [8] "also"    "am"      "an"      "ander"   "andere"  "anderem" "anderen"
## [15] "anderer" "anderes" "anderm"  "andern"  "anderr"  "anders"

head(stopwords::stopwords("de", source = "stopwords-iso"), 20)
##  [1] "a"           "ab"          "aber"        "ach"         "acht"       
##  [6] "achte"       "achten"      "achter"      "achtes"      "ag"         
## [11] "alle"        "allein"      "allem"       "allen"       "aller"      
## [16] "allerdings"  "alles"       "allgemeinen" "als"         "also"

For compatibility with the former quanteda::stopwords():

head(stopwords::stopwords("german"), 20)
##  [1] "aber"    "alle"    "allem"   "allen"   "aller"   "alles"   "als"    
##  [8] "also"    "am"      "an"      "ander"   "andere"  "anderem" "anderen"
## [15] "anderer" "anderes" "anderm"  "andern"  "anderr"  "anders"

Explore sources and languages:

# list all sources
stopwords::stopwords_getsources()
## [1] "snowball"      "stopwords-iso" "misc"          "smart"

# list languages for a specific source
stopwords::stopwords_getlanguages("snowball")
##  [1] "da" "de" "en" "es" "fi" "fr" "hu" "ir" "it" "nl" "no" "pt" "ro" "ru"
## [15] "sv"

Languages available

The following coverage of languages is currently available, by source. Note that the inclusiveness of the stopword lists will vary by source, and the number of languages covered by a stopword list does not necessarily mean that the source is better than one with more limited coverage. (There may be many reasons to prefer the default “snowball” source over the “stopwords-iso” source, for instance.)

The following languages are currently available:

LanguageISO-639-1 Codestopwords-isosnowballSMARTmisc
Afrikaansaf
Arabicar
Armenianhy
Basqueeu
Bengalibn
Bretonbr
Bulgarianbg
Catalanca
Chinesezh
Croatianhr
Czechcs
Danishda
Dutchnl
Englishen
Esperantoeo
Estonianet
Finnishfi
Frenchfr
Galiciangl
Germande
Greekel
Gujaratigu
Hausaha
Hebrewhe
Hindihi
Hungarianhu
Indonesianid
Irishga
Italianit
Japaneseja
Koreanko
Kurdishku
Latinla
Lithuanianlt
Latvianlv
Malayms
Marathimr
Norwegianno
Persianfa
Polishpl
Portuguesept
Romanianro
Russianru
Slovaksk
Sloveniansl
Somaliso
Southern Sothost
Spanishes
Swahilisw
Swedishsv
Thaith
Tagalogtl
Turkishtr
Ukrainianuk
Urduur
Vietnamesevi
Yorubayo
Zuluzu

Contributing

Additional sources can be defined and contributed by adding new data objects, as follows:

  1. Data object. Create a named list of characters, in UTF-8 format, consisting of the stopwords for each language. The ISO-639-1 language code will form the name of the list element, and the values of each element will be the character vector of stopwords for literal matches. The data object should follow the package naming convention, and be called data_stopwords_newsource, where newsource is replaced by the name of the new source.

  2. Documentation. The new source should be clearly documented, especially the source from which was taken.

License

This package as well as the source repositories are licensed under MIT.

Copy Link

Version

Down Chevron

Install

install.packages('stopwords')

Monthly Downloads

13,219

Version

1.0

License

MIT + file LICENSE

Issues

Pull Requests

Stars

Forks

Maintainer

Last Published

July 24th, 2019

Functions in stopwords (1.0)