Portable Tools for 'UTF-8' Character Data
Implements an S3 class for storing 'UTF-8' strings, based on regular character vectors.
Also contains routines to portably read and write 'UTF-8' encoded text files,
to convert all strings in an object to 'UTF-8',
and to create character vectors with various encodings.
Portable tools for UTF-8 character data
R and character encoding
The character encoding of determines the translation of the letters, digits, or other codepoints (atomic components of a text) into a sequence of bytes. A byte sequence may translate into valid text in one character encoding, but give nonsense in other character encodings.
For historic reasons, R can store strings in different ways:
- in the "native" encoding, the default encoding of the operating system
- in UTF-8, the most prevalent and versatile encoding nowadays
- in "latin1", a popular encoding in Western Europe
- as "bytes", leaving the interpretation to the user
On OS X and Linux, the "native" encoding is often UTF-8, but on Windows it is not. To add to the confusion, the encoding is a property of individual strings in a character vector, and not of the entire vector.
When working with text, it is advisable to use UTF-8, because it allows encoding virtually any text, even in foreign languages that contain symbols that cannot be represented in your system's native encoding. The UTF-8 encoding possesses several nice technical properties, and is by far the predominant encoding on the Web. Standardization on a "universal" encoding faciliates data exchange.
Because of R's special handling of strings, some care must be taken to make sure that you're actually using the UTF-8 encoding. Many functions in R will hide encoding issues from you, and transparently convert to UTF-8 as necessary. However, some functions (such as reading and writing files) will stubbornly prefer the native encoding.
The enc pacakge provides helpers for converting all textual components of an object to UTF-8, and for reading and writing files in UTF-8 (with a LF end-of-line terminator by default). It also defines an S3 class for tagging all-UTF-8 character vectors and ensuring that updates maintain the UTF-8 encoding. Examples for other packages that use UTF-8 by default are:
library(enc) utf8(c("a", "ä")) #>  "a" "ä" as_utf8(1) #>  "1" a <- utf8("ä") a <- "ö" class(a) #>  "utf8" data.frame(abc = letters[1:3], utf8 = utf8(letters[1:3])) #> abc utf8 #> 1 a a #> 2 b b #> 3 c c
Install the package from GitHub:
# install.packages("devtools") devtools::install_github("krlmlr/enc")
Functions in enc
|write_lines_enc||Writes to a text file|
|to_encoding||Deep conversion to an encoding|
|utf8||A simple class for storing UTF-8 strings|
|native_eol||The native end-of-line identifier on the current platform|
|transform_lines_enc||Transform a text file|
|read_lines_enc||Reads from a text file|
|enc-package||enc: Portable Tools for 'UTF-8' Character Data|
Last month downloads
|Packaged||2018-03-03 00:03:50 UTC; muelleki|
|Date/Publication||2018-03-03 14:03:13 UTC|
Include our badge in your README