rio: A Swiss-army knife for data I/O
The aim of rio is to make data file I/O in R as easy as possible by implementing three simple functions in Swiss-army knife style:
export
andimport
provide a painless data I/O experience by automatically choosing the appropriate data read or write function based on file extensionconvert
wrapsimport
andexport
to allow the user to easily convert between file formats (thus providing a FOSS replacement for programs like Stat/Transfer or Sledgehammer). Luca Braglia has created a Shiny app called rioweb that provides access to the file conversion features of rio.
The core advantage of rio is that it makes assumptions that the user is probably willing to make. Five of these are important:
- rio uses the file extension of a file name to determine what kind of file it is. This is the same logic used by Windows OS, for example, in determining what application is associated with a given file type. By taking away the need to manually match a file type (which a beginner may not recognize) to a particular import or export function, rio allows almost all common data formats to be read with the same function. The reader package does something similar for reading certain text formats and R binary files and io offers a set of methods for reading and writing to file formats defined by that package, but rio supports a much broader set of commonly used file types for import and export.
- For text-delimited file formats, the package uses
data.table::fread
to automatically determine the file format regardless of the extension. So, a CSV that is actually tab-separated will still be correctly read in. - When importing tabular data (CSV, TSV, etc.), rio does not convert strings to factors.
- The data import functions in base R only support import of local files or web-based data from websites serving HTTP, not SSL (HTTPS). For example, data stored on GitHub as publicly visible files cannot be read directly into R without either manually downloading them or reading them in via RCurl or httr.
import
supports HTTPS automatically. import
reads from single-file .zip and .tar archives automatically, without the need to explicitly decompress them first.
The package also wraps a variety of faster, more stream-lined I/O packages than those provided by base R or the foreign package. Namely, the package uses haven for reading and writing SAS, Stata, and SPSS files, the fread
function from data.table for intuitive import of text-delimited and fixed-width file formats, and readxl for reading from Excel workbooks.
Supported file formats
rio supports a variety of different file formats for import and export.
Format | Import | Export |
---|---|---|
Tab-separated data (.tsv) | Yes | Yes |
Comma-separated data (.csv) | Yes | Yes |
CSVY (CSV + YAML metadata header) (.csvy) | Yes | No |
Pipe-separated data (.psv) | Yes | Yes |
Fixed-width format data (.fwf) | Yes | Yes |
Serialized R objects (.rds) | Yes | Yes |
Saved R objects (.RData) | Yes | Yes |
JSON (.json) | Yes | Yes |
Stata (.dta) | Yes | Yes |
SPSS and SPSS portable | Yes (.sav and .por) | Yes (.sav only) |
"XBASE" database files (.dbf) | Yes | Yes |
Excel (.xls) | Yes | |
Excel (.xlsx) | Yes | Yes |
Weka Attribute-Relation File Format (.arff) | Yes | Yes |
R syntax (.R) | Yes | Yes |
Shallow XML documents (.xml) | Yes | Yes |
SAS and SAS XPORT | Yes (.sas7bdat and .xpt) | |
Minitab (.mtp) | Yes | |
Epiinfo (.rec) | Yes | |
Systat (.syd) | Yes | |
Data Interchange Format (.dif) | Yes | |
OpenDocument Spreadsheet (.ods) | Yes | |
Fortran data (no recognized extension) | Yes | |
Google Sheets | Yes | |
Clipboard (default is tsv) | Yes (Mac and Windows) | Yes (Mac and Windows) |
Additionally, any format that is not supported by rio but that has a known R implementation will produce an informative error message pointing to a package and import or export function. Unrecognized formats will yield a simple "Unrecognized file format" error.
Package Installation
The package is available on CRAN and can be installed directly in R using:
install.packages("rio")
The latest development version on GitHub can be installed using devtools:
if(!require("devtools")){
install.packages("devtools")
library("devtools")
}
install_github("leeper/rio")
Examples
Because rio is meant to streamline data I/O, the package is extremely easy to use. Here are some examples of reading, writing, and converting data files.
Export
Exporting data is handled with one function, export
:
library("rio")
export(mtcars, "mtcars.csv") # comma-separated values
export(mtcars, "mtcars.rds") # R serialized
export(mtcars, "mtcars.sav") # SPSS
Import
Importing data is handled with one function, import
:
x <- import("mtcars.csv")
y <- import("mtcars.rds")
z <- import("mtcars.sav")
# confirm data match
all.equal(x, y, check.attributes = FALSE)
## [1] TRUE
all.equal(x, z, check.attributes = FALSE)
## [1] TRUE
Note: Because of inconsistencies across underlying packages, the data.frame returned by import
might vary slightly (in variable classes and attributes) depending on file type.
Convert
The convert
function links import
and export
by constructing a dataframe from the imported file and immediately writing it back to disk. convert
invisibly returns the file name of the exported file, so that it can be used to programmatically access the new file.
convert("mtcars.sav", "mtcars.dta")
It is also possible to use rio on the command-line by calling Rscript
with the -e
(expression) argument. For example, to convert a file from Stata (.dta) to comma-separated values (.csv), simply do the following:
Rscript -e "rio::convert('iris.dta', 'iris.csv')"