odbc
The goal of the odbc package is to provide a DBI-compliant interface to Open Database Connectivity (ODBC) drivers. This gives a efficient, easy to setup connection to any database with ODBC drivers available, including SQL Server, Oracle, MySQL, PostgreSQL, SQLite and others. The implementation builds on the nanodbc C++ library.
Installation
For Unix and MacOS ODBC drivers should be compiled against unixODBC. Drivers compiled against iODBC may also work, but are not fully supported.
Windows
Windows is bundled with ODBC libraries however drivers for each database need to be installed separately. Windows ODBC drivers typically include a installer that needs to be run and will install the driver to the proper locations.
MacOS
homebrew can be used to easily install database drivers on MacOS.
UnixODBC - Required for all databases
# Install the unixODBC library
brew install unixodbcCommon DB drivers
# SQL Server ODBC Drivers (Free TDS)
brew install freetds --with-unixodbc
# PostgreSQL ODBC ODBC Drivers
brew install psqlodbc
# MySQL ODBC Drivers (and database)
brew install mysql
# SQLite ODBC Drivers
brew install sqliteodbcLinux - Debian / Ubuntu
apt-get can be used to easily install database drivers on Linux distributions which support it, such as Debian and Ubuntu.
UnixODBC - Required for all databases
# Install the unixODBC library
apt-get install unixodbc unixodbc-devCommon DB drivers
# SQL Server ODBC Drivers (Free TDS)
apt-get install tdsodbc
# PostgreSQL ODBC ODBC Drivers
apt-get install odbc-postgresql
# MySQL ODBC Drivers
apt-get install libmyodbc
# SQLite ODBC Drivers
apt-get install libsqliteodbcR
odbc is not yet available on CRAN, however devtools can be used to install the latest version.
# install.packages(devtools)
devtools::install_github("rstats-db/odbc")Connecting to a Database
Databases can be connect by specifying a connection string directly, or with DSN Configuration files.
Connection Strings
Specify a connection string as named arguments directly in the dbConnect() method.
library(DBI)
con <- dbConnect(odbc::odbc(),
driver = "PostgreSQL Driver",
database = "test_db",
uid = "postgres",
pwd = "password",
host = "localhost",
port = 5432)Alternatively you can pass a complete connection string as the .connection_string argument. The Connection Strings Reference is a useful resource that has example connection strings for a large variety of databases.
library(DBI)
con <- dbConnect(odbc::odbc(),
.connection_string = "Driver={PostgreSQL Driver};Uid=postgres;Pwd=password;Host=localhost;Port=5432;Database=test_db;")DSN Configuration files
ODBC configuration files are another option to specify connection parameters and allow one to use a Data Source Name (DSN) to make it easier to connect to a database.
con <- dbConnect(odbc::odbc(), "PostgreSQL")Windows
The ODBC Data Source Administrator application is used to manage ODBC data sources on Windows.
MacOS / Linux
On MacOS and Linux there are two separate text files that need to be edited. UnixODBC includes an command line executable odbcinst which can be used to query and modify the DSN files. However these are plain text files you can also edit by hand if desired.
There are two different files used to setup the DSN information. - odbcinst.ini - which defines driver options - odbc.ini - which defines connection options
The DSN configuration files can be defined globally for all users of the system, often at /etc/odbc.ini or /opt/local/etc/odbc.ini, the exact location depends on what option was used when compiling unixODBC. odbcinst -j can be used to find the exact location. Alternatively the ODBCSYSINI environment variable can be used to specify the location of the configuration files. Ex. ODBCSYSINI=~/ODBC
A local DSN file can also be used with the files ~/.odbc.ini and ~/.odbcinst.ini.
odbcinst.ini
Contains driver information, particularly the name of the driver library. Multiple drivers can be specified in the same file.
[PostgreSQL]
Driver = /usr/local/lib/psqlodbcw.so
[SQLite Driver]
Driver = /usr/local/lib/libsqlite3odbc.dylibodbc.ini
Contains connection information, particularly the username, password, database and host information. The Driver line corresponds to the driver defined in odbcinst.ini.
[PostgreSQL]
Driver = PostgreSQL Driver
Database = test_db
Servername = localhost
UserName = postgres
Password = password
Port = 5432
[SQLite]
Driver = SQLite Driver
Database=/tmp/testingSee also: unixODBC without the GUI for more information and examples.
Usage
All of the following examples assume you have already created a query con. See Connecting to a database for more information on establishing a connection.
Table and Field information
dbListTables() is used for listing tables in a database.
dbListTables(con)
# List tables beginning with f
dbListTables(con, table_name = "f%")
# List all fields in the 'flights' database
dbListFields(con, "flights")Reading
dbReadTable() will read a full table into an R data.frame().
data <- dbReadTable(con, "flights")Writing
dbWriteTable() will write an R data.frame() to an SQL table.
data <- dbWriteTable(con, "iris", iris)Querying
dbGetQuery() will submit a query and fetch the results. It is also possible to submit the query and fetch separately with dbSendQuery() and dbFetch(). If the n= argument to dbFetch() can be used to fetch only part of a query.
result <- dbSendQuery(con, "SELECT flight, tailnum, origin FROM flights ORDER BY origin")
# Retrieve the first 100 results
first_100 <- dbFetch(result, n = 100)
# Retrieve the rest of the results
rest <- dbFetch(result)Benchmarks
Odbc is often much faster than the existing RODBC and DBI compatible RODBCDBI packages.
Reading
Reading a from a PostgreSQL database with the nytflights13 'flights' database (336,776 rows, 19 columns).
# First using RODBC / RODBCDBI
library(DBI)
library(RODBCDBI)
rodbc <- dbConnect(RODBCDBI::ODBC(), dsn = "PostgreSQL")
system.time(rodbc_result <- dbReadTable(rodbc, "flights"))
#> Warning: closing unused RODBC handle 2
#> user system elapsed
#> 19.203 1.356 21.724
# Now using odbc
odbc <- dbConnect(odbc::odbc(), dsn = "PostgreSQL")
system.time(odbc_result <- dbReadTable(odbc, "flights"))
#> user system elapsed
#> 5.119 0.290 6.771
library(tibble)
as_tibble(odbc_result)
#> # A tibble: 336,776 × 20
#> row.names year month day dep_time sched_dep_time dep_delay arr_time
#> <chr> <int> <int> <int> <int> <int> <dbl> <int>
#> 1 1 2013 1 1 517 515 2 830
#> 2 2 2013 1 1 533 529 4 850
#> 3 3 2013 1 1 542 540 2 923
#> 4 4 2013 1 1 544 545 -1 1004
#> 5 5 2013 1 1 554 600 -6 812
#> 6 6 2013 1 1 554 558 -4 740
#> 7 7 2013 1 1 555 600 -5 913
#> 8 8 2013 1 1 557 600 -3 709
#> 9 9 2013 1 1 557 600 -3 838
#> 10 10 2013 1 1 558 600 -2 753
#> # ... with 336,766 more rows, and 12 more variables: sched_arr_time <int>,
#> # arr_delay <dbl>, carrier <chr>, flight <int>, tailnum <chr>,
#> # origin <chr>, dest <chr>, air_time <dbl>, distance <dbl>, hour <dbl>,
#> # minute <dbl>, time_hour <dttm>
identical(dim(rodbc_result), dim(odbc_result))
#> [1] TRUE
rm(rodbc_result, odbc_result, odbc, rodbc)
gc(verbose = FALSE)
#> Warning: closing unused RODBC handle 3
#> used (Mb) gc trigger (Mb) max used (Mb)
#> Ncells 712236 38.1 1770749 94.6 1770749 94.6
#> Vcells 8991012 68.6 27225095 207.8 33776265 257.7Writing
Writing the same dataset to the database.
#> [1] TRUE
#> [1] TRUE
#> used (Mb) gc trigger (Mb) max used (Mb)
#> Ncells 712146 38.1 1770749 94.6 1770749 94.6
#> Vcells 8990784 68.6 27225095 207.8 33776265 257.7library(nycflights13)
# rodbc does not support writing timestamps natively.
rodbc <- dbConnect(RODBCDBI::ODBC(), dsn = "PostgreSQL")
system.time(dbWriteTable(rodbc, "flights2", as.data.frame(flights[, names(flights) != "time_hour"])))
#> user system elapsed
#> 6.693 3.786 48.423
# Now using odbc
odbc <- dbConnect(odbc::odbc(), dsn = "PostgreSQL")
system.time(dbWriteTable(odbc, "flights3", as.data.frame(flights)))
#> user system elapsed
#> 7.802 3.703 26.016