thor v1.1.0

0

Monthly downloads

0th

Percentile

Interface to 'LMDB'

Key-value store, implemented as a wrapper around 'LMDB'; the "lightning memory-mapped database" <https://symas.com/lmdb/>. 'LMDB' is a transactional key value store that uses a memory map for efficient access. This package wraps the entire 'LMDB' interface (except duplicated keys), and provides objects for transactions and cursors.

Readme

thor

Project Status: Active – The project has reached a stable, usable state and is being actively developed. Linux Build Status Windows Build status codecov.io

An R interface to LMDB. LMDB is an embedded transactional key-value store and this package provides R mappings to it. It wraps the entire LMDB interface, except for support for duplicated keys.

Documentation

The package comes with a vignette that describes the main features of the package and of LMDB - see here for a version online. The package also has reference documentation for all methods. It may be useful to refer to the LMDB documentation along side the reference documentation for some details (but hopefully not too much).

Usage

Everything starts by creating an environment (which lives at a point in the file system), and then using methods of the environment object to interact with the database

env <- thor::mdb_env(tempfile())
env
## <mdb_env>
##   Informational:
##     path()
##     flags()
##     info()
##     stat()
##     maxkeysize()
##     maxreaders()
##   Transactions:
##     begin(db = NULL, write = FALSE, sync = NULL, metasync =  ...
##     with_transaction(fun, db = NULL, write = FALSE)
##   Databases:
##     open_database(key = NULL, reversekey = FALSE, create = TRUE)
##     drop_database(db, delete = TRUE)
##   Management:
##     sync(force = FALSE)
##     copy(path, compact = FALSE)
##     close()
##     destroy()
##     reader_list()
##     reader_check()
##   Helpers:
##     get(key, missing_is_error = TRUE, as_raw = NULL, db = NULL)
##     put(key, value, overwrite = TRUE, append = FALSE, db = NULL)
##     del(key, db = NULL)
##     exists(key, db = NULL)
##     list(starts_with = NULL, as_raw = FALSE, size = NULL, db ...
##     mget(key, as_raw = NULL, db = NULL)
##     mput(key, value, overwrite = TRUE, append = FALSE, db =  ...
##     mdel(key, db = NULL)
env$put("hello", "world")
## NULL
env$exists("hello")
## [1] TRUE
env$get("hello") # world
## [1] "world"
env$del("hello")
## [1] TRUE

LMDB is transactional, and thor exposes this like so:

txn <- env$begin(write = TRUE)
txn
## <mdb_txn>
##   Informational:
##     id()
##     stat()
##   Finish:
##     commit()
##     abort(cache = TRUE)
##   Cursors:
##     cursor()
##   Data:
##     get(key, missing_is_error = TRUE, as_proxy = FALSE, as_r ...
##     put(key, value, overwrite = TRUE, append = FALSE)
##     del(key)
##     exists(key)
##     list(starts_with = NULL, as_raw = FALSE, size = NULL)
##     mget(key, as_proxy = FALSE, as_raw = NULL)
##     mput(key, value, overwrite = TRUE, append = FALSE)
##     mdel(key)
##     replace(key, value, as_raw = NULL)
##     pop(key, as_raw = NULL)
##   Compare:
##     cmp(a, b)

Only one write transaction is active at a given point in time. There can be an unlimited number of read transactions.

txn$put("key", "value")
env$get("key", missing_is_error = FALSE) # NULL - not committed yet
txn$commit()
env$get("key") # new transactions see the value

There is a cursor interface for advanced features (see the vignette). Both keys and values can be strings or binary value, the latter working well with serialize. For efficient use from R, thor extends the LMDB interface to implement bulk reads, writes and deletes (mget, mput and mdel).

Performance

lldb is an extremely fast database, but this package may be much less fast than the underlying library. In order to make the interface safe to use from R, there is quite a bit of error checking, and the length of time involved in calling methods in R6 objects is orders of magnitude slower than performing an action on an lldb database (this is not R6's fault and primarily caused by the cost of S3 method lookup for $ on an object with a class attribute). The vectorised functions will help here (e.g., mget, mput), so prefer these where practical if speed is a concern.

Installation

Install from CRAN with

install.packages("thor")

If you want to try the development version from github, you can install with

devtools::install_github("richfitz/thor", upgrade = FALSE)

License

MIT + file LICENSE © Rich FitzJohn. The package contains included code from lmdb which it itself under the "OpenLDAP Public License" - see inst/LICENSE.lmdb for details

Functions in thor

Name Description
mdb_env Create an mdb_env environment
mdb_cursor Use mdb transactions
mdb_dbi Use mdb transactions
mdb_txn Use mdb transactions
storr_thor Thor driver for storr
mdb_proxy Proxy values
No Results!

Vignettes of thor

Name
thor.Rmd
No Results!

Last month downloads

Details

License MIT + file LICENSE
LazyData true
URL https://github.com/richfitz/thor
BugReports https://github.com/richfitz/thor/issues
RoxygenNote 6.1.1
VignetteBuilder knitr
ByteCompile true
Encoding UTF-8
Language en-GB
NeedsCompilation yes
Packaged 2019-03-11 19:00:25 UTC; rfitzjoh
Repository CRAN
Date/Publication 2019-03-14 11:00:02 UTC

Include our badge in your README

[![Rdoc](http://www.rdocumentation.org/badges/version/thor)](http://www.rdocumentation.org/packages/thor)