mvl_open: Open an MVL file

Description

Open an MVL format file for reading and/or writing.

Usage

mvl_open(filename, append = FALSE, create = FALSE)

Value

handle to opened MVL file

Arguments

filename: path to file.
append: specify TRUE when you intend to write data into the file
create: when TRUE create file if it did not exist

Details

MVL stands for "Mapped vector library" and is a file format designed for efficient memory mapped access. An MVL file can be much larger than physical memory of the machine.

mvl_open returns a handle that can be used to access MVL files. Files opened read-only are memory mapped and do not use a file descriptor, and thus are not subject to limits on the number of open files. Files opened for writing data do use a file descriptor. Once opened for read access the data can be accessed using usual R semantics for lists, data.frames and arrays.

Examples

Run this code

if (FALSE) {
M1<-mvl_open("test1.mvl", append=TRUE, create=TRUE)
mvl_write_object(M1, data.frame(x=1:2, y=rnorm(2)), "test_frame")
mvl_close(M1)

M2<-mvl_open("test1.mvl")
print(names(M2))
print(M2["test_frame"])
mvl_close(M2)

M3<-mvl_open("test2.mvl", append=TRUE, create=TRUE)
L<-list()
df<-data.frame(x=1:1e6, y=rnorm(1e6), s=rep(c("a", "b"), 5e5))
L[["x"]]<-mvl_write_object(M3, df, drop.rownames=TRUE)
L[["description"]]<-"Example of large data frame"
mvl_write_object(M3, L, "test_object")
mvl_close(M3)

M4<-mvl_open("test2.mvl")
print(names(M4))
L<-M4["test_object"]
print(L)
print(L[["x"]][1:20,])
mvl_object_stats(L[["x"]])
# If you need to get the whole x, one can use mvl2R(L[["x"]])
mvl_close(M4)
}