Learn R Programming

⚠️There's a newer version (1.16.4) of this package.Take me there.

data.table

data.table provides a high-performance version of base R's data.frame with syntax and feature enhancements for ease of use, convenience and programming speed.

Why data.table?

  • concise syntax: fast to type, fast to read
  • fast speed
  • memory efficient
  • careful API lifecycle management
  • community
  • feature rich

Features

  • fast and friendly delimited file reader: ?fread, see also convenience features for small data
  • fast and feature rich delimited file writer: ?fwrite
  • low-level parallelism: many common operations are internally parallelized to use multiple CPU threads
  • fast and scalable aggregations; e.g. 100GB in RAM (see benchmarks on up to two billion rows)
  • fast and feature rich joins: ordered joins (e.g. rolling forwards, backwards, nearest and limited staleness), overlapping range joins (similar to IRanges::findOverlaps), non-equi joins (i.e. joins using operators >, >=, <, <=), aggregate on join (by=.EACHI), update on join
  • fast add/update/delete columns by reference by group using no copies at all
  • fast and feature rich reshaping data: ?dcast (pivot/wider/spread) and ?melt (unpivot/longer/gather)
  • any R function from any R package can be used in queries not just the subset of functions made available by a database backend, also columns of type list are supported
  • has no dependencies at all other than base R itself, for simpler production/maintenance
  • the R dependency is as old as possible for as long as possible, dated April 2014, and we continuously test against that version; e.g. v1.11.0 released on 5 May 2018 bumped the dependency up from 5 year old R 3.0.0 to 4 year old R 3.1.0

Installation

install.packages("data.table")

# latest development version:
data.table::update.dev.pkg()

See the Installation wiki for more details.

Usage

Use data.table subset [ operator the same way you would use data.frame one, but...

  • no need to prefix each column with DT$ (like subset() and with() but built-in)
  • any R expression using any package is allowed in j argument, not just list of columns
  • extra argument by to compute j expression by group
library(data.table)
DT = as.data.table(iris)

# FROM[WHERE, SELECT, GROUP BY]
# DT  [i,     j,      by]

DT[Petal.Width > 1.0, mean(Petal.Length), by = Species]
#      Species       V1
#1: versicolor 4.362791
#2:  virginica 5.552000

Getting started

Cheatsheets

Community

data.table is widely used by the R community. It is being directly used by hundreds of CRAN and Bioconductor packages, and indirectly by thousands. It is one of the top most starred R package on GitHub. If you need help, the data.table community is active on StackOverflow.

Stay up-to-date

Contributing

Guidelines for filing issues / pull requests: Contribution Guidelines.

Copy Link

Version

Install

install.packages('data.table')

Monthly Downloads

999,142

Version

1.13.2

License

MPL-2.0 | file LICENSE

Issues

Pull Requests

Stars

Forks

Maintainer

Last Published

October 19th, 2020

Functions in data.table (1.13.2)

as.xts.data.table

Efficient data.table to xts conversion
:=

Assignment by reference
all.equal

Equality Test Between Two Data Tables
address

Address in RAM of a variable
J

Creates a join data.table
IDateTime

Integer based date class
as.data.table

Coerce to data.table
as.matrix

Convert a data.table to a matrix
as.data.table.xts

Efficient xts to as.data.table conversion
cdt

data.table exported C routines
chmatch

Faster match of character vectors
between

Convenience functions for range subsets.
datatable.optimize

Optimisations in data.table
dcast.data.table

Fast dcast for data.table
data.table-class

S4 Definition for data.table
fcoalesce

Coalescing missing values
copy

Copy an entire object
fcase

fcase
fifelse

Fast ifelse
fsort

Fast parallel sort
fwrite

Fast CSV writer
frank

Fast rank
foverlaps

Fast overlap joins
na.omit.data.table

Remove rows with missing values on columns specified
nafill

Fill missing values
data.table-package

Enhanced data.frame
roll

Rolling functions
fread

Fast and friendly file finagler
key<-

Deprecated.
duplicated

Determine Duplicate Rows
last

First/last item of an object
groupingsets

Grouping Set aggregation for data tables
.Last.updated

Number of rows affected by last update
patterns

Obtain matching indices corresponding to patterns
setDF

Coerce a data.table to data.frame by reference
rleid

Generate run-length type group id
rowid

Generate unique row ids within each group
melt.data.table

Fast melt for data.table
merge

Merge two data.tables
setDTthreads

Set or get number of threads that data.table should use
test

Test assertions for equality, exceptions and console output
setDT

Coerce lists and data.frames to data.table by reference
print.data.table

data.table Printing Options
test.data.table

Runs a set of tests.
setcolorder

Fast column reordering of a data.table by reference
transform.data.table

Data table utilities
timetaken

Pretty print of time taken
setattr

Set attributes of objects by reference
setNumericRounding

Change or turn off numeric rounding
subset.data.table

Subsetting data.tables
like

Convenience function for calling grep.
setkey

Create key on a data.table
shouldPrint

For use by packages that mimic/divert auto printing e.g. IRkernel and knitr
tables

Display 'data.table' metadata
tstrsplit

strsplit and transpose the resulting list efficiently
shift

Fast lead/lag for vectors and lists
update.dev.pkg

Perform update of development version of a package
transpose

Efficient transpose of list
truelength

Over-allocation access
rbindlist

Makes one data.table from a list of many
setops

Set operations for data tables
setorder

Fast row reordering of a data.table by reference
split

Split data.table into chunks in a list
special-symbols

Special symbols