Learn R Programming

⚠️There's a newer version (1.16.2) of this package.Take me there.

data.table

data.table provides a high-performance version of base R's data.frame with syntax and feature enhancements for ease of use, convenience and programming speed.

Why data.table?

  • concise syntax: fast to type, fast to read
  • fast speed
  • memory efficient
  • careful API lifecycle management
  • community
  • feature rich

Features

  • fast and friendly delimited file reader: ?fread, see also convenience features for small data
  • fast and feature rich delimited file writer: ?fwrite
  • low-level parallelism: many common operations are internally parallelized to use multiple CPU threads
  • fast and scalable aggregations; e.g. 100GB in RAM (see benchmarks on up to two billion rows)
  • fast and feature rich joins: ordered joins (e.g. rolling forwards, backwards, nearest and limited staleness), overlapping range joins (similar to IRanges::findOverlaps), non-equi joins (i.e. joins using operators >, >=, <, <=), aggregate on join (by=.EACHI), update on join
  • fast add/update/delete columns by reference by group using no copies at all
  • fast and feature rich reshaping data: ?dcast (pivot/wider/spread) and ?melt (unpivot/longer/gather)
  • any R function from any R package can be used in queries not just the subset of functions made available by a database backend, also columns of type list are supported
  • has no dependencies at all other than base R itself, for simpler production/maintenance
  • the R dependency is as old as possible for as long as possible, dated April 2014, and we continuously test against that version; e.g. v1.11.0 released on 5 May 2018 bumped the dependency up from 5 year old R 3.0.0 to 4 year old R 3.1.0

Installation

install.packages("data.table")

# latest development version:
data.table::update.dev.pkg()

See the Installation wiki for more details.

Usage

Use data.table subset [ operator the same way you would use data.frame one, but...

  • no need to prefix each column with DT$ (like subset() and with() but built-in)
  • any R expression using any package is allowed in j argument, not just list of columns
  • extra argument by to compute j expression by group
library(data.table)
DT = as.data.table(iris)

# FROM[WHERE, SELECT, GROUP BY]
# DT  [i,     j,      by]

DT[Petal.Width > 1.0, mean(Petal.Length), by = Species]
#      Species       V1
#1: versicolor 4.362791
#2:  virginica 5.552000

Getting started

Cheatsheets

Community

data.table is widely used by the R community. It is being directly used by hundreds of CRAN and Bioconductor packages, and indirectly by thousands. It is one of the top most starred R packages on GitHub, and was highly rated by the Depsy project. If you need help, the data.table community is active on StackOverflow.

Stay up-to-date

Contributing

Guidelines for filing issues / pull requests: Contribution Guidelines.

Copy Link

Version

Install

install.packages('data.table')

Monthly Downloads

999,142

Version

1.14.2

License

MPL-2.0 | file LICENSE

Issues

Pull Requests

Stars

Forks

Maintainer

Last Published

September 27th, 2021

Functions in data.table (1.14.2)

between

Convenience functions for range subsets.
duplicated

Determine Duplicate Rows
data.table-package

Enhanced data.frame
copy

Copy an entire object
fcoalesce

Coalescing missing values
key<-

Deprecated.
dcast.data.table

Fast dcast for data.table
cdt

data.table exported C routines
data.table-class

S4 Definition for data.table
chmatch

Faster match of character vectors
datatable.optimize

Optimisations in data.table
fcase

fcase
frank

Fast rank
foverlaps

Fast overlap joins
groupingsets

Grouping Set aggregation for data tables
last

First/last item of an object
roll

Rolling functions
na.omit.data.table

Remove rows with missing values on columns specified
fread

Fast and friendly file finagler
rbindlist

Makes one data.table from a list of many
nafill

Fill missing values
print.data.table

data.table Printing Options
fsort

Fast parallel sort
melt.data.table

Fast melt for data.table
fwrite

Fast CSV writer
merge

Merge two data.tables
setDTthreads

Set or get number of threads that data.table should use
.Last.updated

Number of rows affected by last update
like

Convenience function for calling grep.
rleid

Generate run-length type group id
rowid

Generate unique row ids within each group
setorder

Fast row reordering of a data.table by reference
setops

Set operations for data tables
fifelse

Fast ifelse
setNumericRounding

Change or turn off numeric rounding
setattr

Set attributes of objects by reference
subset.data.table

Subsetting data.tables
tables

Display 'data.table' metadata
special-symbols

Special symbols
split

Split data.table into chunks in a list
shouldPrint

For use by packages that mimic/divert auto printing e.g. IRkernel and knitr
transpose

Efficient transpose of list
shift

Fast lead/lag for vectors and lists
setcolorder

Fast column reordering of a data.table by reference
setkey

Create key on a data.table
test

Test assertions for equality, exceptions and console output
truelength

Over-allocation access
test.data.table

Runs a set of tests.
timetaken

Pretty print of time taken
setDF

Coerce a data.table to data.frame by reference
patterns

Obtain matching indices corresponding to patterns
transform.data.table

Data table utilities
tstrsplit

strsplit and transpose the resulting list efficiently
setDT

Coerce lists and data.frames to data.table by reference
update.dev.pkg

Perform update of development version of a package
all.equal

Equality Test Between Two Data Tables
as.data.table.xts

Efficient xts to as.data.table conversion
as.xts.data.table

Efficient data.table to xts conversion
as.matrix

Convert a data.table to a matrix
address

Address in RAM of a variable
:=

Assignment by reference
as.data.table

Coerce to data.table
J

Creates a join data.table
IDateTime

Integer based date class