Learn R Programming

⚠️There's a newer version (1.16.0) of this package.Take me there.

data.table

data.table provides a high-performance version of base R's data.frame with syntax and feature enhancements for ease of use, convenience and programming speed.

Why data.table?

  • concise syntax: fast to type, fast to read
  • fast speed
  • memory efficient
  • careful API lifecycle management
  • community
  • feature rich

Features

  • fast and friendly delimited file reader: ?fread, see also convenience features for small data
  • fast and feature rich delimited file writer: ?fwrite
  • low-level parallelism: many common operations are internally parallelized to use multiple CPU threads
  • fast and scalable aggregations; e.g. 100GB in RAM (see benchmarks on up to two billion rows)
  • fast and feature rich joins: ordered joins (e.g. rolling forwards, backwards, nearest and limited staleness), overlapping range joins (similar to IRanges::findOverlaps), non-equi joins (i.e. joins using operators >, >=, <, <=), aggregate on join (by=.EACHI), update on join
  • fast add/update/delete columns by reference by group using no copies at all
  • fast and feature rich reshaping data: ?dcast (pivot/wider/spread) and ?melt (unpivot/longer/gather)
  • any R function from any R package can be used in queries not just the subset of functions made available by a database backend, also columns of type list are supported
  • has no dependencies at all other than base R itself, for simpler production/maintenance
  • the R dependency is as old as possible for as long as possible, dated April 2014, and we continuously test against that version; e.g. v1.11.0 released on 5 May 2018 bumped the dependency up from 5 year old R 3.0.0 to 4 year old R 3.1.0

Installation

install.packages("data.table")

# latest development version that has passed all tests:
data.table::update_dev_pkg()

See the Installation wiki for more details.

Usage

Use data.table subset [ operator the same way you would use data.frame one, but...

  • no need to prefix each column with DT$ (like subset() and with() but built-in)
  • any R expression using any package is allowed in j argument, not just list of columns
  • extra argument by to compute j expression by group
library(data.table)
DT = as.data.table(iris)

# FROM[WHERE, SELECT, GROUP BY]
# DT  [i,     j,      by]

DT[Petal.Width > 1.0, mean(Petal.Length), by = Species]
#      Species       V1
#1: versicolor 4.362791
#2:  virginica 5.552000

Getting started

Cheatsheets

Community

data.table is widely used by the R community. It is being directly used by hundreds of CRAN and Bioconductor packages, and indirectly by thousands. It is one of the top most starred R packages on GitHub, and was highly rated by the Depsy project. If you need help, the data.table community is active on StackOverflow.

Stay up-to-date

Contributing

Guidelines for filing issues / pull requests: Contribution Guidelines.

Copy Link

Version

Install

install.packages('data.table')

Monthly Downloads

891,324

Version

1.14.6

License

MPL-2.0 | file LICENSE

Issues

Pull Requests

Stars

Forks

Maintainer

Last Published

November 16th, 2022

Functions in data.table (1.14.6)

J

Creates a join data.table
address

Address in RAM of a variable
:=

Assignment by reference
IDateTime

Integer based date class
datatable.optimize

Optimisations in data.table
cdt

data.table exported C routines
fcoalesce

Coalescing missing values
key<-

Deprecated.
duplicated

Determine Duplicate Rows
chmatch

Faster match of character vectors
data.table-package

Enhanced data.frame
dcast.data.table

Fast dcast for data.table
copy

Copy an entire object
data.table-class

S4 Definition for data.table
fcase

fcase
fsort

Fast parallel sort
groupingsets

Grouping Set aggregation for data tables
frank

Fast rank
last

First/last item of an object
roll

Rolling functions
fwrite

Fast CSV writer
fread

Fast and friendly file finagler
fifelse

Fast ifelse
foverlaps

Fast overlap joins
.Last.updated

Number of rows affected by last update
print.data.table

data.table Printing Options
rbindlist

Makes one data.table from a list of many
setDTthreads

Set or get number of threads that data.table should use
nafill

Fill missing values
melt.data.table

Fast melt for data.table
merge

Merge two data.tables
na.omit.data.table

Remove rows with missing values on columns specified
like

Convenience function for calling grep.
patterns

Obtain matching indices corresponding to patterns
setDT

Coerce lists and data.frames to data.table by reference
setNumericRounding

Change or turn off numeric rounding
setops

Set operations for data tables
setcolorder

Fast column reordering of a data.table by reference
setorder

Fast row reordering of a data.table by reference
setDF

Coerce a data.table to data.frame by reference
setattr

Set attributes of objects by reference
test

Test assertions for equality, exceptions and console output
shift

Fast lead/lag for vectors and lists
test.data.table

Runs a set of tests.
special-symbols

Special symbols
split

Split data.table into chunks in a list
subset.data.table

Subsetting data.tables
tables

Display 'data.table' metadata
setkey

Create key on a data.table
rleid

Generate run-length type group id
transpose

Efficient transpose of list
timetaken

Pretty print of time taken
tstrsplit

strsplit and transpose the resulting list efficiently
update_dev_pkg

Perform update of development version of a package
rowid

Generate unique row ids within each group
shouldPrint

For use by packages that mimic/divert auto printing e.g. IRkernel and knitr
transform.data.table

Data table utilities
truelength

Over-allocation access
as.xts.data.table

Efficient data.table to xts conversion
all.equal

Equality Test Between Two Data Tables
as.matrix

Convert a data.table to a matrix
between

Convenience functions for range subsets.
as.data.table.xts

Efficient xts to as.data.table conversion
as.data.table

Coerce to data.table