Learn R Programming

R package stream - Infrastructure for Data Stream Mining

Maintainer: Michael Hahsler

Introduction

The package provides support for modeling and simulating data streams as well as an extensible framework for implementing, interfacing and experimenting with algorithms with a focus on data stream clustering tasks. The main advantage of stream is that it seamlessly integrates with the large existing infrastructure provided by R. The package provides:

  • Stream Sources: streaming from files, databases, in-memory data, URLs, pipes, socket connections and several data stream generators including dynamically streams with concept drift.
  • Stream Processing with filters (convolution, scaling, exponential moving average, …)
  • Stream Aggregation: sampling, windowing.
  • Stream Clustering: BICO, BIRCH, D-Stream, DBSTREAM, and evoStream.
  • Stream Outlier Detection based on D-Stream, DBSTREAM.
  • Stream Mining Evaluation with prequential error estimation.

Additional packages in the stream family are:

  • streamConnect: Connect stream mining components using sockets and web services.
  • streamMOA: Interface to clustering algorithms implemented in the MOA framework. The package interfaces clustering algorithms like of DenStream, ClusTree, CluStream and MCOD.
  • rEMM: Provides implementations of threshold nearest neighbor clustering (tNN) and Extensible Markov Model (EMM) for modelling temporal relationships between clusters.

To cite package ‘stream’ in publications use:

Hahsler M, Bolaños M, Forrest J (2017). “Introduction to stream: An Extensible Framework for Data Stream Clustering Research with R.” Journal of Statistical Software, 76(14), 1-50. doi:10.18637/jss.v076.i14 https://doi.org/10.18637/jss.v076.i14.

@Article{,
  title = {Introduction to {stream}: An Extensible Framework for Data Stream Clustering Research with {R}},
  author = {Michael Hahsler and Matthew Bola{\~n}os and John Forrest},
  journal = {Journal of Statistical Software},
  year = {2017},
  volume = {76},
  number = {14},
  pages = {1--50},
  doi = {10.18637/jss.v076.i14},
}

Installation

Stable CRAN version: Install from within R with

install.packages("stream")

Current development version: Install from r-universe.

install.packages("stream",
    repos = c("https://mhahsler.r-universe.dev",
              "https://cloud.r-project.org/"))

Usage

Load the package and a random data stream with 3 Gaussian clusters and 10% noise and scale the data to z-scores.

library("stream")
set.seed(2000)

stream <- DSD_Gaussians(k = 3, d = 2, noise = 0.1) %>%
    DSF_Scale()
get_points(stream, n = 5)
##       X1     X2 .class
## 1 -0.267 -0.802      2
## 2  0.531  1.078     NA
## 3 -0.706  1.427      3
## 4 -0.781  1.355      3
## 5  1.170 -0.712      1
plot(stream)

Cluster a stream of 1000 points using D-Stream which estimates point density in grid cells.

dsc <- DSC_DStream(gridsize = 0.1)
update(dsc, stream, 1000)
plot(dsc, stream, grid = TRUE)

evaluate_static(dsc, stream, n = 100)
## Evaluation results for micro-clusters.
## Points were assigned to micro-clusters.
## 
##             numPoints      numMicroClusters      numMacroClusters 
##              100.0000               65.0000                3.0000 
##        noisePredicted                   SSQ            silhouette 
##               23.0000                0.1696                0.0786 
##       average.between        average.within          max.diameter 
##                1.7809                0.5816                3.9368 
##        min.separation ave.within.cluster.ss                    g2 
##                0.0146                0.5217                0.1596 
##          pearsongamma                  dunn                 dunn2 
##                0.0637                0.0037                0.0154 
##               entropy              wb.ratio            numClasses 
##                3.1721                0.3266                4.0000 
##           noiseActual        noisePrecision        outlierJaccard 
##               16.0000                0.6957                0.6957 
##             precision                recall                    F1 
##                0.6170                0.1618                0.2563 
##                purity             Euclidean             Manhattan 
##                0.9920                0.1633                0.3000 
##                  Rand                 cRand                   NMI 
##                0.7620                0.1688                0.5551 
##                    KP                 angle                  diag 
##                0.2651                0.3000                0.3000 
##                    FM               Jaccard                    PS 
##                0.3159                0.1470                0.0541 
##                    vi 
##                2.2264

Outlier detection using DBSTREAM which uses micro-clusters with a given radius.

dso <- DSOutlier_DBSTREAM(r = 0.1)
update(dso, stream, 1000)
plot(dso, stream)

evaluate_static(dso, stream, n = 100, measure = c("numPoints", "noiseActual", "noisePredicted",
    "noisePrecision"))
## Evaluation results for micro-clusters.
## Points were assigned to micro-clusters.
## 
##      numPoints    noiseActual noisePredicted noisePrecision 
##            100              7              7              1

Preparing complete stream process pipelines that can be run using a single update() call.

pipeline <- DSD_Gaussians(k = 3, d = 2, noise = 0.1) %>%
    DSF_Scale() %>%
    DST_Runner(DSC_DStream(gridsize = 0.1))
pipeline
## DST pipline runner
## DSD: Gaussian Mixture (d = 2, k = 3)
## + scaled
## DST: D-Stream 
## Class: DST_Runner, DST
update(pipeline, n = 500)
pipeline$dst
## D-Stream 
## Class: DSC_DStream, DSC_Micro, DSC_R, DSC 
## Number of micro-clusters: 160 
## Number of macro-clusters: 13

Acknowledgments

The development of the stream package was supported in part by NSF IIS-0948893, NSF CMMI 1728612, and NIH R21HG005912.

References

Copy Link

Version

Install

install.packages('stream')

Monthly Downloads

2,126

Version

2.0.7

License

GPL-3

Issues

Pull Requests

Stars

Forks

Maintainer

Michael Hahsler

Last Published

November 10th, 2026

Functions in stream (2.0.7)

DSD

Data Stream Data Generator Base Classes
DSC_Sample

Extract a Fixed-size Sample from a Data Stream
DSC_Static

Create as Static Copy of a Clustering
DSClassifier

Abstract Class for Data Stream Classifiers
DSC_Window

A sliding window from a Data Stream
DSC_evoStream

evoStream - Evolutionary Stream Clustering
DSClassifier_SlidingWindow

DSClassifier_SlidingWindow -- Data Stream Classifier Using a Sliding Window
DSD_MG

DSD Moving Generator
DSD_Benchmark

Data Stream Generator for Dynamic Data Stream Benchmarks
DSD_BarsAndGaussians

Data Stream Generator for Bars and Gaussians
DSD_Cubes

Static Cubes Data Stream Generator
DSD_Gaussians

Mixture of Gaussians Data Stream Generator
DSD_NULL

Placeholder for a DSD Stream
DSD_ReadDB

Read a Data Stream from an open DB Query
DSD_ScaleStream

Deprecated DSD_ScaleStream
DSD_UniformNoise

Uniform Noise Data Stream Generator
DSD_Memory

A Data Stream Interface for Data Stored in Memory
DSD_mlbenchData

Stream Interface for Data Sets From mlbench
DSD_ReadStream

Read a Data Stream from a File or a Connection
DSD_Target

Target Data Stream Generator
DSD_mlbenchGenerator

mlbench Data Stream Generator
DSF_Func

Apply a Function to Transform a Data Stream
DSF_dplyr

Apply a dplyr Transformation to a Data Stream
DSF_Downsample

Downsample a Data Stream
DSD_Mixture

Mixes Data Points from Several Streams into a Single Stream
DSF_ExponentialMA

Exponential Moving Average over a Data Stream
DSF

Data Stream Filter Base Classes
DSF_Convolve

Apply a Filter to a Data Stream
DSFP

Abstract Class for Frequent Pattern Mining Algorithms for Data Streams
DSF_FeatureSelection

Select Features for a Data Stream
DSF_Scale

Scale a Data Stream
DSOutlier

Abstract Class for Data Stream Outlier Detectors
DSRegressor_SlidingWindow

DSRegressor_SlidingWindow -- Data Stream Regressor Using a Sliding Window
DST_SlidingWindow

DST_SlidingWindow -- Call R Functions on a Sliding Window
DST_WriteStream

Task to Write a Stream to a File or a Connection
DSRegressor

Abstract Class for Data Stream Regressors
MGC

Moving Generator Cluster
agreement

Agreement-based Measures for Clustering
DST_Multi

Apply Multiple Task to the Same Data Stream
animate_cluster

Animates Plots of the Clustering Process
DST

Conceptual Base Class for All Data Stream Mining Tasks
animate_data

Animates the Plotting of a Data Streams
%>%

Pipe operator
plot.DSC

Plot Results of a Data Stream Clustering
evaluate

Evaluate a Data Stream Mining Task
close_stream

Close a Data Stream
evaluate.DSC

Evaluate a Stream Clustering Task
get_assignment

prune_clusters

Prune Clusters from a Clustering
get_points

Get Points from a Data Stream Generator
plot.DSD

Plot Data Stream Data
predict

Make a Prediction for a Data Stream Mining Task
read_saveDSC

Save and Read DSC Objects
recluster

Re-clustering micro-clusters
stream_pipeline

Create a Data Stream Pipeline
stream-package

stream: Infrastructure for Data Stream Mining
reset_stream

Reset a Data Stream to its Beginning
write_stream

Write a Data Stream to a File
update

Update a Data Stream Mining Task Model with Points from a Stream
DSC_BIRCH

Balanced Iterative Reducing Clustering using Hierarchies
DSC_DBSCAN

DBSCAN Macro-clusterer
DSC_DStream

D-Stream Data Stream Clustering Algorithm
DSC_BICO

BICO - Fast computation of k-means coresets in a data stream
DSC_DBSTREAM

DBSTREAM Clustering Algorithm
DSAggregate_Window

Sliding Window (Data Stream Operator)
DSC_EA

Reclustering using an Evolutionary Algorithm
DSAggregate_Sample

Sampling from a Data Stream (Data Stream Operator)
DSC

Data Stream Clustering Base Class
DSAggregate

Data Stream Aggregator Base Classes
DSC_SlidingWindow

DSC_SlidingWindow -- Data Stream Clusterer Using a Sliding Window
DSC_Macro

Abstract Class for Macro Clusterers (Offline Component)
DSC_TwoStage

TwoStage Clustering Process
DSC_R

Abstract Class for Implementing R-based Clusterers
DSC_Reachability

Reachability Micro-Cluster Reclusterer
DSC_Kmeans

Kmeans Macro-clusterer
DSC_Hierarchical

Hierarchical Micro-Cluster Reclusterer
DSC_Micro

Abstract Class for Micro Clusterers (Online Component)