sparklyr (version 1.3.0)

stream_generate_test: Generate Test Stream

Description

Generates a local test stream, useful when testing streams locally.

Usage

stream_generate_test(
  df = rep(1:1000),
  path = "source",
  distribution = floor(10 + 1e+05 * stats::dbinom(1:20, 20, 0.5)),
  iterations = 50,
  interval = 1
)

Arguments

df

The data frame used as a source of rows to the stream, will be cast to data frame if needed. Defaults to a sequence of one thousand entries.

path

Path to save stream of files to, defaults to "source".

distribution

The distribution of rows to use over each iteration, defaults to a binomial distribution. The stream will cycle through the distribution if needed.

iterations

Number of iterations to execute before stopping, defaults to fifty.

interval

The inverval in seconds use to write the stream, defaults to one second.

Details

This function requires the callr package to be installed.