stream_watermark

0th

Percentile

Watermark Stream

Ensures a stream has a watermark defined, which is required for some operations over streams.

Usage
stream_watermark(x, column = "timestamp", threshold = "10 minutes")
Arguments
x

An object coercable to a Spark Streaming DataFrame.

column

The name of the column that contains the event time of the row, if the column is missing, a column with the current time will be added.

threshold

The minimum delay to wait to data to arrive late, defaults to ten minutes.

Aliases
  • stream_watermark
Documentation reproduced from package sparklyr, version 1.0.2, License: Apache License 2.0 | file LICENSE

Community examples

Looks like there are no examples yet.