sdf_sample: Randomly Sample Rows from a Spark DataFrame
Description
Draw a random sample of rows (with or without replacement)
from a Spark DataFrame.
Usage
sdf_sample(x, fraction = 1, replacement = TRUE, seed = NULL)
Arguments
x
An object coercable to a Spark DataFrame.
fraction
The fraction to sample.
replacement
Boolean; sample with replacement?
seed
An (optional) integer seed.
Transforming Spark DataFrames
The family of functions prefixed with sdf_
generally access the Scala
Spark DataFrame API directly, as opposed to the dplyr
interface which
uses Spark SQL. These functions will 'force' any pending SQL in a
dplyr
pipeline, such that the resulting tbl_spark
object
returned will no longer have the attached 'lazy' SQL operations. Note that
the underlying Spark DataFrame does execute its operations lazily, so
that even though the pending set of operations (currently) are not exposed at
the R level, these operations will only be executed when you explicitly
collect()
the table.