spark_apply

0th

Percentile

Apply an R Function in Spark

Applies an R function to a Spark object (typically, a Spark DataFrame).

Usage
spark_apply(x, f, names = colnames(x), memory = TRUE, group_by = NULL,
  packages = TRUE, ...)
Arguments
x

An object (usually a spark_tbl) coercable to a Spark DataFrame.

f

A function that transforms a data frame partition into a data frame.

names

The column names for the transformed object, defaults to the names from the original object.

memory

Boolean; should the table be cached into memory?

group_by

Column name used to group by data frame partitions.

packages

Boolean; distribute .libPaths() packages to nodes?

...

Optional arguments; currently unused.

Aliases
  • spark_apply
Documentation reproduced from package sparklyr, version 0.6.0, License: Apache License 2.0 | file LICENSE

Community examples

Looks like there are no examples yet.