summary

0th

Percentile

summary

Computes specified statistics for numeric and string columns. Available statistics are:

  • count

  • mean

  • stddev

  • min

  • max

  • arbitrary approximate percentiles specified as a percentage (eg, "75%")

If no statistics are given, this function computes count, mean, stddev, min, approximate quartiles (percentiles at 25%, 50%, and 75%), and max. This function is meant for exploratory data analysis, as we make no guarantee about the backward compatibility of the schema of the resulting Dataset. If you want to programmatically compute summary statistics, use the agg function instead.

Usage
summary(object, ...)

# S4 method for SparkDataFrame summary(object, ...)

Arguments
object

a SparkDataFrame to be summarized.

...

(optional) statistics to be computed for all columns.

Value

A SparkDataFrame.

Note

summary(SparkDataFrame) since 1.5.0

The statistics provided by summary were change in 2.3.0 use describe for previous defaults.

See Also

describe

Other SparkDataFrame functions: SparkDataFrame-class, agg(), alias(), arrange(), as.data.frame(), attach,SparkDataFrame-method, broadcast(), cache(), checkpoint(), coalesce(), collect(), colnames(), coltypes(), createOrReplaceTempView(), crossJoin(), cube(), dapplyCollect(), dapply(), describe(), dim(), distinct(), dropDuplicates(), dropna(), drop(), dtypes(), exceptAll(), except(), explain(), filter(), first(), gapplyCollect(), gapply(), getNumPartitions(), group_by(), head(), hint(), histogram(), insertInto(), intersectAll(), intersect(), isLocal(), isStreaming(), join(), limit(), localCheckpoint(), merge(), mutate(), ncol(), nrow(), persist(), printSchema(), randomSplit(), rbind(), rename(), repartitionByRange(), repartition(), rollup(), sample(), saveAsTable(), schema(), selectExpr(), select(), showDF(), show(), storageLevel(), str(), subset(), take(), toJSON(), unionByName(), union(), unpersist(), withColumn(), withWatermark(), with(), write.df(), write.jdbc(), write.json(), write.orc(), write.parquet(), write.stream(), write.text()

Aliases
  • summary
  • summary,SparkDataFrame-method
Examples
# NOT RUN {
sparkR.session()
path <- "path/to/file.json"
df <- read.json(path)
summary(df)
summary(df, "min", "25%", "75%", "max")
summary(select(df, "age", "height"))
# }
Documentation reproduced from package SparkR, version 2.4.6, License: Apache License (== 2.0)

Community examples

Looks like there are no examples yet.