corr

0th

Percentile

corr

Computes the Pearson Correlation Coefficient for two Columns.

Calculates the correlation of two columns of a SparkDataFrame. Currently only supports the Pearson Correlation Coefficient. For Spearman Correlation, consider using RDD methods found in MLlib's Statistics.

Usage
corr(x, ...)

# S4 method for Column corr(x, col2)

# S4 method for SparkDataFrame corr(x, colName1, colName2, method = "pearson")

Arguments
x

a Column or a SparkDataFrame.

...

additional argument(s). If x is a Column, a Column should be provided. If x is a SparkDataFrame, two column names should be provided.

col2

a (second) Column.

colName1

the name of the first column

colName2

the name of the second column

method

Optional. A character specifying the method for calculating the correlation. only "pearson" is allowed now.

Value

The Pearson Correlation Coefficient as a Double.

Note

corr since 1.6.0

corr since 1.6.0

See Also

Other math_funcs: acos, asin, atan2, atan, bin, bround, cbrt, ceil, conv, cosh, cos, covar_pop, cov, expm1, exp, factorial, floor, hex, hypot, log10, log1p, log2, log, pmod, rint, round, shiftLeft, shiftRightUnsigned, shiftRight, signum, sinh, sin, sqrt, tanh, tan, toDegrees, toRadians, unhex

Other stat functions: approxQuantile, cov, crosstab, freqItems, sampleBy

Aliases
  • corr
  • corr
  • corr,Column-method
  • corr
  • corr,SparkDataFrame-method
Examples
# NOT RUN {
corr(df$c, df$d)
# }
# NOT RUN {
df <- read.json("/path/to/file.json")
corr <- corr(df, "title", "gender")
corr <- corr(df, "title", "gender", method = "pearson")
# }
Documentation reproduced from package SparkR, version 2.1.2, License: Apache License (== 2.0)

Community examples

Looks like there are no examples yet.