union

0th

Percentile

Return a new SparkDataFrame containing the union of rows

Return a new SparkDataFrame containing the union of rows in this SparkDataFrame and another SparkDataFrame. This is equivalent to UNION ALL in SQL. Input SparkDataFrames can have different schemas (names and data types).

Usage
union(x, y)

unionAll(x, y)

# S4 method for SparkDataFrame,SparkDataFrame union(x, y)

# S4 method for SparkDataFrame,SparkDataFrame unionAll(x, y)

Arguments
x

A SparkDataFrame

y

A SparkDataFrame

Details

Note: This does not remove duplicate rows across the two SparkDataFrames. Also as standard in SQL, this function resolves columns by position (not by name).

Value

A SparkDataFrame containing the result of the union.

Note

union since 2.0.0

unionAll since 1.4.0

See Also

rbind unionByName

Other SparkDataFrame functions: SparkDataFrame-class, agg(), alias(), arrange(), as.data.frame(), attach,SparkDataFrame-method, broadcast(), cache(), checkpoint(), coalesce(), collect(), colnames(), coltypes(), createOrReplaceTempView(), crossJoin(), cube(), dapplyCollect(), dapply(), describe(), dim(), distinct(), dropDuplicates(), dropna(), drop(), dtypes(), exceptAll(), except(), explain(), filter(), first(), gapplyCollect(), gapply(), getNumPartitions(), group_by(), head(), hint(), histogram(), insertInto(), intersectAll(), intersect(), isLocal(), isStreaming(), join(), limit(), localCheckpoint(), merge(), mutate(), ncol(), nrow(), persist(), printSchema(), randomSplit(), rbind(), rename(), repartitionByRange(), repartition(), rollup(), sample(), saveAsTable(), schema(), selectExpr(), select(), showDF(), show(), storageLevel(), str(), subset(), summary(), take(), toJSON(), unionByName(), unpersist(), withColumn(), withWatermark(), with(), write.df(), write.jdbc(), write.json(), write.orc(), write.parquet(), write.stream(), write.text()

Aliases
  • union
  • unionAll
  • union,SparkDataFrame,SparkDataFrame-method
  • unionAll,SparkDataFrame,SparkDataFrame-method
Examples
# NOT RUN {
sparkR.session()
df1 <- read.json(path)
df2 <- read.json(path2)
unioned <- union(df, df2)
unions <- rbind(df, df2, df3, df4)
# }
Documentation reproduced from package SparkR, version 2.4.6, License: Apache License (== 2.0)

Community examples

Looks like there are no examples yet.