SparkR (version 3.1.2)

union: Return a new SparkDataFrame containing the union of rows

Description

Return a new SparkDataFrame containing the union of rows in this SparkDataFrame and another SparkDataFrame. This is equivalent to UNION ALL in SQL. Input SparkDataFrames can have different schemas (names and data types).

Usage

union(x, y)
# S4 method for SparkDataFrame,SparkDataFrame
union(x, y)

Arguments

x

A SparkDataFrame

y

A SparkDataFrame

Value

A SparkDataFrame containing the result of the union.

Details

Note: This does not remove duplicate rows across the two SparkDataFrames. Also as standard in SQL, this function resolves columns by position (not by name).

See Also

rbind unionByName

Other SparkDataFrame functions: SparkDataFrame-class, agg(), alias(), arrange(), as.data.frame(), attach,SparkDataFrame-method, broadcast(), cache(), checkpoint(), coalesce(), collect(), colnames(), coltypes(), createOrReplaceTempView(), crossJoin(), cube(), dapplyCollect(), dapply(), describe(), dim(), distinct(), dropDuplicates(), dropna(), drop(), dtypes(), exceptAll(), except(), explain(), filter(), first(), gapplyCollect(), gapply(), getNumPartitions(), group_by(), head(), hint(), histogram(), insertInto(), intersectAll(), intersect(), isLocal(), isStreaming(), join(), limit(), localCheckpoint(), merge(), mutate(), ncol(), nrow(), persist(), printSchema(), randomSplit(), rbind(), rename(), repartitionByRange(), repartition(), rollup(), sample(), saveAsTable(), schema(), selectExpr(), select(), showDF(), show(), storageLevel(), str(), subset(), summary(), take(), toJSON(), unionAll(), unionByName(), unpersist(), withColumn(), withWatermark(), with(), write.df(), write.jdbc(), write.json(), write.orc(), write.parquet(), write.stream(), write.text()

Examples

# NOT RUN {
sparkR.session()
df1 <- read.json(path)
df2 <- read.json(path2)
unioned <- union(df, df2)
unions <- rbind(df, df2, df3, df4)
# }

Run the code above in your browser using DataCamp Workspace