unionByName

0th

Percentile

Return a new SparkDataFrame containing the union of rows, matched by column names

Return a new SparkDataFrame containing the union of rows in this SparkDataFrame and another SparkDataFrame. This is different from union function, and both UNION ALL and UNION DISTINCT in SQL as column positions are not taken into account. Input SparkDataFrames can have different data types in the schema.

Usage
unionByName(x, y)

# S4 method for SparkDataFrame,SparkDataFrame unionByName(x, y)

Arguments
x

A SparkDataFrame

y

A SparkDataFrame

Details

Note: This does not remove duplicate rows across the two SparkDataFrames. This function resolves columns by name (not by position).

Value

A SparkDataFrame containing the result of the union.

Note

unionByName since 2.3.0

See Also

rbind union

Other SparkDataFrame functions: SparkDataFrame-class, agg(), alias(), arrange(), as.data.frame(), attach,SparkDataFrame-method, broadcast(), cache(), checkpoint(), coalesce(), collect(), colnames(), coltypes(), createOrReplaceTempView(), crossJoin(), cube(), dapplyCollect(), dapply(), describe(), dim(), distinct(), dropDuplicates(), dropna(), drop(), dtypes(), exceptAll(), except(), explain(), filter(), first(), gapplyCollect(), gapply(), getNumPartitions(), group_by(), head(), hint(), histogram(), insertInto(), intersectAll(), intersect(), isLocal(), isStreaming(), join(), limit(), localCheckpoint(), merge(), mutate(), ncol(), nrow(), persist(), printSchema(), randomSplit(), rbind(), rename(), repartitionByRange(), repartition(), rollup(), sample(), saveAsTable(), schema(), selectExpr(), select(), showDF(), show(), storageLevel(), str(), subset(), summary(), take(), toJSON(), union(), unpersist(), withColumn(), withWatermark(), with(), write.df(), write.jdbc(), write.json(), write.orc(), write.parquet(), write.stream(), write.text()

Aliases
  • unionByName
  • unionByName,SparkDataFrame,SparkDataFrame-method
Examples
# NOT RUN {
sparkR.session()
df1 <- select(createDataFrame(mtcars), "carb", "am", "gear")
df2 <- select(createDataFrame(mtcars), "am", "gear", "carb")
head(unionByName(df1, df2))
# }
Documentation reproduced from package SparkR, version 2.4.6, License: Apache License (== 2.0)

Community examples

Looks like there are no examples yet.