sdf_schema
From sparklyr v1.5.1
by Yitao Li
Read the Schema of a Spark DataFrame
Read the schema of a Spark DataFrame.
Usage
sdf_schema(x, expand_nested_cols = FALSE)
Arguments
- x
A
spark_connection
,ml_pipeline
, or atbl_spark
.- expand_nested_cols
Whether to expand columns containing nested array of structs (which are usually created by tidyr::nest on a Spark data frame)
Details
The type
column returned gives the string representation of the
underlying Spark type for that column; for example, a vector of numeric
values would be returned with the type "DoubleType"
. Please see the
Spark Scala API Documentation
for information on what types are available and exposed by Spark.
Value
An R list
, with each list
element describing the
name
and type
of a column.
Community examples
Looks like there are no examples yet.