ft_standard_scaler
Feature Tranformation -- StandardScaler (Estimator)
Standardizes features by removing the mean and scaling to unit variance using column summary statistics on the samples in the training set. The "unit std" is computed using the corrected sample standard deviation, which is computed as the square root of the unbiased sample variance.
Usage
ft_standard_scaler(x, input_col, output_col, with_mean = FALSE,
with_std = TRUE, dataset = NULL,
uid = random_string("standard_scaler_"), ...)
Arguments
- x
A
spark_connection
,ml_pipeline
, or atbl_spark
.- input_col
The name of the input column.
- output_col
The name of the output column.
- with_mean
Whether to center the data with mean before scaling. It will build a dense output, so take care when applying to sparse input. Default: FALSE
- with_std
Whether to scale the data to unit standard deviation. Default: TRUE
- dataset
(Optional) A
tbl_spark
. If provided, eagerly fit the (estimator) feature "transformer" againstdataset
. See details.- uid
A character string used to uniquely identify the feature transformer.
- ...
Optional arguments; currently unused.
Details
When dataset
is provided for an estimator transformer, the function
internally calls ml_fit()
against dataset
. Hence, the methods for
spark_connection
and ml_pipeline
will then return a ml_transformer
and a ml_pipeline
with a ml_transformer
appended, respectively. When
x
is a tbl_spark
, the estimator will be fit against dataset
before
transforming x
.
When dataset
is not specified, the constructor returns a ml_estimator
, and,
in the case where x
is a tbl_spark
, the estimator fits against x
then
to obtain a transformer, which is then immediately used to transform x
.
Value
The object returned depends on the class of x
.
spark_connection
: Whenx
is aspark_connection
, the function returns aml_transformer
, aml_estimator
, or one of their subclasses. The object contains a pointer to a SparkTransformer
orEstimator
object and can be used to composePipeline
objects.ml_pipeline
: Whenx
is aml_pipeline
, the function returns aml_pipeline
with the transformer or estimator appended to the pipeline.tbl_spark
: Whenx
is atbl_spark
, a transformer is constructed then immediately applied to the inputtbl_spark
, returning atbl_spark
See Also
See http://spark.apache.org/docs/latest/ml-features.html for more information on the set of transformations available for DataFrame columns in Spark.
Other feature transformers: ft_binarizer
,
ft_bucketizer
,
ft_chisq_selector
,
ft_count_vectorizer
, ft_dct
,
ft_elementwise_product
,
ft_feature_hasher
,
ft_hashing_tf
, ft_idf
,
ft_imputer
,
ft_index_to_string
,
ft_interaction
, ft_lsh
,
ft_max_abs_scaler
,
ft_min_max_scaler
, ft_ngram
,
ft_normalizer
,
ft_one_hot_encoder
, ft_pca
,
ft_polynomial_expansion
,
ft_quantile_discretizer
,
ft_r_formula
,
ft_regex_tokenizer
,
ft_sql_transformer
,
ft_stop_words_remover
,
ft_string_indexer
,
ft_tokenizer
,
ft_vector_assembler
,
ft_vector_indexer
,
ft_vector_slicer
, ft_word2vec