RBERT (version 0.1.11)

layer_norm: Run layer normalization

Description

Run layer normalization on the last dimension of the tensor.

Usage

layer_norm(input_tensor, name = NULL)

Arguments

input_tensor

Tensor to perform layor normalization on.

name

Optional variable_scope for layer_norm.

Value

A Tensor of the same shape and type as `input_tensor`, with normalization applied.

Details

Wrapper around tensorflow layer_norm function. From tensorflow documentation: Adds a Layer Normalization layer. Based on the paper: https://arxiv.org/abs/1607.06450.

Note: begin_norm_axis: The first normalization dimension: normalization will be performed along dimensions (begin_norm_axis : rank(inputs) )

begin_params_axis: The first parameter (beta, gamma) dimension: scale and centering parameters will have dimensions (begin_params_axis : rank(inputs) ) and will be broadcast with the normalized inputs accordingly.

Examples

# NOT RUN {
tfx <- tensorflow::tf$get_variable("example", tensorflow::shape(10L))
layer_norm(tfx)
# }