torch (version 0.2.0)

nn_gelu: GELU module

Description

Applies the Gaussian Error Linear Units function: $$\text{GELU}(x)=x\ast \mathrm{\Phi }(x)$$

Usage

nn_gelu()

Shape

• Input: $(N,\ast )$ where * means, any number of additional dimensions

• Output: $(N,\ast )$, same shape as the input

Details

where $\mathrm{\Phi }(x)$ is the Cumulative Distribution Function for Gaussian Distribution.

Examples

# NOT RUN {
if (torch_is_installed()) {
m = nn_gelu()
input <- torch_randn(2)
output <- m(input)

}
# }