mindspore.mint.nn.functional.glu

View Source On Gitee
mindspore.mint.nn.functional.glu(input, dim=- 1)[source]

Computes GLU (Gated Linear Unit activation function) of the input tensor.

\[{GLU}(a, b)= a \otimes \sigma(b)\]

where \(a\) is the first half of the input Tensor after input is split and \(b\) is the second half.

Here \(\sigma\) is the sigmoid function, and \(\otimes\) is the Hadamard product. See Language Modeling with Gated Convluational Networks.

Parameters

  • input (Tensor) – Tensor to be calculated. Dtype is floating point and the shape is \((\ast_1, N, \ast_2)\) where * means, any number of additional dimensions. \(N\) is required to be an even number, where \(N\) is the size of input on the dimension selected by dim.

  • dim (int, optional) – The dimension to split the input input. The value range is [-r, r) where r is the number of dimensions of input. Default: -1 , the last dimension in input.

Returns

Tensor, the same dtype as the input input. The shape is \((\ast_1, M, \ast_2)\) where \(M=N/2\).

Raises

  • TypeError – If input is not a Tensor or dim is not an int.

  • IndexError – If the value of dim is out of the range of [-r, r), where r is the number of dimensions of input.

  • RuntimeError – If dtype of input is not supported.

  • RuntimeError – If the length of input in the dimension selected by dim is not even.

Supported Platforms:

Ascend

Examples

>>> from mindspore import Tensor, mint
>>> input = Tensor([[0.1, 0.2, 0.3, 0.4], [0.5, 0.6, 0.7, 0.8]])
>>> output = mint.nn.functional.glu(input)
>>> print(output)
[[0.05744425 0.11973753]
 [0.33409387 0.41398472]]