mindspore.mint.nn.GLU

class mindspore.mint.nn.GLU(dim=- 1)

Computes GLU (Gated Linear Unit activation function) of the input tensor.

\[{GLU}(a, b)= a \otimes \sigma(b)\]

where \(a\) is the first half of the input Tensor after input is split and \(b\) is the second half.

Here \(\sigma\) is the sigmoid function, and \(\otimes\) is the Hadamard product. See Language Modeling with Gated Convolutional Networks .

Parameters

dim (int, optional) – The dimension to split the input input. The value range is [-r, r) where r is the number of dimensions of input. Default: -1 , the last dimension in input.

Inputs:

• input (Tensor) - Tensor to be calculated. Dtype is floating point and the shape is \((\ast_1, N, \ast_2)\) where * means, any number of additional dimensions. \(N\) is required to be an even number, where \(N\) is the size of input on the dimension selected by dim.

Outputs:

Tensor, the same dtype as the input, with the shape \((\ast_1, M, \ast_2)\) where \(M=N/2\).

Raises

• IndexError – If the value of dim is out of the range of [-r, r), where r is the number of dimensions of input.

• RuntimeError – If dtype of input is not supported.

• RuntimeError – If the length of input in the dimension selected by dim is not even.

Supported Platforms:

Ascend

Examples

>>> import mindspore
>>> glu = mindspore.mint.nn.GLU()
>>> input = mindspore.tensor([[0.1, 0.2, 0.3, 0.4], [0.5, 0.6, 0.7, 0.8]])
>>> output = glu(input)
>>> print(output)
[[0.05744425 0.11973753]
 [0.33409387 0.41398472]]