mindscience.models.layers.MaskedLayerNorm

class mindscience.models.layers.MaskedLayerNorm[source]

Masked layer normalization. Applies layer normalization with mask to the input tensor.

Inputs:

  • act (Tensor) - Tensor of shape \((*, in\_channels)\).

  • gamma (Tensor) - Scale parameter of shape \((in\_channels,)\).

  • beta (Tensor) - Offset parameter of shape \((in\_channels,)\).

  • mask (Tensor, optional) - Mask tensor of shape \((*, 1)\). Default: None.

Outputs:

Tensor of shape \((*, in\_channels)\).

Examples

>>> import mindspore as ms
>>> import mindspore.numpy as mnp
>>> from mindspore import Tensor
>>> from mindscience.models.layers import MaskedLayerNorm
>>> ms.set_context(mode=ms.GRAPH_MODE, device_target="GPU")
>>> x = Tensor(mnp.random.randn(2, 3, 4).astype(mnp.float32))
>>> gamma = Tensor(mnp.ones((4,)).astype(mnp.float32))
>>> beta = Tensor(mnp.zeros((4,)).astype(mnp.float32))
>>> mask = Tensor(mnp.ones((2, 3)).astype(mnp.float32))
>>> net = MaskedLayerNorm()
>>> output = net(x, gamma, beta, mask)
>>> print(output.shape)
(2, 3, 4)