mindspore.nn.InstanceNorm2d

class mindspore.nn.InstanceNorm2d(num_features, eps=1e-05, momentum=0.1, affine=True, gamma_init='ones', beta_init='zeros', moving_mean_init='zeros', moving_var_init='ones', use_batch_statistics=True)[source]

Instance Normalization layer over a 4D input.

This layer applies Instance Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension) as described in the paper Instance Normalization: The Missing Ingredient for Fast Stylization. It rescales and recenters the feature using a mini-batch of data and the learned parameters which can be described in the following formula.

\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]

gamma and beta are learnable parameter vectors of size num_features if affine is True. The standard-deviation is calculated via the biased estimator.

By default, this layer uses instance statistics computed from input data in both training and evaluation modes.

If use_batch_statistics is set to True, it means training phases, and this layer keeps running estimates of its computed mean and variance, which are then used for normalization during evaluation. The running estimates are kept with a default momentum of 0.1.

InstanceNorm2d and BatchNorm2d are very similar, but have some differences. InstanceNorm2d is applied on each channel of channeled data like RGB images, but BatchNorm2d is usually applied on each batch of batched data.

Note

Note that the formula for updating the running_mean and running_var is \(\hat{x}_\text{new} = (1 - \text{momentum}) \times x_t + \text{momentum} \times \hat{x}\), where \(\hat{x}\) is the estimated statistic and \(x_t\) is the new observed value.

Parameters

  • num_features (int) – C from an expected input of size (N, C, H, W).

  • eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.

  • momentum (float) – A floating hyperparameter of the momentum for the running_mean and running_var computation. Default: 0.1.

  • affine (bool) – A bool value. When set to True, gamma and beta can be learned. Default: True.

  • gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘ones’.

  • beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘zeros’.

  • moving_mean_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving mean. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘zeros’.

  • moving_var_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving variance. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘ones’.

  • use_batch_statistics (bool) – If true, use the mean value and variance value of current batch data. If false, use the mean value and variance value of specified value. Default: True.

Inputs:

  • input (Tensor) - Tensor of shape \((N, C, H, W)\). Data type: float16 or float32.

Outputs:

Tensor, the normalized, scaled, offset tensor, of shape \((N, C, H, W)\). Same type and shape as the input_x.

Supported Platforms:

GPU

Raises

  • TypeError – If num_features is not an int.

  • TypeError – If eps is not a float.

  • TypeError – If momentum is not a float.

  • TypeError – If affine is not a bool.

  • TypeError – If the type of gamma_init/beta_init/moving_mean_init/moving_var_init is not same, or if the initialized element type is not float32.

  • ValueError – If num_features is less than 1.

  • ValueError – If momentum is not in range [0, 1].

  • KeyError – If any of gamma_init/beta_init/moving_mean_init/moving_var_init is str and the homonymous class inheriting from Initializer not exists.

Examples

>>> import mindspore
>>> import numpy as np
>>> import mindspore.nn as nn
>>> from mindspore import Tensor
>>> net = nn.InstanceNorm2d(3)
>>> input_tensor = Tensor(np.ones([2, 3, 2, 2]), mindspore.float32)
>>> output = net(input_tensor)
>>> print(output.shape)
(2, 3, 2, 2)