Parameter

Ascend GPU CPU Beginner

 

Parameter is a variable tensor, indicating the parameters that need to be updated during network training. Typically, it consists of weight and bias.

For example, the weight and bias for nn.Conv2d is a tensor with the shape of [out_channel, in_channel, kernel_size, kernel_size] and a scalar with the shape of [out_channel], respectively. For nn.Dense, the shapes are [out_channel, in_channel] and [out_channel].

import numpy as np
from mindspore import Tensor
from mindspore import nn

conv_net = nn.Conv2d(in_channels=120, out_channels=240, kernel_size=4, has_bias=True, weight_init='normal')
dense_net = nn.Dense(in_channels=3, out_channels=4, weight_init='normal', bias_init='zeros', has_bias=True)

print("conv_net, weight shape:", conv_net.weight.shape, ", bias shape:", conv_net.bias.shape)
print("dense_net, weight_shape：", dense_net.weight.shape, ", bias shape:", dense_net.bias.shape)

conv_net, weight shape: (240, 120, 4, 4) , bias shape:  (240,)
dense_net, weight_shape: (4, 3) , bias shape:  (4,)

Usually, weight and bias are integrated in a nn operator, users can specify a kind of initialization method to initialize these parameters.

During training, the optimizer will continually update the parameters according to the backward propagating gradients. The optimal parameters will be finally saved for evaluation usage.

Parameter does not support sparse Tensor, such as mindspore.RowTensor and mindspore.SparseTensor. More information about Parameter initializing and updating can refer to Parameter initialize and Parameter update.