mindspore.nn.Cell

class mindspore.nn.Cell(auto_prefix=True, flags=None)

Base class for all neural networks.

A ‘Cell’ could be a single neural network cell, such as mindspore.nn.Conv2d, mindspore.nn.ReLU, mindspore.nn.BatchNorm, etc. or a composition of cells to constructing a network.

Note

In general, the autograd algorithm will automatically generate the implementation of the gradient function, but if back-propagation(bprop) method is implemented, the gradient function will be replaced by the bprop. The bprop implementation will receive a tensor dout containing the gradient of the loss w.r.t. the output, and a tensor out containing the forward result. The bprop needs to compute the gradient of the loss w.r.t. the inputs, gradient of the loss w.r.t. Parameter variables are not supported currently. The bprop method must contain the self parameter.

Parameters

• auto_prefix (bool) – Recursively generate namespaces. It will affect the name of the parameter in the network. If set to True, the network parameter name will be prefixed, otherwise it will not. Default: True.

• flags (dict) – Network configuration information, currently it is used for the binding of network and dataset. Users can also customize network attributes by this parameter. Default: None.

Supported Platforms:

Ascend GPU CPU

Examples

>>> import mindspore.nn as nn
>>> import mindspore.ops as ops
>>> class MyCell(nn.Cell):
...     def __init__(self, forward_net):
...         super(MyCell, self).__init__(auto_prefix=False)
...         self.net = forward_net
...         self.relu = ops.ReLU()
...
...     def construct(self, x):
...         y = self.net(x)
...         return self.relu(y)
>>>
>>> inner_net = nn.Conv2d(120, 240, 4, has_bias=False, weight_init='normal')
>>> my_net = MyCell(inner_net)
>>> print(my_net.trainable_params())
... # If the 'auto_prefix' set to True or not set when call the '__init__' method of the parent class,
... # the parameter's name will be 'net.weight'.
[Parameter (name=weight, shape=(240, 120, 4, 4), dtype=Float32, requires_grad=True)]

add_flags(**flags)

Add customized attributes for cell.

This method is also called when the cell class is instantiated and the class parameter ‘flags’ is set to True.

Parameters

flags (dict) – Network configuration information, currently it is used for the binding of network and dataset. Users can also customize network attributes by this parameter. Default: None.

add_flags_recursive(**flags)

If a cell contains child cells, this method can recursively customize attributes of all cells.

Parameters

flags (dict) – Network configuration information, currently it is used for the binding of network and dataset. Users can also customize network attributes by this parameter. Default: None.

auto_parallel_compile_and_run()

Whether or not to execute compile and run in ‘AUTO_PARALLEL’ or ‘SEMI_AUTO_PARALLEL’ mode.

Returns

bool, _auto_parallel_compile_and_run value.

property bprop_debug

Get whether cell custom bprop debug is enabled.

cast_inputs(inputs, dst_type)

Cast inputs to specified type.

Parameters

• inputs (tuple[Tensor]) – The cell inputs.

• dst_type (mindspore.dtype) – The specified data type.

Returns

tuple[Tensor], the result with destination data type.

cast_param(param)

Cast parameter according to auto mix precision level in pynative mode.

This interface is currently used in the case of auto mix precision and usually needs not to be used explicitly.

Parameters

param (Parameter) – Parameters, the type of which should be cast.

Returns

Parameter, the input parameter with type automatically cast.

cells()

Returns an iterator over immediate cells.

Returns

Iteration, the immediate cells in the cell.

cells_and_names(cells=None, name_prefix='')

Returns an iterator over all cells in the network.

Includes the cell’s name and itself.

Parameters

• cells (str) – Cells to iterate over. Default: None.

• name_prefix (str) – Namespace. Default: ‘’.

Returns

Iteration, all the child cells and corresponding names in the cell.

Examples

>>> n = Net()
>>> names = []
>>> for m in n.cells_and_names():
...     if m[0]:
...         names.append(m[0])

check_names()

Check the names of cell parameters.

compile(*inputs)

Compiles cell.

Parameters

inputs (tuple) – Inputs of the Cell object.

compile_and_run(*inputs)

Compiles and runs cell.

Parameters

inputs (tuple) – Inputs of the Cell object.

Returns

Object, the result of executing.

construct(*inputs, **kwargs)

Defines the computation to be performed. This method must be overridden by all subclasses.

Returns

Tensor, returns the computed result.

exec_checkpoint_graph()

Executes saving checkpoint graph operation.

extend_repr()

Sets the extended representation of the Cell.

To print customized extended information, re-implement this method in your own cells.

generate_scope()

Generate the scope for each cell object in the network.

get_flags()

Get the self_defined attributes of the cell, which can be added by add_flags method.

get_func_graph_proto()

Return graph binary proto.

get_parameters(expand=True)

Returns an iterator over cell parameters.

Yields parameters of this cell. If expand is true, yield parameters of this cell and all subcells.

Parameters

expand (bool) – If true, yields parameters of this cell and all subcells. Otherwise, only yield parameters that are direct members of this cell. Default: True.

Returns

Iteration, all parameters at the cell.

Examples

>>> net = Net()
>>> parameters = []
>>> for item in net.get_parameters():
...     parameters.append(item)

get_scope()

Returns the scope of a cell object in one network.

Returns

String, scope of the cell.

infer_param_pipeline_stage()

Infer pipeline stages of all parameters in the cell.

Note

• If a parameter does not belong to any cell which has been set pipeline_stage, the parameter should use add_pipeline_stage to add it’s pipeline_stage information.

• If a parameter P has been used by two operators in different stages “stageA” and “stageB”, the parameter P should use P.add_pipeline_stage(stageA) and P.add_pipeline_stage(stageB) to add it’s stage information before using infer_param_pipeline_stage.

Returns

The params belong to current stage in pipeline parallel.

Raises

RuntimeError – If there is a parameter does not belong to any stage.

init_parameters_data(auto_parallel_mode=False)

Initialize all parameters and replace the original saved parameters in cell.

Note

trainable_params() and other similar interfaces may return different parameter instance after init_parameters_data, do not save these results.

Parameters

auto_parallel_mode (bool) – If running in auto_parallel_mode. Default: False.

Returns

Dict[Parameter, Parameter], returns a dict of original parameter and replaced parameter.

insert_child_to_cell(child_name, child_cell)

Adds a child cell to the current cell with a given name.

Parameters

• child_name (str) – Name of the child cell.

• child_cell (Cell) – The child cell to be inserted.

Raises

• KeyError – Child Cell’s name is incorrect or duplicated with the other child name.

• TypeError – Child Cell’s type is incorrect.

insert_param_to_cell(param_name, param, check_name_contain_dot=True)

Adds a parameter to the current cell.

Inserts a parameter with given name to the cell. The method is currently used in mindspore.nn.Cell.__setattr__.

Parameters

• param_name (str) – Name of the parameter.

• param (Parameter) – Parameter to be inserted to the cell.

• check_name_contain_dot (bool) – Determines whether the name input is compatible. Default: True.

Raises

• KeyError – If the name of parameter is null or contains dot.

• TypeError – If the type of parameter is not Parameter.

load_parameter_slice(params)

Replace parameters with sliced tensors by parallel strategies.

Please refer to the usage in source code of mindspore.common._CellGraphExecutor.compile.

Parameters

params (dict) – The parameters dictionary used for initializing the data graph.

name_cells()

Returns an iterator over all immediate cells in the network.

Include name of the cell and cell itself.

Returns

Dict, all the child cells and corresponding names in the cell.

property param_prefix

Param prefix is the prefix of current cell’s direct child parameter.

property parameter_layout_dict

parameter_layout_dict represents the tensor layout of a parameter, which is inferred by shard strategy and distributed operator information.

parameters_and_names(name_prefix='', expand=True)

Returns an iterator over cell parameters.

Includes the parameter’s name and itself.

Parameters

• name_prefix (str) – Namespace. Default: ‘’.

• expand (bool) – If true, yields parameters of this cell and all subcells. Otherwise, only yield parameters that are direct members of this cell. Default: True.

Returns

Iteration, all the names and corresponding parameters in the cell.

Examples

>>> n = Net()
>>> names = []
>>> for m in n.parameters_and_names():
...     if m[0]:
...         names.append(m[0])

parameters_broadcast_dict(recurse=True)

Gets the parameters broadcast dictionary of this cell.

Parameters

recurse (bool) – Whether contains the parameters of subcells. Default: True.

Returns

OrderedDict, return parameters broadcast dictionary.

parameters_dict(recurse=True)

Gets parameters dictionary.

Gets the parameters dictionary of this cell.

Parameters

recurse (bool) – Whether contains the parameters of subcells. Default: True.

Returns

OrderedDict, return parameters dictionary.

recompute(**kwargs)

Set the cell recomputed. All the primitive in the cell will be set recomputed. If a primitive set recomputed feeds into some backward nodes for computing gradient, rather than storing the intermediate activation computed in forward pass, we will recompute it in backward pass.

Note

• If the computation involves something like randomization or global variable, the equivalence is not guaranteed currently.

• If the recompute api of a primitive in this cell is also called, the recompute mode of this primitive is subject to the recompute api of the primitive.

• The interface can be configured only once. Therefore, when the parent cell is configured, the child cell should not be configured.

• When the memory remains after applying the recomputation, configuring ‘mp_comm_recompute=False’ to improve performance if necessary.

• When the memory still not enough after applying the recompute, configuring ‘parallel_optimizer_comm_recompute=True’ to save more memory if necessary. Cells in the same fusion group should have the same parallel_optimizer_comm_recompute configures.

Parameters

• mp_comm_recompute (bool) – Specifies whether the model parallel communication operators in the cell are recomputed in auto parallel or semi auto parallel mode. Default: True.

• parallel_optimizer_comm_recompute (bool) – Specifies whether the communication operator allgathers introduced by optimizer shard are recomputed in auto parallel or semi auto parallel mode. Default: False.

register_backward_hook(fn)

Set the cell backward hook function. Note that this function is only supported in pynative mode.

Note

fn must be defined as the following code. cell_name is the name of registered cell. grad_input is gradient passed to the cell. grad_output is the gradient computed and passed to the next cell or primitive, which may be modified and returned. hook_fn(cell_name, grad_input, grad_output) -> Tensor or None.

Parameters

fn (function) – Specifies the hook function with grad as input.

remove_redundant_parameters()

Remove the redundant parameters.

This interface usually needs not to be used explicitly.

set_auto_parallel()

Set the cell to auto parallel mode.

Note

If a cell needs to use the auto parallel or semi auto parallel mode for training, evaluation or prediction, this interface needs to be called by the cell.

set_boost(boost_type)

In order to improve the network performance, configure the network auto enable to accelerate the algorithm in the algorithm library.

If boost_type is not in the algorithm library. Please view the algorithm in the algorithm library through algorithm library.

Note

Some acceleration algorithms may affect the accuracy of the network, please choose carefully.

Parameters

boost_type (str) – accelerate algorithm.

Returns

Cell, the cell itself.

Raises

ValueError – If boost_type is not in the algorithm library.

set_broadcast_flag(mode=True)

Set parameter broadcast mode for this cell.

Parameters

mode (bool) – Specifies whether the mode is parameter broadcast. Default: True.

set_comm_fusion(fusion_type, recurse=True)

Set comm_fusion for all the parameters in this cell. Please refer to the description of

mindspore.Parameter.comm_fusion.

Note

The value of attribute will be overwritten when the function is called multiply.

Parameters

• fusion_type (int) – The value of comm_fusion.

• recurse (bool) – Whether sets the trainable parameters of subcells. Default: True.

set_data_parallel()

For all primitive ops in this cell(including ops of cells that wrapped by this cell), if parallel strategy is not specified, then instead of auto-searching, data parallel strategy will be generated for those primitive ops.

Note

Only effective while using auto_parallel_context = ParallelMode.AUTO_PARALLEL.

Examples

>>> import mindspore.nn as nn
>>> net = nn.Dense(3, 4)
>>> net.set_data_parallel()

set_grad(requires_grad=True)

Sets the cell flag for gradient. In pynative mode, this parameter specifies whether the network requires gradients. If true, the backward network needed to compute the gradients will be generated when the forward network is executed.

Parameters

requires_grad (bool) – Specifies if the net need to grad, if it is true, the cell will construct backward network in pynative mode. Default: True.

Returns

Cell, the cell itself.

set_parallel_input_with_inputs(*inputs)

Slice inputs tensors by parallel strategies.

Parameters

inputs (tuple) – inputs of construct method.

set_param_fl(push_to_server=False, pull_from_server=False, requires_aggr=True)

Set the way of parameter and server interaction.

Parameters

• push_to_server (bool) – Whether the parameter should be pushed to server. Default: False.

• pull_from_server (bool) – Whether the parameter should be pulled from server. Default: False.

• requires_aggr (bool) – Whether the parameter should be aggregated in the server. Default: True.

set_param_ps(recurse=True, init_in_server=False)

Set whether the trainable parameters are updated by parameter server and whether the trainable parameters are initialized on server.

Note

It only works when a running task is in the parameter server mode.

Parameters

• recurse (bool) – Whether sets the trainable parameters of subcells. Default: True.

• init_in_server (bool) – Whether trainable parameters updated by parameter server are initialized on server. Default: False.

set_train(mode=True)

Sets the cell to training mode.

The cell itself and all children cells will be set to training mode. Layers that have different constructions for training and predicting, such as BatchNorm, will distinguish between the branches by this attribute. If set to true, the training branch will be executed, otherwise another branch.

Parameters

mode (bool) – Specifies whether the model is training. Default: True.

Returns

Cell, the cell itself.

to_float(dst_type)

Add cast on all inputs of cell and child cells to run with certain float type.

If dst_type is mindspore.dtype.float16, all the inputs of Cell, including input, Parameter and Tensor, will be cast to float16. Please refer to the usage in source code of mindspore.build_train_network().

Note

Multiple calls will overwrite.

Parameters

dst_type (mindspore.dtype) – Transfer cell to run with dst_type. dst_type can be mstype.float16 or mstype.float32.

Returns

Cell, the cell itself.

Raises

ValueError – If dst_type is not mstype.float32 or mstype.float16.

Examples

>>> import mindspore.nn as nn
>>> from mindspore import dtype as mstype
>>>
>>> net = nn.Conv2d(120, 240, 4, has_bias=False, weight_init='normal')
>>> net.to_float(mstype.float16)

trainable_params(recurse=True)

Returns all trainable parameters.

Returns a list of all trainable parameters.

Parameters

recurse (bool) – Whether contains the trainable parameters of subcells. Default: True.

Returns

List, the list of trainable parameters.

untrainable_params(recurse=True)

Returns all untrainable parameters.

Returns a list of all untrainable parameters.

Parameters

recurse (bool) – Whether contains the untrainable parameters of subcells. Default: True.

Returns

List, the list of untrainable parameters.

update_cell_prefix()

Update the all child cells’ self.param_prefix.

After being invoked, it can get all the cell’s children’s name prefix by ‘_param_prefix’.

update_cell_type(cell_type)

The current cell type is updated when a quantization aware training network is encountered.

After being invoked, it can set the cell type to ‘cell_type’.

Parameters

cell_type (str) – The type of cell to be updated, cell_type can be “quant” or “second-order”.

update_parameters_name(prefix='', recurse=True)

Updates the names of parameters with given prefix string.

Adds the given prefix to the names of parameters.

Parameters

• prefix (str) – The prefix string. Default: ‘’.

• recurse (bool) – Whether contains the parameters of subcells. Default: True.