mindspore.parallel

This interface is ONLY used in Auto-parallel procedure.

mindspore.parallel.set_fusion_strategy_by_idx(idxList, group='hccl_world_group')[source]

A function set gradient segment strategy according to the index list.

Note

In the back propagation, the fusion of the allreduce operators with a fusion attribute equals 1, will be performed according to the idxList, to achieve the effect of parallel between calculation and communication.

Parameters
  • idxList (list) – The index list of the gradient.

  • group (str) – The hccl communication group.

Raises
mindspore.parallel.set_fusion_strategy_by_size(dataSizeList, group='hccl_world_group')[source]

A function set gradient segment strategy according to the data size percentage list.

Note

In the back propagation, the fusion of the allreduce operators with a fusion attribute equals 1, will be performed according to dataSizeList, to achieve the effect of parallel between calculation and communication.

Parameters
  • dataSizeList (list) – The data size percentage list of the gradient.

  • group (str) – The hccl communication group.

Raises
  • TypeError – If group is not a python str.

  • TypeError – If dataSizeList is not a python list.

  • TypeError – If type of dataSizeList item is not int or float.

  • ValueError – If group name length is out of range.

  • ValueError – If dataSizeList length is 0.

  • ValueError – If dataSizeList item is less than 0.

  • RuntimeError – If allreduce split failed.

mindspore.parallel.get_algo_parameters(attr_key)[source]

Get algo parameter config attributes.

Note

Return value according to the attribute value.

Parameters

attr_key (str) – The key of the attribute.

Raises

ValueError – If context keyword is not recognized.

mindspore.parallel.reset_algo_parameters()[source]

Reset algo parameter attributes.

mindspore.parallel.set_algo_parameters(**kwargs)[source]

Set algo parameter config.

Note

Attribute name is needed.

Parameters
  • simplify_cal (bool) – Whether simplifying calculations in strategy-searching algorithm. Default: True

  • tensor_slice_align_enable (bool) – Whether checking tensor slice shape. Default: False

  • tensor_slice_align_size (int) – The minimum tensor slice shape, the value must be in [1, 1024]. Default: 16

  • not_fully_use_devices (bool) – Whether generating strategies that not fully use devices. Default: False

  • elementwise_op_strategy_follow (bool) – Whether the elementwise operator have the same strategies as its subsequent operators. Default: False

Raises

ValueError – If context keyword is not recognized.