Differences with torch.distributed.all_to_all

torch.distributed.all_to_all

torch.distributed.all_to_all(
    output_tensor_list,
    input_tensor_list,
    group=None,
    async_op=False)

For more details, see torch.distributed.all_to_all.

mindspore.communication.comm_func.all_to_all_with_output_shape

mindspore.communication.comm_func.all_to_all_with_output_shape(
    output_shape_list,
    input_tensor_list,
    group=None
    async_op=False
)

For more details, see mindspore.communication.comm_func.all_to_all_with_output_shape.

Usage

PyTorch: This interface takes the list of tensors to be received, the list of tensors to be sent, the communication group, and whether to execute asynchronously. The received data is modified in place in the tensors of the output_tensor_list argument. If asynchronous execution is enabled, a handler is returned for subsequent synchronization operations.

MindSpore: This interface takes the list of shapes for the tensors to be received, the list of tensors to be sent, and the communication group. It returns the received tensor tuple through the return value. If asynchronous execution is enabled, a tuple is returned, which includes the received data and a handler; if asynchronous execution is not enabled, the tuple contains the received data and None.

Category	Subcategory	PyTorch	MindSpore	Difference
Parameter	Parameter 1	output_tensor_list	output_shape_list	The functionality is inconsistent, and the types are different. PyTorch takes the list of tensors to be received, and the received data is assigned in place to the tensors of the input list; MindSpore takes the shape list of the tensors to be received, and the received data is returned through a new list of tensors.
	Parameter 2	input_tensor_list	input_tensor_list	The functionality is consistent.
	Parameter 3	group	group	The functionality is consistent, but the types are different. PyTorch takes a communication group object; MindSpore takes the string name of the communication group.
	Parameter 4	async_op	async_op	The functionality is consistent