mindspore.nn.SoftmaxCrossEntropyWithLogits

class mindspore.nn.SoftmaxCrossEntropyWithLogits(sparse=False, reduction='none')[source]

Computes softmax cross entropy between logits and labels.

Measures the distribution error between the probabilities of the input (computed with softmax function) and the labels where the classes are mutually exclusive (only one class is positive) using cross entropy loss.

Typical input into this function is unnormalized scores denoted as \(x\) whose shape is \((N, C)\) , and the corresponding targets.

Typically, the input to this function is the fractional value of each category and the corresponding target value, and the input format is \((N, C)\) .

For each instance \(x_i\), \(i\) ranges from 0 to N-1, the loss is given as:

\[\ell(x_i, c) = - \log\left(\frac{\exp(x_i[c])}{\sum_j \exp(x_i[j])}\right) = -x_i[c] + \log\left(\sum_j \exp(x_i[j])\right)\]

where \(x_i\) is a 1D score Tensor, \(c\) is the index of 1 in one-hot.

Note

While the labels classes are mutually exclusive, i.e., only one class is positive in the labels, the predicted probabilities does not need to be exclusive. It is only required that the predicted probability distribution of entry is a valid one.

Parameters

sparse (bool, optional) – Specifies whether labels use sparse format or not. Default: False .
reduction (str, optional) –
Apply specific reduction method to the output: 'none' , 'mean' , 'sum' . Default: 'none' .
- 'none': no reduction will be applied.
- 'mean': compute and return the mean of elements in the output.
- 'sum': the output elements will be summed.

Inputs:

logits (Tensor) - Tensor of shape \((N, C)\) . Data type must be float16 or float32.
labels (Tensor) - Tensor of shape \((N, )\) . If sparse is True, The type of labels is int32 or int64. Otherwise, the type of labels is the same as the type of logits.

Outputs:

Tensor, a tensor of the same shape and type as logits with the component-wise logistic losses.

Raises

TypeError – If sparse is not a bool.
TypeError – If sparse is True and dtype of labels is neither int32 nor int64.
TypeError – If sparse is False and dtype of labels is neither float16 not float32.
ValueError – If reduction is not one of 'none', 'mean', 'sum'.

Supported Platforms:: Ascend GPU CPU

Examples

>>> import mindspore
>>> from mindspore import Tensor, nn
>>> import numpy as np
>>> # case 1: sparse=True
>>> loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True)
>>> logits = Tensor(np.array([[3, 5, 6, 9, 12, 33, 42, 12, 32, 72]]), mindspore.float32)
>>> labels_np = np.array([1]).astype(np.int32)
>>> labels = Tensor(labels_np)
>>> output = loss(logits, labels)
>>> print(output)
[67.]
>>> # case 2: sparse=False
>>> loss = nn.SoftmaxCrossEntropyWithLogits(sparse=False)
>>> logits = Tensor(np.array([[3, 5, 6, 9, 12, 33, 42, 12, 32, 72]]), mindspore.float32)
>>> labels_np = np.array([[0, 0, 0, 0, 0, 0, 1, 0, 0, 0]]).astype(np.float32)
>>> labels = Tensor(labels_np)
>>> output = loss(logits, labels)
>>> print(output)
[30.]