mindspore.mint.nn.functional.cross_entropy

View Source On AtomGit
mindspore.mint.nn.functional.cross_entropy(input, target, weight=None, ignore_index=-100, reduction='mean', label_smoothing=0.0)[source]

The cross entropy loss between input and target.

The cross entropy supports two kinds of targets:

  • Class indices (int) in the range \([0, C)\) where \(C\) is the number of classes. If reduction is 'none', then

    \[\ell(x, y) = L = \{l_1,\dots,l_N\}^\top, \quad l_n = - w_{y_n} \log \frac{\exp(x_{n,y_n})}{\sum_{c=1}^C \exp(x_{n,c})} \cdot \mathbb{1}\{y_n \not= \text{ignore_index}\}\]

    where \(x\) is the inputs, \(y\) is the target, \(w\) is the weight, \(N\) is the batch size, \(c\) belonging to \([0, C-1]\) is class index, where \(C\) is the number of classes.

    If reduction is not 'none' (default 'mean' ), then

    \[\begin{split}\ell(x, y) = \begin{cases} \sum_{n=1}^N \frac{1}{\sum_{n=1}^N w_{y_n} \cdot \mathbb{1}\{y_n \not= \text{ignore_index}\}} l_n, & \text{if reduction} = \text{'mean',}\\ \sum_{n=1}^N l_n, & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]

  • Probabilities (float) for each class, useful when labels beyond a single class per minibatch item are required. If reduction is 'none', then

    \[\ell(x, y) = L = \{l_1,\dots,l_N\}^\top, \quad l_n = - \sum_{c=1}^C w_c \log \frac{\exp(x_{n,c})}{\sum_{i=1}^C \exp(x_{n,i})} y_{n,c}\]

    where \(x\) is the inputs, \(y\) is the target, \(w\) is the weight, N is the batch size, \(c\) belonging to \([0, C-1]\) is class index, where \(C\) is the number of classes.

    If reduction is not 'none' (default 'mean' ), then

    \[\begin{split}\ell(x, y) = \begin{cases} \frac{\sum_{n=1}^N l_n}{N}, & \text{if reduction} = \text{'mean',}\\ \sum_{n=1}^N l_n, & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]

Warning

This is an experimental API that is subject to change or deletion.

Note

Dynamic shape, dynamic rank and variable constant inputs are not supported in strict graph mode (jit_syntax_level=mindspore.STRICT).

Parameters:

  • input (Tensor) – \((N)\) or \((N, C)\) where C = number of classes or \((N, C, H, W)\) in case of 2D Loss, or \((N, C, d_1, d_2, ..., d_K)\). input is expected to be unnormalized logits, supported data types include float16, float32 or bfloat16.

  • target (Tensor) – For class indices, tensor of shape \(()\), \((N)\) or \((N, d_1, d_2, ..., d_K)\) , data type must be int32 or int64. For probabilities, tensor of shape \((N,)\) , \((N, C)\) or \((N, C, d_1, d_2, ..., d_K)\) , supported data types include float16, float32 or bfloat16.

  • weight (Tensor, optional) – A rescaling weight applied to the loss of each batch element. If not None, the shape is \((C,)\), supported data types include float16, float32 or bfloat16. Default: None .

  • ignore_index (int, optional) – Specifies a target value that is ignored and does not contribute to the input gradient. Only valid in class indices, please set it to a negative number in probabilities. Default: -100 .

  • reduction (str, optional) –

    Apply specific reduction method to the output: 'none' , 'mean' , 'sum' . Default: 'mean' .

    • 'none': no reduction will be applied.

    • 'mean': compute and return the weighted mean of elements in the output.

    • 'sum': the output elements will be summed.

  • label_smoothing (float, optional) – Label smoothing values, a regularization tool used to prevent the model from overfitting when calculating Loss. The value range is [0.0, 1.0]. Default: 0.0 .

Returns:

Tensor. The output data type is the same as input.

When weight is not None and its data type differs from input, the output data type is the common type promoted from both. For example, the output data type is float32 when input is bfloat16 and weight is float32.

Raises:

  • ValueError – If label_smoothing is not in the range [0.0, 1.0].

  • ValueError – If input is 0-dim or the size of its first dimension is 0.

  • ValueError – If target is probabilities and its shape is not the same as the shape of input.

Supported Platforms:

Ascend

Examples

>>> import mindspore as ms
>>> from mindspore import mint, Tensor
>>> import numpy as np
>>> # Case 1: Indices labels
>>> inputs = Tensor(np.random.randn(3, 5), ms.float32)
>>> target = Tensor(np.array([1, 0, 4]), ms.int32)
>>> output = mint.nn.functional.cross_entropy(inputs, target)
>>> # Case 2: Probability labels
>>> inputs = Tensor(np.random.randn(3, 5), ms.float32)
>>> target = Tensor(np.random.randn(3, 5), ms.float32)
>>> output = mint.nn.functional.cross_entropy(inputs, target)