Function Differences with tf.nn.dropout

tf.nn.dropout

tf.nn.dropout(
    x,
    rate,
    noise_shape=None,
    seed=None,
    name=None
) -> Tensor

For more information, see tf.nn.dropout.

mindspore.ops.dropout

mindspore.ops.dropout(x, p=0.5, seed0=0, seed1=0) -> Tensor

For more information, see mindspore.ops.dropout.

Differences

TensorFlow: dropout is a function used to prevent or mitigate overfitting by dropping a random portion of neurons at different training sessions. That is, the neuron output is randomly set to 0 with a certain probability p, which serves to reduce the neuron correlation. The remaining parameters that are not set to 0 will be scaled with \(\frac{1}{1-rate}\).

MindSpore: MindSpore API basically implements the same function as TensorFlow, but TensorFlow has an additional noise_shape parameter to control the retention/discard dimension.

Categories	Subcategories	TensorFlow	MindSpore	Differences
Parameters	Parameter 1	x	x	-
	Parameter 2	rate	p	Same function, different parameter names
	Parameter 3	noise_shape	-	A 1 int32 tensor representing a randomly generated “keep/drop” flag for the shape. MindSpore does not have this parameter
	Parameter 4	seed	seed0	Same function, different parameter names
	Parameter 5	name		Not involved
	Parameter 6	-	seed1	The global random seed, which together with the random seed of the operator layer determines the final generated random number. Default value: 0

Code Example 1

When the value of noise_shape is None, the two APIs functions are the same.

# TensorFlow
import tensorflow as tf
import numpy as np
neuros = np.array([[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]],dtype=np.float32)
neuros_drop = tf.nn.dropout(neuros, rate=0.2)
print(neuros_drop.shape)
# (10, 10)

# MindSpore
import mindspore
x = mindspore.Tensor([[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]], mindspore.float32)
output, mask = mindspore.ops.dropout(x, p=0.2)
print(output.shape)
# (10, 10)