mindspore.mint.nn.Embedding

class mindspore.mint.nn.Embedding(num_embeddings, embedding_dim, padding_idx=None, max_norm=None, norm_type=2.0, scale_grad_by_freq=False, sparse=False, _weight=None, _freeze=False, dtype=None)[source]

The value in input is used as the index, and the corresponding embedding vector is queried from weight .

Warning

This is an experimental API that is subject to change or deletion.
On Ascend, the behavior is unpredictable when the value of input is invalid.

Parameters

num_embeddings (int) – Size of the dictionary of embeddings.
embedding_dim (int) – The size of each embedding vector.
padding_idx (int, optional) – If specified, the embedding vector at padding_idx is not updated during training. For a newly constructed Embedding, the embedding vector at padding_idx will default to all zeros. The value should be in range [-num_embeddings, num_embeddings). Default: None.
max_norm (float, optional) – If given, each embedding vector with norm larger than max_norm is renormalized to have norm max_norm. vector specified by input where p is specified by norm_type; if the result is larger than max_norm, update the embedding vector with \(\frac{max\_norm}{result+1e^{-7}}\). Default: None.
norm_type (float, optional) – Indicates the value of p in p-norm. Default: 2.0.
scale_grad_by_freq (bool, optional) – If True, the gradient will be scaled by the inverse of frequency of the index in input. Default: False.
sparse (bool, optional) – If True, gradient w.r.t. weight matrix will be a sparse tensor which has not been supported. Default: False.
_weight (Tensor, optional) – Use to initialize the weight of Embedding. If None, the weight will be initialized from normal distribution \({N}(\text{sigma=1.0}, \text{mean=0.0})\). Default: None.
_freeze (bool, optional) – If weight, the learnable weight of this module, should be frozen. Default: False.
dtype (mindspore.dtype, optional) – Dtype of Embedding's weight. It is meaningless when _weight is not None. Default: None.

Variables:

weight (Parameter) - The learnable weights of this module of shape (num_embeddings, embedding_dim), which initialized from \({N}(\text{sigma=1.0}, \text{mean=0.0})\) or _weight .

Inputs:

input (Tensor) - The input tensor containing the indices to extract, which in range [0, num_embeddings).

Outputs:

Tensor.

Supported Platforms:

Ascend

Examples

>>> import mindspore
>>> mindspore.set_seed(0)
>>> input = mindspore.tensor([[1, 0, 1, 1], [0, 0, 1, 0]])
>>> embedding = mindspore.mint.nn.Embedding(num_embeddings=10, embedding_dim=3)
>>> output = embedding(input)
>>> print(output)
[[[ 0.6712398   0.5407775   1.0317237]
  [-0.49091062 -0.42302188 -1.4807187]
  [ 0.6712398   0.5407775   1.0317237]
  [ 0.6712398   0.5407775   1.0317237]]
 [[-0.49091062 -0.42302188 -1.4807187]
  [-0.49091062 -0.42302188 -1.4807187]
  [ 0.6712398   0.5407775   1.0317237]
  [-0.49091062 -0.42302188 -1.4807187]]]