mindspore.dataset.text.SPieceTokenizerOutType

class mindspore.dataset.text.SPieceTokenizerOutType[source]

An enumeration for mindspore.dataset.text.SentencePieceTokenizer .

Possible enumeration values are: SPieceTokenizerOutType.STRING, SPieceTokenizerOutType.INT.

  • SPieceTokenizerOutType.STRING: means output type of SentencePiece Tokenizer is string.

  • SPieceTokenizerOutType.INT: means output type of SentencePiece Tokenizer is int.