mindspore.dataset.text.SentencePieceModel

class mindspore.dataset.text.SentencePieceModel[source]

Subword algorithms for SentencePiece.

Available values are as follows:

SentencePieceModel.UNIGRAM: Unigram Language Model subword algorithm.
SentencePieceModel.BPE: Byte-Pair-Encoding subword algorithm.
SentencePieceModel.CHAR: Character-based subword algorithm.
SentencePieceModel.WORD: Word-based subword algorithm.