mindspore.dataset.text.SPieceTokenizerLoadType

View Source On Gitee
class mindspore.dataset.text.SPieceTokenizerLoadType[source]

Model input type for the SentencePiece tokenizer.

Available values are as follows:

  • SPieceTokenizerLoadType.FILE: Load model from specified file path.

  • SPieceTokenizerLoadType.MODEL: Load model from specified vocab object.