Class AddToken

Inheritance Relationships

Base Type

Class Documentation

class AddToken : public mindspore::dataset::TensorTransform

Add token to beginning or end of sequence.

Public Functions

explicit AddToken(const std::string &token, bool begin = true)

Constructor.

Parameters
  • token[in] The token to be added.

  • begin[in] Whether to insert token at start or end of sequence. Default: true.

Example
/* Define operations */
auto add_token_op = text::AddToken(token='TOKEN', begin=True);

/* dataset is an instance of Dataset object */
dataset = dataset->Map({add_token_op},   // operations
                       {"text"});       // input columns
~AddToken() override = default

Destructor.