Class AddToken

Defined in File text.h

Inheritance Relationships

Base Type

public mindspore::dataset::TensorTransform (Class TensorTransform)

Class Documentation

class AddToken : public mindspore::dataset::TensorTransform 

Add token to beginning or end of sequence.

Public Functions

explicit AddToken(const std::string &token, bool begin = true)

Constructor.

参数

token – [in] The token to be added.
begin – [in] Whether to insert token at start or end of sequence. Default: true.

样例

/* Define operations */
auto add_token_op = text::AddToken(token='TOKEN', begin=True);

/* dataset is an instance of Dataset object */
dataset = dataset->Map({add_token_op},   // operations
                       {"text"});       // input columns

~AddToken() override = default: Destructor.