Class ModelParallelRunner

Class Documentation

class ModelParallelRunner

The ModelParallelRunner class is used to define a MindSpore ModelParallelRunner, facilitating Model management.

Public Functions

inline Status Init(const std::string &model_path, const std::shared_ptr<RunnerConfig> &runner_config = nullptr)

build a model parallel runner from model path so that it can run on a device. Supports importing the ms model (exported by the converter_lite tool) and the mindir model (exported by MindSpore or exported by the converter_lite tool). The support for the ms model will be removed in future iterations, and it is recommended to use the mindir model for inference. When using the ms model for inference, please keep the suffix name of the model as .ms, otherwise it will not be recognized.

Parameters

  • model_path – [in] Define the model path.

  • runner_config – [in] Define the config used to store options during model pool init.

Returns

Status.

Status Init(const void *model_data, const size_t data_size, const std::shared_ptr<RunnerConfig> &runner_config = nullptr)

build a model parallel runner from model buffer so that it can run on a device. This interface only supports passing in mindir model file data.

Parameters

  • model_data – [in] Define the buffer read from a model file.

  • data_size – [in] Define bytes number of model buffer.

  • runner_config – [in] Define the config used to store options during model pool init.

Returns

Status.

std::vector<MSTensor> GetInputs()

Obtains all input tensors information of the model.

Returns

The vector that includes all input tensors.

std::vector<MSTensor> GetOutputs()

Obtains all output tensors information of the model.

Returns

The vector that includes all output tensors.

Status Predict(const std::vector<MSTensor> &inputs, std::vector<MSTensor> *outputs, const MSKernelCallBack &before = nullptr, const MSKernelCallBack &after = nullptr)

Inference ModelParallelRunner.

Parameters

  • inputs – [in] A vector where model inputs are arranged in sequence.

  • outputs – [out] Which is a pointer to a vector. The model outputs are filled in the container in sequence.

  • before – [in] CallBack before predict.

  • after – [in] CallBack after predict.

Returns

Status.