mindspore_lite.MultiModelRunner

class mindspore_lite.MultiModelRunner[source]

The MultiModelRunner class is used to create mindir with multiple Models and provides a way to schedule multiple models.

Raises

TypeError – model_path is not str.
TypeError – model_type is not ModelType.
TypeError – context is not Context or None .
TypeError – config_path is not str.
RuntimeError – model_path file path not exist.
RuntimeError – config_path file path not exist.
RuntimeError – load config_path failed.
RuntimeError – load and build MultiModelRunner failed.

Examples

>>> import mindspore_lite as mslite
>>> import numpy as np
>>> dtype_map = {
>>>     mslite.DataType.FLOAT32:np.float32,
>>>     mslite.DataType.INT32:np.int32,
>>>     mslite.DataType.FLOAT16:np.float16,
>>>     mslite.DataType.INT8:np.int8
>>> }
>>> model_type=mslite.ModelType.MINDIR
>>> context = mslite.Context()
>>> context.ascend.device_id = 2
>>> model_path = "path_to_model1"
>>> model_runner = mslite.MultiModelRunner()
>>> model_runner.build_from_file(model_path, mslite.ModelType.MINDIR, context)
>>> execs = runner.get_model_ececutor()
>>> for exec_ in execs:
>>>     exec_inputs = exec_.get_inputs()
>>>     exec_outputs = exec_.get_outputs()
>>>     for i, input in enumerate(exec_inputs):
>>>         data = np.random.randn(*input.shape).astype(dtype_map[input.dtype])
>>>         input.set_data_from_numpy(data)
>>>     exec.predict(exec_inputs)

build_from_file(model_path, model_type, context=None, config_path='', config_dict=None)[source]

Load and build a runner from file.

Parameters

model_path (str) – Path of the input model when build from file. For example, "/home/user/model.mindir". Model should use .mindir as suffix.
model_type (ModelType) – Define The type of input model file. Option is ModelType.MINDIR. For details, see ModelType .
context (Context, optional) – Define the context used to transfer options during execution. Default: None. None means the Context with cpu target.

config_path (str, optional) –

Define the config file path. the config file is used to transfer user defined options during build model. In the following scenarios, users may need to set the parameter. For example, "/home/user/config.txt". Default: "".

Usage 1: Set mixed precision inference. The content and description of the configuration file are as

follows:

[execution_plan]
[op_name1]=data_Type: float16 (The operator named op_name1 sets the data type as float16)
[op_name2]=data_Type: float32 (The operator named op_name2 sets the data type as float32)

Usage 2: When GPU inference, set the configuration of TensorRT. The content and description of the

configuration file are as follows:

[ms_cache]
serialize_Path=[serialization model path](storage path of serialization model)
[gpu_context]
input_shape=input_Name: [input_dim] (Model input dimension, for dynamic shape)
dynamic_Dims=[min_dim~max_dim] (dynamic dimension range of model input, for dynamic shape)
opt_Dims=[opt_dim] (the optimal input dimension of the model, for dynamic shape)

config_dict (dict, optional) –
When you set config in this dict, the priority is higher than the configuration items in config_path.

Set rank table file for inference. The content of the configuration file is as follows:
```
[ascend_context]
rank_table_file=[path_a](storage initial path of the rank table file)
```
When set
```
config_dict = {"ascend_context" : {"rank_table_file" : "path_b"}}
```
The the path_b from the config_dict will be used to compile the model.