mindspore_lite.MultiModelRunner

class mindspore_lite.MultiModelRunner[source]

The MultiModelRunner class is used to create mindir with multiple Models and provides a way to schedule multiple models.

Examples

>>> import mindspore_lite as mslite
>>> import numpy as np
>>> dtype_map = {
>>>     mslite.DataType.FLOAT32:np.float32,
>>>     mslite.DataType.INT32:np.int32,
>>>     mslite.DataType.FLOAT16:np.float16,
>>>     mslite.DataType.INT8:np.int8
>>> }
>>> model_type=mslite.ModelType.MINDIR
>>> context = mslite.Context()
>>> context.ascend.device_id = 2
>>> model_path = "path_to_model1"
>>> model_runner = mslite.MultiModelRunner()
>>> model_runner.build_from_file(model_path, mslite.ModelType.MINDIR, context)
>>> execs = model_runner.get_model_ececutor()
>>> for exec_ in execs:
>>>     exec_inputs = exec_.get_inputs()
>>>     exec_outputs = exec_.get_outputs()
>>>     for i, input in enumerate(exec_inputs):
>>>         data = np.random.randn(*input.shape).astype(dtype_map[input.dtype])
>>>         input.set_data_from_numpy(data)
>>>     exec.predict(exec_inputs)

build_from_file(model_path, model_type, context=None, config_path='', config_dict=None)[source]

Load and build a runner from file.

Parameters

model_path (str) – Path of the input model when build from file. For example, "/home/user/model.mindir". Model should use .mindir as suffix.
model_type (ModelType) – Define The type of input model file. Option is ModelType.MINDIR. For details, see ModelType .
context (Context, optional) – Define the context used to transfer options during execution. Default: None. None means the Context with cpu target.
config_path (str, optional) –
Define the config file path. the config file is used to transfer user defined options during build model. In the following scenarios, users may need to set the parameter. For example, "/home/user/config.txt". Default: "".

Set mixed precision inference. The content and description of the configuration file are as follows:
```
[execution_plan]
[op_name1]=data_Type: float16 (The operator named op_name1 sets the data type as float16)
[op_name2]=data_Type: float32 (The operator named op_name2 sets the data type as float32)
```
config_dict (dict, optional) –
When you set config in this dict, the priority is higher than the configuration items in config_path.

Set rank table file for inference. The content of the configuration file is as follows:
```
[ascend_context]
rank_table_file=[path_a](storage initial path of the rank table file)
```
When set
```
config_dict = {"ascend_context" : {"rank_table_file" : "path_b"}}
```
The the path_b from the config_dict will be used to compile the model.

Raises

TypeError – model_path is not str.
TypeError – model_type is not ModelType.
TypeError – context is not Context or None .
TypeError – config_path is not str.
RuntimeError – model_path file path not exist.
RuntimeError – config_path file path not exist.
RuntimeError – load config_path failed.
RuntimeError – load and build MultiModelRunner failed.

get_model_executor()[source]

Get ModelExecutors from MultiModelRunner.

Returns: list[ModelExecutor], all Executors in MultiModelRunner.