This article introduces the related features of the cloud-side inference model conversion tool in Ascend back-end, such as profile options, dynamic shape, AOE, custom operators.

Graph Kernel Fusion Configuration Instructions (Beta Feature)

Graph kernel fusion is a unique network performance optimization technique in MindSpore. It can automatically analyze and optimize the existing network computational graph logic and combine with the target hardware capabilities to perform optimizations, such as computational simplification and substitution, operator splitting and fusion, operator special case compilation, to improve the utilization of device computational resources and achieve the overall optimization of network performance.

Quick Start to Device-side Inference

This document uses a model inference example to describe how to use basic device-side MindSpore Lite functions.

Quick Start to Cloud-Side Inference

This document uses a model inference example to describe how to use basic cloud-side MindSpore Lite functions.

Experiencing C++ Simplified Inference Demo

This tutorial provides a MindSpore Lite inference demo. It demonstrates the basic on-device inference process using C++ by inputting random data, executing inference, and printing the inference result.

Experiencing Java Simplified Inference Demo

This tutorial provides an example program for MindSpore Lite to perform inference. It demonstrates the basic process of performing inference on the device side using MindSpore Lite Java interface by random inputting data, executing inference, and printing the inference result.

Experiencing C-language Simplified Inference Demo

This tutorial provides a sample program for MindSpore Lite to perform inference, which demonstrates the basic process of end-side inference with C-language by randomly typing, performing inference, and printing inference results, so that users can quickly understand the use of MindSpore Lite to perform inference-related interfaces.

Android Application Development Based on JNI Interface

It is recommended that you start from the image classification demo on the Android device to understand how to build the MindSpore Lite application project, configure dependencies, and use related APIs.

Android Application Development Based on Java Interface

This tutorial demonstrates the on-device deployment process based on the image segmentation demo on the Android device provided by the MindSpore team.

Post Training Quantization

Converting a trained 'float32' model into an 'int8' model through quantization after training can reduce the model size and improve the inference performance. This tutorial introduces how to use the function.

Preprocessing Image Data

This tutorial introduces how to process the image data before inference to meet the data format requirements for model inference by creating a LiteMat object.

Using C++ Interface to Perform Inference

After model conversion using MindSpore Lite, the model inference process needs to be completed in Runtime. This tutorial introduces how to use C++ interface to write inference code.

Using Java Interface to Perform Inference

After model conversion using MindSpore Lite, the model inference process needs to be completed in Runtime. This tutorial introduces how to use Java interface to write inference code.

Performing Inference or Training on MCU or Small Systems

MindSpore Lite provides a light-weight Micro solution for deploying AI models to IOT devices. In this solution, the model is generated into pure C code called by simple operators, and online model parsing and graph compilation are no longer required, which is suitable for the environment with limited memory and computing power.

NPU Integration Information

This tutorial introduces the instructions for using the integrated NPU, including steps to use, supported chips, and supported operators.

Usage Description of the Integrated NNIE

This tutorial introduces instructions for using integrated NNIE.

TensorRT Integration Information

This tutorial introduces the instructions for using the integrated TensorRT.

Integrated Ascend

This document describes how to use MindSpore Lite to perform inference and use the dynamic shape function on Linux in the Ascend environment.

Implementing Device Training Based On C++ Interface

This tutorial explains the code that trains a LeNet model using Training-on-Device infrastructure.

Implementing Device Training Based On Java Interface

This tutorial demonstrates how to use the Java interface on MindSpore Lite by building and deploying LeNet of the Java version.

Using C++ Interface to Perform Training

After model conversion using MindSpore Lite, the model training process needs to be completed in Runtime. This tutorial introduces how to use C++ interface to write training code.

Using Java Interface to Perform Training

After model conversion using MindSpore Lite, the model training process needs to be completed in Runtime. This tutorial introduces how to use Java interface to write training code.

Building Custom Operators Offline

MindSpore Lite provides a highly flexible tool for offline model conversion. It supports users to expand such as combining your own specific hardware with MindSpore Lite Inference Engine.

Building Custom Operators Online

MindSpore Lite provides a southbound operator registration mechanism. This document describes how to schedule your own operators through the MindSpore Lite framework.

Using Delegate to Support Third-party AI Framework

Delegate of MindSpore Lite is used to support third-party AI frameworks (such as NPU, TensorRT) to quickly access to the inference process in MindSpore Lite.

Offline Converting Models for Inference

MindSpore Lite provides a tool for offline model conversion. It supports conversion of multiple types of models. The converted models can be used for inference.

Converting MindSpore Lite Models

This tutorial introduces that how to convert your MindSpore ToD(Train on Device) model.

benchmark

After model conversion and before inference, you can use the Benchmark tool to perform benchmark testing on a MindSpore Lite model. It can not only perform quantitative analysis (performance) on the forward inference execution duration of a MindSpore Lite model, but also perform comparative error analysis (accuracy) based on the output of the specified model.

benchmark_train

You can use the net_train tool to perform benchmark testing on a MindSpore ToD (Train on Device) model. It can not only perform quantitative analysis (performance) on the execution duration the model, but also perform comparative error analysis (accuracy) based on the output of the specified model.

Static Library Cropper Tool

MindSpore Lite provides the libmindspore-lite.a static library cropping tool for runtime, which can filter out the operators in the ms model, crop the static library files, and effectively reduce the size of the library files.

Visualization Tool

Netron is a neural network model visualization tool developed based on the Electron platform. Netron supports MindSpore Lite models, allowing you to easily view model information.

Model Obfuscation Tool

MindSpore Lite provides a lightweight offline model obfuscator to protect the confidentiality of model files deployed on the IoT devices.

Using C++ Interface to Perform Cloud-side Inference

This tutorial introduces how to use C++ interface to write cloud-side inference code.

Using Java Interface to Perform Cloud-side Inference

This tutorial introduces how to use Java interface to write cloud-side inference code.

Using Python Interface to Perform Cloud-side Inference

This tutorial introduces how to use Python interface to write cloud-side inference code.

Using C++ Interface to Perform Concurrent Inference

This tutorial provides an example program for MindSpore Lite to multi-model concurrent inference by using C++ interface.

Using Java Interface to Perform Concurrent Inference

This tutorial provides an example program for MindSpore Lite to multi-model concurrent inference by using Java interface.

Using Python Interface to Perform Concurrent Inference

This tutorial provides an example program for MindSpore Lite to multi-model concurrent inference by using Python interface.

Offline Conversion of Inference Models

MindSpore Lite provides a tool for cloud-side offline model conversion. It supports conversion of multiple types of models. The converted models can be used for inference.

Using Python Interface to Perform Model Conversions

MindSpore Lite supports cloud-side model conversion via Python interface, supporting multiple types of model conversion, and the converted models can be used for inference.

benchmark

After model conversion and before cloud-side inference, you can use the Benchmark tool to perform benchmark testing on a MindSpore Lite model. It can not only perform quantitative analysis (performance) on the forward inference execution duration of a MindSpore Lite model, but also perform comparative error analysis (accuracy) based on the output of the specified model.

Overall Architecture (Lite)

MindSpore Lite is an ultra-fast, intelligent, and simplified AI engine that enables intelligent applications in all scenarios, provides E2E solutions for users, and helps users enable AI capabilities.

Lite Operator List

This article lists the operators supported by MindSpore Lite.

Codegen Operator List

This article lists the operators supported by MindSpore Lite Codegen.

Image Classification Model

Image classification is to identity what an image represents, to predict the object list and the probabilities.

Object Detection Model

Object detection can identify the object in the image and its position in the image.

Image Segmentation Model

Image segmentation is used to detect the position of the object in the picture or a pixel belongs to which object.

Style Transfer Model

The style transfer model can change the artistic style of the user’s target image according to the standard image built in this demo, and display it in the App image preview interface.

Scene Detection Model

Scene detection can identify the type of scene in the device’s camera.

Troubleshooting

If you encounter an issue when using MindSpore Lite, you can view logs first. In most scenarios, you can locate the issue based on the error information reported in logs.

Log

Only server inference version and windows version support environment variables below, except GLOG_v.

Release Notes