Sichuan University and MindSpore to Jointly Build a Multimodal Animal Monitoring Algorithm Platform

The College of Computer Science of Sichuan University (SCU) and MindSpore have recently decided to build a multimodal animal monitoring algorithm platform based on Rongcheng.Panda, a foundation model for audio and visual feature recognition of animals. The platform will be used for recording, analyzing and managing the massive data generated by various species including giant pandas, red pandas, golden snub-nosed monkeys, and yaks.

Ecological protection is one of the key strategies of China, and wildlife protection is an indispensable part of ecological protection. In response to this strategy, more than 10 national parks (including pilot sites) has been established, covering 220,000 square kilometers across 12 provinces.

In practice, traditional wildlife monitoring and protection methods face many problems, such as time-consuming manual monitoring, high costs, and poor timeliness. Electronic identification of animals, a relatively new monitoring method, also has many disadvantages and is not wildlife friendly. Therefore, a real-time monitoring algorithm with high accuracy is required to make up for the disadvantages of traditional methods.

Such an algorithm is the goal of the cooperation between MindSpore and the College of Computer Science of SCU. The joint team plans to develop a general-purpose animal feature identification algorithm running on the MindSpore all-scenario AI framework. The core of the algorithm is the multimodal convergence AI technology that can be used in individual identification, group counting, and vital signs monitoring scenarios based on accurately identified multimodal animal features, such as audio, video, image, and body posture features.

Professor Zhao Qijun, Vice Dean of the College of Computer Science of SCU, said that Rongcheng.Panda, an Ascend-based foundation model, would be the best starting point for the joint research of SCU and MindSpore, and a multimodal animal monitoring algorithm platform could significantly improve the real-time performance, safety, and friendliness of wildlife monitoring. The College of Computer Science of SCU has incorporated MindSpore courses into the undergraduate education to deepen the integration of AI and education and better cultivate AI talent.

As stated by Tian Kunyang, Product Director of MindSpore, the protection of wildlife such as giant pandas, red pandas, golden snub-nosed monkeys, and yaks is an important measure to maintain the ecosystem sustainability. The application of AI in the ecosystem protection industry would boost the efficiency of animal protection research and operation, and better integrate industry resources through digitalization.

Universities, research institutes, enterprises, and ecosystem protection institutions will work together to accelerate the implementation of end-to-end solutions and the maturity of industrialization. With the solid academic research background of SCU and the underlying hardware support of the Ascend AI platform, the joint team will make remarkable achievements in ecosystem protection and meet the requirements in various animal identification scenarios.

Ascend AI has entered a new phase of development. Together with other partners, MindSpore will continue to innovate AI technologies, to drive the digital economy and promote the intelligent upgrade for all industries.