Idea Sharing: Advancements Made by AI in Marine Meteorology

Background

Humans have been predicting and understanding the world. From philosophical reasoning in ancient Greece, to pseudoscience methods in the Middle Ages, and to modern scientific discourses including hypothesis testing, theoretical development, and computer modeling, these efforts have been made based on statistical and physical relationships, namely related laws. Weather forecast of geoscience is a good example, which mainly depends on the growth of theories, observation systems, and computing capabilities. Even so, accurate weather forecasts only can be done by day instead of by month. Earth data keeps surging. The CMIP5 dataset, which is widely used for periodic climate assessment, has output more than 3 PB data, while the next-generation CMIP6 dataset is estimated to generate 30 PB. Similar to big data in other fields, massive data of geoscience, such as remote sensing, fixed-point observation, and pattern data, also features large volume, high velocity, rich variety, and high veracity. How to extract and interpret effective information from the big data and how to use deep learning to improve the prediction capability of models have become great challenges faced by geoscience systems.

The following parts provide a quick review of recent progress of deep learning models in short-term precipitation, weather prediction, and extreme weather identification.

1. Pixel-Level Segmentation of Climate Data Using E-Level Deep Learning Model

In 2018, Lawrence Berkeley National Laboratory won the ACM Gordon Bell Prize by successfully identifying extreme weather through deep learning. The achievement runs on the world's best Summit System, reaching an unprecedented computing scale. The difficulty of this work lies in the analysis of the CAM5 climate dataset, which has high resolution, large number of elements, low probability of target events, and unbalanced positive and negative samples. To address these issues, the Laboratory has optimized the image segmentation model DeepLabv3 at the system level (data pipeline and data specification optimization) and model level (weighted loss function, adaptive learning rate, and gradient delay update), significantly improving the model throughput and parallel efficiency.

For details, visit:

https://ieeexplore.ieee.org/abstract/document/8665799

Figure 1 Throughput of the DeepLabv3+ model

As shown in Figure 1, the throughput of the model reaches 325.8 PF/s, which is at the level of 10 to the power of 15. The parallel efficiency reaches 90.7%. To deal with 3.5 TB climate data, 27,360 GPUs can run on this model, with the throughput reaching 1.13 EF/s.

2. El Niño Prediction Based on Deep Learning

The AI-based El Niño-Southern Oscillation (ENSO) forecast was published in Nature in 2019. ENSO is an irregular periodic variation in winds and sea surface temperatures over the tropical eastern Pacific Ocean. It has a quasi-period of 2 to 7 years and is divided into the neutral, warm (positive), and cool (negative) phases. According to numerical simulation results, seasonal variation of solar radiation, Asian monsoon, global warming and many other factors may incur ENSO. One of the biggest limitations of deep learning in climate forecast is that observations of global ocean temperature distribution is short as it began from 1871. Moreover, the ENSO period ranges from 2 to 7 years, resulting in a small number of samples.

South Korean scientists and professor Luo Jingjia from Nanjing University of Information Science & Technology explored ENSO prediction using a convolutional neural network (CNN) and transfer learning. Their work uses the sea surface temperature reanalysis data from 1871 to 1973 and climate simulation data from CMIP5 to increase the number of training data. The model is first trained on CMIP5, and then the training result is used as the initial value and is trained with reanalysis data. According to the final results, the correlation coefficient exceeds 0.5 in the 17-month-ahead ENSO forecast, and the correlation coefficient exceeds 0.7 in the 6-month-ahead ENSO forecast.

For details, visit:

https://www.nature.com/articles/s41586-019-1559-7

3. Precipitation Forecasts Enabled by MindSpore

This is an innovative achievement of Huawei in short-term precipitation forecast. An adaptive recursive neural network (AdaRNN) is proposed to address the challenges of the AI extrapolation short-term prediction technology, such as fast attenuation of long-range prediction and wind-affected movement of clouds. The model selects historical radar echograms as training data. Through model parameter optimization and cross-check of test sets, the optimal radar echo prediction model is generated. In real-time services, the first 20 frames (the first two hours) radar echoes are used as input data to realize the service operation of 0–2 hour radar reflectivity factor extrapolation.

To solve problems such as cloud formation/disappearance and fuzzy prediction graph, the model is optimized as follows:

1. Modeling of formation/disappearance: The spatio-temporal model is added with the module of cloud formation/disappearance modeling.

2. The adversarial loss training model based on special design can generate radar echo diagrams with clear details.

3. Texture transfer is implemented using neural networks to increase details while maintaining regional features.

4. Mid-term Weather Forecast: NVIDIA FourCastNet & Huawei Pangu-Weather

In this spring, researchers from NVIDIA, Lawrence Berkeley National Laboratory, University of Michigan (Ann Arbor), and Rice University released FourCastNet, a Fourier-based neural network prediction model. The model unprecedentedly achieves the precision comparable with NWP, and increases the AI mid-term prediction resolution from 2° to 0.25°. High-precision grid input is critical to capturing subtle structures and helps improve the precipitation forecast precision. Compared with traditional numerical models, AI models shorten the prediction time by more than 45,000 times.

For details, visit:

https://arxiv.org/abs/2202.11214

Not long ago, Huawei upgraded FourCastNet and launched Pangu-Weather. Based on FourCastNet, the Pangu research team found two factors that would affect the precision of AI weather forecast models. First, existing AI-based systems for weather forecasting are all based on 2D neural networks and cannot properly handle 3D weather data featuring uneven spatial distribution. Second, without real-world constraints, medium-range weather forecast can suffer from cumulative forecast errors when the model is called too many times. Addressing these problems, Pangu-Weather was proposed with a 3D earth-specific transformer (3DEST) architecture to process 3D data. It also uses a hierarchical temporal aggregation algorithm to reduce the number of iterations needed to make a medium-range forecast. In this way, cumulative forecast errors are alleviated. The model was built with 192 GPUs, and the training took up to two weeks. Its prediction precision exceeds that of FourCastNet and traditional numerical methods for the first time, ending the debate about whether AI models can outperform numerical prediction models.

For details, visit:

https://arxiv.org/abs/2211.02556

References

[1] Reichstein M, Camps-Valls G, Stevens B, et al. Deep learning and process understanding for data-driven Earth system science[J]. Nature, 2019, 566(7743): 195-204.

[2] Kurth, Thorsten, et al. "Exascale deep learning for climate analytics." SC18: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 2018.

[3] Ham, Yoo-Geun, Jeong-Hwan Kim, and Jing-Jia Luo. "Deep learning for multi-year ENSO forecasts." Nature 573.7775 (2019): 568-572.

[4] Pathak J, Subramanian S, Harrington P, et al. Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators[J]. arXiv preprint arXiv:2202.11214, 2022.

[5] Bi K, Xie L, Zhang H, et al. Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast[J]. arXiv preprint arXiv:2211.02556, 2022.