[{"data":1,"prerenderedAt":281},["ShallowReactive",2],{"content-query-DfsIfxjZvw":3},{"_path":4,"_dir":5,"_draft":6,"_partial":6,"_locale":7,"title":8,"description":9,"date":10,"cover":11,"type":12,"body":13,"_type":275,"_id":276,"_source":277,"_file":278,"_stem":279,"_extension":280},"/technology-blogs/en/2871","en",false,"","Idea Sharing: Generative Models Are Shining in Precipitation Nowcasting","From the perspective of data distribution or mechanism equations, the precipitation nowcasting models driven by data-knowledge integration will gradually become the future evolution direction.","2023-10-23","https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2023/11/15/fc2c7155e9b54d188a5423329658c9ec.png","technology-blogs",{"type":14,"children":15,"toc":272},"root",[16,24,34,39,44,49,57,70,78,83,88,93,100,105,113,118,125,204,216,224,229,237,250,261],{"type":17,"tag":18,"props":19,"children":21},"element","h1",{"id":20},"idea-sharing-generative-models-are-shining-in-precipitation-nowcasting",[22],{"type":23,"value":8},"text",{"type":17,"tag":25,"props":26,"children":27},"p",{},[28],{"type":17,"tag":29,"props":30,"children":31},"strong",{},[32],{"type":23,"value":33},"Background",{"type":17,"tag":25,"props":35,"children":36},{},[37],{"type":23,"value":38},"Global climate change has caused frequent extreme precipitation in recent years. People have started to keep a close eye on more accurate and refined precipitation nowcasts with longer lead times. Precipitation nowcasting refers to the forecasting of precipitation in an area over a period from the present up to 6 hours ahead. However, physics-based numerical methods struggle to forecast the extreme precipitation at the kilometer scale. Generally, methods based on optical flow vectors are used in practice, where the radar echo diagrams of two radars are used to estimate the optical flow vectors, based on which radar echo extrapolation is performed to obtain the forecast. Compared with numerical methods, these methods have higher forecast accuracy in nowcasting with a lead time of one or two hours.",{"type":17,"tag":25,"props":40,"children":41},{},[42],{"type":23,"value":43},"Common deep learning methods, such as ConvLSTM, RNN, and Transformer, use radar observations to directly predict the future rainfall rate. Although low-intensity rainfall can be accurately predicted, the predictions are blurry and smooth, lacking the capability of predicting strong convection and moderate to heavy rain events.",{"type":17,"tag":25,"props":45,"children":46},{},[47],{"type":23,"value":48},"In 2021, DeepMind worked with the Meteorological Office (UK) to develop Deep Generative Models of Radar (DGMR) [1], which can accurately produce nowcasts of precipitation with a lead time of 90 minutes. Compared with previous forecasting models, this generative model solves the problem of blurry prediction. However, as a pure black-box generative model, DGMR lacks the deterministic constraint of physical mechanisms on the output of predictions, so there may be large deviation in strength and position. This blog introduces two new generative models integrated with prior knowledge: NowcastNet [2] and PreDiff [3].",{"type":17,"tag":25,"props":50,"children":51},{},[52],{"type":17,"tag":29,"props":53,"children":54},{},[55],{"type":23,"value":56},"1. NowcastNet",{"type":17,"tag":25,"props":58,"children":59},{},[60,62,68],{"type":23,"value":61},"NowcastNet is a precipitation nowcasting model proposed by Shengming Long's team at Tsinghua University and the National Meteorological Center of China. The model has been published in ",{"type":17,"tag":63,"props":64,"children":65},"em",{},[66],{"type":23,"value":67},"Nature",{"type":23,"value":69},". NowcastNet is a deep learning model for nowcasting extreme precipitation. It unifies the physical evolution equation and generative model into one framework to optimize the end-to-end nowcasting of precipitation. NowcastNet produces precipitation nowcasts with sharp multiscale patterns with lead times of up to 3 hours. In a systematic evaluation by 62 professional meteorologists from across China, the model ranks first in 71% of cases against the leading methods. The following figure shows the model structure.",{"type":17,"tag":25,"props":71,"children":72},{},[73],{"type":17,"tag":74,"props":75,"children":77},"img",{"alt":7,"src":76},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2023/11/15/e5a957baa26f408c99e928572d801f55.png",[],{"type":17,"tag":25,"props":79,"children":80},{},[81],{"type":23,"value":82},"NowcastNet consists of an evolution network and a generative network. The evolution network predicts physics-constrained deterministic changes at the 20-km scale based on past radar observations. The generative network uses historical radar observations and the prediction result of the evolution network to generate the convection details at a 1 to 2-km scale through the GAN and obtain the final prediction result.",{"type":17,"tag":25,"props":84,"children":85},{},[86],{"type":23,"value":87},"The general idea of the evolution network is to decompose precipitation into intensity changes caused by the advection motion and addition of the precipitation system. The evolution network focuses on mesoscale precipitation processes with more significant physical properties, such as advection motion. The intensity decoder and motion decoder are used to simulate the precipitation intensity and convection trend, respectively, to generate the corresponding intensity field and moving field. Then, by using an evolution operator based on the continuity equation, the motion field is applied to the last frame of radar observations, and an intensity field is added so as to generate a prediction result at a large scale.",{"type":17,"tag":25,"props":89,"children":90},{},[91],{"type":23,"value":92},"The following figure shows the structure of the generative network, which is a convective-scale generative network based on the SPADE network. The generative network uses the prediction result of the mesoscale evolution network as a condition and uses a probabilistic generative model to further capture the kilometer-scale precipitation process with more significant chaotic effects such as convective generation and elimination.",{"type":17,"tag":25,"props":94,"children":95},{},[96],{"type":17,"tag":74,"props":97,"children":99},{"alt":7,"src":98},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2023/11/15/68cbb74d1072449c8bae6ac64cabaa60.png",[],{"type":17,"tag":25,"props":101,"children":102},{},[103],{"type":23,"value":104},"The structure diagram shows that the mesoscale prediction is not only used as the input of the encoder but can also be regarded as the semantic guidance in the generative model, which constrains the prediction result of a finer resolution. Thanks to this integration design, NowcastNet has the advantages of both deep learning and physical modeling. For the first time in the world, the model extends the valid lead time of precipitation nowcasting to 3 hours and makes up for the weakness of extreme precipitation forecasting.",{"type":17,"tag":25,"props":106,"children":107},{},[108],{"type":17,"tag":29,"props":109,"children":110},{},[111],{"type":23,"value":112},"2. PreDiff",{"type":17,"tag":25,"props":114,"children":115},{},[116],{"type":23,"value":117},"As a type of generative model, diffusion models have attracted more and more attention from AI academia and industry since 2020 and have been applied to precipitation nowcasting. PreDiff [3] is a precipitation nowcasting model proposed by Boson AI and Amazon based on latent diffusion models. The model structure is as follows:",{"type":17,"tag":25,"props":119,"children":120},{},[121],{"type":17,"tag":74,"props":122,"children":124},{"alt":7,"src":123},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2023/11/15/4b88250c7d99459a85183c2d784ccbfe.png",[],{"type":17,"tag":25,"props":126,"children":127},{},[128,130,135,137,142,144,148,150,155,157,162,164,169,171,175,177,182,184,189,191,195,197,202],{"type":23,"value":129},"Based on the training idea of the latent diffusion model, PreDiff is first trained on radar echo data based on the VAE, to obtain an encoder ",{"type":17,"tag":29,"props":131,"children":132},{},[133],{"type":23,"value":134},"E",{"type":23,"value":136}," and a decoder ",{"type":17,"tag":29,"props":138,"children":139},{},[140],{"type":23,"value":141},"D",{"type":23,"value":143},". The encoder ",{"type":17,"tag":29,"props":145,"children":146},{},[147],{"type":23,"value":134},{"type":23,"value":149}," encodes the observation sequence y into a latent context ",{"type":17,"tag":29,"props":151,"children":152},{},[153],{"type":23,"value":154},"zcond",{"type":23,"value":156},". The latent diffusion model then generates the prediction result ",{"type":17,"tag":29,"props":158,"children":159},{},[160],{"type":23,"value":161},"z0",{"type":23,"value":163}," by autoregressively denoising Gaussian noise ",{"type":17,"tag":29,"props":165,"children":166},{},[167],{"type":23,"value":168},"zT",{"type":23,"value":170}," conditioned on ",{"type":17,"tag":29,"props":172,"children":173},{},[174],{"type":23,"value":154},{"type":23,"value":176},". The transition distribution of each step from ",{"type":17,"tag":29,"props":178,"children":179},{},[180],{"type":23,"value":181},"zt+1",{"type":23,"value":183}," to ",{"type":17,"tag":29,"props":185,"children":186},{},[187],{"type":23,"value":188},"zt",{"type":23,"value":190}," can be further refined via knowledge control, according to auxiliary prior knowledge. ",{"type":17,"tag":29,"props":192,"children":193},{},[194],{"type":23,"value":161},{"type":23,"value":196}," is decoded back to pixel space by the decoder D to produce the final prediction ",{"type":17,"tag":29,"props":198,"children":199},{},[200],{"type":23,"value":201},"xb",{"type":23,"value":203},". In addition, the Earthformer-UNet architecture is used in PreDiff.",{"type":17,"tag":25,"props":205,"children":206},{},[207,209,214],{"type":23,"value":208},"In the PreDiff model, the author focuses on the importance of prior knowledge integration. Though diffusion models hold great promise for diverse and realistic generation, the generated predictions may violate physical behaviors by producing implausible forecasts or disregarding domain-specific expertise. One possible reason for this is that diffusion models are not necessarily trained on data fully compliant with domain knowledge. When trained on curated data, there is no guarantee that the generations sampled from the learned distribution will remain physically realizable. To achieve knowledge control, PreDiff trains a neural network U, whose training loss is the difference between the control constraint of prior knowledge and the network output. At inference time, knowledge control is applied as a plug-in without impacting the trained VAE and the LDM. This modular approach allows the training lightweight knowledge control network ",{"type":17,"tag":29,"props":210,"children":211},{},[212],{"type":23,"value":213},"U",{"type":23,"value":215}," to impose different constraints without retraining the full model. The key advantage over incorporating constraints into model architectures or training losses is the flexibility to quickly explore diverse domain knowledge.",{"type":17,"tag":25,"props":217,"children":218},{},[219],{"type":17,"tag":29,"props":220,"children":221},{},[222],{"type":23,"value":223},"3. Summary",{"type":17,"tag":25,"props":225,"children":226},{},[227],{"type":23,"value":228},"Since the advent of DGMR, generative models have been widely used in precipitation nowcasting. This type of models can effectively solve the problem of blurry prediction results of existing AI models. At the same time, prior knowledge also plays an important role in the precipitation nowcasting models. From the perspective of data distribution or mechanism equations, the precipitation nowcasting models driven by data-knowledge integration will gradually become the future evolution direction.",{"type":17,"tag":25,"props":230,"children":231},{},[232],{"type":17,"tag":29,"props":233,"children":234},{},[235],{"type":23,"value":236},"References",{"type":17,"tag":25,"props":238,"children":239},{},[240,242],{"type":23,"value":241},"[1]Ravuri S, Lenc K, Willson M, et al. Skilful precipitation nowcasting using deep generative models of radar[J]. Nature, 2021, 597(7878): 672-677.",{"type":17,"tag":243,"props":244,"children":248},"a",{"href":245,"rel":246},"https://doi.org/10.1038/s41586-021-03854-z",[247],"nofollow",[249],{"type":23,"value":245},{"type":17,"tag":25,"props":251,"children":252},{},[253,255],{"type":23,"value":254},"[2]Zhang Y, Long M, Chen K, et al. Skilful nowcasting of extreme precipitation with NowcastNet[J]. Nature, 2023, 619(7970): 526-532.",{"type":17,"tag":243,"props":256,"children":259},{"href":257,"rel":258},"https://doi.org/10.1038/s41586-023-06184-4",[247],[260],{"type":23,"value":257},{"type":17,"tag":25,"props":262,"children":263},{},[264,266],{"type":23,"value":265},"[3]Gao Z, Shi X, Han B, et al. PreDiff: Precipitation Nowcasting with Latent Diffusion Models[J]. arXiv preprint arXiv:2307.10422, 2023.",{"type":17,"tag":243,"props":267,"children":270},{"href":268,"rel":269},"https://arxiv.org/abs/2307.10422",[247],[271],{"type":23,"value":268},{"title":7,"searchDepth":273,"depth":273,"links":274},4,[],"markdown","content:technology-blogs:en:2871.md","content","technology-blogs/en/2871.md","technology-blogs/en/2871","md",1776506107803]