[{"data":1,"prerenderedAt":360},["ShallowReactive",2],{"content-query-V3nTwiTwM6":3},{"_path":4,"_dir":5,"_draft":6,"_partial":6,"_locale":7,"title":8,"description":9,"date":10,"cover":11,"type":12,"category":13,"body":14,"_type":354,"_id":355,"_source":356,"_file":357,"_stem":358,"_extension":359},"/technology-blogs/en/3413","en",false,"","Spectral neural operator opens source in MindSpore Flow","Author: Yu Fan | Source: Zhihu","2024-06-28","https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/10/11/c1672c30b2eb4adca3452231960da799.png","technology-blogs","Practices",{"type":15,"children":16,"toc":349},"root",[17,25,35,40,45,60,68,73,78,86,91,99,104,109,114,122,127,134,139,153,158,166,171,178,183,191,196,203,208,213,220,225,232,237,242,249,254,268,273,278,288,301,313,325,337],{"type":18,"tag":19,"props":20,"children":22},"element","h1",{"id":21},"spectral-neural-operator-opens-source-in-mindspore-flow",[23],{"type":24,"value":8},"text",{"type":18,"tag":26,"props":27,"children":28},"p",{},[29],{"type":18,"tag":30,"props":31,"children":32},"strong",{},[33],{"type":24,"value":34},"Background",{"type":18,"tag":26,"props":36,"children":37},{},[38],{"type":24,"value":39},"Recently, spectral neural operator (SNO) and its use cases in Burgers 1D/Navier-Stokes 2D/Navier-Stokes 3D are open-sourced in MindSpore Flow.",{"type":18,"tag":26,"props":41,"children":42},{},[43],{"type":24,"value":44},"SNO is an FNO-like architecture that uses a polynomial to transform computation into a spectral space (Chebyshev, Legendre and more). Compared with FNO, the system deviation caused by obfuscation error is smaller in SNO. One of the most important benefits is that the base selection in SNO is broader, and you can find a set of polynomials that is most convenient to represent. For example, select an adaptive base according to the symmetry of the problem or the time interval. In addition, when the input is defined on unstructured grids, neural operators based on orthogonal polynomials are more competitive than other operators.",{"type":18,"tag":26,"props":46,"children":47},{},[48,53,55],{"type":18,"tag":30,"props":49,"children":50},{},[51],{"type":24,"value":52},"01",{"type":24,"value":54}," ",{"type":18,"tag":30,"props":56,"children":57},{},[58],{"type":24,"value":59},"SNO Principles",{"type":18,"tag":26,"props":61,"children":62},{},[63],{"type":18,"tag":30,"props":64,"children":65},{},[66],{"type":24,"value":67},"1.1 Neural Operator",{"type":18,"tag":26,"props":69,"children":70},{},[71],{"type":24,"value":72},"In recent years, the rapid development of neural network (NN) offers a new paradigm for scientific computing. The classical neural network performs mapping in a space of a limited dimension, and learns only solutions related to specific discretization. Unlike the classical neural network, the Fourier neural operator (FNO) is a new deep learning architecture that can learn an infinite-dimensional function space mapping. With stronger generalization capability, it can learn the mapping from an arbitrary function parameter to a solution and solve partial differential equation (PDE).",{"type":18,"tag":26,"props":74,"children":75},{},[76],{"type":24,"value":77},"However, the FNO has some disadvantages. For example, linear truncation after transformation to Fourier space causes aliasing errors, and complex number operation involved in Fourier transform consumes a relatively large quantity of computing resources.",{"type":18,"tag":26,"props":79,"children":80},{},[81],{"type":18,"tag":30,"props":82,"children":83},{},[84],{"type":24,"value":85},"1.2 Spectral Neural Operator (SNO)",{"type":18,"tag":26,"props":87,"children":88},{},[89],{"type":24,"value":90},"The solution provided by the SNO is to replace Fourier transform with polynomial approximation (such as Chebyshev approximation and Legendre polynomial) where only basic linear transform is required rather than Fourier transform and complex number calculations. For details about the specific architecture, see Figure 1.",{"type":18,"tag":26,"props":92,"children":93},{},[94],{"type":18,"tag":95,"props":96,"children":98},"img",{"alt":7,"src":97},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/10/11/f2f2c631191249dca5ea5f0282baaa89.png",[],{"type":18,"tag":26,"props":100,"children":101},{},[102],{"type":24,"value":103},"Figure 1",{"type":18,"tag":26,"props":105,"children":106},{},[107],{"type":24,"value":108},"Time series forecast in two-dimensional physical field is used as an example in Figure 1. First, in order to calculate the forward and inverse polynomial transform matrices of the spectral convolution, the input should be interpolated at the corresponding Gauss orthogonal node (Chebyshev grid and more). Then, the interpolated input is promoted to a higher dimension by using a convolutional encoder. The encoded result is used as an input to a series of SNO layers, each of which performs linear convolution on its truncated spectral representation. Next, the output of the SNO layer is projected back to the target dimension by the convolutional decoder, and finally interpolated back to the original node.",{"type":18,"tag":26,"props":110,"children":111},{},[112],{"type":24,"value":113},"The following operations will be performed at the SNO layer: First, apply polynomial transform to A (Chebyshev, Legendre, and more) to map the physical field from the physical space to the spectral space. Then, the L layer performs a linear convolution operation on a polynomial low-order mode and a filtering operation on a high-order mode. Next, an inverse transform S = A^{-1} (mapping from the spectral space back to the physical space) is applied. Finally, the output result is summed up with the direct linear convolution W of the input layer, and the nonlinear activation layer is used to complete the calculation of an SNO layer.",{"type":18,"tag":26,"props":115,"children":116},{},[117],{"type":18,"tag":30,"props":118,"children":119},{},[120],{"type":24,"value":121},"1.3 U-SNO",{"type":18,"tag":26,"props":123,"children":124},{},[125],{"type":24,"value":126},"Particularly, in a two-dimensional use case, adding several U-SNO layers to the nested SNO layers can achieve a better performance. An architecture of the U-SNO layer is similar to that of the SNO layer, and a difference lies in that the U-SNO layer uses UNet2D instead of direct linear convolution W. For details, see Figure 2.",{"type":18,"tag":26,"props":128,"children":129},{},[130],{"type":18,"tag":95,"props":131,"children":133},{"alt":7,"src":132},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/10/11/3927baffcd37471db7ccacb41b538d9e.png",[],{"type":18,"tag":26,"props":135,"children":136},{},[137],{"type":24,"value":138},"Figure 2",{"type":18,"tag":26,"props":140,"children":141},{},[142,147,148],{"type":18,"tag":30,"props":143,"children":144},{},[145],{"type":24,"value":146},"02",{"type":24,"value":54},{"type":18,"tag":30,"props":149,"children":150},{},[151],{"type":24,"value":152},"Model Performance",{"type":18,"tag":26,"props":154,"children":155},{},[156],{"type":24,"value":157},"Currently, the SNO cell is open-sourced in MindSpore Flow, as well as use cases in Burgers 1D/Navier-Stokes 2D/Navier-Stokes 3D.",{"type":18,"tag":26,"props":159,"children":160},{},[161],{"type":18,"tag":30,"props":162,"children":163},{},[164],{"type":24,"value":165},"2.1 Equation Solving",{"type":18,"tag":26,"props":167,"children":168},{},[169],{"type":24,"value":170},"In a two-dimensional linear equation and nonlinear equations including a hyperbolic equation, a parabolic equation, and an elliptic equation, accuracy of the SNO is improved by more than 1.5 times compared with that of the FNO, and a performance loss is within 10%. For details, see Figure 3",{"type":18,"tag":26,"props":172,"children":173},{},[174],{"type":18,"tag":95,"props":175,"children":177},{"alt":7,"src":176},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/10/11/ef0f44d00b5c44dc8bf0ac0a636d9c3d.png",[],{"type":18,"tag":26,"props":179,"children":180},{},[181],{"type":24,"value":182},"Figure 3 Comparison of Equation Solving Effects",{"type":18,"tag":26,"props":184,"children":185},{},[186],{"type":18,"tag":30,"props":187,"children":188},{},[189],{"type":24,"value":190},"2.2 Use Cases",{"type":18,"tag":26,"props":192,"children":193},{},[194],{"type":24,"value":195},"The following three open source datasets of MindSpore Flow are generated based on Zongyi Li's articles. In the Burgers1D case, an initial condition u_0(x) that satisfies the following distribution is generated based on a periodic boundary:",{"type":18,"tag":26,"props":197,"children":198},{},[199],{"type":18,"tag":95,"props":200,"children":202},{"alt":7,"src":201},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/10/11/4061a8669424457dbeb2b06d34f6d005.png",[],{"type":18,"tag":26,"props":204,"children":205},{},[206],{"type":24,"value":207},"The coefficient of viscosity v = 0.1 is selected, and the equation is solved by using the fractional step method, where the heat equation part is accurately solved in the Fourier space, and then the nonlinear part is solved by using the forward Euler method.",{"type":18,"tag":26,"props":209,"children":210},{},[211],{"type":24,"value":212},"In the Navier-Stokes2D and 3D cases, an initial condition w_0(x) that satisfies the following distribution is generated based on a periodic boundary:",{"type":18,"tag":26,"props":214,"children":215},{},[216],{"type":18,"tag":95,"props":217,"children":219},{"alt":7,"src":218},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/10/11/42ff98ec22364bd095de3a1883ce384f.png",[],{"type":18,"tag":26,"props":221,"children":222},{},[223],{"type":24,"value":224},"Set the external force as follows:",{"type":18,"tag":26,"props":226,"children":227},{},[228],{"type":18,"tag":95,"props":229,"children":231},{"alt":7,"src":230},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/10/11/6e95e8cec9cf492db8c9f100ff177862.png",[],{"type":18,"tag":26,"props":233,"children":234},{},[235],{"type":24,"value":236},"The Crank-Nicolson method is used to generate data, the time step size is set to 1e-4, and the final data is recorded in every t = 1 time unit. All data is generated on a 256 x 256 grid and downsampled to a 64 x 64 grid. The coefficient of viscosity v = 1e-5 is selected.",{"type":18,"tag":26,"props":238,"children":239},{},[240],{"type":24,"value":241},"The test result is shown in Figure-4. In the two-dimensional case, the SNO precision is slightly improved, and the performance is significantly improved. In the one-dimensional and three-dimensional cases, the FNO performance is better than that of the SNO.",{"type":18,"tag":26,"props":243,"children":244},{},[245],{"type":18,"tag":95,"props":246,"children":248},{"alt":7,"src":247},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/10/11/785fbeb8c0c44b2497c71c916a427e4a.png",[],{"type":18,"tag":26,"props":250,"children":251},{},[252],{"type":24,"value":253},"Figure 4 Comparison of Use Cases Effect",{"type":18,"tag":26,"props":255,"children":256},{},[257,262,263],{"type":18,"tag":30,"props":258,"children":259},{},[260],{"type":24,"value":261},"03",{"type":24,"value":54},{"type":18,"tag":30,"props":264,"children":265},{},[266],{"type":24,"value":267},"Conclusion",{"type":18,"tag":26,"props":269,"children":270},{},[271],{"type":24,"value":272},"The SNO uses polynomial approximation to replace the Fourier transform in FNO, and can complete the mapping from the physical field to the spectral space through basic linear calculation. Orthogonal basis has more options, and can be selected based on practice. In the validation case, SNO is particularly suitable for the physical field prediction problem of the two-dimensional Navier-Stokes equation.",{"type":18,"tag":26,"props":274,"children":275},{},[276],{"type":24,"value":277},"SNO cells and related use cases are open-sourced in MindSpore Flow. Feel free to join our community and use those cases.",{"type":18,"tag":279,"props":280,"children":282},"h2",{"id":281},"references",[283],{"type":18,"tag":30,"props":284,"children":285},{},[286],{"type":24,"value":287},"References",{"type":18,"tag":26,"props":289,"children":290},{},[291,293],{"type":24,"value":292},"[1] Fanaskov V S, Oseledets I V. Spectral neural operators[C]//Doklady Mathematics. Moscow: Pleiades Publishing, 2023, 108(Suppl 2): S226-S232. ",{"type":18,"tag":294,"props":295,"children":299},"a",{"href":296,"rel":297},"https://arxiv.org/abs/2205.10573",[298],"nofollow",[300],{"type":24,"value":296},{"type":18,"tag":26,"props":302,"children":303},{},[304,306],{"type":24,"value":305},"[2] SNO cell: ",{"type":18,"tag":294,"props":307,"children":310},{"href":308,"rel":309},"https://gitee.com/mindspore/mindscience/blob/master/MindFlow/mindflow/cell/neural%5C_operators/sno.py",[298],[311],{"type":24,"value":312},"https://gitee.com/mindspore/mindscience/blob/master/MindFlow/mindflow/cell/neural\\_operators/sno.py",{"type":18,"tag":26,"props":314,"children":315},{},[316,318],{"type":24,"value":317},"[3] SNO Burgers1D application: ",{"type":18,"tag":294,"props":319,"children":322},{"href":320,"rel":321},"https://gitee.com/mindspore/mindscience/tree/master/MindFlow/applications/data%5C_driven/burgers/sno1d",[298],[323],{"type":24,"value":324},"https://gitee.com/mindspore/mindscience/tree/master/MindFlow/applications/data\\_driven/burgers/sno1d",{"type":18,"tag":26,"props":326,"children":327},{},[328,330],{"type":24,"value":329},"[4] SNO Navier-Stokes2D application: ",{"type":18,"tag":294,"props":331,"children":334},{"href":332,"rel":333},"https://gitee.com/mindspore/mindscience/tree/master/MindFlow/applications/data%5C_driven/navier%5C_stokes/sno2d",[298],[335],{"type":24,"value":336},"https://gitee.com/mindspore/mindscience/tree/master/MindFlow/applications/data\\_driven/navier\\_stokes/sno2d",{"type":18,"tag":26,"props":338,"children":339},{},[340,342],{"type":24,"value":341},"[5] SNO Navier-Stokes3D application: ",{"type":18,"tag":294,"props":343,"children":346},{"href":344,"rel":345},"https://gitee.com/mindspore/mindscience/tree/master/MindFlow/applications/data%5C_driven/navier%5C_stokes/sno3d",[298],[347],{"type":24,"value":348},"https://gitee.com/mindspore/mindscience/tree/master/MindFlow/applications/data\\_driven/navier\\_stokes/sno3d",{"title":7,"searchDepth":350,"depth":350,"links":351},4,[352],{"id":281,"depth":353,"text":287},2,"markdown","content:technology-blogs:en:3413.md","content","technology-blogs/en/3413.md","technology-blogs/en/3413","md",1776506111246]