{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# FNO for 1D Burgers\n",
    "\n",
    "[![DownloadNotebook](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0.0-alpha/resource/_static/logo_notebook_en.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/r2.0.0-alpha/mindflow/en/data_driven/mindspore_FNO1D.ipynb) [![DownloadCode](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0.0-alpha/resource/_static/logo_download_code_en.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/r2.0.0-alpha/mindflow/en/data_driven/mindspore_FNO1D.py) [![ViewSource](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0.0-alpha/resource/_static/logo_source_en.png)](https://gitee.com/mindspore/docs/blob/r2.0.0-alpha/docs/mindflow/docs/source_en/data_driven/FNO1D.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Overview\n",
    "\n",
    "Computational fluid dynamics is one of the most important techniques in the field of fluid mechanics in the 21st century. The flow analysis, prediction and control can be realized by solving the governing equations of fluid mechanics by numerical method. Traditional finite element method (FEM) and finite difference method (FDM) are inefficient because of the complex simulation process (physical modeling, meshing, numerical discretization, iterative solution, etc.) and high computing costs. Therefore, it is necessary to improve the efficiency of fluid simulation with AI.\n",
    "\n",
    "Machine learning methods provide a new paradigm for scientific computing by providing a fast solver similar to traditional methods. Classical neural networks learn mappings between finite dimensional spaces and can only learn solutions related to a specific discretization. Different from traditional neural networks, Fourier Neural Operator (FNO) is a new deep learning architecture that can learn mappings between infinite-dimensional function spaces. It directly learns mappings from arbitrary function parameters to solutions to solve a class of partial differential equations.  Therefore, it has a stronger generalization capability. More information can be found in the paper, [Fourier Neural Operator for Parametric Partial Differential Equations](https://arxiv.org/abs/2010.08895).\n",
    "\n",
    "This tutorial describes how to solve the 1-d Burgers' equation using Fourier neural operator."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Burgers' equation\n",
    "\n",
    "The 1-d Burgers’ equation is a non-linear PDE with various applications including modeling the one\n",
    "dimensional flow of a viscous fluid. It takes the form\n",
    "\n",
    "$$\n",
    "\\partial_t u(x, t)+\\partial_x (u^2(x, t)/2)=\\nu \\partial_{xx} u(x, t), \\quad x \\in(0,1), t \\in(0, 1]\n",
    "$$\n",
    "\n",
    "$$\n",
    "u(x, 0)=u_0(x), \\quad x \\in(0,1)\n",
    "$$\n",
    "\n",
    "where $u$ is the velocity field, $u_0$ is the initial condition and $\\nu$ is the viscosity coefficient.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem Description\n",
    "\n",
    "We aim to learn the operator mapping the initial condition to the solution at time one:\n",
    "\n",
    "$$\n",
    "u_0 \\mapsto u(\\cdot, 1)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Technology Path\n",
    "\n",
    "MindFlow solves the problem as follows:\n",
    "\n",
    "1. Training Dataset Construction.\n",
    "2. Model Construction.\n",
    "3. Optimizer and Loss Function.\n",
    "4. Define Solver.\n",
    "5. Define Callback.\n",
    "6. Model Training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fourier Neural Operator\n",
    "\n",
    "The Fourier Neural Operator consists of the Lifting Layer, Fourier Layers, and the Decoding Layer.\n",
    "\n",
    "![Fourier Neural Operator model structure](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0.0-alpha/docs/mindflow/docs/source_en/data_driven/images/FNO.png)\n",
    "\n",
    "Fourier layers: Start from input V. On top: apply the Fourier transform $\\mathcal{F}$; a linear transform R on the lower Fourier modes and filters out the higher modes; then apply the inverse Fourier transform $\\mathcal{F}^{-1}$. On the bottom: apply a local linear transform W.  Finally, the Fourier Layer output vector is obtained through the activation function.\n",
    "\n",
    "![Fourier Layer structure](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0.0-alpha/docs/mindflow/docs/source_en/data_driven/images/FNO-2.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "from mindspore import context, nn, Tensor, set_seed\n",
    "from mindspore import DynamicLossScaleManager, LossMonitor, TimeMonitor\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following `src` pacakage can be downloaded in [applications/data_driven/burgers/src](https://gitee.com/mindspore/mindscience/tree/r0.2.0-alpha/MindFlow/applications/data_driven/burgers/src)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from mindflow import FNO1D, RelativeRMSELoss, Solver, load_yaml_config, get_warmup_cosine_annealing_lr\n",
    "\n",
    "from src import PredictCallback, create_training_dataset\n",
    "\n",
    "\n",
    "set_seed(0)\n",
    "np.random.seed(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "context.set_context(mode=context.GRAPH_MODE, device_target='GPU', device_id=4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "config = load_yaml_config(\"burgers1d.yaml\")\n",
    "data_params = config[\"data\"]\n",
    "model_params = config[\"model\"]\n",
    "optimizer_params = config[\"optimizer\"]\n",
    "callback_params = config[\"callback\"]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Training Dataset Construction\n",
    "\n",
    "Download the training and test dataset: [data_driven/burgers/dataset](https://download.mindspore.cn/mindscience/mindflow/dataset/applications/data_driven/burgers/dataset/) .\n",
    "\n",
    "In this case, training datasets and test datasets are generated according to Zongyi Li's dataset in [Fourier Neural Operator for Parametric Partial Differential Equations](https://arxiv.org/pdf/2010.08895.pdf) . The settings are as follows:\n",
    "\n",
    "the initial condition $u_0(x)$ is generated according to periodic boundary conditions:\n",
    "\n",
    "$$\n",
    "u_0 \\sim \\mu, \\mu=\\mathcal{N}\\left(0,625(-\\Delta+25 I)^{-2}\\right)\n",
    "$$\n",
    "\n",
    "We set the viscosity to $\\nu=0.1$ and solve the equation using a split step method where the heat equation part is solved exactly in Fourier space then the non-linear part is advanced, again in Fourier space, using a very fine forward Euler method. The number of samples in the training set is 1000, and the number of samples in the test set is 200.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data preparation finished\n",
      "input_path:  (1000, 1024, 1)\n",
      "label_path:  (1000, 1024)\n"
     ]
    }
   ],
   "source": [
    "# create training dataset\n",
    "train_dataset = create_training_dataset(data_params, shuffle=True)\n",
    "\n",
    "# create test dataset\n",
    "test_input, test_label = np.load(os.path.join(data_params[\"path\"], \"test/inputs.npy\")), \\\n",
    "                         np.load(os.path.join(data_params[\"path\"], \"test/label.npy\"))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Model Construction\n",
    "\n",
    "The network is composed of 1 lifting layer, multiple Fourier layers and 1 decoding layer:\n",
    "\n",
    "- The Lifting layer corresponds to the `FNO1D.fc0` in the case, and maps the output data $x$ to the high dimension;\n",
    "\n",
    "- Multi-layer Fourier Layer corresponds to the `FNO1D.fno_seq` in the case. Discrete Fourier transform is used to realize the conversion between time domain and frequency domain;\n",
    "\n",
    "- The Decoding layer corresponds to `FNO1D.fc1` and `FNO1D.fc2` in the case to obtain the final predictive value.\n",
    "\n",
    "The initialization of the model based on the network above, parameters can be modified in [configuration file](https://gitee.com/mindspore/mindscience/blob/r0.2.0-alpha/MindFlow/applications/data_driven/burgers/burgers1d.yaml)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "model = FNO1D(in_channels=model_params[\"in_channels\"],\n",
    "              out_channels=model_params[\"out_channels\"],\n",
    "              resolution=model_params[\"resolution\"],\n",
    "              modes=model_params[\"modes\"],\n",
    "              channels=model_params[\"width\"],\n",
    "              depth=model_params[\"depth\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Optimizer and Loss Function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "steps_per_epoch = train_dataset.get_dataset_size()\n",
    "lr = get_warmup_cosine_annealing_lr(lr_init=optimizer_params[\"initial_lr\"],\n",
    "                                    last_epoch=optimizer_params[\"train_epochs\"],\n",
    "                                    steps_per_epoch=steps_per_epoch,\n",
    "                                    warmup_epochs=1)\n",
    "optimizer = nn.Adam(model.trainable_params(), learning_rate=Tensor(lr))\n",
    "loss_scale = DynamicLossScaleManager()\n",
    "\n",
    "loss_fn = RelativeRMSELoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Define Solver\n",
    "\n",
    "Solver class is the interface for model training and evaluation. Given the optimizer, network model, loss function, loss scaling strategy, etc. the solver object can be defined easily. Parameters of `optimizer_params` and `model_params` can be modified in [configuration file](https://gitee.com/mindspore/mindscience/blob/r0.2.0-alpha/MindFlow/applications/data_driven/burgers/burgers1d.yaml)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "solver = Solver(model,\n",
    "                optimizer=optimizer,\n",
    "                loss_scale_manager=loss_scale,\n",
    "                loss_fn=loss_fn,\n",
    "                )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Define Callback"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./FNO1D\n",
      "check test dataset shape: (200, 1024, 1), (200, 1024)\n"
     ]
    }
   ],
   "source": [
    "summary_dir = os.path.join(callback_params[\"summary_dir\"], \"FNO1D\")\n",
    "print(summary_dir)\n",
    "pred_cb = PredictCallback(model=model,\n",
    "                          inputs=test_input,\n",
    "                          label=test_label,\n",
    "                          config=config,\n",
    "                          summary_dir=summary_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Model Training\n",
    "\n",
    "Invoke the Solver interface for model training and callback interface for evaluation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 1 step: 125, loss is 2.377823\n",
      "Train epoch time: 5782.938 ms, per step time: 46.264 ms\n",
      "epoch: 2 step: 125, loss is 0.88470775\n",
      "Train epoch time: 1150.446 ms, per step time: 9.204 ms\n",
      "epoch: 3 step: 125, loss is 0.98071647\n",
      "Train epoch time: 1135.464 ms, per step time: 9.084 ms\n",
      "epoch: 4 step: 125, loss is 0.5404751\n",
      "Train epoch time: 1114.245 ms, per step time: 8.914 ms\n",
      "epoch: 5 step: 125, loss is 0.39976493\n",
      "Train epoch time: 1125.107 ms, per step time: 9.001 ms\n",
      "epoch: 6 step: 125, loss is 0.508416\n",
      "Train epoch time: 1127.477 ms, per step time: 9.020 ms\n",
      "epoch: 7 step: 125, loss is 0.42839915\n",
      "Train epoch time: 1125.775 ms, per step time: 9.006 ms\n",
      "epoch: 8 step: 125, loss is 0.28270185\n",
      "Train epoch time: 1118.428 ms, per step time: 8.947 ms\n",
      "epoch: 9 step: 125, loss is 0.24137405\n",
      "Train epoch time: 1121.705 ms, per step time: 8.974 ms\n",
      "epoch: 10 step: 125, loss is 0.22623646\n",
      "Train epoch time: 1118.699 ms, per step time: 8.950 ms\n",
      "================================Start Evaluation================================\n",
      "mean rms_error: 0.03270653011277318\n",
      "=================================End Evaluation=================================\n",
      "...\n",
      "predict total time: 0.5012176036834717 s\n",
      "epoch: 91 step: 125, loss is 0.026378194\n",
      "Train epoch time: 1119.095 ms, per step time: 8.953 ms\n",
      "epoch: 92 step: 125, loss is 0.057838168\n",
      "Train epoch time: 1116.712 ms, per step time: 8.934 ms\n",
      "epoch: 93 step: 125, loss is 0.034773324\n",
      "Train epoch time: 1107.931 ms, per step time: 8.863 ms\n",
      "epoch: 94 step: 125, loss is 0.029720988\n",
      "Train epoch time: 1109.336 ms, per step time: 8.875 ms\n",
      "epoch: 95 step: 125, loss is 0.02933883\n",
      "Train epoch time: 1111.804 ms, per step time: 8.894 ms\n",
      "epoch: 96 step: 125, loss is 0.03140598\n",
      "Train epoch time: 1116.788 ms, per step time: 8.934 ms\n",
      "epoch: 97 step: 125, loss is 0.03695058\n",
      "Train epoch time: 1115.020 ms, per step time: 8.920 ms\n",
      "epoch: 98 step: 125, loss is 0.039841708\n",
      "Train epoch time: 1120.316 ms, per step time: 8.963 ms\n",
      "epoch: 99 step: 125, loss is 0.039001673\n",
      "Train epoch time: 1134.618 ms, per step time: 9.077 ms\n",
      "epoch: 100 step: 125, loss is 0.038434036\n",
      "Train epoch time: 1116.549 ms, per step time: 8.932 ms\n",
      "================================Start Evaluation================================\n",
      "mean rms_error: 0.005707952339434996\n",
      "=================================End Evaluation=================================\n",
      "predict total time: 0.5055065155029297 s\n"
     ]
    }
   ],
   "source": [
    "solver.train(epoch=optimizer_params[\"train_epochs\"],\n",
    "             train_dataset=train_dataset,\n",
    "             callbacks=[LossMonitor(), TimeMonitor(), pred_cb],\n",
    "             dataset_sink_mode=True)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.0 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  },
  "vscode": {
   "interpreter": {
    "hash": "df0893f56f349688326838aaeea0de204df53a132722cbd565e54b24a8fec5f6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}