{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# MindSpore NumPy函数\n",
    "\n",
    "`Ascend` `GPU` `CPU` `模型开发`"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "[![在线运行](https://gitee.com/mindspore/docs/raw/r1.6/resource/_static/logo_modelarts.png)](https://authoring-modelarts-cnnorth4.huaweicloud.com/console/lab?share-url-b64=aHR0cHM6Ly9taW5kc3BvcmUtd2Vic2l0ZS5vYnMuY24tbm9ydGgtNC5teWh1YXdlaWNsb3VkLmNvbS9ub3RlYm9vay9tYXN0ZXIvcHJvZ3JhbW1pbmdfZ3VpZGUvemhfY24vbWluZHNwb3JlX251bXB5LmlweW5i&imageid=65f636a0-56cf-49df-b941-7d2a07ba8c8c) [![下载Notebook](https://gitee.com/mindspore/docs/raw/r1.6/resource/_static/logo_notebook.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r1.6/programming_guide/zh_cn/mindspore_numpy.ipynb) [![下载样例代码](https://gitee.com/mindspore/docs/raw/r1.6/resource/_static/logo_download_code.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r1.6/programming_guide/zh_cn/mindspore_numpy.py) [![查看源文件](https://gitee.com/mindspore/docs/raw/r1.6/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/r1.6/docs/mindspore/programming_guide/source_zh_cn/numpy.ipynb)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 概述\n",
    "\n",
    "MindSpore NumPy工具包提供了一系列类NumPy接口。用户可以使用类NumPy语法在MindSpore上进行模型的搭建。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "> 本样例适用于CPU、GPU和Ascend环境。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 算子介绍\n",
    "\n",
    "MindSpore Numpy具有四大功能模块：张量生成、张量操作、逻辑运算和其他常用数学运算。算子的具体相关信息可以参考[NumPy接口列表](https://www.mindspore.cn/docs/api/zh-CN/r1.6/api_python/mindspore.numpy.html)。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 张量生成\n",
    "\n",
    "生成类算子用来生成和构建具有指定数值、类型和形状的数组(Tensor)。\n",
    "\n",
    "构建数组代码示例："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "source": [
    "import mindspore.numpy as np\n",
    "import mindspore.ops as ops\n",
    "input_x = np.array([1, 2, 3], np.float32)\n",
    "print(\"input_x =\", input_x)\n",
    "print(\"type of input_x =\", ops.typeof(input_x))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "input_x = [1. 2. 3.]\n",
      "type of input_x = Tensor[Float32]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "除了使用上述方法来创建外，也可以通过以下几种方式创建。\n",
    "\n",
    "#### 生成具有相同元素的数组\n",
    "\n",
    "生成具有相同元素的数组代码示例："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "source": [
    "input_x = np.full((2, 3), 6, np.float32)\n",
    "print(input_x)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[[6. 6. 6.]\n",
      " [6. 6. 6.]]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "生成指定形状的全1数组，示例："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "source": [
    "input_x = np.ones((2, 3), np.float32)\n",
    "print(input_x)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[[1. 1. 1.]\n",
      " [1. 1. 1.]]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 生成具有某个范围内的数值的数组\n",
    "\n",
    "生成指定范围内的等差数组代码示例："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "source": [
    "input_x = np.arange(0, 5, 1)\n",
    "print(input_x)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[0 1 2 3 4]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 生成特殊类型的数组\n",
    "\n",
    "生成给定对角线处下方元素为1，上方元素为0的矩阵，示例："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "source": [
    "input_x = np.tri(3, 3, 1)\n",
    "print(input_x)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[[1. 1. 0.]\n",
      " [1. 1. 1.]\n",
      " [1. 1. 1.]]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "生成对角线为1，其他元素为0的二维矩阵，示例："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "source": [
    "input_x = np.eye(2, 2)\n",
    "print(input_x)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[[1. 0.]\n",
      " [0. 1.]]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 张量操作\n",
    "\n",
    "变换类算子主要进行数组的维度变换，分割和拼接等。\n",
    "\n",
    "#### 数组维度变换\n",
    "\n",
    "矩阵转置，代码示例："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "source": [
    "input_x = np.arange(10).reshape(5, 2)\n",
    "output = np.transpose(input_x)\n",
    "print(output)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[[0 2 4 6 8]\n",
      " [1 3 5 7 9]]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "交换指定轴，代码示例："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "source": [
    "input_x = np.ones((1, 2, 3))\n",
    "output = np.swapaxes(input_x, 0, 1)\n",
    "print(output.shape)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "(2, 1, 3)\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 数组分割\n",
    "\n",
    "将输入数组平均切分为多个数组，代码示例："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "source": [
    "input_x = np.arange(9)\n",
    "output = np.split(input_x, 3)\n",
    "print(output)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "(Tensor(shape=[3], dtype=Int32, value= [0, 1, 2]), Tensor(shape=[3], dtype=Int32, value= [3, 4, 5]), Tensor(shape=[3], dtype=Int32, value= [6, 7, 8]))\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 数组拼接\n",
    "\n",
    "将两个数组按照指定轴进行拼接，代码示例："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "source": [
    "input_x = np.arange(0, 5)\n",
    "input_y = np.arange(10, 15)\n",
    "output = np.concatenate((input_x, input_y), axis=0)\n",
    "print(output)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[ 0  1  2  3  4 10 11 12 13 14]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 逻辑运算\n",
    "\n",
    "逻辑计算类算子主要进行逻辑运算。\n",
    "\n",
    "相等（equal）和小于（less）计算代码示例如下："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "source": [
    "input_x = np.arange(0, 5)\n",
    "input_y = np.arange(0, 10, 2)\n",
    "output = np.equal(input_x, input_y)\n",
    "print(\"output of equal:\", output)\n",
    "output = np.less(input_x, input_y)\n",
    "print(\"output of less:\", output)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "output of equal: [ True False False False False]\n",
      "output of less: [False  True  True  True  True]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 数学运算\n",
    "\n",
    "数学计算类算子主要进行各类数学计算：\n",
    "加减乘除乘方，以及指数、对数等常见函数等。\n",
    "\n",
    "数学计算支持类似NumPy的广播特性。\n",
    "\n",
    "#### 加法\n",
    "\n",
    "以下代码实现了`input_x`和`input_y`两数组相加的操作："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "source": [
    "input_x = np.full((3, 2), [1, 2])\n",
    "input_y = np.full((3, 2), [3, 4])\n",
    "output = np.add(input_x, input_y)\n",
    "print(output)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[[4 6]\n",
      " [4 6]\n",
      " [4 6]]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 矩阵乘法\n",
    "\n",
    "以下代码实现了`input_x`和`input_y`两矩阵相乘的操作："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "source": [
    "input_x = np.arange(2*3).reshape(2, 3).astype('float32')\n",
    "input_y = np.arange(3*4).reshape(3, 4).astype('float32')\n",
    "output = np.matmul(input_x, input_y)\n",
    "print(output)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[[20. 23. 26. 29.]\n",
      " [56. 68. 80. 92.]]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 求平均值\n",
    "\n",
    "以下代码实现了求`input_x`所有元素的平均值的操作："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "source": [
    "input_x = np.arange(6).astype('float32')\n",
    "output = np.mean(input_x)\n",
    "print(output)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "2.5\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 指数\n",
    "\n",
    "以下代码实现了自然常数`e`的`input_x`次方的操作："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "source": [
    "input_x = np.arange(5).astype('float32')\n",
    "output = np.exp(input_x)\n",
    "print(output)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[ 1.         2.7182817  7.389056  20.085537  54.59815  ]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## MindSpore Numpy与MindSpore特性结合\n",
    "\n",
    "`mindspore.numpy`能够充分利用MindSpore的强大功能，实现算子的自动微分，并使用图模式加速运算，帮助用户快速构建高效的模型。同时，MindSpore还支持多种后端设备，包括`Ascend`、`GPU`和`CPU`等，用户可以根据自己的需求灵活设置。以下提供了几种常用方法：\n",
    "\n",
    "- `ms_function`: 将代码包裹进图模式，用于提高代码运行效率。\n",
    "- `GradOperation`: 用于自动求导。\n",
    "- `mindspore.context`: 用于设置运行模式和后端设备等。\n",
    "- `mindspore.nn.Cell`: 用于建立深度学习模型。\n",
    "\n",
    "### ms_function使用示例\n",
    "\n",
    "首先，以神经网络里经常使用到的矩阵乘与矩阵加算子为例："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "source": [
    "import mindspore.numpy as np\n",
    "\n",
    "x = np.arange(8).reshape(2, 4).astype('float32')\n",
    "w1 = np.ones((4, 8))\n",
    "b1 = np.zeros((8,))\n",
    "w2 = np.ones((8, 16))\n",
    "b2 = np.zeros((16,))\n",
    "w3 = np.ones((16, 4))\n",
    "b3 = np.zeros((4,))\n",
    "\n",
    "def forward(x, w1, b1, w2, b2, w3, b3):\n",
    "    x = np.dot(x, w1) + b1\n",
    "    x = np.dot(x, w2) + b2\n",
    "    x = np.dot(x, w3) + b3\n",
    "    return x\n",
    "\n",
    "print(forward(x, w1, b1, w2, b2, w3, b3))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[[ 768.  768.  768.  768.]\n",
      " [2816. 2816. 2816. 2816.]]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "对上述示例，我们可以借助`ms_function`将所有算子编译到一张静态图里以加快运行效率，示例如下："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "source": [
    "from mindspore import ms_function\n",
    "\n",
    "forward_compiled = ms_function(forward)\n",
    "print(forward(x, w1, b1, w2, b2, w3, b3))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[[ 768.  768.  768.  768.]\n",
      " [2816. 2816. 2816. 2816.]]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "> 目前静态图不支持在Python交互式模式下运行，并且有部分语法限制。`ms_function`的更多信息可参考[API: ms_function](https://www.mindspore.cn/docs/api/zh-CN/r1.6/api_python/mindspore/mindspore.ms_function.html)。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### GradOperation使用示例\n",
    "\n",
    "`GradOperation` 可以实现自动求导。以下示例可以实现对上述没有用`ms_function`修饰的`forward`函数定义的计算求导。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "source": [
    "from mindspore import ops\n",
    "\n",
    "grad_all = ops.composite.GradOperation(get_all=True)\n",
    "print(grad_all(forward)(x, w1, b1, w2, b2, w3, b3))"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "(Tensor(shape=[2, 4], dtype=Float32, value=\n",
       " [[ 5.12000000e+02,  5.12000000e+02,  5.12000000e+02,  5.12000000e+02],\n",
       "  [ 5.12000000e+02,  5.12000000e+02,  5.12000000e+02,  5.12000000e+02]]),\n",
       " Tensor(shape=[4, 8], dtype=Float32, value=\n",
       " [[ 2.56000000e+02,  2.56000000e+02,  2.56000000e+02 ...  2.56000000e+02,  2.56000000e+02,  2.56000000e+02],\n",
       "  [ 3.84000000e+02,  3.84000000e+02,  3.84000000e+02 ...  3.84000000e+02,  3.84000000e+02,  3.84000000e+02],\n",
       "  [ 5.12000000e+02,  5.12000000e+02,  5.12000000e+02 ...  5.12000000e+02,  5.12000000e+02,  5.12000000e+02]\n",
       "  [ 6.40000000e+02,  6.40000000e+02,  6.40000000e+02 ...  6.40000000e+02,  6.40000000e+02,  6.40000000e+02]]),\n",
       "  ...\n",
       " Tensor(shape=[4], dtype=Float32, value= [ 2.00000000e+00,  2.00000000e+00,  2.00000000e+00,  2.00000000e+00]))"
      ]
     },
     "metadata": {},
     "execution_count": 18
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "如果要对`ms_function`修饰的`forward`计算求导，需要提前使用`context`设置运算模式为图模式，示例如下："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "source": [
    "from mindspore import ms_function, context\n",
    "\n",
    "context.set_context(mode=context.GRAPH_MODE)\n",
    "grad_all = ops.composite.GradOperation(get_all=True)\n",
    "print(grad_all(ms_function(forward))(x, w1, b1, w2, b2, w3, b3))"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "(Tensor(shape=[2, 4], dtype=Float32, value=\n",
       " [[ 5.12000000e+02,  5.12000000e+02,  5.12000000e+02,  5.12000000e+02],\n",
       "  [ 5.12000000e+02,  5.12000000e+02,  5.12000000e+02,  5.12000000e+02]]),\n",
       " Tensor(shape=[4, 8], dtype=Float32, value=\n",
       " [[ 2.56000000e+02,  2.56000000e+02,  2.56000000e+02 ...  2.56000000e+02,  2.56000000e+02,  2.56000000e+02],\n",
       "  [ 3.84000000e+02,  3.84000000e+02,  3.84000000e+02 ...  3.84000000e+02,  3.84000000e+02,  3.84000000e+02],\n",
       "  [ 5.12000000e+02,  5.12000000e+02,  5.12000000e+02 ...  5.12000000e+02,  5.12000000e+02,  5.12000000e+02]\n",
       "  [ 6.40000000e+02,  6.40000000e+02,  6.40000000e+02 ...  6.40000000e+02,  6.40000000e+02,  6.40000000e+02]]),\n",
       "  ...\n",
       " Tensor(shape=[4], dtype=Float32, value= [ 2.00000000e+00,  2.00000000e+00,  2.00000000e+00,  2.00000000e+00]))"
      ]
     },
     "metadata": {},
     "execution_count": 19
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "更多细节可参考[API: GradOperation](https://www.mindspore.cn/docs/api/zh-CN/r1.6/api_python/ops/mindspore.ops.GradOperation.html)。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### mindspore.context使用示例\n",
    "\n",
    "MindSpore支持多后端运算，可以通过`mindspore.context`进行设置。`mindspore.numpy` 的多数算子可以使用图模式或者PyNative模式运行，也可以运行在CPU，CPU或者Ascend等多种后端设备上。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "```python\n",
    "from mindspore import context\n",
    "\n",
    "# Execucation in static graph mode\n",
    "context.set_context(mode=context.GRAPH_MODE)\n",
    "\n",
    "# Execucation in PyNative mode\n",
    "context.set_context(mode=context.PYNATIVE_MODE)\n",
    "\n",
    "# Execucation on CPU backend\n",
    "context.set_context(device_target=\"CPU\")\n",
    "\n",
    "# Execucation on GPU backend\n",
    "context.set_context(device_target=\"GPU\")\n",
    "\n",
    "# Execucation on Ascend backend\n",
    "context.set_context(device_target=\"Ascend\")\n",
    "...\n",
    "```"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "更多细节可参考[API: mindspore.context](https://www.mindspore.cn/docs/api/zh-CN/r1.6/api_python/mindspore.context.html)。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### mindspore.numpy使用示例\n",
    "\n",
    "这里提供一个使用`mindspore.numpy`构建网络模型的示例。\n",
    "\n",
    "`mindspore.numpy` 接口可以定义在`nn.Cell`代码块内进行网络的构建，示例如下："
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "source": [
    "import mindspore.numpy as np\n",
    "from mindspore import context\n",
    "from mindspore.nn import Cell\n",
    "\n",
    "context.set_context(mode=context.GRAPH_MODE)\n",
    "\n",
    "x = np.arange(8).reshape(2, 4).astype('float32')\n",
    "w1 = np.ones((4, 8))\n",
    "b1 = np.zeros((8,))\n",
    "w2 = np.ones((8, 16))\n",
    "b2 = np.zeros((16,))\n",
    "w3 = np.ones((16, 4))\n",
    "b3 = np.zeros((4,))\n",
    "\n",
    "class NeuralNetwork(Cell):\n",
    "    def construct(self, x, w1, b1, w2, b2, w3, b3):\n",
    "        x = np.dot(x, w1) + b1\n",
    "        x = np.dot(x, w2) + b2\n",
    "        x = np.dot(x, w3) + b3\n",
    "        return x\n",
    "\n",
    "net = NeuralNetwork()\n",
    "\n",
    "print(net(x, w1, b1, w2, b2, w3, b3))\n"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[[ 768.  768.  768.  768.]\n",
      " [2816. 2816. 2816. 2816.]]\n"
     ]
    }
   ],
   "metadata": {}
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "MindSpore",
   "language": "python",
   "name": "mindspore"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}