{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 算子分类\n",
    "\n",
    "[![在线运行](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.7/resource/_static/logo_modelarts.png)](https://authoring-modelarts-cnnorth4.huaweicloud.com/console/lab?share-url-b64=aHR0cHM6Ly9taW5kc3BvcmUtd2Vic2l0ZS5vYnMuY24tbm9ydGgtNC5teWh1YXdlaWNsb3VkLmNvbS9ub3RlYm9vay9yMS43L3R1dG9yaWFscy9leHBlcnRzL3poX2NuL29wZXJhdGlvbi9taW5kc3BvcmVfb3BfY2xhc3NpZmljYXRpb24uaXB5bmI=&imageid=9d63f4d1-dc09-4873-b669-3483cea777c0) [![下载Notebook](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.7/resource/_static/logo_notebook.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r1.7/tutorials/experts/zh_cn/operation/mindspore_op_classification.ipynb) [![下载样例代码](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.7/resource/_static/logo_download_code.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r1.7/tutorials/experts/zh_cn/operation/mindspore_op_classification.py) [![查看源文件](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.7/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/r1.7/tutorials/experts/source_zh_cn/operation/op_classification.ipynb)\n",
    "\n",
    "算子主要分为Primitivie算子和nn算子。所有的算子在Ascend AI处理器、GPU和CPU的支持情况，参见[算子支持列表](https://www.mindspore.cn/docs/zh-CN/r1.7/note/operator_list.html)。\n",
    "\n",
    "## Primitive算子\n",
    "\n",
    "Primitive算子是开放给用户的最低阶算子接口，一个Primitive算子对应一个原语，它封装了底层的Ascend、GPU、AICPU、CPU等多种算子的具体实现，为用户提供基础算子能力。\n",
    "\n",
    "Primitive算子接口是构建高阶接口、自动微分、网络模型等能力的基础。\n",
    "\n",
    "Primitive算子可以分为[计算算子](#计算相关的算子)和[框架算子](#框架相关的算子)。计算算子主要负责具体的计算，而框架算子主要用于构图，自动微分等功能。\n",
    "\n",
    "functional接口是为简化没有属性的Primitive算子调用流程而提供的，functional接口、composite接口和Primitive算子都可以从mindspore.ops中导入。\n",
    "\n",
    "例如用户想使用pow功能，若使用Primitive算子，用户需要先实例化Pow算子，此时用户可以直接使用functional接口的tensor_pow来简化流程，代码示例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-04T11:29:05.058954Z",
     "start_time": "2022-01-04T11:29:03.387110Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1.  8. 64.]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import mindspore\n",
    "from mindspore import Tensor\n",
    "import mindspore.ops as ops\n",
    "\n",
    "input_x = mindspore.Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32)\n",
    "input_y = 3.0\n",
    "# 使用Primitive算子需先实例化\n",
    "pow = ops.Pow()\n",
    "output = pow(input_x, input_y)\n",
    "\n",
    "# 直接使用functional接口\n",
    "output = ops.tensor_pow(input_x, input_y)\n",
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "更多functional接口参见[functional接口](https://mindspore.cn/docs/zh-CN/r1.7/api_python/mindspore.ops.functional.html)。\n",
    "\n",
    "### 计算相关的算子\n",
    "\n",
    "计算算子按功能主要分为神经网络算子、数学算子、数组算子、通信算子等。\n",
    "\n",
    "#### 神经网络算子\n",
    "\n",
    "神经网络算子主要用于构建网络模型，比如卷积算子Conv2D，最大池化算子MaxPool等，参见[神经网络算子](https://www.mindspore.cn/docs/zh-CN/r1.7/api_python/mindspore.ops.html#神经网络层算子)。\n",
    "\n",
    "以下代码展示了最大池化算子MaxPool的使用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-04T11:29:05.071605Z",
     "start_time": "2022-01-04T11:29:05.061024Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[[ 5.  6.  7.]\n",
      "   [ 9. 10. 11.]]\n",
      "\n",
      "  [[17. 18. 19.]\n",
      "   [21. 22. 23.]]\n",
      "\n",
      "  [[29. 30. 31.]\n",
      "   [33. 34. 35.]]]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import mindspore\n",
    "from mindspore import Tensor\n",
    "import mindspore.ops as ops\n",
    "\n",
    "x = Tensor(np.arange(1 * 3 * 3 * 4).reshape((1, 3, 3, 4)), mindspore.float32)\n",
    "maxpool_op = ops.MaxPool(pad_mode=\"VALID\", kernel_size=2, strides=1)\n",
    "output = maxpool_op(x)\n",
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 数学算子\n",
    "\n",
    "数学算子主要是针对数学运算开发的算子，比如相加算子Add、求对数算子Log等，参见[数学算子](https://www.mindspore.cn/docs/zh-CN/r1.7/api_python/mindspore.ops.html#数学运算算子)。\n",
    "\n",
    "以下代码展示了求对数算子Log的使用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-04T11:29:05.095875Z",
     "start_time": "2022-01-04T11:29:05.073720Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-1.4305115e-06  6.9314575e-01  1.3862929e+00]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import mindspore\n",
    "from mindspore import Tensor\n",
    "import mindspore.ops as ops\n",
    "\n",
    "x = Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32)\n",
    "log_oo = ops.Log()\n",
    "output = log_oo(x)\n",
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 数组算子\n",
    "\n",
    "数组算子主要是针对数组类操作的算子，比如排序算子Sort、转置算子Transpose等，参见[数组算子](https://www.mindspore.cn/docs/zh-CN/r1.7/api_python/mindspore.ops.html#array操作)。\n",
    "\n",
    "以下代码展示了转置算子Transpose的使用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-04T11:29:05.106800Z",
     "start_time": "2022-01-04T11:29:05.098979Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[ 1.  4.]\n",
      "  [ 2.  5.]\n",
      "  [ 3.  6.]]\n",
      "\n",
      " [[ 7. 10.]\n",
      "  [ 8. 11.]\n",
      "  [ 9. 12.]]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import mindspore\n",
    "from mindspore import Tensor\n",
    "import mindspore.ops as ops\n",
    "\n",
    "input_x = Tensor(np.array([[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]]), mindspore.float32)\n",
    "input_perm = (0, 2, 1)\n",
    "transpose_op = ops.Transpose()\n",
    "output = transpose_op(input_x, input_perm)\n",
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 通信算子\n",
    "\n",
    "通信算子主要是针对[多卡训练](https://www.mindspore.cn/tutorials/experts/zh-CN/r1.7/parallel/train_ascend.html#多机多卡训练)时对各个卡进行通信的算子，比如收集算子AllGather、广播算子Broadcast等，参见[通信算子](https://www.mindspore.cn/docs/zh-CN/r1.7/api_python/mindspore.ops.html#communication-operators#通信算子)。\n",
    "\n",
    "以下代码展示了收集算子AllGather的使用：\n",
    "\n",
    "```python\n",
    "# This example should be run with two devices. Refer to the tutorial > Distributed Training on mindspore.cn\n",
    "import numpy as np\n",
    "import mindspore.ops as ops\n",
    "import mindspore.nn as nn\n",
    "from mindspore.communication import init\n",
    "from mindspore import Tensor, context\n",
    "\n",
    "context.set_context(mode=context.GRAPH_MODE)\n",
    "init()\n",
    "class Net(nn.Cell):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.allgather = ops.AllGather()\n",
    "    def construct(self, x):\n",
    "        return self.allgather(x)\n",
    "\n",
    "input_x = Tensor(np.ones([2, 8]).astype(np.float32))\n",
    "net = Net()\n",
    "output = net(input_x)\n",
    "print(output)\n",
    "```\n",
    "\n",
    "运行结果如下：\n",
    "\n",
    "```text\n",
    "[[1. 1. 1. 1. 1. 1. 1. 1.]\n",
    " [1. 1. 1. 1. 1. 1. 1. 1.]\n",
    " [1. 1. 1. 1. 1. 1. 1. 1.]\n",
    " [1. 1. 1. 1. 1. 1. 1. 1.]]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 框架相关的算子\n",
    "\n",
    "`mindspore.ops.composite`中提供了一些涉及图变换的组合类算子，例如`MultitypeFuncGraph`、`HyperMap`和`GradOperation`等。\n",
    "\n",
    "`MultitypeFuncGraph`用于定义一组重载的函数，用户可以使用该算子，根据不同类型，采用不同实现，参见[MultitypeFuncGraph](https://www.mindspore.cn/tutorials/experts/zh-CN/r1.7/operation/op_overload.html#multitypefuncgraph)。\n",
    "\n",
    "`HyperMap`可以对一组或多组输入做指定的运算，可以配合`MultitypeFuncGraph`一起使用，参见[HyperMap](https://www.mindspore.cn/tutorials/experts/zh-CN/r1.7/operation/op_overload.html#hypermap)。\n",
    "\n",
    "`GradOperation`用于生成输入函数的梯度，利用get_all、get_by_list和sens_param参数控制梯度的计算方式，参见[GradOperation](https://www.mindspore.cn/tutorials/zh-CN/r1.7/beginner/autograd.html)。\n",
    "\n",
    "## nn算子\n",
    "\n",
    "nn算子是对低阶API的封装，主要包括卷积层算子、池化层算子、损失函数、优化器等。\n",
    "\n",
    "nn算子还提供了部分与Primitive算子同名的接口，主要作用是对Primitive算子进行进一步封装，为用户提供更友好的API，当nn算子功能满足用户的要求时可以直接使用nn算子，而当nn算子功能无法满足用户特定要求时可以使用低阶的Primitive算子实现特定的功能。\n",
    "\n",
    "### 卷积层算子\n",
    "\n",
    "卷积层算子主要是在模型卷积层中使用的算子，比如卷积算子Conv2d、转置卷积算子Conv2dTranspose等，参见[卷积层算子](https://www.mindspore.cn/docs/zh-CN/r1.7/api_python/mindspore.nn.html#卷积神经网络层)。\n",
    "\n",
    "以下代码展示了卷积算子Conv2d的使用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-04T11:29:07.040406Z",
     "start_time": "2022-01-04T11:29:05.107864Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 240, 1024, 640)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import mindspore\n",
    "from mindspore import Tensor\n",
    "import mindspore.nn as nn\n",
    "\n",
    "net = nn.Conv2d(120, 240, 4, has_bias=False, weight_init='normal')\n",
    "x = Tensor(np.ones([1, 120, 1024, 640]), mindspore.float32)\n",
    "output = net(x).shape\n",
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 池化层算子\n",
    "\n",
    "池化层算子主要是在模型池化层中使用的算子，比如平均池化算子AvgPool2d、最大池化算子MaxPool2d等，参见[池化层算子](https://www.mindspore.cn/docs/zh-CN/r1.7/api_python/mindspore.nn.html#池化层)。\n",
    "\n",
    "以下代码展示了最大池化算子MaxPool2d的使用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-04T11:29:07.058829Z",
     "start_time": "2022-01-04T11:29:07.041413Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 2, 2, 2)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import mindspore\n",
    "from mindspore import Tensor\n",
    "import mindspore.nn as nn\n",
    "\n",
    "pool = nn.MaxPool2d(kernel_size=3, stride=1)\n",
    "x = Tensor(np.random.randint(0, 10, [1, 2, 4, 4]), mindspore.float32)\n",
    "output = pool(x)\n",
    "print(output.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 损失函数\n",
    "\n",
    "损失函数主要是用来评价模型的预测值和真实值的差异程度，常用的损失函数有BCEWithLogitsLoss、SoftmaxCrossEntropyWithLogits等，参见[损失函数](https://www.mindspore.cn/docs/zh-CN/r1.7/api_python/mindspore.nn.html#损失函数])。\n",
    "\n",
    "以下代码展示了SoftmaxCrossEntropyWithLogits损失函数的使用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-04T11:29:07.136953Z",
     "start_time": "2022-01-04T11:29:07.061949Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[30.]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import mindspore\n",
    "from mindspore import Tensor\n",
    "import mindspore.nn as nn\n",
    "\n",
    "loss = nn.SoftmaxCrossEntropyWithLogits()\n",
    "logits = Tensor(np.array([[3, 5, 6, 9, 12, 33, 42, 12, 32, 72]]), mindspore.float32)\n",
    "labels_np = np.array([[0, 0, 0, 0, 0, 0, 1, 0, 0, 0]]).astype(np.float32)\n",
    "labels = Tensor(labels_np)\n",
    "output = loss(logits, labels)\n",
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 优化器\n",
    "\n",
    "优化器主要是用于计算和更新梯度，常用的优化器有Adam、Momentum等，参见[优化器](https://www.mindspore.cn/docs/zh-CN/r1.7/api_python/mindspore.nn.html#优化器)。\n",
    "\n",
    "以下代码展示了Momentum优化器的使用：\n",
    "\n",
    "```python\n",
    "import mindspore.nn as nn\n",
    "from mindspore import Model\n",
    "\n",
    "net = Net()\n",
    "#1) All parameters use the same learning rate and weight decay\n",
    "optim = nn.Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9)\n",
    "\n",
    "#2) Use parameter groups and set different values\n",
    "conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params()))\n",
    "no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params()))\n",
    "group_params = [{'params': conv_params, 'weight_decay': 0.01, 'grad_centralization':True},\n",
    "                {'params': no_conv_params, 'lr': 0.01},\n",
    "                {'order_params': net.trainable_params()}]\n",
    "optim = nn.Momentum(group_params, learning_rate=0.1, momentum=0.9, weight_decay=0.0)\n",
    "# The conv_params's parameters will use a learning rate of default value 0.1 and a weight decay of 0.01 and\n",
    "# grad centralization of True.\n",
    "# The no_conv_params's parameters will use a learning rate of 0.01 and a weight decay of default value 0.0\n",
    "# and grad centralization of False..\n",
    "# The final parameters order in which the optimizer will be followed is the value of 'order_params'.\n",
    "\n",
    "loss = nn.SoftmaxCrossEntropyWithLogits()\n",
    "model = Model(net, loss_fn=loss, optimizer=optim, metrics=None)\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "MindSpore",
   "language": "python",
   "name": "mindspore"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}