{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 实现一个图片分类应用\n",
    "\n",
    "`Linux` `Windows` `Ascend` `GPU` `CPU` `全流程` `初级` `中级` `高级`\n",
    "\n",
    "[![](https://gitee.com/mindspore/docs/raw/r1.3/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/r1.3/docs/mindspore/programming_guide/source_zh_cn/quick_start/quick_start.ipynb) [![](https://gitee.com/mindspore/docs/raw/r1.3/resource/_static/logo_notebook.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/r1.3/programming_guide/zh_cn/mindspore_quick_start.ipynb) [![](https://gitee.com/mindspore/docs/raw/r1.3/resource/_static/logo_modelarts.png)](https://authoring-modelarts-cnnorth4.huaweicloud.com/console/lab?share-url-b64=aHR0cHM6Ly9vYnMuZHVhbHN0YWNrLmNuLW5vcnRoLTQubXlodWF3ZWljbG91ZC5jb20vbWluZHNwb3JlLXdlYnNpdGUvbm90ZWJvb2svbW9kZWxhcnRzL21pbmRzcG9yZV9xdWlja19zdGFydC5pcHluYg==&imageid=65f636a0-56cf-49df-b941-7d2a07ba8c8c) [![](https://gitee.com/mindspore/docs/raw/r1.3/resource/_static/logo_online_experience.png)](https://ascend.huawei.com/zh/#/college/onlineExperiment/codeLabMindSpore/mindSpore)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 概述\n",
    "\n",
    "下面我们通过一个实际样例，带领大家体验MindSpore基础的功能，对于一般的用户而言，完成整个样例实践会持续20~30分钟。\n",
    "\n",
    "本例子会实现一个简单的图片分类的功能，整体流程如下：\n",
    "\n",
    "1. 处理需要的数据集，这里使用了MNIST数据集。\n",
    "2. 定义一个网络，这里我们使用LeNet网络。\n",
    "3. 自定义回调函数收集模型的损失值和精度值。\n",
    "4. 定义损失函数和优化器。\n",
    "5. 加载数据集并进行训练，训练完成后，查看结果及保存模型文件。\n",
    "6. 加载保存的模型，进行推理。\n",
    "7. 验证模型，加载测试数据集和训练后的模型，验证结果精度。\n",
    "\n",
    "这是简单、基础的应用流程，其他高级、复杂的应用可以基于这个基本流程进行扩展。\n",
    "\n",
    "> 本文档适用于CPU、GPU和Ascend环境。  \n",
    "> 你可以在这里找到完整可运行的样例代码：<https://gitee.com/mindspore/docs/tree/r1.3/docs/sample_code/lenet> 。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 准备环节\n",
    "\n",
    "在动手进行实践之前，确保，你已经正确安装了MindSpore。如果没有，可以通过[MindSpore安装页面](https://www.mindspore.cn/install)将MindSpore安装在你的电脑当中。  \n",
    "\n",
    "同时希望你拥有Python编码基础和概率、矩阵等基础数学知识。\n",
    "\n",
    "那么接下来，就开始MindSpore的体验之旅吧。\n",
    "\n",
    "### 下载数据集\n",
    "\n",
    "我们示例中用到的`MNIST`数据集是由10类$28*28$的灰度图片组成，训练数据集包含60000张图片，测试数据集包含10000张图片。\n",
    "\n",
    "> MNIST数据集下载页面：<http://yann.lecun.com/exdb/mnist/>。页面提供4个数据集下载链接，其中前2个文件是训练数据需要，后2个文件是测试结果需要。\n",
    "\n",
    "在Jupyter Notebook中执行如下命令下载MNIST数据集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./datasets/MNIST_Data\n",
      "├── test\n",
      "│   ├── t10k-images-idx3-ubyte\n",
      "│   └── t10k-labels-idx1-ubyte\n",
      "└── train\n",
      "    ├── train-images-idx3-ubyte\n",
      "    └── train-labels-idx1-ubyte\n",
      "\n",
      "2 directories, 4 files\n"
     ]
    }
   ],
   "source": [
    "!mkdir -p ./datasets/MNIST_Data/train ./datasets/MNIST_Data/test\n",
    "!wget -NP ./datasets/MNIST_Data/train https://mindspore-website.obs.myhuaweicloud.com/notebook/datasets/mnist/train-labels-idx1-ubyte --no-check-certificate \n",
    "!wget -NP ./datasets/MNIST_Data/train https://mindspore-website.obs.myhuaweicloud.com/notebook/datasets/mnist/train-images-idx3-ubyte --no-check-certificate\n",
    "!wget -NP ./datasets/MNIST_Data/test https://mindspore-website.obs.myhuaweicloud.com/notebook/datasets/mnist/t10k-labels-idx1-ubyte --no-check-certificate\n",
    "!wget -NP ./datasets/MNIST_Data/test https://mindspore-website.obs.myhuaweicloud.com/notebook/datasets/mnist/t10k-images-idx3-ubyte --no-check-certificate\n",
    "!tree ./datasets/MNIST_Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 导入Python库&模块\n",
    "\n",
    "在使用前，需要导入需要的Python库。\n",
    "\n",
    "目前使用到`os`库，为方便理解，其他需要的库，我们在具体使用到时再说明。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:53:40.753790Z",
     "start_time": "2021-02-03T08:53:40.750705Z"
    }
   },
   "outputs": [],
   "source": [
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "详细的MindSpore的模块说明，可以在[MindSpore API页面](https://www.mindspore.cn/docs/api/zh-CN/r1.3/index.html)中搜索查询。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 配置运行信息\n",
    "\n",
    "在正式编写代码前，需要了解MindSpore运行所需要的硬件、后端等基本信息。\n",
    "\n",
    "可以通过`context.set_context`来配置运行需要的信息，譬如运行模式、后端信息、硬件等信息。\n",
    "\n",
    "导入`context`模块，配置运行需要的信息。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:53:41.528022Z",
     "start_time": "2021-02-03T08:53:40.755378Z"
    }
   },
   "outputs": [],
   "source": [
    "from mindspore import context\n",
    "\n",
    "context.set_context(mode=context.GRAPH_MODE, device_target=\"CPU\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在样例中我们配置样例运行使用图模式。根据实际情况配置硬件信息，譬如代码运行在Ascend AI处理器上，则`--device_target`选择`Ascend`，代码运行在CPU、GPU同理。详细参数说明，请参见`context.set_context`接口说明。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据处理\n",
    "数据集对于训练非常重要，好的数据集可以有效提高训练精度和效率，在加载数据集前，通常会对数据集进行一些处理。\n",
    "\n",
    "由于后面会采用LeNet这样的卷积神经网络对数据集进行训练，而采用在训练数据时，对数据格式是有所要求的，所以接下来需要先查看数据集内的数据是什么样的，这样才能构造一个针对性的数据转换函数，将数据集数据转换成符合训练要求的数据形式。\n",
    "\n",
    "执行如下代码查看原始数据集数据。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:53:42.128603Z",
     "start_time": "2021-02-03T08:53:41.530089Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The type of mnist_ds: <class 'mindspore.dataset.engine.datasets.MnistDataset'>\n",
      "Number of pictures contained in the mnist_ds： 60000\n",
      "The item of mnist_ds: dict_keys(['image', 'label'])\n",
      "Tensor of image in item: (28, 28, 1)\n",
      "The label of item: 8\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAEICAYAAACZA4KlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAANvUlEQVR4nO3dfaxkdX3H8fdHXJe42MhKgRWpUEqTamNXc4smNK2NrSI2QZto2DRmSYxrqya1oYmGNpE0NaFN1dpQH9aKLkZRU6FuE6rSTVNC2xAvBGERK5Susu7CatEKVNYFvv1jzrZ3L/dpZ8483P29X8lkzpyHOd85uZ97zpzfOfNLVSHpxPeMaRcgaTIMu9QIwy41wrBLjTDsUiMMu9QIw66nSbIvyW9Muw71y7Br4pKck+TGJD9I8mCSq5M8c9p1negMu8ZmhQB/GDgEbAG2Ar8GvH1SdbXKsK8j3eH1Hya5M8l/J/l8kpOTXJbklkXzVpKf64Y/leTDSf4hyaNJ/iXJmUn+stu7fjPJSxet7peTfKOb/skkJy94799KckeSHyb51yQvWVTju5PcCTy2TODPBb5QVY9X1YPAl4EX97WdtDTDvv68CbiIQWBeAlx2HMv9MXAacBj4N+D27vXfAh9YNP/vAK8BzgN+vluWJC8DrgHeBjwP+BiwO8nGBctuA14HPLeqnuj+0Xx4wfQPAZcmeXaSs4DXMgi8xsiwrz9/VVUHquph4O8ZHAavxQ1VdVtVPQ7cADxeVddW1ZPA54HFe/arq+qBbj3vYxBggLcCH6uqW6vqyaraxeCfxysW1fhAVf0YoKreXlULD9P/mcGe/EfAfmAe+Ls1fg4NybCvPw8uGP4f4JQ1LvfQguEfL/F68fs8sGD428Dzu+EXApd3h/A/TPJD4OwF0xcve4wkzwC+AlwPbGJwZHEq8Gdr/BwakmE/MTwGPPvoiyRn9vCeZy8Y/hngQDf8APC+qnrugsezq+q6BfOvdCvl5u69r66qw1X1X8AngYt7qFkrMOwnhq8DL06ytTuRdmUP7/mOJC9Ishm4gsGhPsDHgd9N8vIMbEryuiTPWcubVtX3gf8Efi/JM5M8F9jefQaNkWE/AVTVt4A/Af4RuBe4ZeUl1uSzwFeB+7vHn3brmmfwvf1q4AfAfaxykjDJR5N8dMGo32ZwkvF73fJPAH/QQ81aQfzxCqkN7tmlRhh2qRGGXWqEYZcaMdE7jZ6VjXUymya5Sqkpj/MYP6nDWWraSGFPchGD65xPAv6mqq5aaf6T2cTL86pRVilpBbfWnmWnDX0Yn+Qk4K8Z3MTwImBbkhcN+36SxmuU7+wXAPdV1f1V9RPgc8Al/ZQlqW+jhP0sjr3hYX837hhJdiSZTzJ/hMMjrE7SKEYJ+1InAZ52OV5V7ayquaqa28DGJRaRNAmjhH0/x94Z9QL+/84oSTNmlLB/DTg/yblJngVcCuzupyxJfRu66a37uaF3MvghgpOAa6rq7t4qk9SrkdrZq+pG4MaeapE0Rl4uKzXCsEuNMOxSIwy71AjDLjXCsEuNMOxSIwy71AjDLjXCsEuNMOxSIwy71AjDLjXCsEuNMOxSIwy71AjDLjXCsEuNMOxSIwy71AjDLjXCsEuNMOxSIwy71AjDLjXCsEuNMOxSIwy71AjDLjVipF5ctTZfOXDHtEuYmtc8f+u0S1BnpLAn2Qc8AjwJPFFVc30UJal/fezZf72qvt/D+0gaI7+zS40YNewFfDXJbUl2LDVDkh1J5pPMH+HwiKuTNKxRD+MvrKoDSU4Hbkryzaq6eeEMVbUT2AnwU9lcI65P0pBG2rNX1YHu+RBwA3BBH0VJ6t/QYU+yKclzjg4Drwb29lWYpH6Nchh/BnBDkqPv89mq+nIvVa0zLbejr2aUbWMbfb+GDntV3Q/8Uo+1SBojm96kRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRvhT0ppZq90e6y2wx8c9u9QIwy41wrBLjTDsUiMMu9QIwy41wrBLjbCd/QQwSnvzev4Z7HHWfiK24btnlxph2KVGGHapEYZdaoRhlxph2KVGGHapEbaznwBWam9erb14tenruR1+FOP+3NNox191z57kmiSHkuxdMG5zkpuS3Ns9nzreMiWNai2H8Z8CLlo07j3Anqo6H9jTvZY0w1YNe1XdDDy8aPQlwK5ueBfw+p7rktSzYU/QnVFVBwG659OXmzHJjiTzSeaPcHjI1Uka1djPxlfVzqqaq6q5DWwc9+okLWPYsD+UZAtA93yov5IkjcOwYd8NbO+GtwNf6qccSeOyajt7kuuAVwKnJdkPvBe4CvhCkrcA3wHeOM4iZ916bque5drWs1m8H37VsFfVtmUmvarnWiSNkZfLSo0w7FIjDLvUCMMuNcKwS43wFtcTnE1rOso9u9QIwy41wrBLjTDsUiMMu9QIwy41wrBLjbCdfQLW8y2w69ks3mY6Te7ZpUYYdqkRhl1qhGGXGmHYpUYYdqkRhl1qhO3sM+BEboe3rXt2uGeXGmHYpUYYdqkRhl1qhGGXGmHYpUYYdqkRhl1qxKphT3JNkkNJ9i4Yd2WS7ya5o3tcPN4yJY1qLXv2TwEXLTH+g1W1tXvc2G9Zkvq2atir6mbg4QnUImmMRvnO/s4kd3aH+acuN1OSHUnmk8wf4fAIq5M0imHD/hHgPGArcBB4/3IzVtXOqpqrqrkNbBxydZJGNVTYq+qhqnqyqp4CPg5c0G9Zkvo2VNiTbFnw8g3A3uXmlTQbVr2fPcl1wCuB05LsB94LvDLJVqCAfcDbxljjuree70cf1Sif3Xvh+7Vq2Ktq2xKjPzGGWiSNkVfQSY0w7FIjDLvUCMMuNcKwS43wp6Q1s1ZrtrNp7vi4Z5caYdilRhh2qRGGXWqEYZcaYdilRhh2qRG2s2tFJ3J30q1xzy41wrBLjTDsUiMMu9QIwy41wrBLjTDsUiMMu9QIwy41wrBLjTDsUiMMu9QIwy41wrBLjTDsUiPW0mXz2cC1wJnAU8DOqvpQks3A54FzGHTb/Kaq+sH4StU0eL/6iWMte/YngMur6heAVwDvSPIi4D3Anqo6H9jTvZY0o1YNe1UdrKrbu+FHgHuAs4BLgF3dbLuA14+rSEmjO67v7EnOAV4K3AqcUVUHYfAPATi97+Ik9WfNYU9yCvBF4F1V9aPjWG5Hkvkk80c4PEyNknqwprAn2cAg6J+pquu70Q8l2dJN3wIcWmrZqtpZVXNVNbeBjX3ULGkIq4Y9SYBPAPdU1QcWTNoNbO+GtwNf6r88SX1Zy09JXwi8GbgrydF2mCuAq4AvJHkL8B3gjeMpcf0btWthm7/Uh1XDXlW3AFlm8qv6LUfSuHgFndQIwy41wrBLjTDsUiMMu9QIwy41wrBLjTDsUiMMu9QIwy41wrBLjTDsUiMMu9QIwy41Yi33s0tTMervAOhY7tmlRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqE7ezrwDjbm/1N+na4Z5caYdilRhh2qRGGXWqEYZcaYdilRhh2qRGrtrMnORu4FjgTeArYWVUfSnIl8Fbge92sV1TVjeMqVOPhPePtWMtFNU8Al1fV7UmeA9yW5KZu2ger6i/GV56kvqwa9qo6CBzshh9Jcg9w1rgLk9Sv4/rOnuQc4KXArd2odya5M8k1SU5dZpkdSeaTzB/h8EjFShremsOe5BTgi8C7qupHwEeA84CtDPb8719quaraWVVzVTW3gY09lCxpGGsKe5INDIL+maq6HqCqHqqqJ6vqKeDjwAXjK1PSqFYNe5IAnwDuqaoPLBi/ZcFsbwD29l+epL6s5Wz8hcCbgbuSHL0f8gpgW5KtQAH7gLeNpUJJvVjL2fhbgCwxyTZ1aR3xCjqpEYZdaoRhlxph2KVGGHapEYZdaoRhlxph2KVGGHapEYZdaoRhlxph2KVGGHapEYZdakSqanIrS74HfHvBqNOA70+sgOMzq7XNal1gbcPqs7YXVtVPLzVhomF/2sqT+aqam1oBK5jV2ma1LrC2YU2qNg/jpUYYdqkR0w77zimvfyWzWtus1gXWNqyJ1DbV7+ySJmfae3ZJE2LYpUZMJexJLkry70nuS/KeadSwnCT7ktyV5I4k81Ou5Zokh5LsXTBuc5KbktzbPS/Zx96UarsyyXe7bXdHkounVNvZSf4pyT1J7k7y+934qW67FeqayHab+Hf2JCcB3wJ+E9gPfA3YVlXfmGghy0iyD5irqqlfgJHkV4FHgWur6he7cX8OPFxVV3X/KE+tqnfPSG1XAo9OuxvvrreiLQu7GQdeD1zGFLfdCnW9iQlst2ns2S8A7quq+6vqJ8DngEumUMfMq6qbgYcXjb4E2NUN72LwxzJxy9Q2E6rqYFXd3g0/AhztZnyq226FuiZiGmE/C3hgwev9zFZ/7wV8NcltSXZMu5glnFFVB2HwxwOcPuV6Flu1G+9JWtTN+Mxsu2G6Px/VNMK+VFdSs9T+d2FVvQx4LfCO7nBVa7OmbrwnZYluxmfCsN2fj2oaYd8PnL3g9QuAA1OoY0lVdaB7PgTcwOx1Rf3Q0R50u+dDU67n/8xSN95LdTPODGy7aXZ/Po2wfw04P8m5SZ4FXArsnkIdT5NkU3fihCSbgFcze11R7wa2d8PbgS9NsZZjzEo33st1M86Ut93Uuz+vqok/gIsZnJH/D+CPplHDMnX9LPD17nH3tGsDrmNwWHeEwRHRW4DnAXuAe7vnzTNU26eBu4A7GQRry5Rq+xUGXw3vBO7oHhdPe9utUNdEtpuXy0qN8Ao6qRGGXWqEYZcaYdilRhh2qRGGXWqEYZca8b+yHlbL9HH54gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import matplotlib\n",
    "import numpy as np\n",
    "import mindspore.dataset as ds\n",
    "\n",
    "train_data_path = \"./datasets/MNIST_Data/train\"\n",
    "test_data_path = \"./datasets/MNIST_Data/test\"\n",
    "mnist_ds = ds.MnistDataset(train_data_path)\n",
    "print('The type of mnist_ds:', type(mnist_ds))\n",
    "print(\"Number of pictures contained in the mnist_ds：\", mnist_ds.get_dataset_size())\n",
    "\n",
    "dic_ds = mnist_ds.create_dict_iterator()\n",
    "item = next(dic_ds)\n",
    "img = item[\"image\"].asnumpy()\n",
    "label = item[\"label\"].asnumpy()\n",
    "\n",
    "print(\"The item of mnist_ds:\", item.keys())\n",
    "print(\"Tensor of image in item:\", img.shape) \n",
    "print(\"The label of item:\", label)\n",
    "\n",
    "plt.imshow(np.squeeze(img))\n",
    "plt.title(\"number:%s\"% item[\"label\"].asnumpy())\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面的运行情况我们可以看到，训练数据集`train-images-idx3-ubyte`和`train-labels-idx1-ubyte`对应的是6万张图片和6万个数字标签，载入数据后经过`create_dict_iterator`转换字典型的数据集，取其中的一个数据查看，这是一个key为`image`和`label`的字典，其中的`image`的张量(高度28，宽度28，通道1)和`label`为对应图片的数字。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 定义数据集及数据操作\n",
    "\n",
    "我们定义一个函数`create_dataset`来创建数据集。在这个函数中，我们定义好需要进行的数据增强和处理操作：\n",
    "\n",
    "1. 定义数据集。\n",
    "2. 定义进行数据增强和处理所需要的一些参数。\n",
    "3. 根据参数，生成对应的数据增强操作。\n",
    "4. 使用`map`映射函数，将数据操作应用到数据集。\n",
    "5. 对生成的数据集进行处理。\n",
    "\n",
    "定义完成后，使用`create_datasets`对原始数据进行增强操作，并抽取一个`batch`的数据，查看数据增强后的变化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:53:42.147143Z",
     "start_time": "2021-02-03T08:53:42.129614Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of groups in the dataset: 1875\n"
     ]
    }
   ],
   "source": [
    "import mindspore.dataset.vision.c_transforms as CV\n",
    "import mindspore.dataset.transforms.c_transforms as C\n",
    "from mindspore.dataset.vision import Inter\n",
    "from mindspore import dtype as mstype\n",
    "\n",
    "\n",
    "def create_dataset(data_path, batch_size=32, repeat_size=1,\n",
    "                   num_parallel_workers=1):\n",
    "    \"\"\" \n",
    "    create dataset for train or test\n",
    "    \n",
    "    Args:\n",
    "        data_path (str): Data path\n",
    "        batch_size (int): The number of data records in each group\n",
    "        repeat_size (int): The number of replicated data records\n",
    "        num_parallel_workers (int): The number of parallel workers\n",
    "    \"\"\"\n",
    "    # define dataset\n",
    "    mnist_ds = ds.MnistDataset(data_path)\n",
    "\n",
    "    # define some parameters needed for data enhancement and rough justification\n",
    "    resize_height, resize_width = 32, 32\n",
    "    rescale = 1.0 / 255.0\n",
    "    shift = 0.0\n",
    "    rescale_nml = 1 / 0.3081\n",
    "    shift_nml = -1 * 0.1307 / 0.3081\n",
    "\n",
    "    # according to the parameters, generate the corresponding data enhancement method\n",
    "    resize_op = CV.Resize((resize_height, resize_width), interpolation=Inter.LINEAR)\n",
    "    rescale_nml_op = CV.Rescale(rescale_nml, shift_nml)\n",
    "    rescale_op = CV.Rescale(rescale, shift)\n",
    "    hwc2chw_op = CV.HWC2CHW()\n",
    "    type_cast_op = C.TypeCast(mstype.int32)\n",
    "\n",
    "    # using map to apply operations to a dataset\n",
    "    mnist_ds = mnist_ds.map(operations=type_cast_op, input_columns=\"label\", num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(operations=resize_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(operations=rescale_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(operations=rescale_nml_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(operations=hwc2chw_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
    "    \n",
    "    # process the generated dataset\n",
    "    buffer_size = 10000\n",
    "    mnist_ds = mnist_ds.shuffle(buffer_size=buffer_size)\n",
    "    mnist_ds = mnist_ds.batch(batch_size, drop_remainder=True)\n",
    "    mnist_ds = mnist_ds.repeat(repeat_size)\n",
    "\n",
    "    return mnist_ds\n",
    "\n",
    "ms_dataset = create_dataset(train_data_path)\n",
    "print('Number of groups in the dataset:', ms_dataset.get_dataset_size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "调用数据增强函数后，查看数据集`size`由60000变成了1875，符合我们的数据增强中`mnist_ds.batch`操作的预期（$60000/32=1875$）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上述增强过程中：\n",
    "\n",
    "- 数据集中的`label`数据增强操作：\n",
    "\n",
    "  - `C.TypeCast`：将数据类型转化为`int32`。\n",
    "\n",
    "- 数据集中的`image`数据增强操作：  \n",
    "\n",
    "  - `datasets.MnistDataset`：将数据集转化为MindSpore可训练的数据。  \n",
    "  - `CV.Resize`：对图像数据像素进行缩放，适应LeNet网络对数据的尺寸要求。  \n",
    "  - `CV.Rescale`：对图像数据进行标准化、归一化操作，使得每个像素的数值大小在（0,1）范围中，可以提升训练效率。  \n",
    "  - `CV.HWC2CHW`：对图像数据张量进行变换，张量形式由`高x宽x通道`（HWC）变为`通道x高x宽`（CHW），方便进行数据训练。\n",
    " \n",
    "- 其他增强操作：\n",
    "\n",
    "  - `mnist_ds.shuffle`：随机将数据存放在可容纳10000张图片地址的内存中进行混洗。  \n",
    "  - `mnist_ds.batch`：从混洗的10000张图片地址中抽取32张图片组成一个`batch`，参数`batch_size`表示每组包含的数据个数，现设置每组包含32个数据。  \n",
    "  - `mnist_ds.repeat`：将`batch`数据进行复制增强，参数`repeat_size`表示数据集复制的数量。\n",
    "\n",
    "先进行`shuffle`、`batch`操作，再进行`repeat`操作，这样能保证1个`epoch`内数据不重复。\n",
    "\n",
    "> MindSpore支持进行多种数据处理和增强的操作，各种操作往往组合使用，具体可以参考[数据处理](https://www.mindspore.cn/docs/programming_guide/zh-CN/r1.3/pipeline.html)和与[数据增强](https://www.mindspore.cn/docs/programming_guide/zh-CN/r1.3/augmentation.html)章节。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 查看增强后的数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从1875组数据中取出一组数据，查看其数据张量及`label`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:53:42.580776Z",
     "start_time": "2021-02-03T08:53:42.149224Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor of image: (32, 1, 32, 32)\n",
      "Labels: [9 8 5 5 1 2 3 5 7 0 6 1 0 3 8 1 2 1 5 1 5 2 8 4 4 6 4 5 5 5 7 8]\n"
     ]
    }
   ],
   "source": [
    "data = next(ms_dataset.create_dict_iterator(output_numpy=True))\n",
    "images = data[\"image\"]\n",
    "labels = data[\"label\"]\n",
    "print('Tensor of image:', images.shape)\n",
    "print('Labels:', labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将张量数据和`label`对应的值进行可视化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:53:43.245788Z",
     "start_time": "2021-02-03T08:53:42.582357Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAADsCAYAAABKZHxbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOyde3hU1bn/P+9MQi4kJCQhQLgHSIhiRYxw8Ip6gFKLxapUhFKwniiKN0DrOfhT65FjqwitRbSoQClIxWuLl4LnVDiACEUuyiESucklXJIAgVyZzKzfH3smmUlmkkkys/cE1+d58mT2/Ttr1n732u9637VEKYVGo9FozMdmtQCNRqP5vqINsEaj0ViENsAajUZjEdoAazQajUVoA6zRaDQWoQ2wRqPRWIQ2wBqNRmMREWuARSRHRP4hIqUisldEbrFaUyBEpLeIfCwip0XkuIjMF5Eoq3X5Q0TWikiViJS5//ZYrSkQbUWriEwTka0iUi0iS6zWEwgRiRGRN0TkOxE5JyLbRWS01boCISLLROSYiJwVkQIRudtqTYFoaV2NSAPsNl5/BT4EUoA8YJmIZFkqLDALgJNAV2AQcB1wn6WKGmeaUirB/ZdttZgmaAtaC4FngUVWC2mCKOAwRv1MAv4fsFJEeluoqTGeA3orpToANwPPisjlFmtqjGbX1SYNsIgcFJGZIvKVuzX6lojEishkEdlQb18lIv3cn5eIyAIR+cT9RNgoIl1E5HfuluI3InJZgMsOADKAeUopp1LqH8BG4OcRqBWgD7BSKVWllDoO/B24OEK1Npu2otUqnUqp95RSHwAlkaxVKVWulHpaKXVQKeVSSn0IHAAaNWoWluv/KaWqPYvuv76RqLWlBNsCHgf8EMPQ/ACY3IzjngDSgGpgE7DNvfwOMNezo/vLL/As+jmXAAMjUCvA74E7RCReRLoBozGMcCRqBXhORIrdlWx4M67ZFrRapbMlWKpVRDoDWcD/RapW97oK4BvgGPBxpGqlJfeVUqrRP+AgMNFr+XngVfeX2lBvXwX0c39eArzmte0BIN9r+RLgTIBrRgP7gcfcn0cC54HVkabVvT0H+BKocZ93CSARqnUokAjEAL8AzgF9LwStVun02u9ZYElT+0WI1mjgv4E/tgGtduBqDOMYHYlam1tXPX/BtoCPe32uABKCPO6E1+dKP8t+z6OUcgBjgZvc154BrASORJpWEbEBq4H3gPYYT8uOwG8jTSuAUmqzUuqcUqpaKfUnDNfOjy4grabrbAWWaHXX2T9jNGqmBXlNy8pVGW7IDUB3YGoQ12wrdbVVnXDlQLxnQUS6tOJcDVBKfaWUuk4plaqUGgVkAltaeLpwak0BegDz3YVfAiwmOKPmj7CWqx8U/l0+wdBWtJqtszWEVauICPAG0Bm41d3YaSlml2sUTfiAGyEi62prDPBO4GIRGSQiscDTrThXA0TkB27nebyIzMSIMFjSwtOFTatSqhijI2OqiESJSDLGK8jOFp4ybFpFJFlERrnLNUpEJgDXYrTgL2St4a6rUe7z2gG7R3MLTxdWrcArGC6zMUqpylaeK5y/f7qI3CEiCSJiF5FRwHjgHxGotcV1tcUGWClVADyD4Uf6FtjQ+BGNIyKvisirXqt+juF0PwncCIxQdT2ikab1pxhO/yJgL4Yv+JEI1BqN4acsAoox/FxjlVItiq9tK1pN+P2fwHhFfRyY6P78REvOHU6tItILuAcjVPK41MWsTog0rRgtyKkYbsfTwBzgYaXUXyNQa4vrqrgdyBqNRqMxmYhMxNBoNJrvA9oAazQajUVoA6zRaDQWoQ2wRqPRWIQ2wBqNRmMRQcUqjrDdHlGhEp+63vYb4NxWdILW2houBK1tRSdora2hMa2gW8AajUZjGdoAazQajUVE5KwNGo0mMOqqQXx7V8Nbt8cqG3EftHS4FI0VaAPchrHFx3PooUG4Yhpu67Bfkbx0k/miLgD8lWunHTUAlho45/DBAByeWsOBa15vsP3a7rdQyRAgvDo9Oo4Oj232sZlvHKLmcDCDGn4/0C4IjUajsQjdAnYTldmbk9d19butY4ExaJRs3GH8j4nhzO2X4YwW0tcdo2b/QbNkAnVaa+KF1VOfp3tUw2FKh+8aC0tNlXXBIIkJvHPPHHLa1Y5eSJ9PjPkgsz6wRpO6ahCHpxqt8D3X+P9h//eS9xnxwBgAzlfm0m711rBoOXqd0fLNz2v+pCA53Efm0ijT75nWYrs0B4DiwclN7ptQ6Ai67MNugO2dOuG4qLvfbdGFpQA4v90fbhkBiephaNs/KSNghaq9+TaCLTGRiuE5rJg9hz7RCQz5j6l0NKky+dcajnHCNZHGt3dF1bodChzlrCz1ncZtXNKXZEW359OcVQAMnzG25YOMhpH8vAWGEX6jJmJcEZ776nxmJwCkRmHbvAsGDcDZPhqAgvHG/wM3v9Lk+YbvCr7sw2aAbfFG6+HYuP5sn+XfsPX9xxQAsqcl4TxTGi4pAbElJvLt/T0AKJgU3NO8ZnA/1v1xIWYbvqa0Hqkpo73Y6GiPb7BNc2GRt2cCMSMP+qz77zUTWDuwrnkeG+XAlpqCs+RU2PU4lYt9NZU4ldAjykaCrXHfcH7eArJip9J3tnHPu86dC7vGQPjeV4ZxLXCU8/DoKVy/ZBOPpuxr9PgDjjKqlK8n93RFHMGO9h42A3zooUEArJ76PIGM1dbrXgZg+JJf0mWs+QZ4z28u4oub57iX2pt+/ebQlNYxv32Ms8Mq2XfDYnOFaSKSd7Le5ZGPb+TQ0PBfa19NJQ+PngIlZ3CtiObvAz5q8pgNE+bwL4nTAeh//+ZwSwyIv/uqb1Qcv/tkMb2i2mEM9RuYydOm037LQZ913aoLcQZ5/ZAb4ILXc/nlkA0MiX8NgO5RCSwszWD5Yz9myfy53Prco1w0KZ9lvdfWttY6xlciuQO5epHhN9lwVy5q665QS2uAinWSbq8zZgtLM3j73lEN9sspNFoR5346lJ/PXhV2XfUpeD2Xt26c76MVjKfv5GnTaXfWQdfd39J1TSIjXjPeKuJKKnGZrrR5nJx2JQC35vmf5ODdhTeQPv9zMyX5pc/f8sh5sRgg6Bsr1OT85hQjlgb+beOmx3Lps+PZOWQFAAm2WHrFlXCI5kcqNEXmokMAjFhr6JEaha1gF6qmBtuDAxiROqXBMec7RLNk/lz6RBuNsXR7e1SsVaVpEOi+souNrOjADTJvOxG/LR9nK1rwITfAvXoW80TaN4zINzoDZi3rTmypi/ZrdjB+1ky6rN7L0QP9uCIjp/aYsh7C5S/t5om0bwAY0X6YJeEZJx0dsK3b3mC9p5pUXd2ZvKRCn22qpbOpNYNePYsZEuP7JF5ZlsTcZ+4lec12VHW1obGoCNu3xvZIM7725CSOLsmgY3zdLDh3ZhiOskCveTH3Olgaa1T0jDnWGeLYY1GW9lOA0U/S2G/r2vUNZ0tyTdHi8d3avHy4nvxf165v/N677TunN3hVtxp/95U/stdPosPf6wxybKmL+HVGq72191lIDfDB2cN4stdKAPYe7AxA1mIjFlUBScu+wAm0W11EClA51ohZTJ54jGW914ZSSkip0+nbadDnb3kM2HYmrMbOu0y9KajqStKyL4ioxHc/eJIGbDFOtl7+cqM+6pPOcq5ePhN7Vd1TrdPeGjNkNo4JD9kLGXv/TPY8lUQXu9VKmkefv+URezyKHmursK8NT0x9SAywJ3D9Dz97jZHxDqYcuoa0jU0/WYoGGZfPd/fcnnZWkLvufrILS8P+qndm0jCGX9y0m6Ny7BCYWgRQ28Ps0ZnzYnHYWkb1y7Q+Q+L3sfzpf/NZ121tFfa128Kip7n4TxowjO+I/DG1D2hvpMpO9uxdlnbKaEKDd7JGVZcaDtywEK9JidsE/VY4sK0Lb+JNZL0TaDQazfeIkLSA6weuf/HJJfRcFITPrt7783EnZM84ivPEyVDIapRTF8G/dtzd6D7nR+US88Cx2pavBzN0+ksG8GZkvKNB3HJ2ziQ69BkGNC8YPJScH5VLWUY0Z39YDhhJAwWOcvL21E28Ky+mkRVAm9W+a3unThy7rR+JNkPJjGOD6bA/0h091uKdmOSh7vcPHN7Zr/cJAE5NGWb4Vd+zLhrCH0eujyMh07if6idjhYqwhKFVda3B3j8TqEuykKgoXEMHoqLEZz8PJ53lzD0xEmrM8fllPr6JJzrdwoTRDXPqPagZxQ2Mr2k6a2qYe2IEszPWNOilDcSea5bCNcbn5gSDhxI1o5h/DvRNF1tb0Z/o2R291jhxXXdZRCTi1MdxUXd33LrRW7/+D0PpqMfUCEj9xKTmUHtvzTYiC945MRLbZiOaIhLwbuBcumU8ABmlA3Dt+iZk1wiLAT5w80L6JriTLGakGytTk3npzZcDhncsPH05h4aWA+XhkBQ8ItjT0sAmxEb5+l7LXFXMKhxpik5nySkODYVZm0cyvfOnfvexi6JvVBx2aehJMjMQv1ZPakqDMgPISyokb0XD+GSrE3HaCvbkJIjxM+JSPWwx5od1hSoxKS+pkOFvvlwXT3ymFFVdHRqRTXDaWQFAuXL5TesHasP7hs8dS8zI0F07bIkYniSL4+63Co+xiHTsaWnkfb6J7OiT9IiygVcc5W0Ft2K7vRIzHxJHfxTDjKix/jemJvO7Txb7faiZGYjvodvH1czr9jEEGXtqdSJOW+Hokgz+MuiNJvczogzaVkeXN54ECKcSpjwxnaRlX5hy3dx19wPQYVNcwKzdcBESA+wqOcVDE+7jttfX1MbJ1iZZBBF6MvHgcIof7A6EP/miMfI6fknMZw5Gx58mRuoqcu3rxxMKZ8lRUzU11oKVklNMu/N+VJTQ+bkDPqF84QzED0SvuJKAaaiepJEJz3/YsI54xQZrfDn1YRZvX/J6o4kB/jDrnoratpcR4xsmXjRG/brqwTsBYvqTK3gueULYEnFOfZjF8uzFQHtc1YaR6rryW0bsqvsut7+6ukHc/8Ls5Uz4cAopPy4IiY6QGGBVU4Ns3MGfZ43htSQbyROPNPCdHqkpY8xvHyOqQnHttM282LUuXOpIWTIxJmS+1af/ohqyEyYBhv803d7enRTgG0J3tsSoFF12md+p1Riechfg6K9zGfFYok+5j0v6krc/uJtukwvD9opvGziAwmeFbpMLgara9Z5EnDPLjIFO7A5F8prtnJzdASj0cybrOT8ql25P7fVZZ0aiTWP8pOdXzTa+YN495Tp3zm/yUmMc/XWuTyIWQFkv8fG5jkso5anUkEj0wZMQ5HmoZa+fRP9Fhs/ZWVSEbV1R7b5/njWGPbP+6WOrsqLb85OeX7E+RA2bkLog4t/bTDxQeXoIOYPu89lmq4aei3fgqqjgy/G9oav18aqycQe2EUY6rKfzypvs9ZOw5SfQY0dkdAo0RrvVWykYlwte9Toruj1/GfQGM2ICuDBaibpqEIWPVrP28jfInX8/Tyf+N56H19VpRnbbO6k96PHX4+Q/nsKJ6y/hmcRXaCq/3irKMqJ9WmZmJNoEy7Vf30LRJv/Dpf5+ov9Y8XDjSbKRKjvZj+8OOn673eqtpNRbl96jO1mxU9kwYU5dp3M4Hn4xMfxl0Bu1DzVbfgKy0X8rO/69zXx5b++w2qqw+IDjPthCTz/jprow8v89KaiRToe/tydlsfVjEQRDsIkloeR0Vhw7hywB4t2DANUZ1qc6GSF+8ZOrWdDvRg6MXujeUrfPr4suAqBkTQYZHDRFc3Pot8KBa2e+1TIAKNrUlZ5P+9ZFT7KOP27K+Jo3p40Kyyt8/SSbk85yrj43k/4vH46YISZDgRm2SidiaDQajUWYPiPGrXn/qB18Zfk5w8lzdFtXMi1oAamrBuHKKfO7beLB4SQUmv9a1xIqfjqUqx/y9auDMa7pHTvuplt1eHyuCYUOJh4c3ug4Ho+m7OPRm30H25lxbDBflvSkZE0GYO1AO20FV04Z6iqjtSsbdxgJI+P6B5wR5dGUfVTn/YP180PfCeuZEcOTZJFub0/BpFcY8ckUnwF6msP5zE7u8Xi9/N0hzn+pn2TT1DW8bZWHtZU23thyNVmEpj/I0imJnthwCwBZj5sf6G4bOIDCR6vZ447vq8/upTl02bbXsuEHvbENHEBNalyDxAVPcsvPZ69q0FsLsLL0crqMzQ/bd2i3eivFJQN5dtGAZh23ZfYVxL+3OWLcDp4ZEcp6GU7HauXgpdMDsJc7Imawoz3XLOXSGCMaJz3qMopyYn0SRgDWVNS5d8LpE44tMf6vLEtiXEJd525JTiypXOazb1RJZYPEBU999qYkJ/zROq5enblz6mo62RuPqfbcV+nRaxpse3rfzWTdHbrOePMMsDvBIcZmGI+TznKkyrrhkSrnVrFzYOAJvrY9+QpDqqbScUlRwH3MonJuFWsH/qUucSGI5JYyVxXfVaYS7phltXUX63/QvJsnnshKOd1/V0+gLvNpv8PBZzf2RZ2wNiwS4LvKVMpcVSTYYmuTAfDTZjhSU8YjbzxWu/zOPXPC9vt7/Mpzz4znCq8MuG1PNpyu54ff3IR9fLrPOtdL5/h0wF8avcaRmjJs50Mk2I3auovPbuzLjzd/TU4742HljFPYEhMNXefOIVFR2LIyG00aCyWmGWBPgsPo+NNANFcvn0n2bKOCR0Ivc1ugOcktViSNaELP0R/FcNvbtzY5y8SY3z5Gz8V14xTMWDTWnS4fvt8/+e3tTC6d7s6E8887We9yeLPvHV4/wckfnu8TbttQf2YO19CBvPTmy6YljZnXArYJ2dEnaxMc7JWihx0MkvqzHTSW3GJl0khb5ODsYfzhZ69ZLSMgzpJTVNf0bnSfwc9Mpeu73+KsqKhb6f05TKjqauLX5nPdPXk+s114k2CLJadd887r9/uEiPpJY+n29vzlR/MBWPPVJaRHr/Hb8vW+r0L5ULDMB3zX7atZWm79bAeNYXUQvgfXrm/o/MIgsh+sSxqpT+b795C61UZn96hNrl2hHbXpQuV8F4clMbTNQV5M44qMqQG3d/lwL84ia1xlrnPniFuzk/GzZjL9yRU+PuHmsLIsiReeuxMI7/fxJC+ddNQlBHlmxRgS03CQnXDfV5YZ4FVHf0DHSJjtoI0gG3fQI9qIv8zJv6/B9uz3T0dMzGpboscqGzmH/SQNnYucB5i/xAVvrO4oVtXVJC37gueSJ/BUWsvOEVsM6e6YezO+z7sLbyDmXkejsx73+VseAxaeDet9ZZkBLtrUlZ4fWNfyLVmTQZ/Ddze6T/+CyBqjwDPbRc+1DbdpP3rLaCxpSNM8ImEC1WBJn/85S2NHsSAncE9fOGe88aATMTQajcYizGsBV1dzx45f1o58ZfUsA5Hqd9ZoNObQlA0wwxVimgF2nin1GfM1JkKC8DUajcYqtAtCo9FoLEKUipRkS41Go/l+oVvAGo1GYxHaAGs0Go1FaAOs0Wg0FqENsEaj0ViENsAajUZjERFrgEWkrN6fU0T+YLUuf4hIioi8LyLlIvKdiNxptabGEJE7RCTfrXefiPiZktR6RGSaiGwVkWoRWWK1nkC0pd9fRJaJyDEROSsiBSLSeD6+hYhIbxH5WEROi8hxEZkvIpZOIhGIltbViPwyAEqp2rHtRKQ9cAJ42zpFjfIycB7oDAwCPhKRnUqp/7NWVkNEZATwW+BnwBbA/1S7kUEh8CwwCjBngNaW0WZ+f+A54JdKqWoRGQCsFZHtSqkvrRbmhwXASYw6mgx8CtwHvGSlqAC0rK4qpRr9Aw4CM4GvgFLgLYzRlCcDG+rtq4B+7s9LMArwE6AM2Ah0AX4HnAa+AS5r6vruc/0C2I87bjmStGJMYnUeyPJa92fgN5Gm1X385xg3YJPlbrVWr3M+CyyJRJ1t7fevd95s4BgwLhK1AvnAj7yWXwD+GIlam1tXPX/BuiDGAT8E+gA/cH+ZYI97AkgDqoFNwDb38jvAXM+OIrJARBYEOM8vgKXK/Q0jTGsW4FRKFXidaydwcaRpFRE7kAt0EpG9InLE/VoXzBPb6joQLPr3b6JM3esqMIzKMeDjCNX6e+AOEYkXkW7AaODvEaq1RQRrgF9SShUqpU4BqzBes4LhfaXUl0qpKuB9oEoptVQp5cR4MtXO4KeUuk8p1WCgWxHpCVwH/ClCtSZgPGm9KQUSI1BrZyAauA24xn29yzAqXaRpbSn692+iTN3LiRh14D0MYxOJWtdhPMjOAkeArUDgiRyt1doigjXAx70+V+A9FWvjnPD6XOlnOZjzTMJ4dTgQ5DXN1loGdKi3rgMQzHxLZmv1DHD8B6XUMaVUMcZT/UcRqLWl6N8/iPMopZxKqQ1AdyDwdBt1mKpVRGzAaowHRHuMVmhHjP6LiNLaGloTBVEOxHsWRKRL6+X4ZRLBt34DEU6tBUCUiPT3Wncp0NIOmLBpVUqdxmhJhGoAELPqQGvRv39gooC+LTw2nFpTgB7AfKVUtVKqBFhMcI0Ff0RkXW2NAd4JXCwig0QkFng6NJLqEJErgW60PvohbFqVUuUYT+lnRKS9iFwF/ASjI6YlhLtcFwMPiEi6iHQEHgY+bOG5wqpVRKLc57UDdhGJbWEYkv79AfdvfoeIJIiIXURGAeOBf0SaVvfb2QFgqrseJGP0Be1s4Skjsq622AC7Ox2eAf4b+BbY0NJzAYjIqyLyar3VvwDeU0q1avpkE7TehxF6chJYAUxVLQxBMkHrfwL/xGi55QPbgdkRqvUJjFe/x4GJ7s/B+KvN1tlWfn+F4W44gtGzPwd4WCn11wjUCvBTjM60ImAvUAM8EqFaW1RX9XCUGo1GYxERmwmn0Wg0FzraAGs0Go1FaAOs0Wg0FqENsEaj0ViENsAajUZjEUHFVI6w3R5RoRKfut4Wf+vbik7QWlvDhaC1regErbU1NKYVIng4Sk3LODNpGABxE4/5rI+bHotr1zdWSNJoNAHQBvgCoXDmlZTlnGf4xbsAWNxzvc/2EalTtL9Jo4kwtAFuhDOThnE2M/AbROYbh6g5fMRERb7Y4uM59NAgXDFw122reTRln2VaWoPt0hwO3tLRZ13v90/j2plvupaoHt3Z/8ueAbd32K9IXrrJREUaM/H3+3dbW4V97bawXE83ijQajcYidAu4HhITw5nbL8MZLVw7bTMvdg385Bvx2RRsFraAJTGBd+6ZQ067+ID7lLmquK3gVqJKKnGZqK0p1FWDOJ1ljANfkuti/y2+Y1tf8d1UUlo67EorOJ/Zify8wONszzg2mC1lQ4l/b7Mpeip+OpSqpJa1k2JLXabp9Mb7t/VH+rpj1Ow/aJ6gZuDv98/OmUQvhzGksGzcEdLraQPshS0xkYrhOayYPYc+0SEf+jNk2Dt1wnFRd6o6RBMrgc3qSWc5swpHYru9EmfJURMVNs23d0VxYPQrAbdLRPVl1/Fi1208+3QF69+LNeV6o55exxNpLes8fbZ4gGk6PdgGDqDw0Wp2DlkScJ8h/zGVjhFqgP2x55ql9Ckz5i7N2hjac7faAEtUFLbUlNadpLoa55n6kwqYi8TEUDE8h3V/XEhj4y5XKwf7HQ7jmBpzrYQt3mjpHhvXn+2zPE9p/1rLXFXMKhzJoaHlGEOhRjbe5QpgM7lswagD5ztEN7lfjM2BvVMPnMXFoAezqsWemoLrpXPsHPBRo/s52gu2+HhcFRUmKWs9thgnAPbkpJDaqlYbYNfQgcxb3rqpke7Y8Uu6jLXWAJ+5/TJWzJ5DU4Pev3R6AJ/daIxfbSvZFbKRzYPh0EPGa9Dqqc/TlM7bCm7FdnslbcH4gm+5AiSf2W5q2ULwdeDBjt/Q//MTLLxyGM6iInPEtQG6fVzNvG4fY8yBGZhVv3qeUR0eo/tzn5sjLARsve5lAIYvCa2tarUBVlHSqA8yGN6+7HUmfDiFlB8XNL1zmHBFSVBuh18kfUXB37pw9EcxqJoaE5TV4Wpn/O8eFVjnpVvGA5DxhIo4twPAqQ+z+EnPr/jPxFcwpqcz8C5XZ8kpUzWdnHYlAP/+4HKfOjD4mamk5ldx8pEqdg5ZUbs+RqLJjj4JtkZj7EPChrtyGdF+WJP7dX7uAMt6rw27nsboFVdCgs3X+E48OJwT/96H8x2iWTJ/Ln2iE+gelcC8X77GAwnGa33vWZEfVdLRbti4jvGVTezZPMLmAx6RPwaAM8u6N9hWv3MrK7o9P+n5FeubeHKGA0/iwjUPBNdZkW5vz7xu/8Ntb9+K7cEBpiU3FM68krtuXx1we+b795C61UbnAqOCuHaFtrMgVPyk51dun6bvq/7aygy+/sMlJJdtN1VP4cwrmTTZKNdxCb4tm9T8KmzrttO5ZhDZD04CDH8gQI8oG64V0WGvA2rrriZDlQpez2VW5/8Nm4bWcKQsmZh124mLiWH8rJlMf3IF4xJKGRnv4HwXR9MnMJHzo3Lp9tRev9s89kxeTMOY+T40tNoAt9tfRM7ChpODdtphtA5TPmj4dNvgHMaU+9o3SBYwm1N3DWNgnpG40Fi0Q30SbLH8fcBHpiU3FM68ktt+vjZgnG+fv+UxYOFZS+JmQ8ELp/ry5qujSF/2ualuh0DletJZztXLZ9J//2FqMHq+bSOMVjLXGP/MrgP+sCUmsuc3F/HWjfMZEuP7QHvhVF/eXXgD6UTGa76qriZp2Rc8lzyB7+6NrJj1yrFDAIh54Jjft4hrv74FXukEQNzqLSG9dqsNcM3hI/R82rpQrJZyZpJhfK1+CARD6shCnuq0O+D2fiscbdb4AnxUeAnp8803FIHKtcQp9Ju3j5oTJ03XFAyexBVnrOKLm+eQbm9fu+3XRRcB8M6fh5Nhcpn+edX19LmtiAmJJbXrbsr4mjenjar9fdPnf85HN18SUQa4aJBhBvNzVvnfvqkrPT8IT1nqRAyNRqOxCEvigOMmHrO85RmMhhnHBvNlSU9uyvgawJKndsVPhzIk9Z9+t0VqkkV97J06UfTjfgBkxb5psZrGKXCUc8eOu+lWXeizvsN+wzky49jgZrmrQkXgxJX2tbq+LOlJyZoMADLmmP9G0XvWJp62j4NbVta2gh9N2UevB5cz98x4klZ2cEAAACAASURBVN/eTvlNgwLWZ8toyu8VRr9YRCRiFDjK+euhH5CCOVEQkjuQ7gmNu00WlmawZfYVxL+3mUVPjQLg0XtaF27XEgIF4tdPsrANHEBNauDsI4Do3UcsCZtyXNSdf85umHSxpdrohPnuUBpZIezYCIZAdWBl6eV0GZuPs956z/gP623D4L/qDHBJTixddncKa7k2ldzgXVczTC7H+mQ+volnnOPo9LPXARgZ72BcQilXzJ7D5NLp/Hz2KvKSCps4y/cH0w2wPTWF2Cjf3s+8PRPMCUETwZ6WxvVLNjVozZa5qjhcU9eOfOfukcRvND+Nsxa31hjbfr+b/1T6A47e3B6iwN45HddL5/h0wF8aPeVls++j6zuCqnBHSpw7F3LZwXLSWc4dH08HIOt+E8u5iTrwXWUqzYmdfvffX2B82UySloXPAFfOrWLnwA8arHcqF/tqKq2vq/XoPWsTj5T9G2DErHePSqBPdII7yUnjjekGONhg7XBgT0sj7/NNjI4/Tf0wqNsKbsU23isTy+Qki/o0phWMZIAfb/66drlHlI1gAuDPPWrjpo8eAaC/mYavHlcvn0n2bCMCxUz3SZN1oJnJK5OnTSd5jflJIwD7aip5ePQUbAXW1lV/9Py9EQY55uxjXlmbmvqYboC9g7W9kwZMuQltQnb0SWKkYeJIVU00MScaJi4cnD2MP/zsNTPU+dKIVjCSAXLaNZ02640ngUPF1n/BDj2e5IZb8/5BenTD+GV7pVjTAvdTrq1JXml31oGqrg6pxGBxKoGSM6YnBAWDJ82468pvGeycyrYnG7qg3rrBWPez16eSdfdWU/V5k7noEAA5cl+jAzGFA0t9wGdLjA6ELrusK/ymAqzPd3EwMr6uZbyyLIkXnrsTgE679zbwFWoMqlKN/y0dSMZMmlMPVfiT34LGkwxSXdMbeTGNdqutu48C4SwqIjW/YTIWUBu7vPj6RUx5/S7LjLBnTO+E73qYfm3TDLAnaPw/E+cD0Vz79S30WGVeFJy9fyZ7nkqii913/d6DnQHI8lN5C2deyeTctT7rCqq6krLY6JDRxrchdTNzfB1wn+z1k+ixrspEVa3HE6yfPNH8mPfKZV2Zct81DaJ2PMkgACMeG0PBuFzSNhpGLWVR5Kf3ehge5+KtG1/hjpenkf34btPfjJr8bcP40DXNAEt8HB/dNK923IhwBjf7w5GRxL4bFgONj1txocwy4Y8R+WPYe7Bz7U0aDlJHFvK1nw4jD5duGU+vl2zIRvNDuVpK5dghMNXoZPvUHax/2llB7rr7yS4sDfuDOHnpJnZFDWNKnrHsL3zy05xVkANTLjZS9b7oeSWxxViS4OKN7dIcCsbX1bdLt4yvfeMYfvEewPg+Q2Ki+eimecx4ZiyYbICbSsQIJzoRQ6PRaCzClBawvVMnjt3Wj0SbV1dbBHTbvnCqLwn57XzWNTbLxNpKG29suZosIs/X1hQTDw6n5vnOfl0toSKYpJGMJ1TEDhQUiMNjXBy45H2fdcedkD3jKE6T0pVTFm1id40xcNTwiYaDPTbKwTtZ7/qMQFbbOs5bz8qypNoECKs6CosHJ3Pg5roOuPR5sXRZZ9TBL55yj69xj8XDAVzoiRiOi7q7Q1GMXviVZUnEljR+jBksensUmW8ZPaDnr7sMoNFZJp7ed7OlvbXNocBRzsrSy2uXix/sTrut4dEuUVG4hg4MGGQfyTNzNIbkDsTZPpoOqZExpnLtZKDGgGzYUlN45OMb6RVXwrikL8mKbu+zv3cCRPzafEuiTiJ1ZpNgCbetsiQKYs5/3Un6EutHaXLGKb693+j5LJjkHSbjO97uaacRUnO6Io4uZolzKfY40smMPk2MRPvoKFcuukcl1AbiO/10zd+x4266jPUeoGdX2KTaUlOYt3yB37eGtjAzR+1sB53Tfdb7S9YAY/aOPY50cFlrXZwlpzg0FA4Ry9sf3M1fBr1Bqt3Q5Bmgx5MAMWL8FGzrzB3qE4yZTQ44ymrHWT7fIZq4mBjLWuTNJdy2KiJSka1iw4Q5XkvtA+6Xu+5+ALKn7Tct8sFZXMzCK4fx7Wf7a42AR0eHTXFsn7WgNhCfkjMNju9WXRgRURptYWYOz2wHx+vlpWRGR+MvCeal0wP47PpMY0qiCKHb5EJmxIxl7yPGrCK+DQrrSH57O5NLp9dmwS2ZP5fxs2aStOwLi5VFBt9rA+w9jJ8/chbeR/fPKskuNAbqNnXeOqVwFhXxP5OHsab9tQC1OjhzhBG7piA1ysiCisBA/EidmcNVcoqHJtzHba+vqXWX1M52YG/sSIOJB4dT/GB3VFH43ij8cXD2MH4+5rNG9jDC+h6Ob2wf81HV1bQ7WxdH3yc6gelPruC55Ak++y0szeCdu0diKzG3XK0m7AbY3yjzVgSzR+8+wmWz72PVr55vdEqfIzVljPntY0RVKDLXFVKz/6ClLUnvGRG8ddjWGWFRkepii4QkG3+omhpk4w7+PGsMe2b9M+iRzTwJOzXPdw6bL70xzndxtCipxVOfu+7+1rJ6XP/eG5dQynf3+mZHnnR0QDbuiLj6HG5bFXYDXJYR3WCU+eSJR6g8PYS4D0I7unxjOIuK6LK4nFEdHsMVE3g/WzX0XLwDV0UFkdeujEzUuTJu++NMn3LtsSOySy/+vc1sSBhGTua/BLW/Z4aXUM+IECw9Vtm4tvstAPxvvYiMQKypiOaRNx6j5+IdOC2cgdj73pv3y9cYGe+IqPj6bu6koBxpOLNP722nwzpMgiUuiE9zVpEz6D56Bo7XDwuuioqgZmKN5LF1I5FgyzXSSF66iWSrRQRJ3AdbqMTI2Ooz5u6gjml3PJrez30eEfXZU0ceSLjb71xw7Y5H0xtrsvfsa423oJ5rG24Ld9npRAyNRqOxCEtawDOODa6dYUCj0QSHx2WXZfKbYyhpC1PQm0nYDXBsqYtniwf4rNsy+wqS39M/hEaj+X4TdgMc/95m1r/nO1B4PJEzer9Go9FYhSilXQEajUZjBboTTqPRaCxCG2CNRqOxCG2ANRqNxiK0AdZoNBqL0AZYo9FoLEIbYI1Go7GIiDTAIhIjIm+IyHcick5EtovIaKt1BUJEponIVhGpFpElVutpDBFZKyJVIlLm/ttjtaZAtJVybStl2gbvq94i8rGInBaR4yIyX0QieghdEenvrgvLgtk/Ig0wRoLIYeA6IAn4f8BKEeltoabGKASeBRZZLSRIpimlEtx/2VaLaYS2VK5toUzb2n21ADgJdAUGYehuOGRZZPEy4H9iRD80aYBF5KCIzBSRr0SkVETeEpFYEZksIhvq7atEpJ/78xIRWSAin7hbBRtFpIuI/M79RPtGRC7zd02lVLlS6mml1EGllEsp9SFwALjc3/5WanXrfU8p9QEQ9OxRVmltCW2lXHWZXlj3FdAHWKmUqlJKHQf+DlwcoVoRkTuAM8D/NLafN8G2gMcBP8QokB8Ak5tx3BNAGlANbAK2uZffAeZ6dnR/+QX+TiIinYEs4P8iXWszsUrrcyJS7K5kwyNca3PRZXrh3Fe/B+4QkXgR6QaMxjDCEadVRDoAzwAzgrwWELwBfkkpVaiUOgWswngdCIb3lVJfKqWqgPeBKqXUUqWUE3gLqH2iKKXuU0o1eL0QkWhgOfAnpVQwUwJYprUFWKH1V0Am0A1YCKwSkb4RqrUl6DK9cO6rdRgt3rPAEWArEMxYcFZo/U/gDaXU4SCvBQRvgI97fa6g/rTBgTnh9bnSz3Kj5xERG/Bn4DwwLchrWqK1hZiuVSm1WSl1TilVrZT6E7AR+FEkam0hukwvgPvKrXE18B7GjLlpQEfgtxGodRDwr8C8IK9TS2t6FMuB2nnIRSSkM7aLiABvAJ2BHymlGg6jHzxh1RpizNaqgJbOfNVWylWXad352sp9lQL0AOYrpaqBahFZjNEp+1gLzhdOrcOB3sAho3hJAOwicpFSanBjB7YmCmIncLGIDBKRWODpVpzLH68AOcAYpVRlK88VVq0iEuU+rx2j4GOl5eEyYdMqIskiMsqjT0QmANditDQiSiuEtFx1mdbRJu4rpVQxRgfhVHe5JgO/cF+zJYSzXBcCfTFcHYOAV4GPgFFNHdhiA6yUKsBwOv838C2wofEjGkdEXhWRV92fewH3YHyZ41IXXzmh0ZNYoNXNExivKI8DE92fn4hArdEYLYgioBh4ABirlGpR3GpbKVddpm32vvopRmdaEbAXqAEeiTStSqkKpdRxzx9QhuE/LmryPHo8YI1Go7GGSE3E0Gg0mgsebYA1Go3GIrQB1mg0GovQBlij0WgsIqiQnhG22yOqp+5T19t+Yyzbik7QWlvDhaC1regErbU1NKYVdAtYo9FoLEMbYI1Go7GIsA1uXDl2CACHx7gC7pO2MRqAlEWbwiVDo7kgsffPJP/xlIDbE/LbkTHncxMVaVpCyA3wmUnDOJspdBp2DIADl7wfcN8pF18DwO6aYSQvjUwjXDl2CEWD/BdT5huHqDl8xGRFbZuT066kKs13XYf9KiJ+f9ulORy8pWOzj+v9/mlcO/PDoMg/tktzyJ+ayIHRCwPu88LQvrxZNYr0+doIRzLaBaHRaDQWEfIWcNzEY2weGMyQnbC453oAhk9MhaWhVtJ6zo/KJeaBY+TnrPK7fcRnU7DpFnCzuDXvHzyR5jv87Ixjg9lSNpT49zZbpMqgeHAy+XnNHw8+s9M9ZC/MCVsr+PyoXMoyomuXS3JdHLj5j40e82jKPno9uJy5Z8aT/PZ2VHV1WLQFwnZpDmCUqT/Stp0x9a2hOUhMDGduvwxntJC+7hg1+w+G7VoRMcFd94QzFOcORG3dZbWUWiR3IN2e2suy3mt91p90lrPwtDGDi73cQUTFvLRRXuy6jWefrmD9e7GWXD+qR3cAynq1bATJ/bf8kZyT99GzpeN0NYGaUcw/g2zUeDMuoZRLnp3LjE/H4jxxMgzKAnNwrOHKyb/H/wOtz9/y6LfCGN88qqQS165gxoQPD57f/3xmJ+N/h2hWzJ5Dn+gEhvzHVDq2JQN8uiKO/PMVje7TI8pGgq3uZlvWey3PLhrA+h9YcwP64+pFWxu01AAWnr7cS2fkPDAiHhHsaWnE2PY32FTmquK7ylSMIVvNZ/9dPQFa1PrVNMSWmIgzrq5pUq0c7HcYww5nRkcTI9EcuHkh3Gxs/+E3N2G/PQVnySlTddqTkyAmhm/v7wFAwaRXvLYGN4a7PTUFouqZ0epqnGdKgzo+5Aa42+RCZsSMbXQf14po/j7go1BfWhPB2NPSyPt8E6PjT2OM4FjHbQW3Yru9EqsMsCa07PnNRXxx8xz3UnteOj2Az240Zmi6/n/28WjKPp/938l6l0c+vpFDQ83VeXRJBn8Z9Aapds/Don2zz9Ht42qmd/7QZ90dO35Jl7EWGeCmLP+pD7NY3ncxLfmymjaMTciOPkmMxDfYVFUTTUzJUQtEwcHZw/jDz15r1jELSzN4+17fsbYz9x+iJpTCguCAo4zJ06Yz4fkPyUsqNPnqgVGxTtLtxv098eBwih/sjjphvC1Wu6Ib7J9gi6VXXAmHMPcNuGN8JTntGtbHYLCnptDt42pmZ6yp/a7e5w0W033AP+n5FVnRvoJnHBvMltlXEI+1nTBgvJYcXZLBuKTXiYSHRMHrufTqWRy28393KI2su7eG7fyNMSJ/DADyYhpw0PTr7//NMJ6+ZSUj431n5TlSU8aY3z5GVIV/D39sqYv4db51NZzGV15M44qMqQ3W2x2K5DXbOTm7AxA5BtibI2XJxHj17ax++jr+mnQ9AGd/aLzx7LnG/B74gtdzWdx3UaP75Cy8j8z/LWzw29oGDsD10jnmdfuYBFvrbEREdMJ9WdLT8h7wWmJi+MugNxo8JABeONWXdxfeQDrmxVb+csgGv77oUPFslwGsN7nl4WHvwc4AZK225gHQbfAxJiSWNFh/zmWj6zt7Te+4CkS71Vvxl3JhS0xkz9xBPJM4n/punUgl/r3NtROzlfW60vhwDYxM/JpFL08j+/HduM6dC9v1bYmJ7PnNRbx143yGxDQsszUV0Ty07N8A/3H+6qpBFD5azc4BH0EI7hvTDLAtPp5DDw1iSHzzXvfMJKpHd769v4eXT8iXjwovMT2w/c+rrqfPbUV+DUVjLD+XyhMbbmmwfnLu5zzVaXeo5AVFU+UaSWypdnDHx4+QXWFuGTUXT5l+cfOcBq/AkcRNGV/z5rS6hJCT066ky/rTlPdJrE3WMhOJj+Ojm+YFdD1sqehLz6cNrf7eak5nxbFzyBK/x/666CIAStZkkBHkG51OxNBoNBqLMK0FLIkJvHPPHJ8nz/JzqQAc3daVTAt8gPU5n9nJHYoSOS2K3rM28bR9HK8NbthaiI1y8E7Wu7UhfS+c6stHhZcARplmPd4wvfedmcP5bGRW7fJ3h9LIIrwugEgs10CsOXcJWdN3cNodiO+P2FKXpS6zqMze7J+UQcGkBTRWpgWOcu7YcTfdqs31Dyfkt+OFoUbUgych5IXyOwH49weX83jOHfQdUMinXglOa85dQv/7NxN45Bhr8JcEE4glWw2XSlYzxuAwxQDbEhMpH9KbWPEV73lF9mcoIokt1UYnjWGsDpp+/cwA5WPrnM7hzS5y2sHKsiTefLXuVS/QAy1jzucwp2453N/H3qkTRTkNfWVrKqJpdzzy/Jbp0WepHHltbSC+P54tHmBZ0ghA0bVdm4xZLnCUM+HrKXQZm4/TJF0eMuZ8zqL2RpTIo/csYFxCKeNm18XYjrvFN4uvwFHOXw/9gBQKTNXpj/Tos7iuG1a77C8Zqz4ry5IoqOraovocUgPsNygZKB/Sm3V/XIh3cPNJZzlSZQ/l5VuFxMRwvoP/AvzZP4weaKuiBeojMTHYkpMgNRm7GH7VOf91J+lLImfgFVu88aZzbFx/ts/yNRZHasp45I3H6P1c5Oj1kJdUSF69ulqfGJsDe6ceOIuLwaRZxW2JiUh8HACO9k1n7OXtmUDKj603aI1x2mkkbN2+/W66jI2MtOS8pELyViwOev8DjjLmPnMvScu+oDfNb0iG1AD7C0oG3C1f3wp99fKZZM82wlMi4bXjzO2XsWL2HILNgLGSM7dfxuJn52IXRd+oOKvl+OXQQ4MAWD31eeqX6ZjfPkbPxTsi4ndvCQ92/Ib+n59g4ZXDcBYVmXLNPb+5iI9umgdAoq3h/dQWyV13PwDZ0/ab3koPFZOnTSd5zfYWD0nQagMsuQO5epHRMszruDHoHtn5t73OA867AcPPaTWuKPH7ytnnb3nkvGjE4UZKJXFFiY8vffAzU+ny0d6I0Qfgamf87x7VsEyjyxWuisbT1SOZGIkmO/ok2Fo2dkRzKXg9l7dunN/ipAGzqZ/c4p24cvurq2uTRlZcZQyn+bM5UyPm7TIYvL9P/LZ8XK0Y6KjVBtjZPtorTjX4TpaR8Q6evG0lAK9dcY1lCQFnJhn+nmse8N+pEnssCue3DccvsIJAWlPzq0xriQVD4cwruev21X63BQputwJPgkNTCQHZ6yfVfrYiaaBXz2K/Mav1yV4/iQ5/N+7BhEIHViS3AJzv4vBJbjnp6IBt3XYA/jxrDK8l2UieeKS2Ey6ciUb1cZ0pZcoT05n+5ArGJQSXLlwf7+/T2rc4SxMxPLGtEwZ+wJb+Dn72huFrzfnNKVOM3qm7hjEwz3CDvNh1m/+dzGnkBMXZTEOMR+tJZzlXL59J//2HI8KgeSjLOd8g399D988qwzq8X3PwJDgkHRgMQE7+fX7367G2iqPD3Z1u15ijzZMwoGKdPNsr8KQGPsfkJ5Cy2Fq/euHMK5mcu7Z2uX7ykicRo+DqXDBGrOTfeq3nmdnjTHkTVtXVJC37goLHukILDHCok7EiIhMOYEhMNAdGvw5AH0cesce70G1tFfa1AQxjCCi+ylE7JrE/RuSPodMO601b/VlGNKHFU8d6rm1kp+FX+iym2hV7H+lL/5fbhXxWlJYkWURKXU0dWchTnXbXJiW88+fhZDSRvDQhsYQDYz5j/SzzIkv+vOp63uhytc+6fr1P+ITGeZhy6BrW/l824J7qKYTJWDoRQ6PRaCzCtBZwmauK2wpuparG8GX9W6/1AdNrD9xsOOezcybRy2H0psvGHSHVc35ULv16n2h0nzPLupPygfUdhIFmGUm3t6dg0iuM+CTyZ+bw/P5RJZWWRD9EZfbm5HVda5c7fbi3eX7zet3c4Sz7liSu7Psmg9SONiqnGP0ECYUO2lk0xga0LCnBTLzdHZ7ZO/bkZdS6Rbz54pNLyPp1eL6HKQb4pLOcWYUjsd1eWTvs4DOzx3FgzGe1+6RHn20wpN6ea5Zyacx4ADJKB4Rs1PxAs114s7A0g9hS6wKlJCoK19CBqCjh+JkatlQ7guqIsRrbwAF0SG04ru/hGhe28Q6cJ8yd+cAz28H+SRk+yQuDY6aSmt896PNUdfV9va9WDl46PSBiZkXZf8sfwWvoj4kHh3Oi6rLa5ahte8M6yE1bxd4/k/ypiQBNTvMUDsJugMtcVcwqHMmhoeV4D7jde9YmH5+PuupfGP7my/SNisMudZ6RnUNWADB87lhiRoZGU6DZLgCcysW+mkreuXsk8RutSze1paYwb/kCctrFc9ns+xh/Po99N/gGiHu0Sk0kmACDyrlV7GzB9DnhItBsF9uefMXf7sGf1+Hgsxv71o5zG2ks670WVqytXR4xfkptz70Z2GKMoEh7cpLPGOGeZC3PdrBuRhR7chJ7nkriwA3+Z5c+4CijStmwnQ+fhrAb4GBnO7Bt3sXDo6fwu08W+x0K0iz21VTy8Ogp2Ap2RUTLBmDVr56nvdgA3zjQSNSq0QBsve5lAIYv8Z0dwpOs1cUOnvps1YwoR5dksPXyl6l/X3mYPG067bccpOe58CUNtboTLmrbXq67J48DjrLadX3+lseI8VMYMX4K8mBiUHM9qZoaXAX7mXbn/SwszWitrBbjVAIlZ1A11vYou0pO8dCE+1hYmkH3qAQ62n0rycLSDKbdeT+ugv2Wa/Vw6sMsFmYvb7B+YWkGD024D5fJc36Fiwvt+4SDjvZ4Otrjefuy17nmqyp6bm6PPTWFXnEl5LSL96nPVTXRps8HB8bMFfXvK2/anXXgPHEyrElDrW4Bu86dI27NTsbPmlk7etQArymnm/PkUDU1yMYdtcHa3pgRWL6yLIm5z9xL8hnzXtUC0VhZgHtEro2bI6rl2+hsJxZp7f3BaQAy0+8x/KQh4KSjA7JxR9i+T/TuI1wxq+EMGK2h025zMiX9Jbc8kfaN0Qn79q2MS/oST+ei1TOiBMIzI0rX3d+GvcxC4oLwBDd7aG1z3XvU/HDw7sIbWJ52Q4P1scWQvuzziDJq4S6LUOKvXDvsVyS/Z10kiachkL0wh5wi/4kWzSW2mLDOiuIsKiJlcWgzG81KU6+f3JLNJPZcs5QEW6x7Il7D+F779S3wijENfNzqLSapa5o1FdE88oYxVonThHT5iEnEMBOzZ7X4vhDJ5eramU/PnVar+P7gSW7p5RhEn7K7G2zvscpG3AfWGt6SNRn0Oeyrrd3xaHo/97lpoZI6EUOj0Wgs4nvZAtZoNOYgG3eQtdFqFf7JiIAkEd0C1mg0GovQBlij0WgsQpRJU6poNBqNxhfdAtZoNBqL0AZYo9FoLEIbYI1Go7EIbYA1Go3GIrQB1mg0GouIeAMsIv1FpEpEllmtpTFE5A4RyReRchHZJyImTd/YMtpCuYrIWrfGMvffHqs1+aOt6IQ2p7Ws3p9TRP5gta76iEhvEflYRE6LyHERmS8iQSW5RbwBBl4G/mm1iMYQkRHAb4EpQCJwLRAZc9kHJuLL1c00pVSC+y/bajGN0FZ0QhvR6qUxAegMVAJvWyzLHwuAk0BXYBBwHRDUyE9NGmAROSgiM0XkKxEpFZG3RCRWRCaLyIZ6+yoR6ef+vEREFojIJ+6n10YR6SIiv3M/Kb4Rkcv8X7X2fHcAZ4D/CebLWKj118AzSqkvlFIupdRRpdTRCNXalsq1WbQVnVpri7TehmHkAk5jbqHOPsBKpVSVUuo48Hfg4mC+VLAt4HHAD90X+gEwuRnHPQGkAdXAJmCbe/kdYK5nR3cBLPBa7gA8A8wI8lqWaBURO5ALdBKRvSJyRIxXkLhI0+pebhPl6sVzIlLsvimGX0A6tdbgtHr4BbBUNZ05ZoXO3wN3iEi8iHQDRmMY4aZRSjX6hzFS8kSv5eeBV91fbEO9fRXQz/15CfCa17YHgHyv5UuAM41c9/fAr9yfnwaWRaJWIMN9rq0YryBpwEZgdqRpbUvl6t4+FMOlE4NxA54D+rZ1nVprcFq99uuJMaRxn0jUiTGX8pdAjfu8S3BnGTf1F2wL+LjX5wogIcjjvOd9r/Sz7Pc8IjII+FdgXpDX8cZUre5tAH9QSh1TShVjPC1/FGla21i5opTarJQ6p5SqVkr9CePB1lS5thWdWmsTWr2YhGFADwSxr9n3lA1YDbyHMdp8GtARo0+oSVozHGU5XrPZiUiXVpyrPsOB3sAhEQHjy9tF5CKl1OAWnC9sWpVSp0XkCIRsIg1droFRgLTguLaiE7RWf0wCftOK48OpMwXoAcxXSlUD1SKyGHgWeKypg1sTBbETuFhEBolILMbrbKhYCPTF6FEchPEa8REwqoXnC6dWgMXAAyKSLiIdgYeBD1t4Ll2ugIgki8godydKlIhMwIguWX0B69Ra6yEiVwLdaF30Q9h0ut94DwBT3WWajOHaCWr+lRYbYKVUAUZnzn8D3wIbGj+icUTkVRF51X3uCqXUcc8fUAZUKaVaNFFWOLW6+U+MkK4CIB/YDsyONK1trFyjMVoRRUAxhl9urFKq2XGrbUWn1trgvgLDmL2nlDrX0vOaoPOnGB1/RcBeDF/wHDygEgAAEJxJREFUI0Gdy+1E1mg0Go3JtIVEDI1Go7kg0QZYo9FoLEIbYI1Go7EIbYA1Go3GIrQB1mg0GosIKhFjhO32iAqV+NT1tt/A8baiE7TW1nAhaG0rOkFrbQ2NaQXdAtZoNBrL0AZYo9FoLKI1Y0G0aQpnXklZznnSNkYDkLJok8WKNBrN943vrQFOHVnI1wM/oA93A5CyyGJBmjaFLT6eQw8NwhXTcFuH/YrkpfqBrmka7YLQaDQai/jetoA99OttDPt5flQu7VZvtVgNRGX25uR1XWuXO324F2dRi8bKiRjsnTpR9ON+Te/nUCS/vR1VXW2CquZhuzQHgOLByQDUxAurpz5P96iGw8QO3zUWlobu2udH5VKWER3Uvh0LjOGpZeOO0AnQhA3LDLC9fyaOjKS65XIHausuU64tuQPpnnAEgE9zVgEwfMbYlg/KF0KKru3KP2e/Urs8OGYqXd4laCMc1aM7AOczO5lapoGwd+rE8Vv7se3JV5rcN/98BTM+HYvzxEkTlDWN5A7E2d4wfAXjjf8Hbvb+HsGO9d2Ca0dF4Ro6EBUldHtqL8t6rw3quD6fGC61rI1hk1aLvVMnHBd1b9Gx7fYXUXP4SIgVhQbvsg+G6N1HWtxICpsBtsUb4x9LoruSuhTO4mJwj76W/3gKB0a/Xrv/s8UDWP+D2HDJ8eHqRVt5Iu0bU64VLJ7ycrT3/dG3PfkKQ6qm0nFJcD/w/rt6ApB/zwJeONWXz67P9Cl3s/B8n2Pj+rN9Vt30WQccZVQp/56vPY50cEVAGKcI9rQ0rl+yiUdT9gV92GlnhfG/Io7WjPgtUVHYsjJ56c2XyYpu34ozhZfim/qx5b+afrD6I2vpVPrOLgXAda7FI02GnJaU/WWz76PL4nJcFRXNvl7YDPChhwYB8M49cwDj5lp45bA2/zodLjzltXrq84SqZfVgx2/o//kJS8o90PeZPG067bcc9H+Q5yFtMfa0NPI+38To+NMYQ+gGR+66+wHInrYfZyuu7xo6kJfefJm+UcHM69o22TBhDv+SOB2A/vdvtlhNHS0p+1W/ep5RHR6j+3OfN/t6YTPArnbG/5x2nplAToLNaN0VvJ7LWze8QnMqd6QguQO5epGvr/jdhTeQPr/5he+Np7z8+RRbSoxEMzr+NN9+tp//mTzMVHeE9/dZWJrB2/cak27Eb8vHGUEtHm8qfjqUUU+vI8a2n9Hxp4mRpuvn4GemkppfBUB2odGic54pbZUOFSVNtr68y9SbnMJThoZWKQiOtI/2Mjh2alDupfqk29vzlx/NB+Bnr08l627r+18qfjqUn89e1ey3ju5RCcz75Ws8kGC4f3rPCj4CJiwGuHDmldx1e2CHaq+exQyJiQzjOyJ/DADyYhrGpKqBOT8ql25P7W3gvlieekOY1Bmols7YhWGEH03Zx5r211oW8nLS0QHbuu0AuCzSUJ+C13Pp1dNobZesyQAgtkR5/bb+6+eMY4P53/lDa5e7eHWShtPoZa+fRIe/1xmG2FIX8esathzNMLy11yoqosu7cEX11Cb3rYkXVv3Kt9PSYwM8v4NVnJk0DICrH9pMXlJhg+31y76sl5Cft8Bnn5HxDs53cTT72mExwGU555vlO3vhVF+jFUnrWpEtYe/BzgBkBYiAOHXXMIqvMgq2X+8TfjtD/mX01+w6bPyILU7oCGBk+/wtjwHbzgRvuFphrEOKl46RiV+z6OVpZD++23J/ny0xkT2/uYi3bpxfawD6HDZaLrEbGhrd7PWTsOXXGY0O+xUpXjG+4TB47fYXkbV0KhsmzCHdbtz4tvwEUhabf380hbOoiJTFTbu3bPHxjOrwGPN++Roj45tvqMLJ2Uyjsr7YdZvP+j5/yyP2eBQ91lZhX1v3m6f36E5WrO/v01JMiYLYUu3gjo8fIbtiNyenXcmdGb6t448KL2n1K3y4KL7K4dNZCEZnS+66+1lx1UKGxESzuOd6+lyVDYQ+oaPfCgeunfmhPakJdNpRAxhvGJ/mrOKjm+Yx45mxYKEBjurRnW/v78EXNzd+43h+X1e1nf6LapCN5tbNmsNH6DfvPCV3COl2Uy8dNlwVFXR/7nO2jO/LyPjI6QCvHDuETsOO+d3Wb4UD27otADiHG5OGHx3uCRQITWexTsTQaDQaizClBbzm3CX0v38zLuDWvH80yz0RTl441ZeE/HYBt58flVubqOFNuXLRYVMcB4ekMSSmtMnzBEUjD9T6gfgJhQ6/SSPqqkG4cspapyNExH1gtBzOV+Yy8anh/Kb7hxy7rR9dVyrLImHOZ3aiYNIrgP/Wb0Khg+G7xnK6Is6IZGhlZ1ooceWUcWrKsGYdE1vqIv69yIkwiEQOj3Fx4JL3fdaVuaq4reBWTlweR0ymUeZnf1gOwJ5rvH2/dfWopTbA0ky4LdWGL+i7Q2lkNdEBFgo8Adbp0WsAWPT2KHrO8f96KbkDGwTAFzjKWVl6OTE2B1/8x++JkWhWliXx5qujyGilCyW2xPi/siyJcQl1N35JTiwXTcr30THx4HBOVF3mc3z07iPk3xXFgWt83SVW0271Vk5UXUb3FQlsn7WAwc7mJZaYQYdU4+aKqoqF2R3pXKNwlZVbrMqXPdcshWuad8zC0gzeLjEiJaK27bXc/x5p2AYOqP3tvSly1lA5rxvTnv/Ab6ecNwtLMzjp6MC7C29okQ0IiwGWKjsnncYXS7e3J8bmwN7ZSBCIsdW1fn/2D6P31KwQFFtqCvOWL/AKjQuMJ1njpLOcEqfhpL9jx910ve1bbFmZ/OSTr4immrnP3Ev6stb7CD0+8DlldzLOK7jdX4jPst5rYcVan3WXzb4PW0xlq3WEA6lRFDjK6RsV1+zEEjPYOWSF8cH9r8BRzsOjp0DJGVxnSiMyNToY8pIKyVuxGIAR46fURqJYgju5Jca23zoN9aicW8XOgR80WN8nOoF1f1zY6LFO5WJfTSXv3D0S2bijxQEEYTHA2Y/v5upzMwEomPQKD3b8hh9v/hqAzOho2lL879XLZ9JvnvHQ6FZdSI1XoPYN995L8prtIXLHt45Vv3qe9mIDmn64mI1t8y4eHj2F332yOKIzuzz0jYrjd58sxqmEKU9MJ2nZF1ZLavO0NLklUtlXU8nDo6f8//bOPaapK47j3962QLSMRxXGQ4Qqj5s9NAw0LOrQZBq3mUjiYwuGTWWI9Q8jPubm5mMJYUvcTHBihqIJ6tycUYwTA8sC6lBxzIgzNKJUQARnZdDx0Mptuz8ut6Uvnr0PsvP5r/cWenrvOeee/s7v+/uCqr8zpvHPywRs6epC7IGHAAD6uRa6rALQPuPjosvVwYgoNSErqAp04VbEFjWD6a9NwCVqK2HFguxsTKjUwSKR1ZE3BRzexsowQHsnzP0JzVs++wF5qnTJZr7IZZTtQZGz8yTyAoVvq6X9H2xM18KqkCE074FrCOrTGLd/d99Ws2LwFZzgUDLEK5/AVya9BcJwmHHjAwBAyD42C0LGWNnJl2HG9H95iwFzhTY0xQrQ0LokLsdfyUDskbE1fqysWV6G4h42RhbOxYIVCuSE/oIQ+USomqxoXhkF9UL2Ms1S/4GsgFboXlCYeKORF0XX5MttoAsdr5fm7Dqoa+wJK+1JFujTvvf6Z/OJpdOI1Z/nIGfnSaxQGbFLLXwblHUtSN7hKhqwb7C4ljBboTKiKbsMxX5O/YRnrAwDWdUtyAA82pOE5HDadk7V2gefS+7Ddgmd7Ps05nWS7yMjEUF5m/rDSTg6bfCcUbpQC1UTu74NtVWZs4dxvPHLl/dNOEbfiMiKICDL8TilUwmeX+nM1uAGFNAvPJ4PXNWCOZMasGtynWBtYvSN0BQxoKG1HYs/2+GQCzzpJg3aoHX4uzXLyiSTXeIOq8mEgOPXUb8tDFAZvSNeGSGeRAMBD9gcz3hkuJ2Eh+onfONTVoPgYb63J8YfADAtYfDNIykwlAjK28hjNdBtZ6/kTwsOelTjPjH3YM6JLeyvX54rtomSBfG2boktUV9IrL3P8O6FTfjxHbsKKvWVuwCAuowUFxcDrlTlQAaKSviCediCqN32G++sgrPU6hBV63isuHsRCugXtu9zNOoKb+0bC8fOz0fMMgOv4pWRIq9kFVBT+2YipjsTlK8ZNW8dQJDc/nN5sH4iJQwz2SGtc9N3xYITwKjl4u2WUDNo6Nb748FiLjTjOSTabpZh+r4GW+iR13bx/gkEAoFAcAvvK2CFJhr6+Y6l3TqPRyK4RPhVhKWrC7EbqlF++zXM8mXlkNxKcfPGHlyWp4CZIIM/5b7yQuUzCqsr1iOuX1QiJbjY5PVdb7IH1kljBazQROPfmaE2QUD0jms4lDwX6W7Sf8RGVnULcVWAPDQEj6uBoAEyYK6fpK5Se9XtwptISYjDodBEQ58RjvqMAngSwAjB08RAp2L6bGGlRFUT0v3bRWqVABOwYV6Yw4ZSoTEcfkZxp69zza9jRcCfDilR34TdBHK5YhyOGQWcYGR1hTTK5o2GdtoPL9dNFlwAYZgXhsxPzuH03wtBVdt3jct7lfB5LGxmDOXvDyZxaGuk5y8p4SeT2iN2cKhXE9C61YS7XE5zPyZrH/I7Elh3FBHa5Tz+OYS+/zI3X/7K/tk4MzcR6YvFEy/x7ojh7PDwc/Yit2X0hCT4vXosL8lE5RtFDnE+Z1qYbnRZKLxfugkAECehwtEjRUwBRFZAK+aeKMDm2UsBhoGfog8bj32M6D0CF7lJnI5f+4UJQ+P4EPaW2wUfyNXBsOR3oTbhgsNxk7UPF3uDUDFfA6tBWGsqT+OfQ4z770zfRBkoXyELeLrCuyOGNx0evEnER61I+m4DGhZ4HpBLvt6GsNP3bRtu42tNJE0iSk3YF1GK5Es5YjdlRHjL7YIPuGsKOFp65Xck2C2pBEbq4x+QhniJd0cMqQoEzJ1GxO8xIqabzY9zl7iu7LFKxiByvMK5Jlz7Ih/zf2vAhwG3oaKkr4ZzxmJiA8JiFOipP5yEtbN+93g+K6jK7TU1WZSi1NxozE3B/pWHALgf/3ShFpojzRAyD8qde4cU5ibBHDFamG52RVl3TzIrCPM9PRIO+gIAkqtdE/RDLrcJ2km8QXRJBwBAE+KaiD8WZ43RYjYYoNZF2pw5xNyIGQ2cCCa2Xrw6G1Ojng5hIut6TTe3JeJGbjImQNiwmf6rFOxOO+Wx6DpdqIWmuFVwR2SuHw7Fqe4AfPtlNgI7hambIYgjRnmvEpuKtiHq6C2YR+EcyiecwCG41vXceJt8Afv3UdekAGkiN2YwRHgY+OgNoAu1Q79xAM4iGCkz7680GK6FAWCdOwLPCJ9pFJHY5jarQEhxgyeGc//9ngIhx68KtmEpjCNG7zRE5l0lMVQBCap/hpiLmQ7HxFrFKVuNLm2ZIoIQx1ngMhyk0Gfby8NttkmDMeU8hagS6dXXYMVLOYjPvQNGxJKYo7n/fEOEGAQCgSASvKyAVTofpE5ZanvNFlwfn/mz4xVOVCAFzPf0iFsrnTqw4w2hCgB5g0c3w5CKpQ7HmponSVK8JAV4mYDD914F9tpfC+F2QSAQxEez3TXuTMa/Z0gIgkAgEERCZrVKwc+BQCAQ/n+QFTCBQCCIBJmACQQCQSTIBEwgEAgiQSZgAoFAEAkyARMIBIJIkAmYQCAQROI/84Wo3sDO604AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 32 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "count = 1\n",
    "for i in images:\n",
    "    plt.subplot(4, 8, count) \n",
    "    plt.imshow(np.squeeze(i))\n",
    "    plt.title('num:%s'%labels[count-1])\n",
    "    plt.xticks([])\n",
    "    count += 1\n",
    "    plt.axis(\"off\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过上述查询操作，看到经过变换后的图片，数据集内分成了1875组数据，每组数据中含有32张图片，每张图片像数值为32×32，数据全部准备好后，就可以进行下一步的数据训练了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 定义网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们选择相对简单的LeNet网络。LeNet网络不包括输入层的情况下，共有7层：2个卷积层、2个下采样层（池化层）、3个全连接层。每层都包含不同数量的训练参数，如下图所示："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://gitee.com/mindspore/docs/raw/r1.3/docs/mindspore/programming_guide/source_zh_cn/quick_start/images/LeNet_5.png\" alt=\"LeNet5\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 更多的LeNet网络的介绍不在此赘述，希望详细了解LeNet网络，可以查询<http://yann.lecun.com/exdb/lenet/>。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在构建LeNet前，我们对全连接层以及卷积层采用Normal进行参数初始化。\n",
    "\n",
    "MindSpore支持`TruncatedNormal`、`Normal`、`Uniform`等多种参数初始化方法，具体可以参考MindSpore API的`mindspore.common.initializer`模块说明。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用MindSpore定义神经网络需要继承`mindspore.nn.Cell`，`Cell`是所有神经网络（`Conv2d`等）的基类。\n",
    "\n",
    "神经网络的各层需要预先在`__init__`方法中定义，然后通过定义`construct`方法来完成神经网络的前向构造，按照LeNet的网络结构，定义网络各层如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:53:43.367791Z",
     "start_time": "2021-02-03T08:53:43.248322Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "layer conv1: Conv2d<input_channels=1, output_channels=6, kernel_size=(5, 5),stride=(1, 1),  pad_mode=valid, padding=0, dilation=(1, 1), group=1, has_bias=Falseweight_init=normal, bias_init=zeros, format=NCHW>\n",
      "****************************************\n",
      "layer fc1: Dense<input_channels=400, output_channels=120, has_bias=True>\n"
     ]
    }
   ],
   "source": [
    "import mindspore.nn as nn\n",
    "from mindspore.common.initializer import Normal\n",
    "\n",
    "class LeNet5(nn.Cell):\n",
    "    \"\"\"Lenet network structure.\"\"\"\n",
    "    # define the operator required\n",
    "    def __init__(self, num_class=10, num_channel=1):\n",
    "        super(LeNet5, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(num_channel, 6, 5, pad_mode='valid')\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5, pad_mode='valid')\n",
    "        self.fc1 = nn.Dense(16 * 5 * 5, 120, weight_init=Normal(0.02))\n",
    "        self.fc2 = nn.Dense(120, 84, weight_init=Normal(0.02))\n",
    "        self.fc3 = nn.Dense(84, num_class, weight_init=Normal(0.02))\n",
    "        self.relu = nn.ReLU()\n",
    "        self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2)\n",
    "        self.flatten = nn.Flatten()\n",
    "\n",
    "    # use the preceding operators to construct networks\n",
    "    def construct(self, x):\n",
    "        x = self.max_pool2d(self.relu(self.conv1(x)))\n",
    "        x = self.max_pool2d(self.relu(self.conv2(x)))\n",
    "        x = self.flatten(x)\n",
    "        x = self.relu(self.fc1(x))\n",
    "        x = self.relu(self.fc2(x))\n",
    "        x = self.fc3(x) \n",
    "        return x\n",
    "    \n",
    "network = LeNet5()\n",
    "print(\"layer conv1:\", network.conv1)\n",
    "print(\"*\"*40)\n",
    "print(\"layer fc1:\", network.fc1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "构建完成后，可以使用`print(LeNet5())`将神经网络中的各层参数全部打印出来，也可以使用`LeNet().{layer名称}`打印相应的参数信息。本例选择打印第一个卷积层和第一个全连接层的相应参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 自定义回调函数收集模型的损失值和精度值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "自定义一个数据收集的回调类`StepLossAccInfo`，用于收集两类信息：\n",
    "\n",
    "1. 训练过程中`step`和`loss`值之间关系的信息；\n",
    "2. 每训练125个`step`和对应模型精度值`accuracy`的信息。\n",
    "\n",
    "该类继承了`Callback`类，可以自定义训练过程中的操作，等训练完成后，可将数据绘成图查看`step`与`loss`的变化情况，以及`step`与`accuracy`的变化情况。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以下代码会作为回调函数，在模型训练函数`model.train`中调用，本文验证模型阶段会将收集到的信息，进行可视化展示。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:53:43.400967Z",
     "start_time": "2021-02-03T08:53:43.369836Z"
    }
   },
   "outputs": [],
   "source": [
    "from mindspore.train.callback import Callback\n",
    "\n",
    "# custom callback function\n",
    "class StepLossAccInfo(Callback):\n",
    "    def __init__(self, model, eval_dataset, steps_loss, steps_eval):\n",
    "        self.model = model\n",
    "        self.eval_dataset = eval_dataset\n",
    "        self.steps_loss = steps_loss\n",
    "        self.steps_eval = steps_eval\n",
    "        \n",
    "    def step_end(self, run_context):\n",
    "        cb_params = run_context.original_args()\n",
    "        cur_epoch = cb_params.cur_epoch_num\n",
    "        cur_step = (cur_epoch-1)*1875 + cb_params.cur_step_num\n",
    "        self.steps_loss[\"loss_value\"].append(str(cb_params.net_outputs))\n",
    "        self.steps_loss[\"step\"].append(str(cur_step))\n",
    "        if cur_step % 125 == 0:\n",
    "            acc = self.model.eval(self.eval_dataset, dataset_sink_mode=False)\n",
    "            self.steps_eval[\"step\"].append(cur_step)\n",
    "            self.steps_eval[\"acc\"].append(acc[\"Accuracy\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中：\n",
    "\n",
    "- `model`：计算图模型Model。\n",
    "- `eval_dataset`：验证数据集。\n",
    "- `steps_loss`：收集step和loss值之间的关系，数据格式`{\"step\": [], \"loss_value\": []}`。\n",
    "- `steps_eval`：收集step对应模型精度值`accuracy`的信息，数据格式为`{\"step\": [], \"acc\": []}`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 定义损失函数及优化器\n",
    "\n",
    "在进行定义之前，先简单介绍损失函数及优化器的概念。\n",
    "\n",
    "- 损失函数：又叫目标函数，用于衡量预测值与实际值差异的程度。深度学习通过不停地迭代来缩小损失函数的值。定义一个好的损失函数，可以有效提高模型的性能。\n",
    "\n",
    "- 优化器：用于最小化损失函数，从而在训练过程中改进模型。\n",
    "\n",
    "定义了损失函数后，可以得到损失函数关于权重的梯度。梯度用于指示优化器优化权重的方向，以提高模型性能。\n",
    "\n",
    "MindSpore支持的损失函数有`SoftmaxCrossEntropyWithLogits`、`L1Loss`、`MSELoss`等。这里使用`SoftmaxCrossEntropyWithLogits`损失函数。\n",
    "\n",
    "MindSpore支持的优化器有`Adam`、`AdamWeightDecay`、`Momentum`等。这里使用流行的`Momentum`优化器。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:53:43.478821Z",
     "start_time": "2021-02-03T08:53:43.402997Z"
    }
   },
   "outputs": [],
   "source": [
    "import mindspore.nn as nn\n",
    "from mindspore.nn import SoftmaxCrossEntropyWithLogits\n",
    "\n",
    "lr = 0.01\n",
    "momentum = 0.9 \n",
    "\n",
    "# create the network\n",
    "network = LeNet5()\n",
    "\n",
    "# define the optimizer\n",
    "net_opt = nn.Momentum(network.trainable_params(), lr, momentum)\n",
    "\n",
    "# define the loss function\n",
    "net_loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "完成神经网络的构建后，就可以着手进行网络训练了，通过MindSpore提供的`Model.train`接口可以方便地进行网络的训练，参数主要包含：\n",
    "\n",
    "1. 每个`epoch`需要遍历完成图片的batch数：`epoch_size`；  \n",
    "2. 训练数据集`ds_train`；  \n",
    "3. MindSpore提供了callback机制，回调函数`callbacks`，包含`ModelCheckpoint`、`LossMonitor`和`Callback`模型检测参数；其中`ModelCheckpoint`可以保存网络模型和参数，以便进行后续的fine-tuning（微调）操作；    \n",
    "4. 数据下沉模式`dataset_sink_mode`，此参数默认`True`需设置成`False`，因为此模式不支持CPU计算平台。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:54:23.928574Z",
     "start_time": "2021-02-03T08:53:43.479829Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 1 step: 125, loss is 2.2961428\n",
      "epoch: 1 step: 250, loss is 2.2972755\n",
      "epoch: 1 step: 375, loss is 2.2992194\n",
      "epoch: 1 step: 500, loss is 2.3089285\n",
      "epoch: 1 step: 625, loss is 2.304193\n",
      "epoch: 1 step: 750, loss is 2.3023324\n",
      "epoch: 1 step: 875, loss is 0.69262105\n",
      "epoch: 1 step: 1000, loss is 0.23356618\n",
      "epoch: 1 step: 1125, loss is 0.35567114\n",
      "epoch: 1 step: 1250, loss is 0.2065609\n",
      "epoch: 1 step: 1375, loss is 0.19551893\n",
      "epoch: 1 step: 1500, loss is 0.1836512\n",
      "epoch: 1 step: 1625, loss is 0.028234977\n",
      "epoch: 1 step: 1750, loss is 0.1124336\n",
      "epoch: 1 step: 1875, loss is 0.026502304\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "from mindspore import Tensor, Model\n",
    "from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor\n",
    "from mindspore.nn import Accuracy\n",
    "\n",
    "epoch_size = 1\n",
    "mnist_path = \"./datasets/MNIST_Data\"\n",
    "model_path = \"./models/ckpt/mindspore_quick_start/\"\n",
    "\n",
    "repeat_size = 1\n",
    "ds_train = create_dataset(os.path.join(mnist_path, \"train\"), 32, repeat_size)\n",
    "eval_dataset = create_dataset(os.path.join(mnist_path, \"test\"), 32)\n",
    "\n",
    "# clean up old run files before in Linux\n",
    "os.system('rm -f {0}*.ckpt {0}*.meta {0}*.pb'.format(model_path))\n",
    "\n",
    "# define the model\n",
    "model = Model(network, net_loss, net_opt, metrics={\"Accuracy\": Accuracy()} )\n",
    "\n",
    "# save the network model and parameters for subsequence fine-tuning\n",
    "config_ck = CheckpointConfig(save_checkpoint_steps=375, keep_checkpoint_max=16)\n",
    "# group layers into an object with training and evaluation features\n",
    "ckpoint_cb = ModelCheckpoint(prefix=\"checkpoint_lenet\", directory=model_path, config=config_ck)\n",
    "\n",
    "steps_loss = {\"step\": [], \"loss_value\": []}\n",
    "steps_eval = {\"step\": [], \"acc\": []}\n",
    "# collect the steps,loss and accuracy information\n",
    "step_loss_acc_info = StepLossAccInfo(model , eval_dataset, steps_loss, steps_eval)\n",
    "\n",
    "model.train(epoch_size, ds_train, callbacks=[ckpoint_cb, LossMonitor(125), step_loss_acc_info], dataset_sink_mode=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练完成后，会在设置的模型保存路径上生成多个模型文件。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:54:23.979139Z",
     "start_time": "2021-02-03T08:54:23.929589Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./models/ckpt/mindspore_quick_start/\n",
      "├── checkpoint_lenet-1_1125.ckpt\n",
      "├── checkpoint_lenet-1_1500.ckpt\n",
      "├── checkpoint_lenet-1_1875.ckpt\n",
      "├── checkpoint_lenet-1_375.ckpt\n",
      "├── checkpoint_lenet-1_750.ckpt\n",
      "└── checkpoint_lenet-graph.meta\n",
      "\n",
      "0 directories, 6 files\n"
     ]
    }
   ],
   "source": [
    "!tree $model_path"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-01T03:42:45.627283Z",
     "start_time": "2021-02-01T03:42:45.606466Z"
    }
   },
   "source": [
    "文件名称具体含义`{ModelCheckpoint中设置的自定义名称}-{第几个epoch}_{第几个step}.ckpt`。\n",
    "\n",
    "> 使用自由控制循环的迭代次数、遍历数据集等，可以参照官网编程指南《[训练](https://www.mindspore.cn/docs/programming_guide/zh-CN/r1.3/train.html#%E8%87%AA%E5%AE%9A%E4%B9%89%E8%AE%AD%E7%BB%83%E5%BE%AA%E7%8E%AF)》的自定义循环训练部分。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 查看模型损失值随着训练步数的变化情况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:54:24.122898Z",
     "start_time": "2021-02-03T08:54:23.980168Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEWCAYAAACEz/viAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd7wcZb3H8c8vhR5qQkkoCQFLuEoLTYooiAlKuypFLxELoFcQKSo2jIhcEUQuUnIBaVIMGMSAIM1QQj+EFEIISYCYQjopkARIzu/+8cxk5+yZ3bN7zs7unrPf9+u1r52d+tvZc+Y3z/PMPGPujoiINLZutQ5ARERqT8lARESUDERERMlARERQMhAREZQMREQEJYMuz8yGm9lttY6jvczsFDMbW+s4kszsIjNbZGbzarBtN7NdSpjvUDOb3dH11JKZ9Y/i7FHrWBqBkkEXYGZfNbMmM3vXzN42swfN7KBax1WP2jpIlrD8DsC5wCB337ZykYnUlpJBJ2dm5wBXABcD2wA7AtcAx9QyrnpUoTPMnYDF7r6gAusSqRtKBp2YmW0GXAh8z93vcff33P1Dd7/P3X+YmHU9M7vVzFaY2WQzG5xYx/lmNiOa9qqZHZeYdoqZjTWzy8zsHTN708yGJqYPMLMno2UfNbOrk1VSZra/mT1jZkvNbIKZHVrku+xgZveY2UIzW2xmV+VNLxTDN8xsShTDG2Z2emLaoWY228x+HFXp3Ak8CPSNSlHvmlnftP0a7a+FZjbTzH5uZt3M7HDgkcTyN6csG2/zR2a2ICqpHWtmR5rZ62a2xMx+mph/fTO7wszmRq8rzGz9xPQfRuuYa2bfzNvW+tF++beZzTezEWa2YaF9XGTfp37faNouZvaEmS2LqsZGRuPNzP4QfcdlZjbRzP4jZd0nmllT3rizzWx0NPwFM3vZzJab2SwzG14kzrei3yD+3KIKtJy/N0nh7np10hcwBFgD9Cgyz3BgNXAk0B34H+C5xPSvAH0JJwYnAO8B20XTTgE+BE6Nlv0uMBewaPqzwGXAesBBwHLgtmhaP2BxtN1uwOeiz31SYuwOTAD+AGwMbAAcVGIMXwAGAgZ8GlgJ7BVNOzTaP5cA6wMbRuNmt7FfbwX+DvQC+gOvA99KrLPg8oltXgD0jOJeCNwRrW+36PfYOZr/QuA5YGugD/AM8OvE7zsf+I9ov9wBOLBLNP0KYDSwZbTu+4D/KTHO5HqKfd87gZ9Fv2Hyd/k88BKwebTvP070d5O3nY2AFcCuiXEvAicm4vxEtP5PRt/32Gha/yjOHtHnt4DD8/62y/5706vA30StA9CrAz8efA2Y18Y8w4FHE58HAauKzD8eOCYaPgWYnpi2UfTPuS2hOmoNsFFi+m2Jf84fA3/OW/dDwNdTtnlAdMBsldSKxVAg/nuBs6LhQ4EPgA0S09s6SHYH3ie0CcTjTgceL3H5Q4FVQPfoc68o3v0S87yUOODNAI5MTPs88FY0fCPw28S0j0Tr2oVwAH4PGJi3H98sMc54PW1931uB64Dt85b/LCFp7A90a+Nv8Dbggmh4V0Jy2KjAvFcAf4iG+1N6Mij5702v9JeqiTq3xUDvEurCk1e9rAQ2iJcxs2FmNj4qWi8lnIX2TlvW3VdGg5sQShNLEuMAZiWGdwK+Eq83WvdBwHYp8e0AzHT3NW3FnxcDZjbUzJ6Lql+WEs4Mk/EvdPfVBdabpjehpDMzMW4m4cyzVIvdfW00vCp6n5+YviqOn7Af87fVNzFtVt60WB9CYnwpsX//GY0vR1vf90eExPOChSrGbwK4+7+Aq4Crgflmdp2ZbVpgG3cAJ0XDXwXujX9HM9vPzMZEVVTLgO/Q8vcrVTl/b5JCyaBze5ZQ5XBsexY2s52A64EzgK3cfXPgFcI/f1veBrY0s40S43ZIDM8inKltnnht7O6/TVnXLGDHEpJafvzrA6MIVVXbRPE/kBd/fre8bXXTu4hQLbVTYtyOwJxyYivD3JRtzY2G36blPt0xMbyIkFR2S+zfzdx9E8pT9Pu6+zx3P9Xd+xJKDNdYdEmqu1/p7nsTqr4+AvyQdA8TTlr2ICSFOxLT7iBUde3g7psBIyj89/ceIQHGkldzlfP3JimUDDoxd19GqJu+Omqk3MjMekZny78rYRUbEw6OCyE0xhJKBqVseybQBAw3s/XM7ADgqMQstwFHmdnnzay7mW0QNa5un7K6FwgHvt+a2cbRvAeWEMZ6hLaAhcAaCw3LR7SxzHxgKwuN72nfay1wF/AbM+sVJcxzou+ThTuBn5tZHzPrTfg9423dBZxiZoOipPvLRJzNhET+BzPbGsDM+pnZ58vZeFvf18y+kvjN3iH8vaw1s32is/qehIP0amBt6y1AVOL7K3ApoX3jkcTkXoQS5moz25dQcihkPHBi9Dc+GPhyYlo5f2+SQsmgk3P3ywn/vD8nHBRnEc707y1h2VeB3xNKGPMJDXlPl7H5rxHqqRcDFwEjCfXPuPsswuWtP03E9UNS/uaiA9JRhDrsfwOzCY3ZbcW/Avg+4WD2DuFAMrqNZV4jHIDfiKoTWl1NBJxJOMC9AYwlnL3e2FY87XQRIalOBCYB46JxuPuDhDr0fwHTo/ekH0fjnzOz5cCjwEfbEUOx77sP8LyZvUvYt2e5+5vApoRk9A6hWmkxoYRWyB3A4cDdedWB/w1caGYrCInwriLr+AXhYoF3gF+RKGGU8/cm6eIrMkQ6LLrs8DV3/2WbM4tIXVHWlHaLqgoGWrgGfwjhzKzNEomI1B/1+SEdsS1wD7AVoWrnu+7+cm1DEpH2UDWRiIiomkhERDppNVHv3r29f//+tQ5DRKRTeemllxa5e+qNiZ0yGfTv35+mpqa2ZxQRkXXMbGahaaomEhGRbJOBhW6Jx1joYniymZ2VMs+hURe446PXBVnGJCIirWVdTbQGONfdx5lZL0KnWo9Ed74mPeXuX8w4FhERKSDTkoG7v+3u46LhFcAUyuv9UUREqqBqbQZm1h/YE3g+ZfIB0ZOJHjSz3Qosf5qF5/w2LVy4MMNIRUQaT1WSgZltQuhq+Afuvjxv8jhgJ3ffHfgjBbozcPfr3H2wuw/u06fcLttFRKSYzJNB1MXtKOB2d78nf7q7L3f3d6PhB4CeUVe+IiJSJVlfTWTAn4ApUVfLafNsG81H1J95N0J3uCLV99RTMHlyraMQqbqsryY6EDgZmGRm46NxPyV6YpO7jyA8oOK7ZraG8OSmE10dJkmtHHJIeNefoDSYTJOBu4+ljUcouvtVhGepiohIjegOZBERUTIQERElAxERQclARERQMhAREZQMREQEJQMREUHJoLpWr4aXX06f9s47Yfr//A8sWZJ9LNOnw6pVuc8jR8K9qd1CtVYsvnnzYNw4mDMH4g4F990XfvITmDYN7r4bPvyw5TKPPw4zZ0Jzc+5mrzfeqM5+EJHA3Tvda++99/ZMPf20++DB7pMnu48b5z5vXhi/bJn77ru7X321+9ix7u++637YYe733humz57tfv757i+80Hqdy5a5H3+8O7iPHOm+ZIn7xRe79+vnft99YXzPnuEd3GfNCu+9ernvuaf7D3/ofsMN4X3OHPc1a9ynTw/TR4xwnzLFvanJvbnZ/YIL3O+6y/36690vusj9/ffD9q++2v3FF91PPjms+7DDwjT33Hbvv9999Wr3hx4K45980v2QQ8J3evNN99//Psx3333uo0a5P/GE+5lnuvfokVtH8nXFFa3H7bef+xe/GGJatKjltAED3I8+Ogxvv33Yp/lmzHCfNi23z0eOTN/fzzzjfuON4bcsVRyHSBcENHmB46p5J7ztfvDgwd6uZyDPmAEvvABjxsD11+fGDxkC//xn5QIE2G8/aGqCtWsru17J+dOf4FvfCsO33Ra6khg6NPQt1KsXrFiRm/eDD6Bnz7bXadEN853w/0KkLWb2krsPTp3WUMng8svh3HMrH5DUv9NPhxEj2p5PyUC6sGLJoLHaDL71LRg7Fi68MJy133QTHHZYbvp22+WGkyWHfBtskP75O9+BTTapXLydRd++tY6gbaW2h4g0qMZKBpttBgceCL/4Bey9N5xyCjz6aK7Geu5ceP11eP55+Pa3YenS0KgbT1+1KlQ9rFrVshY8/nzttWF6vMy8efDII7B8OUydGmI4+WS4+OIw/H//B489BpdeCg8/HBpY+0VPBT3vPFi5MjS8vvdeaFy991549tnQ2OoOr7wCr72W+34//3mYPn58bpx7qBp74olcvEuWwNZbw+jR4fMVV8BnPxvmP+KI8D5qFHzpS2H/PPUU3H8/vP8+vPoq/O53sO228JnPhLPtOXPgl79suXzsiCPgllvC8AsvhP0xblyI8/XXc/PttFNI0gCHHx6qehYvDvt26lR48smwT+J9fO65cMAB4bd87jl4+univ/38+XDNNW3+iYg0qsaqJqq1GTPCQa9Hkc5im5tDVYUV7ey1bf/4R0gU5VaLrV0bktnGG5e3nHtYbsMNw+f58+H44+HGG2HgwMLLzZgR4hw6FLp1Cwlv6FBYf/3ytg8hefz2t3DCCfCFL4Srlt55p+U8ixbBVlsVXoeqiaQLU5uBNKbf/Q5+/OOW47beOiSqQpQMpAtTm4E0pnPOaV19tGABvPhibeIRqWNKBtJ19egBn/pUaC9JUjIQaUXJQLq+Qw4Jjd2x9rRHiHRxSgbSGH7/+9xw/qXBIqJkIA1ijz1yVxEl+2QSEUDJQBqFWe5ejxdeqG0sInVIyUAaR3x3eLG7y0UalJKBNI711qt1BCJ1S8lAGkfyru6VK2sXh0gdUjKQxrR6da0jEKkrSgbSmD7xiVpHIFJXlAykMc2dW+sIROqKkoE0lsMPr3UEInVJyUAay3HH1ToCkbqkZCCNRV1Ti6RSMpDG0txc6whE6pKSgTSWtWvTh0UaXKbJwMx2MLMxZjbFzCab2Vkp85iZXWlm081sopntlWVM0uCSJQN1WCeyTtYlgzXAue7+cWB/4HtmNihvnqHArtHrNODajGOSRpZMBrrxTGSdTJOBu7/t7uOi4RXAFKBf3mzHALd68BywuZltl2Vc0sCSVUMqGYisU7U2AzPrD+wJPJ83qR8wK/F5Nq0TBmZ2mpk1mVnTwoULswpTuro1a3LDSgYi61QlGZjZJsAo4Afuvjx/csoira7/c/fr3H2wuw/u06dPFmFKI1i8ODesZCCyTubJwMx6EhLB7e5+T8oss4EdEp+3B9RXgGTj5JNzw0oGIutkfTWRAX8Cprj75QVmGw0Mi64q2h9Y5u5vZxmXNLA998wNKxmIrNMj4/UfCJwMTDKz8dG4nwI7Arj7COAB4EhgOrAS+EbGMYkESgYi62SaDNx9LOltAsl5HPhelnGIpFIyEFlHdyBL40peWSTS4JQMpHGpnyKRdZQMpHEpGYiso2Qgjev222sdgUjdUDKQxvXgg3DLLXDDDbWORKTmsr60VKS+nXJKeP/2t2sahkitqWQgIiJKBiIiomQgIiIoGYiICEoG0oi23rrWEYjUHSUDaTyTJtU6ApG6o2QgjWfDDWsdgUjdUTKQxtO9e60jEKk7SgbSeLrpz14kn/4rpPEoGYi0ov8KaTyqJhJpRclAGk/37rDxxrWOQqSuKBlIYzrwwFpHIFJXlAykMamqSKQFJQNpTGpEFmlB/xHSmMxqHYFIXVEyEBERJQMREVEykEblXusIROqKkoGIiCgZSINSyUCkBSUDERFRMpAGpZKBSAtKBtKYlAxEWlAyEBGRbJOBmd1oZgvM7JUC0w81s2VmNj56XZBlPCLrlFoymDgR/v73bGMRqQM9Sp3RzLYBLgb6uvtQMxsEHODufyqy2M3AVcCtReZ5yt2/WGocIplYsAC23rr1+N13D++qVpIurpySwc3AQ0Df6PPrwA+KLeDuTwJL2hWZSDVts02tIxCpqXKSQW93vwtoBnD3NcDaCsRwgJlNMLMHzWy3QjOZ2Wlm1mRmTQsXLqzAZqWh6UxfpIVyksF7ZrYV4ABmtj+wrIPbHwfs5O67A38E7i00o7tf5+6D3X1wnz59OrhZaXhXXll8+oUXVicOkTpRTjI4BxgNDDSzpwntAGd2ZOPuvtzd342GHwB6mlnvjqxTpCQf/Sh861uFp196afViEakDJTcgu/s4M/s08FHAgKnu/mFHNm5m2wLz3d3NbF9CclrckXWKlKzY085UjSQNppyriYbljdrLzHD3glcKmdmdwKFAbzObDfwS6Ang7iOALwPfNbM1wCrgRHf9F0qV6NGXIuuUnAyAfRLDGwCHEer8CyYDdz+p2Ard/SrCpaci1adkILJOOdVELdoHzGwz4M8Vj0ikWpQMRNbpyB3IK4FdKxWISNV1U28sIrFy2gzuI7qslJBEBgF3ZRGUSFWoZCCyTjltBpclhtcAM919doXjEakeXU0ksk45bQZPZBmISNWpZCCyTpvJwMxWkKseajEJcHfftOJRiVRDfjJY1tEb6kU6rzaTgbv3qkYgIlWXnwz22Sd9PpEGUE6bAQBmtjXhPgMA3P3fFY1IpFryryaaNq02cYjUgZKvrTOzo81sGvAm8ATwFvBgRnGJZE9tBiLrlHOh9a+B/YHX3X0A4Q7kpzOJSqQadDWRyDrlJIMP3X0x0M3Murn7GGCPjOISyZ5KBiLrlNNmsNTMNgGeBG43swWE+w1EOiclA5F1yikZHEPoguJs4J/ADOCoLIISqQolA5F1yikZnAbcHd11fEtG8YhUj/omElmnnP+GTYGHzOwpM/uemekJ4tK5jRlT6whE6kbJycDdf+XuuwHfA/oCT5jZo5lFJpK1p54qPE1XE0mDaU85eQEwj/B4yq0rG45IFe2/f60jEKkb5dx09l0zexx4DOgNnOrun8wqMJHM/eIXhaetXl29OETqQDkNyDsBP3D38WkTzWwLd3+nMmGJVEHPnrWOQKRulNOF9fltzPIYsFfHwhGpIl1NJLJOJf8brILrEsme7jMQWaeSyUCXX0jnopKByDr6b5DG1VbJQJeXSgNRNZE0rvySQY+8JrTm5urFIlJj5VxaOtDM1o+GDzWz75vZ5olZDqt4dCJZyk8G+Qf/c8/NDauUIF1cOSWDUcBaM9sF+BMwALgjnujuSyocm0i2tsnrUSX/gP+//1t4mkgXU04yaHb3NcBxwBXufjawXTZhiVTBxhvDqafmPhc74CsZSBdX1sNtzOwk4OvA/dE43bUjjUHJQLq4cpLBN4ADgN+4+5tmNgC4LZuwRKqk1MtLlQykiyvnDuRXge9D6HoC6OXuv80qMJGqUDIQAcq7muhxM9vUzLYEJgA3mdnl2YUmUgVW4hXRSgbSxZVTTbSZuy8H/hO4yd33Bg4vtoCZ3WhmC8zslQLTzcyuNLPpZjbRzNS3kVRXuSWD5maYMye7eERqpJxk0MPMtgOOJ9eA3JabgSFFpg8Fdo1epwHXlhGPSMeVmwwuvBC23x7+/e/sYhKpgXKSwYXAQ8AMd3/RzHYGphVbwN2fBIrdf3AMcKsHzwGbRwlHpDrKTQYPPRTeVTqQLqacBuS7gbsTn98AvtTB7fcDZiU+z47Gvd3B9YqUptSeS+NkECePtWuziUekRsppQN7ezP4WtQHMN7NRZrZ9B7ef1nqX2lJnZqeZWZOZNS1cuLCDmxWJlFsyiJOH+i2SLqacaqKbgNFAX8LZ+33RuI6YDeyQ+Lw9MDdtRne/zt0Hu/vgPn36dHCzIpFyk0E8v5KBdDHlJIM+7n6Tu6+JXjcDHT0qjwaGRVcV7Q8sc3dVEUn1lPtMAyUD6aLKeQbyIjP7L+DO6PNJwOJiC5jZncChQG8zmw38kqgLC3cfATwAHAlMB1YS7nIWqZ72lgzUZiBdTDnJ4JvAVcAfCPX6z9DGwdvdT2pjugPfKyMGkcoqJxn84x/w2GPhs0oG0sWUXEZ293+7+9Hu3sfdt3b3Ywk3oIl0XuUkgyuuaPlZpAvp6JPOzqlIFCK1MmwYbL55+rRtt80N5x/8VTKQLqajyUCPupTObeed4Z130qfNm5cbzk8GM2bAJZdkF5dIlZXTZpBGZWVpDPnJ4PvfD+9f+1ronkKkk2uzZGBmK8xsecprBeGeA5Guzz29h9OudFXRFlu0bBeRhtJmMnD3Xu6+acqrl7t3tGQh0jkUajDuSg3JS5fC2WfXOgqpkY62GYg0hq500BdJoWQgUopGKBlIQ1MyEClFqcnggw9g6FCYMCH7mEQqSHX+IhB6Iy3WGFxqMnj5ZfjnP2HJEnj++crFJ5IxlQxEoO07kUeNgqeeaj2+q1QTdZXvIe2mZCACbSeDM8+E1atbj+/oQXTNGjjhBJg4sWPr6Sglg4anaiIRKL8r61hHD6KTJ8Ndd8GUKbVPCNLQVDIQgfYng9NOC89FXrOmfcun3chWCyoZNDwlAxFofzJ44gkYMgSGD69oOFWnZNDwlAxEoONn6K+/3rHl3eHmm2G77bLtEfWqq+COO9K3Lw1NbQYi0P6SQSz/YFrqwTWZhE49NVQ3rV3b8XgKOfPM8P7Vr7Ycr2TQ8FQyEAE48cSWnz/+8drEAbU5MNcqGaxeDXvvDc88U5vtyzpKBiIAf/wjTJqU+/zrX5e3vDu88QZMm9a+7Sd7RU0emJubq/MgnVolg8mTYdy4XIlFakbJQASgRw/o3Tv3uXv38tcxcCCcfHJ5yyQTQDx8111w3HFheNNNYZddyo9FpExKBiKxZP19uXX2aWfWTU2hS+hCT1LL32Zs2DC4994w/N578Oab5cXSHqWWDNasqU48xbjD2LFq56gwJQORWPLA/IlPlLds/oHJDPbZJzwsZsst216+uTl0clcrpR5Yf/rT8KjQWbOyjaeYG2+Egw+Gu++uXQxdkJKBSCxOBptsAgMGVH7dEyfCihXpN6i99lplt1euUpPBo4+G9wULsoulLfFlvLUuoXQxSgYisTgZtOeeg1IOpnfeGdoAklcuddY7kOslbqkYJQORWEcOcGkH00LtDqNGtX87bXn7bbj++txn99CldltXJJWaDOqhnj6OQQmpopQMRGLxwbtSB7z2XJGU1J44jjkm9Jc0e3b4fM894WE7V17ZsVjy1fJArGSQCSUDkVh8cGnPdf3llAzStlnqOpPGjIFjj20Z78KF4f3DD8N7nBTaql+fObP4dOny1B2FSCztpq+OyE8GaestlgzaSkpHHw3vvhsuP+3Vq+X64mVL/S6lXj1VD9VEkgklA5FYpdsMOlpNVE4J5f334aWXCie0SlepVHp9SjI1p2QgEutIySB/mfffD2ftaeuPjRgR5it1ncWmn302XHtt6cu2lw7aXZaSgUisI8ng/vtbfp4wofU8+ev97neLr7OcksH48e1ftj3KKRksWADHHw9/+Qtsu23H16eElAk1IIvEKt1mkO+SS8qbv9TLQZP9GmWtPfvm//4vPATo6qsrG4uuJqqozJOBmQ0xs6lmNt3Mzk+ZfoqZLTSz8dHr21nHJJKq0peWFrPeem3Pc911rcc1NaU/OyH/wJj1d6jlmbxKBpnINBmYWXfgamAoMAg4ycwGpcw60t33iF43ZBmTSEFZlwyS4ks/iznnnJaf//nP0N/RNdfAf/83rFwZxjc3Vz8ZtEexBDJuHPz975Vbn5Qt65LBvsB0d3/D3T8A/gIck/E2RdqnmsmgPWbMCO+33966sTj/wFjupaWlitfX3AxLl1Z23b//fWXXJ2XJOhn0A5LdG86OxuX7kplNNLO/mtkOaSsys9PMrMnMmhbGN9aIVFJ+MrjggtrFUo5SSgblnkXfcUfuBrY0550HW2zR8oqppUvDdm66qe31u7cuHZXa6F2vybqTyzoZpP0F5v+S9wH93f2TwKPALWkrcvfr3H2wuw/u06dPhcMUofUB81e/qs+EkHWbwauvwte+Bkcc0XL82rXwyith+LHHwvt774WD+vrrh/0Fodvutlx/fWg3ie+Qzo957Fi47LLi6yiW4JYuDU+ra26GKVPCvK++mj6ve3YJ5r33wrOtK12KykDWyWA2kDzT3x6Ym5zB3Re7e3yx9fXA3hnHJJIurZqolLr9Wps4sXA1UXucdVZ4z79cdd681vN26xYe3vPBB6Ulgdjtt4f3uOoLWu73gw+GH/6w9PXlO/vskMjvuw9Gjgzj4vekRYvCd7jmmvZvq5hrr4UbboCLL85m/RWUdTJ4EdjVzAaY2XrAicDo5Axmtl3i49HAlIxjEknX3mTwwAPZxFNI/oH/8MMrWzLITyRLl8Jzz4WSQb5u3dpXJRWvK3mXdiXPzuPqqw8+KN6x3Zw54T3tyq1KSF7+Gxs+vHWirQOZJgN3XwOcATxEOMjf5e6TzexCMzs6mu37ZjbZzCYA3wdOyTImkYLSDhalJINqPaO42MGyvcng3XdzB8RY/kH/C1+AAw6AVavSt1vqtsaObb2NZDJobg6vESNKW18xyZiKJYP4Et9id4JXKg4I3/tXv4L99stmex2Q+X0G7v6Au3/E3Qe6+2+icRe4++ho+Cfuvpu77+7un3H3Gj/ySRpWfLBIHqD6pV3vkPDVr1bnEscxY8IdvIXkxzB9esuz3Q8+gGXLWi/3qU/B9tu3HJefAF98MbynPaHNvfQqqTFjcsPxMsnO/NxDVU5bd2YnD7BmxW/mSyarZL39ddfBXntlnwyScUDue9dh9aPuQBaJmYWztqam3LhzzoG//S29KmjgwFD3XUpX1R312c/C00+H4bQz8SeeaPn5+OPh9NNzz1W+5hrYfPPWy02a1Hpc8vsnpVUTtbfxNV5Xct81N7dOWGPHtj5Q55/pn9/qXtZ0yTaN00+Hl1/OrSvrZBDLupuQDlAyEEm64ALYY4/c5+7dwzMDhg5tPW9HHpPZEWkH37Sz9kLztiVOIKVswx3eeKPluAkT0quUkuKDYrIU9tJLMG1ay/kOPhh+8IP0daR9tzlzQvcXpX7vOCnF323NmmzP2tMSaltWrw6X+mZ8Sa2SgUhHVaNkkPT886XPW+gAcvnlpS0/aVJuHWnJoLkZDjmk9fi2GmTjg2L+JZdpcSVLL0cdBf/7v2E47bsdfTR85zvh8Z+x/N5jk+KkFCf0j+ig2q4AABGcSURBVH+8tK5C2qs9NwP+6EfhUt9//SubmCJKBiLtFSeBeu4WodBB59xzS1v+k59sedZc6vqLnQGPHJl78tqPftR2DMn9m+wdNq3KZfHi8J6s9sm/5DW+HyItzunT246nHPn7pz3VRP/+d3hfvrzj8RShZCBSquQNUpA7SFW7ZFCOStZRl5MMCnGHE0+EFStKXybez6/lXVtS7IFCcaxpiXr48Nxwe/bP5Mlhvcl7JObNCw3Zbe2PtrY3cSI8+2zLcfE6M/47q+O/YpE6k39lUWcuGUC44aoc7UkG8+a1PKNtb7337NmhCicp7cAaJ4NS6+aT86XdVJfm5pvD+6hRuXEnnRQastOeY1Foe2l23z1c4ZWUduVVBvRwG5FybLJJrg66M5QM7ryz8LSddipvXWkNy4UO7nGd/XbbtRz/0EPlbRPCfs4vlRXadnywLbUROJ7fDB55JDd+yhTYeefQzUZaPElTpsDjj4fhQg358b5rT0kkv10jI3X8VyxSh159NdfI2RlKBlOK3NAfd4FdqrQDbKGD22WXpc9/5JHlbRMKn7Enk0HcEB1f2VToiqh8550X3ufPbzl+0KDQEJ2WcOJxEyeGq88efrjw+uN540bvcpPBkiW5fqBUTSRSR3bYIVzzD52jZFBJaQf3YtU+P/tZZbb7+uvp45MH1i22aFltU2oyyL8/I+nmm8NvW+hqpNtvD53hJZNqWwf7ci8tPfroXGO4koFIndkh6nvx9NPDez2XDCrpS19qPa5YMkh2P5GF/G3HZ9DQsvqno955p+Xn/HUmq4b2269443i5JYO4l1hQm4FI3dlyy5YHokYpGaQplgzyr4qp9HaKbTs+6C5Z0vFtt5VQ8ktM3/gG/PWvxeMqVfKmPJUMROpco5QM0sTXwGct7cD/y1+2/Jw8Q49LBqee2vFtJ3/fKVPg0ktbTs+vkorvocj3zDOFk8GECS3vcYjbepIJQMlApM41csng05+uznZKOaO+/vrccHu6fRg2rO157r679bj8kkFaj6kABx5YOK499oBdd819nhs99iVZMshYA/8Vi1RIsZJBfo+g0j7l3p9QKHn8/e/lbzv5+6Yl/vySwfvvFy4xlfNoz6lTW85/2GGwYEFpy7eDkoFIRxUrGeh5vZVRbl17oUbcY4/tWBylJINXXw33cKT1hJr8HmeeWXg7b70FH/tY6+dQjxtXcqjlUjIQ6ahGbjOollom1Ysuyg2n/dbl9PKabFS/6qrC28y/7yGWYRfYSgYiWVLJoDJq+RyAa6/NDZfzNLx77mk9Lr+aaurU8m7mUzIQqWNZdnkswXHH1Xb7ZqHqJv/ZDVD4wTjDhrXudjr/2ccf+xicdVbrZfN7Wo0VK010kJKBSEett1762V3yjFI6JuPum0syYEDLK5Zixe52zq/ueeut1vOk3QVd6P6I9vTtVCIlA5FK6NGj9WMlS30k5L77ZhOTVMfo0YWnlfI4zXIvg017VGkFKBmIVMo554T3r341PGz9K18pbbkrr8wuJqmt1avbnmfq1PLWedNN7YulDUoGIpUycGB4/9znwvN8e/cOXV4npT0ics894YEH4JZbOrb9Rr75rV6VkgzqhP56RCrlpJNC/e/Xv54b17dvy3nS6od79oShQ2Gjjdq/7Vdeqe0VN+3RowG6RivluQp1ksTrIwqRrsAsnPknLz+84462n/Mbzx+3L2y2WXnb7dYNdtutvGUK+fa3c8PF2jI23bTj2zrmmI6vo97l93iaptwknlHSVzIQyVLfvuHZuEl33w1jxrSeN34qWLE7U9NU8uCwzz654f/6r8LzJZNGe+lmvfZpT79LJVAyEKm2L38ZDj0UXngh99Q0gIMOCtel5/fGOWtWaevt37/0GI46qu150h75GKvEgTx+VKSUR8lApIvZZx84++yW4z7zmdZ16cUOygAnnBDeN9ggffruu7f8/PDDcOut6fOawRFHhOFil8VW4oC04YYdX0cjUjIQaTAbbwxPPgl9+uTG/ehH4XGMsSVL4M9/DsOFztbzSwGf/GTLefOTyEc+Et4XLSocW6EHv5ej3LaRSpk+PVzt1VkpGYg0kGnTYOZMOPjgluMvuaTl1UpbbBGuRoLWyWC99UKDdlqf+MkuNL75zfC4xlh8xl7soJPWhUL+sw1+85vCy0PhkkzWBg4M94EMGVKb7XeUkoFIJ1dO/zq77AJbbZX7PHYsTJ5cfJk4GVx1FTz/fLj79YknYOedW8+74YYwfHgY7tkzdzWSWe4gXeigs+WWIT53+M//zI1//PHQdXMsvwosFieibt3Cc4ufey49vvY46ig477zC07fZJjf8sY+1bxu1pmQg0omtXQujRrV/+QMPhEGDis9z0UWhveGUU1peFnryyfDgg+EmuKSNNw7v3bvn2hV23jl3IE7rc2faNHj99dznuIoqluyYLe2AfswxcN11YbhbN/jsZ1uWSiAkvbQO4QAuuyx9fGy33cI6k048MTecrB5atar4utqSfDJZNXXWZGBmQ8xsqplNN7PzU6avb2Yjo+nPm1n/rGMSqbpu3Sp7KeWwYS3bEiA8uOXDD3MH+ZhZqBLJv6ntoIPC+5Ah4XLWpqZwII2rpuLpSfkllvx1DhjQ8nN+Q/W99+aqZy68MP27DRoE224btpXvP/4jfZlY376tG77vvBPmzAntHP365cavXBneb745XOGVZsSI8B63oyQ9/ngoiVSyO5GTToIzzig+T0bJAHfP7AV0B2YAOwPrAROAQXnz/DcwIho+ERjZ1nr33ntvF5EyTZ3qPny4e3Nzbtzq1enzLlsW3nPd7YVXmvxp4H7qqbnPzzwTxu2yS+HYrrnG/XOfc587N33d8evpp93Hj3f/y19y4yZNCt/pr391X7vW/f77w/jdd3efNq3wNo8+Osw3apT7qlXu55zjfthhufU+9FBu3sWLC++LuXNbTyv2uuWW3HZ+85uW06ZMCeu8+OLw+dOfbr38CScU/k5tAJq80PG60IRKvIADgIcSn38C/CRvnoeAA6LhHsAiwIqtV8lApEpmzQoHwksvdX/rrfR55sxxf+ON3OdksnF3X7nSfeut3e+7r/ztNzWFg/2qVeGgnbR4cS5pJS1ZErb37LPF1x0ng+eey4378MMw7vzzW867fHnhZBAvA+59+hROAmPH5vbNGWeEcY89lpv+/vutv5976/VMnFj8exVRLBlYmJ4NM/syMMTdvx19PhnYz93PSMzzSjTP7OjzjGieRXnrOg04DWDHHXfce+bMmZnFLSINYN48uO02OPfcllV4cTVM/lVYf/0r7L8//OMf8KlPwSc+kZv24YehDWLTTcPwDTeEqq7jjoP77guXsyYb1FevDo/A/MxnQvXV2rWF7yeZMiVcQNCvX7gooAMP+jGzl9x9cOq0jJPBV4DP5yWDfd39zMQ8k6N5kslgX3dfXGi9gwcP9qampsziFhHpioolg6wbkGcDOyQ+bw/MLTSPmfUANgMKPOZHRESykHUyeBHY1cwGmNl6hAbi/McCjQbiu2i+DPzLsyyuiIhIK5l2KO7ua8zsDEIjcXfgRnefbGYXEhoyRgN/Av5sZtMJJYITC69RRESykPnTJdz9AeCBvHEXJIZXAyU+H1BERLKgO5BFRETJQERElAxERAQlAxERIeObzrJiZguB9t6C3JvQ5UU9q/cY6z0+UIyVUO/xQf3HWG/x7eTufdImdMpk0BFm1lToDrx6Ue8x1nt8oBgrod7jg/qPsd7jS1I1kYiIKBmIiEhjJoPrah1ACeo9xnqPDxRjJdR7fFD/MdZ7fOs0XJuBiIi01oglAxERyaNkICIijZUMzGyImU01s+lmdn6NYtjBzMaY2RQzm2xmZ0Xjh5vZHDMbH72OTCzzkyjmqWb2+SrF+ZaZTYpiaYrGbWlmj5jZtOh9i2i8mdmVUYwTzWyvjGP7aGI/jTez5Wb2g1rvQzO70cwWRE/vi8eVvc/M7OvR/NPM7Otp26pwjJea2WtRHH8zs82j8f3NbFVif45ILLN39PcxPfoelra9CsVX9u+a5f96gRhHJuJ7y8zGR+Orvg/brdDzMLvai9CF9gxgZ2A9YAIwqAZxbAfsFQ33Al4HBgHDgfNS5h8Uxbo+MCD6Dt2rEOdbQO+8cb8Dzo+GzwcuiYaPBB4EDNgfeL7Kv+s8YKda70PgEGAv4JX27jNgS+CN6H2LaHiLjGM8AugRDV+SiLF/cr689bxAeMa5Rd9jaIbxlfW7Zv2/nhZj3vTfAxfUah+299VIJYN9genu/oa7fwD8BTim2kG4+9vuPi4aXgFMAfoVWeQY4C/u/r67vwlMJ3yXWjgGuCUavgU4NjH+Vg+eAzY3s+2qFNNhwAx3L3ZHelX2obs/Seun9JW7zz4PPOLuS9z9HeARYEiWMbr7w+6+Jvr4HOGJhAVFcW7q7s96OKrdmvheFY+viEK/a6b/68VijM7ujwfuLLaOLPdhezVSMugHzEp8nk3xg3DmzKw/sCfwfDTqjKiofmNcnUDt4nbgYTN7ycxOi8Zt4+5vQ0hqwNY1jhHCw5CS/3j1tA+h/H1W67/TbxLOUmMDzOxlM3vCzA6OxvWL4opVI8Zyftda7sODgfnuPi0xrl72YVGNlAzS6uNqdl2tmW0CjAJ+4O7LgWuBgcAewNuEoibULu4D3X0vYCjwPTM7pMi8NYnRwqNUjwbujkbV2z4splBMNYvVzH4GrAFuj0a9Dezo7nsC5wB3mNmmNYix3N+1lr/3SbQ8OamXfdimRkoGs4EdEp+3B+bWIhAz60lIBLe7+z0A7j7f3de6ezNwPblqjJrE7e5zo/cFwN+ieObH1T/R+4JaxkhIVOPcfX4Ua13tw0i5+6wmsUYN1V8EvhZVWxBVvyyOhl8i1MN/JIoxWZWUaYzt+F1rtQ97AP8JjIzH1cs+LEUjJYMXgV3NbEB0RnkiMLraQUR1in8Cprj75YnxyTr244D4SoXRwIlmtr6ZDQB2JTQ8ZRnjxmbWKx4mNDC+EsUSX93ydeDviRiHRVfI7A8si6tGMtbiLKye9mFCufvsIeAIM9siqg45IhqXGTMbAvwYONrdVybG9zGz7tHwzoT99kYU5woz2z/6ex6W+F5ZxFfu71qr//XDgdfcfV31T73sw5LUsvW62i/CFRyvE7Lzz2oUw0GE4uBEYHz0OhL4MzApGj8a2C6xzM+imKdShSsOCFdhTIhek+N9BWwFPAZMi963jMYbcHUU4yRgcBVi3AhYDGyWGFfTfUhITG8DHxLO/L7Vnn1GqLefHr2+UYUYpxPq2OO/xxHRvF+Kfv8JwDjgqMR6BhMOyjOAq4h6M8govrJ/1yz/19NijMbfDHwnb96q78P2vtQdhYiINFQ1kYiIFKBkICIiSgYiIqJkICIiKBmIiAhKBiJFmdnPLPQuOzHqdXI/Cz2kblTr2EQqSZeWihRgZgcAlwOHuvv7Ztab0AvmM4T7AhbVNECRClLJQKSw7YBF7v4+QHTw/zLQFxhjZmMAzOwIM3vWzMaZ2d1Rv1PxMyEuMbMXotcu0fivmNkrZjbBzJ6szVcTaUklA5ECooP6WMLdzo8CI939CTN7i6hkEJUW7iHc/fqemf0YWN/dL4zmu97df2Nmw4Dj3f2LZjYJGOLuc8xsc3dfWpMvKJKgkoFIAe7+LrA3cBqwEBhpZqfkzbY/4SErT1t4utXXCQ/aid2ZeD8gGn4auNnMTiU8iEWk5nrUOgCReubua4HHgcejM/r8x1Aa4WE0JxVaRf6wu3/HzPYDvgCMN7M9POrZUqRWVDIQKcDCs5Z3TYzaA5gJrCA8shTCk8EOTLQHbGRmH0ksc0Li/dlonoHu/ry7XwAsomV3yyI1oZKBSGGbAH+08ID4NYTePU8jdJ39oJm97e6fiaqO7jSz9aPlfk7oMRNgfTN7nnDiFZceLo2SjBF6Mp1QlW8jUoQakEUykmxornUsIm1RNZGIiKhkICIiKhmIiAhKBiIigpKBiIigZCAiIigZiIgI8P8McjSkUsZIJQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "steps = steps_loss[\"step\"]\n",
    "loss_value = steps_loss[\"loss_value\"]\n",
    "steps = list(map(int, steps))\n",
    "loss_value = list(map(float, loss_value))\n",
    "plt.plot(steps, loss_value, color=\"red\")\n",
    "plt.xlabel(\"Steps\")\n",
    "plt.ylabel(\"Loss_value\")\n",
    "plt.title(\"Change chart of model loss value\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面可以看出来大致分为三个阶段：\n",
    "\n",
    "阶段一：训练开始时，loss值在2.2上下浮动，训练收益感觉并不明显。\n",
    "\n",
    "阶段二：训练到某一时刻，loss值迅速减少，训练收益大幅增加。\n",
    "\n",
    "阶段三：loss值收敛到一定小的值后，开始振荡在一个小的区间上无法趋0，再继续增加训练并无明显收益，至此训练结束。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  验证模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "得到模型文件后，通过运行测试数据集得到的结果，验证模型的泛化能力。\n",
    "\n",
    "搭建测试网络来验证模型的过程主要为：\n",
    "\n",
    "1. 载入模型`.ckpt`文件中的参数`param_dict`；\n",
    "2. 将参数`param_dict`载入到神经网络LeNet中；\n",
    "3. 载入测试数据集；\n",
    "4. 调用函数`model.eval`传入参数测试数据集`ds_eval`，生成模型`checkpoint_lenet-{epoch}_1875.ckpt`的精度值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:54:25.073201Z",
     "start_time": "2021-02-03T08:54:24.124960Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "============== Starting Testing ==============\n",
      "============== Accuracy:{'Accuracy': 0.9697516025641025} ==============\n"
     ]
    }
   ],
   "source": [
    "from mindspore import load_checkpoint, load_param_into_net\n",
    "\n",
    "# testing relate modules \n",
    "def test_net(network, model, mnist_path):\n",
    "    \"\"\"Define the evaluation method.\"\"\"\n",
    "    print(\"============== Starting Testing ==============\")\n",
    "    # load the saved model for evaluation\n",
    "    param_dict = load_checkpoint(\"./models/ckpt/mindspore_quick_start/checkpoint_lenet-1_1875.ckpt\")\n",
    "    # load parameter to the network\n",
    "    load_param_into_net(network, param_dict)\n",
    "    # load testing dataset\n",
    "    ds_eval = create_dataset(os.path.join(mnist_path, \"test\"))\n",
    "    acc = model.eval(ds_eval, dataset_sink_mode=False)\n",
    "    print(\"============== Accuracy:{} ==============\".format(acc))\n",
    "\n",
    "test_net(network, model, mnist_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中：\n",
    "\n",
    "- `load_checkpoint`：通过该接口加载CheckPoint模型参数文件，返回一个参数字典。\n",
    "\n",
    "- `checkpoint_lenet-1_1875.ckpt`：之前保存的CheckPoint模型文件名称。\n",
    "\n",
    "- `load_param_into_net`：通过该接口把参数加载到网络中。\n",
    "\n",
    "经过1875步训练后生成的模型精度超过95%，模型优良。\n",
    "\n",
    "我们可以看一下模型随着训练步数变化，精度随之变化的情况。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`eval_show`将绘制每25个`step`与模型精度值的折线图，其中`steps_eval`存储着模型的step数和对应模型精度值信息。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:54:25.213489Z",
     "start_time": "2021-02-03T08:54:25.078949Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de5gcZZn38e+PhIQAgUASDCEkEyGwRBDEyOFlPaxyCiKIi3IQOYiy7oLKoruLi6/L4uouKKKu7CIgclIQEDXvCgaWo6AoiQYhnGYIgQQChEMIEExIcr9/PNWkZ9LdUzPTNd09/ftcV11dXae+u3qm7nqeeuopRQRmZta+Nmh0AGZm1lhOBGZmbc6JwMyszTkRmJm1OScCM7M250RgZtbmnAisTyR1SApJw3Mse7ykuwYjrnYh6eOSbhrA+jdKOq6eMfXyebn/XqxxnAiGMEkLJa2SNK7H9HnZP2dHYyKz/oqIH0XE/nmWlXSmpCt7rD8zIi4rJrrBkf3tbt/oOIYSJ4Kh73HgqNIbSbsAoxoXTnNoxTPUVoy5ntr9+xfJiWDouwI4tuz9ccDl5QtI2lzS5ZKWSnpC0pclbZDNGybpm5Kel7QA+GCFdX8gaYmkpyT9m6RheQKTdK2kZyS9LOlOSW8rmzdK0rlZPC9LukvSqGzeX0r6jaRlkhZJOj6bfrukT5Vto1vVVHYmebKkTqAzm/adbBvLJc2V9O6y5YdJ+mdJj0l6JZu/raTzJZ3b47v8P0mnVviOF0j6Zo9pv5B0WjZ+etn2H5R0WI/475Z0nqQXgTMrfKeK8Us6EPhn4AhJr0q6r+c+krRB9ls/Iem57G9g82xeqUrnOElPZr//GTV+y6q/V+bjlbYjaQ9Jv81+yyWSvidpRLXfTNKd2az7su91RLWYrA8iwsMQHYCFwL7AI8BOwDBgETAFCKAjW+5y4BfAaKADeBQ4MZv3GeBhYFtgS+C2bN3h2fyfA98HNgG2An4P/E0273jgrhrxfTL7zJHAt4F5ZfPOB24Htsni/j/ZcpOBV0ilnA2BscBu2Tq3A58q20a3z8/ivjn7HqOyacdk2xgOfAF4Btgom/cPwP3AjoCAXbNl9wCeBjbIlhsHrADeUuE7vifb58rebwG8DkzM3n8UmEg6KTsCeA3Yuiz+1cBns/hGVfhOteI/E7iyRzxv7qNs/3cBbwU2Ba4HrsjmdWT766Lsc3cFVgI7Vfktq/1eNbcDvBPYK4u/A3gIOLWX3yyA7Rv9/zWUhoYH4KHAH3ddIvgy8O/Agdk/1fDsn6kj+6ddCUwvW+9vgNuz8VuBz5TN2z9bdzjwlmzdUWXzjwJuy8a7HbR6iXVMtt3Ns4Pi68CuFZb7EvCzKtt48yBX6fOz7b+/lzheKn0uKYEeWmW5h4D9svFTgBuqLCfgSeA92ftPA7fW+Px5pc/M4n+yx/ya+7RH/GdSOxHcAvxd2bwdgTfKDsoBTCqb/3vgyAqfWev3yr2dbN6p5b9vpd8MJ4K6D64aag9XAEeTDiKX95g3DhgBPFE27QnSmR2ks9VFPeaVTCGdlS/JivbLSKWDrXoLKKt2+Y+sWmQ5KWmV4hkHbAQ8VmHVbatMz6v8uyDpC5IeyqozlpESUenieq3Puox0Nk72ekWlhSIdua5m3XWao4EflX3+sUoX70v7b+eyz18v3p56ib83E1n/dy8l+JJnysZXkEoOPdX6vWpuR9IOkv4nqyJcDny9Qvw194ENnBNBG4iIJ0gXjQ8iFf/LPU86C5xSNm0y8FQ2voR0QCyfV7KIVCIYFxFjsmGziHgbvTsaOJRUYtmcdOYI6Qz6eeDPwHYV1ltUZTqkapWNy95PqLDMm93tZvXp/wR8DNgiIsYAL2cx9PZZVwKHStqVVO328yrLAVwFHC5pCrAn8NPs86eQqkxOAcZmn/9A2ed3i7enHPH31rXw06z/u68Gnu1lvZ5q/V69+W9S1eO0iNiMdF1DPZZxF8kFcyJoHyeSitivlU+MiDXANcDXJI3ODk6nkQ50ZPM+J2mSpC2A08vWXQLcBJwrabPs4uN2kt6bI57RpCTyAung/fWy7a4FLgG+JWliVnrYW9JI0tn0vpI+Jmm4pLGSdstWnQd8RNLGSs0LT8wRw2pgKTBc0leAzcrmXwx8VdI0JW+XNDaLcTFwL6kk8NOIeL3ah0TEH7PPuBiYHRHLslmbkA5ySwEknUAqEeTVW/zPAh3KLvxXcBXw95KmStqU9Bv8JCJW9yGG3n6vPN9hOfCqpL8A/jbHOs+SrmtYnTgRtImIeCwi5lSZ/VnS2fQC4C7gx6R/bEhnrLOB+4A/sH6J4lhS1dKDpPrp64Ctc4R0Oakq4qls3Xt6zP8i6ULtvcCLwNmki7NPkko2X8imzyNdgAQ4D1hFOlBcRlkVTBWzgRtJF8efIJ3VlldDfIuUCG8iHax+QPemt5cBu1ClWqiHq0ilnx+XJkTEg8C5wG+zmHcB7s6xrbzxX5u9viDpDxXWvySL/U5SifHPpL+F/qj4e+Vc72hSA4CLgJ/kWOdM4LKsOu1j/YrWuim1ZDCzPpL0HlLJqSM7KzZrSS4RmPWDpA2BzwMXOwlYq3MiMOsjSTsBy0hVYN9ucDhmA+aqITOzNucSgZlZmyusEydJlwAHA89FxHpN4iQJ+A6pBcgK4PiIqNSyoZtx48ZFR0dHnaM1Mxva5s6d+3xEjK80r8je/C4Fvsf6d7KWzASmZcOepBtL9uxtox0dHcyZU60VpJmZVSLpiWrzCqsaiog7Se2JqzkUuDySe4AxkvK0Pzczszpq5DWCbeh+88ti1vVv042kkyTNkTRn6dKlgxKcmVm7aGQi6NmfCFTpUyQiLoyIGRExY/z4ilVcZmbWT41MBIvp3pnZJFInWGZmNogamQhmAcdmnXntBbycdWJmZmaDqMjmo1cB7wPGSVoM/Aup73oi4gLgBlLT0S5S89ETiorFzMyqKywRRMRRvcwP4OSiPt/MzPIp8j4CMzPLa80aWL4cli3rPrz88rrxgw+GGTPq/tFOBGbWPlasgAcegPvug3nz4OmnYcQIGDkyvZYPlablnb5yZeUDebUD/LJl8Morvcc/YYITgZlZLhHpID9vXjrol4bOTlib9Ro+ejRMmQKrV6cD96pV6w9r1tQvpg02gM03hzFj1g3bb7/+tDFjKk8bPRqGDatfPGWcCMysta1aBQ8+2P2Af9998MIL65aZOhV23RWOPDK97rordHSkg3Mta9asnxyqJY3S9JUrU+mg54F8001BlW6fajwnAjMbmD//GZ58EhYuhEWL0tn4RhvBqFFpKI1Xm7bhhvkPkEuXdj/Yz5sHDz2UzuohbXPnneGww2C33dIBf5dd0hl2fwwbti7mIcyJwMxqe+01eOKJNCxcuO61NP7MMwPb/gYbVE4Y5YljzRqYPx+WlN1qNHFiOtgffPC6s/xp0wqrPhnKnAjM2t3y5esf5MsP9s8/3335DTdMdetTpsAHP5heOzrS6+TJ6UD8+uuppFD+Wmlantdly1K9/n77rTvg77orjBs36LtqqHIiMBuK3ngjHcCXLoXnnqs8LF6cDvQvvdR93Y02Wndw3333dQf5jo40TJjQe926tRQnArNWEJHOjKsd1HsOL1bpAX74cNhqqzRsvTXsvXf3M/qOjjSvSS9qWjGcCMya0Zo1cMstcOmlcMcd6eBeuiDa09ix6eA9fny6UFo60FcaxozxQd7W40Rg1kwefTQd/C+/HJ56CrbcMtXDT5pU+cA+dmyqszcbACcCs0ZbvhyuuQZ++EP4zW9S/fvMmfDtb8OHPpTapJsVyInArBHWroXbbksH/+uvTy1kdtoJzjkHjjkm1d+bDRInArPB9Nhj66p+nnwy1dkff3wa3vUu199bQzgRmBXtlVfguuvS2f+vf50O9vvvn87+Dz00Ndc0ayAnArMirF0Ld96ZDv7XXZd6vdxhB/j61+ETn0gXf82ahBOBWT09/jhcdlkaFi6EzTaDj38cTjgB9trLVT/WlJwIzOph4UI48US49dZ0sP/AB+BrX4MPfxg23rjR0ZnV5ERgNlBLl6Y6/6VL4atfhWOPTX3umLUIJwKzgXj11XTD16JF8L//C/vs0+iIzPrMicCsv954Aw4/HObOhZ/9zEnAWpYTgVl/rF2brgnMng0XXQSHHNLoiMz6zX3JmvXH6afDFVekawKf+lSjozEbECcCs7467zz4xjfg5JPhjDMaHY3ZgDkRmPXFj38Mp52Wrg185zu+L8CGBCcCs7xuvjn1CfTe96ZqIT8b14YIJwKzPObOhY98JPUQ+otfuH8gG1KcCMx609mZng8wdizceCNsvnmjIzKrKycCs1qeeQYOOCA1F509GyZObHREZnXn+wjMqlm+PJUEnn029SG0446NjsisEE4EZpWsXAmHHQYPPACzZsGeezY6IrPCOBGY9bR2beo47tZbU3fSM2c2OiKzQvkagVm5CPj7v08Pkz/nnJQQzIY4JwKzcmefDd/9bkoGX/xio6MxGxROBGYll14KX/oSHH00fPObvmvY2oYTgRnAL3+ZOo/bb7/0nOEN/K9h7aPQv3ZJB0p6RFKXpNMrzJ8s6TZJf5T0J0kHFRmPWUX33AMf/Sjsthv89KcwYkSjIzIbVIUlAknDgPOBmcB04ChJ03ss9mXgmoh4B3Ak8F9FxWNW0cMPpyeMTZwIN9wAo0c3OiKzQVdkiWAPoCsiFkTEKuBq4NAeywSwWTa+OfB0gfGYdffUU+mu4eHD013DW23V6IjMGqLI+wi2ARaVvV8M9Lwr50zgJkmfBTYB9q20IUknAScBTPZDwa0eli2DAw+EF1+EO+6A7bZrdERmDVNkiaBSk4vo8f4o4NKImAQcBFwhab2YIuLCiJgRETPGjx9fQKjWVl5/PT1a8pFH0rOGd9+90RGZNVSRiWAxsG3Z+0msX/VzInANQET8FtgIGFdgTGZw3HFw113pmQL7ViyEmrWVIhPBvcA0SVMljSBdDJ7VY5kngQ8ASNqJlAiWFhiTtbsXXoBrr4V//Ec44ohGR2PWFApLBBGxGjgFmA08RGodNF/SWZIOyRb7AvBpSfcBVwHHR0TP6iOz+unsTK/77NPYOMyaSKGdzkXEDcANPaZ9pWz8QcD/kTZ4urrS67RpjY3DrIn49klrL52d6a7hqVMbHYlZ03AisPbS1QWTJ8PIkY2OxKxpOBFYe+nshO23b3QUZk3FicDaS1eXrw+Y9eBEYO3jhRfgpZdcIjDrwYnA2odbDJlV5ERg7aN0D4FLBGbdOBFY++jqSk8de+tbGx2JWVNxIrD20dnppqNmFTgRWPtwiyGzipwIrH34HgKzipwIrD28+GJqOuoSgdl6nAisPbjFkFlVTgTWHnwPgVlVTgTWHjo7U9NR9zpqth4nAmsPXV2w7baw0UaNjsSs6TgRWHtw01GzqpwIrD246ahZVb0mAklzJJ0saYvBCMis7l58MQ0uEZhVlKdEcCQwEbhX0tWSDpCkguMyq59SiyGXCMwq6jURRERXRJwB7AD8GLgEeFLSv0rasugAzQbMTUfNasp1jUDS24FzgW8APwUOB5YDtxYXmlmdlJqOutdRs4qG97aApLnAMuAHwOkRsTKb9TtJ+xQZnFlduOmoWU29JgLgoxGxoNKMiPhIneMxqz+3GDKrKU/V0KckjSm9kbSFpH8rMCaz+vI9BGY15UkEMyNiWelNRLwEHFRcSGZ19NJL6aH1LhGYVZUnEQyT9OYjnSSNAvyIJ2sNbjFk1qs81wiuBG6R9EMggE8ClxUalVm9uPtps171mggi4hxJ9wMfAAR8NSJmFx6ZWT2UHli/3XaNjsSsaeUpERARNwI3FhyLWf11dsKkSW46alZDnr6G9pJ0r6RXJa2StEbS8sEIzmzA3GLIrFd5LhZ/DzgK6ARGAZ8C/rPIoMzqxvcQmPUqb9VQl6RhEbEG+KGk3xQcl9nAlZqOukRgVlOeRLBC0ghgnqRzgCXAJsWGZVYH7nXULJc8VUOfyJY7BXgN2Bb46yKDMqsL30NglkvNEoGkYcDXIuIY4M/Avw5KVGb1ULqHwL2OmtVUs0SQXRMYn1UNmbWWUq+jo0Y1OhKzppbnGsFC4G5Js0hVQwBExLd6W1HSgcB3gGHAxRHxHxWW+RhwJumu5fsi4uhckZv1xi2GzHLJkwiezoYNgNF5N5xVK50P7AcsJj3qclZEPFi2zDTgS8A+EfGSpK36ErxZTV1dcNhhjY7CrOnl6WKiv9cF9gC6Ss8ykHQ1cCjwYNkynwbOz3o0JSKe6+dnmXW3bBk8/7wvFJvlkOcJZbeRqm26iYj397LqNsCisveLgT17LLND9hl3k6qPzoyIX1WI4STgJIDJkyf3FrKZm46a9UGeqqEvlo1vRGo6ujrHeqowrWdCGQ5MA94HTAJ+LWnn8ucfAETEhcCFADNmzFgvKZmtp9RiyCUCs17lqRqa22PS3ZLuyLHtxaR7Dkomka419Fzmnoh4A3hc0iOkxHBvju2bVVcqEbjpqFmv8nQ6t2XZME7SAcCEHNu+F5gmaWrW/PRIYFaPZX4O/FX2OeNIVUUVn49s1ielXkc33rjRkZg1vTxVQ3NJVToiVQk9DpzY20oRsVrSKcBsUv3/JRExX9JZwJyImJXN21/Sg8Aa4B8i4oX+fRWzMl1dvj5gllOeqqGp/d14RNwA3NBj2lfKxgM4LRvM6qez001HzXLKUzV0sqQxZe+3kPR3xYZlNgClpqMuEZjlkqfTuU+Xt+LJ2vx/uriQzAboscfSq1sMmeWSJxFsIOnNpqDZHcPue8ialx9Yb9YneS4WzwaukXQB6aLxZ4D1bvoyaxqlpqN+YL1ZLnkSwT+R7ur9W1LLoZuAi4sMymxAOjthm23cdNQspzyJYBRwUURcAG9WDY0EVhQZmFm/+YH1Zn2S5xrBLaRkUDIK+N9iwjGrA3c/bdYneRLBRhHxaulNNu4ytzWnl1+GpUtdIjDrgzyJ4DVJu5feSHon8HpxIZkNgHsdNeuzPNcITgWulVTqMG5r4IjiQjIbAD+w3qzP8nQxca+kvwB2JLUaejjrLdSs+ZTuIXDTUbPc8pQIICWB6aTnEbxDEhFxeXFhmfVTV5ebjpr1UZ4nlP0L6cEx00kdyM0E7gKcCKz5uMWQWZ/luVh8OPAB4JmIOAHYlXQfgVnz8T0EZn2WJxG8HhFrgdWSNgOeA/zYJ2s+y5fDc8+5RGDWR3muEczJuqG+iPSQmleB3xcalVl/uMWQWb/kaTVUevbABZJ+BWwWEX8qNiyzfnCvo2b9krfVEAARsbCgOMwGzr2OmvVLnmsEZq2hsxMmToRNNml0JGYtxYnAhg4/sN6sX6pWDUnastaKEfFi/cMxG4DOTvjQhxodhVnLqXWNYC7piWSqMC9wE1JrJm46atZvVRNBREwdzEDMBsRNR836rddrBEqOkfR/s/eTJe1RfGhmfeDup836Lc/F4v8C9gaOzt6/ApxfWERm/eF7CMz6Lc99BHtGxO6S/ggQES9JGlFwXGZ909UFW2/tpqNm/ZCnRPBG9sD6AJA0HlhbaFRmfdXZ6esDZv2UJxF8F/gZsJWkr5G6oP56oVGZ9ZXvITDrtzx9Df1I0lxSV9QCPhwRDxUemVley5fDs8+6RGDWT3lvKHsOuKp8nm8os6bx2GPp1SUCs37Je0PZZOClbHwM8CTg+wysOZRaDLlEYNYvVa8RRMTUiHgrMBv4UESMi4ixwMHA9YMVoFmv3Ouo2YDkuVj8roi4ofQmIm4E3ltcSGZ9VGo6uummjY7ErCXluY/geUlfBq4kVRUdA7xQaFRmfeEH1psNSJ4SwVHAeFIT0p8DW2XTzJqDH1hvNiB5mo++CHw+e3D92oh4tfiwzHJ65RV45hmXCMwGIE+nc7tk3UvcD8yXNFfSznk2LulASY9I6pJ0eo3lDpcUkmbkD92MdU1HXSIw67c8VUPfB06LiCkRMQX4AnBhbytl3VKcD8wEpgNHSZpeYbnRwOeA3/UlcDPAnc2Z1UGeRLBJRNxWehMRtwN5evbaA+iKiAURsQq4Gji0wnJfBc4B/pxjm2bduftpswHLkwgWSPq/kjqy4cvA4znW2wZYVPZ+cTbtTZLeAWwbEf9Ta0OSTpI0R9KcpUuX5vhoaxudnTBhgpuOmg1AnkTwSVKroetJLYfGAyfkWK/aIy7TTGkD4DxSVVNNEXFhRMyIiBnjx4/P8dHWNtxiyGzA8rQaeolUh99Xi4Fty95PAp4uez8a2Bm4XRLABGCWpEMiYk4/Ps/aUWcnzJzZ6CjMWlqtTudm1VoxIg7pZdv3AtMkTQWeAo5k3VPOiIiXgXFln3c78EUnAcvt1VfddNSsDmqVCPYm1fFfRWrRU6mqp6qIWC3pFFJfRcOASyJivqSzgDkRUTPRmPXKD6w3q4taiWACsB/pLuKjgV8CV0XE/Lwbz/oouqHHtK9UWfZ9ebdrBrjFkFmd1Op9dE1E/CoijgP2ArpI9fmfHbTozGrxPQRmdVHzYrGkkcAHSaWCDtJjK90FtTWHri54y1tg9OhGR2LW0mpdLL6M1KrnRuBfI+KBQYvKLA8/sN6sLmqVCD4BvAbsAHwua+IJ6aJxRMRmBcdmVltXFxxwQKOjMGt5VRNBROS52cysMV59FZYscYnArA58sLfW5AfWm9WNE4G1Jj+w3qxunAisNfkeArO6cSKw1tTZ6aajZnXiRGCtqavLpQGzOnEisNbkewjM6saJwFrPa6+lpqMuEZjVhROBtR73OmpWV04E1nrcYsisrpwIrPW411GzunIisNbT1QVbbQWbubsrs3pwIrDW4xZDZnXlRGCtx/cQmNWVE4G1ltdeg6efdonArI6cCKy1uNdRs7pzIrDW4qajZnXnRGCtxU1HzerOicBaS1cXjB8Pm2/e6EjMhgwnAmstbjpqVndOBNZa3HTUrO6cCKx1rFgBTz3lEoFZnTkRWOtw01GzQjgRWOvwA+vNCuFEYK3D9xCYFcKJwFpHZ6ebjpoVwInAWodbDJkVwonAWofvITArhBOBtYZS01GXCMzqzonAWkOp6ahLBGZ150RgrcEthswK40RgrcG9jpoVptBEIOlASY9I6pJ0eoX5p0l6UNKfJN0iaUqR8VgL6+qCceNgzJhGR2I25BSWCCQNA84HZgLTgaMkTe+x2B+BGRHxduA64Jyi4rEW5xZDZoUpskSwB9AVEQsiYhVwNXBo+QIRcVtErMje3gNMKjAea2W+h8CsMEUmgm2ARWXvF2fTqjkRuLHAeKxVrVgBixe7RGBWkOEFblsVpkXFBaVjgBnAe6vMPwk4CWDy5Mn1is9axYIF6dUlArNCFFkiWAxsW/Z+EvB0z4Uk7QucARwSESsrbSgiLoyIGRExY/z48YUEa03MvY6aFarIRHAvME3SVEkjgCOBWeULSHoH8H1SEniuwFislfkeArNCFZYIImI1cAowG3gIuCYi5ks6S9Ih2WLfADYFrpU0T9KsKpuzdtbZ6aajZgUq8hoBEXEDcEOPaV8pG9+3yM+3IcIthswK5TuLrfl1djoRmBXIicCa2+uvu+moWcGcCKy5+YH1ZoVzIrDmVmox5BKBWWGcCKy5uemoWeGcCKy5dXbC2LGwxRaNjsRsyHIisObmpqNmhXMisObm7qfNCudEYM3r9ddh0SKXCMwK5kRgzavU66hLBGaFciKw5uUWQ2aDwonAmpe7nzYbFE4E1pzWroVHH4Utt3TTUbOCFdr76JC2bBnMnw8PPgjPP999nlT9fa15ld5HVH+tNa/aMpX0N77y8bVrYdUqWLkyDQMZX7kSVq9O291rr+pxm1ldOBH05uWX1x3w589fNzy93sPWDGDkyDSMGFF9fNSo9GyBWsuUxvfbr9HfyGzIcyIoWb688gH/qafWLbPxxrDTTrDvvvC2t60bJkxYd3bc88y7/H2teZWWLW2z1mt/lqlXfD1tuGHlzzCzptZ+iWD5cnjooe4H+/nzU1fHJaNGpQP++9/f/YA/ZQpsMMQuq/jAbdb22icRXHwxnHVWukGpZKON0gH/fe9LB/rp09NrRwcMG9aoSM3MBlX7JIIJE+Dd7+5+hj91qg/4Ztb22icRHHxwGszMrJshVuFtZmZ95URgZtbmnAjMzNqcE4GZWZtzIjAza3NOBGZmbc6JwMyszTkRmJm1OUWt7ombkKSlwBONjqOHccDzvS7VPFopXsdanFaKt5ViheaMd0pEjK80o+USQTOSNCciZjQ6jrxaKV7HWpxWireVYoXWi9dVQ2Zmbc6JwMyszTkR1MeFjQ6gj1opXsdanFaKt5VihRaL19cIzMzanEsEZmZtzonAzKzNORH0QtK2km6T9JCk+ZI+n00/U9JTkuZlw0Fl63xJUpekRyQd0ICYF0q6P4trTjZtS0k3S+rMXrfIpkvSd7N4/yRp90GMc8ey/TdP0nJJpzbTvpV0iaTnJD1QNq3P+1LScdnynZKOG8RYvyHp4Syen0kak03vkPR62T6+oGydd2Z/P13Z9ynkwdZV4u3zby/pwGxal6TTBzHWn5TFuVDSvGx6w/dtn0WEhxoDsDWwezY+GngUmA6cCXyxwvLTgfuAkcBU4DFg2CDHvBAY12PaOcDp2fjpwNnZ+EHAjYCAvYDfNWg/DwOeAaY0074F3gPsDjzQ330JbAksyF63yMa3GKRY9weGZ+Nnl8XaUb5cj+38Htg7+x43AjMHcd/26bfPhseAtwIjsmWmD0asPeafC3ylWfZtXweXCHoREUsi4g/Z+CvAQ8A2NVY5FLg6IlZGxONAF7BH8ZH26lDgsmz8MuDDZdMvj+QeYIykrRsQ3weAxyKi1l3jg75vI+JO4MUKcfRlXx4A3BwRL0bES8DNwIGDEWtE3BQRq7O39wCTam0ji3eziPhtpCPX5az7foXHW0O1334PoCsiFkTEKuDqbNlBizU7q/8YcFWtbQzmvu0rJ4I+kNQBvAP4XTbplKzIfUmpeoCUJBaVrbaY2omjCAHcJGmupJOyaW+JiCWQkhuwVTa9GeIFOJLu/0jNum+h7/uyWeL+JOkstGSqpD9KukPSu7Np25Zge5kAAAULSURBVJDiK2lErH357Zth374beDYiOsumNeu+rciJICdJmwI/BU6NiOXAfwPbAbsBS0hFQ0hFvp4Gu43uPhGxOzATOFnSe2os2/B4JY0ADgGuzSY1876tpVp8DY9b0hnAauBH2aQlwOSIeAdwGvBjSZvR+Fj7+ts3Ol6Ao+h+EtOs+7YqJ4IcJG1ISgI/iojrASLi2YhYExFrgYtYV0WxGNi2bPVJwNODGW9EPJ29Pgf8LIvt2VKVT/b6XLZ4w+MlJaw/RMSz0Nz7NtPXfdnQuLOL0wcDH8+qJMiqWF7IxueS6tl3yGItrz4a1Fj78ds3et8OBz4C/KQ0rVn3bS1OBL3I6v9+ADwUEd8qm15ej34YUGpNMAs4UtJISVOBaaQLRIMV7yaSRpfGSRcLH8jiKrVWOQ74RVm8x2YtXvYCXi5VewyibmdUzbpvy/R1X84G9pe0RVbVsX82rXCSDgT+CTgkIlaUTR8vaVg2/lbSvlyQxfuKpL2yv/1jy77fYMTb19/+XmCapKlZyfLIbNnBsi/wcES8WeXTrPu2pkZfrW72AfhLUvHtT8C8bDgIuAK4P5s+C9i6bJ0zSGcBjzDIrQJIrSfuy4b5wBnZ9LHALUBn9rplNl3A+Vm89wMzBjnejYEXgM3LpjXNviUlqCXAG6QzuhP7sy9J9fNd2XDCIMbaRapDL/3tXpAt+9fZ38d9wB+AD5VtZwbpAPwY8D2yHggGKd4+//bZ/+Oj2bwzBivWbPqlwGd6LNvwfdvXwV1MmJm1OVcNmZm1OScCM7M250RgZtbmnAjMzNqcE4GZWZtzIrC2oNSr6caNjqOWrNfKB3pf0qy+nAisXZxKumdhyMrucjXrMycCG1KyO6t/Kek+SQ9IOkLS54CJwG2SbsuW21/SbyX9QdK1WV9SpWc5nC3p99mwfYXPODPrEO12SQuy7a93Ri/pi5LOzMZvl3SepDuVnm3xLknXKz2f4N/KNj9c0mVZp2vXlUoxSv3Y35F1JDi7rIuL2yV9XdIdwOcL2ak25DkR2FBzIPB0ROwaETsDv4qI75L6dPmriPgrSeOALwP7Ruqcbw6pc7CS5RGxB+nOz29X+Zy/IHUvvQfwL1l/VL1ZFRHvAS4gdS1wMrAzcLyksdkyOwIXRsTbgeXA32Xb/k/g8Ih4J3AJ8LWy7Y6JiPdGxLmY9YOLkjbU3A98U9LZwP9ExK8rLLMX6UEnd6cuXxgB/LZs/lVlr+dV+ZxfRsRKYKWk54C35Iit1AfO/cD8yPp0krSA1HHaMmBRRNydLXcl8DngV6SEcXMW7zBSdwclP8FsAJwIbEiJiEclvZPU/8y/S7opIs7qsZhID4o5qtpmqoyXW1k2vob0v7Sa7qXsjaqss7bH+mtZ97/Y8/NKXS3Pj4i9q8TyWpXpZrm4asiGFEkTgRURcSXwTdLjBQFeIT1qFNKTuvYp1f9L2ljSDmWbOaLstbyk0Jtnga0kjZU0ktT1c19NllQ64B8F3EXqZG18abqkDSW9rR/bNqvIJQIbanYBviFpLamnyL/Npl8I3ChpSXad4HjgquyADemawaPZ+EhJvyOdKFUrNawnIt6QdBbpCXaPAw/3I/6HgOMkfZ/Uu+l/R8QqSYcD35W0Oen/9tukHi7NBsy9j5qVkbSQ1H30842OxWywuGrIzKzNuURgZtbmXCIwM2tzTgRmZm3OicDMrM05EZiZtTknAjOzNvf/AUE9FLljBccsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def eval_show(steps_eval):\n",
    "    plt.xlabel(\"step number\")\n",
    "    plt.ylabel(\"Model accuracy\")\n",
    "    plt.title(\"Model accuracy variation chart\")\n",
    "    plt.plot(steps_eval[\"step\"], steps_eval[\"acc\"], \"red\")\n",
    "    plt.show()\n",
    "\n",
    "eval_show(steps_eval)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从图中可以看出训练得到的模型精度变化分为三个阶段：\n",
    "\n",
    "阶段一：训练开始时，模型精度缓慢震荡上升。\n",
    "\n",
    "阶段二：训练到某一时刻，模型精度迅速上升。\n",
    "\n",
    "阶段三：缓慢上升趋近于不到1的某个值时附近振荡。\n",
    "\n",
    "整个训练过程，随着训练数据的增加，会对模型精度有着正相关的影响，但是随着精度到达一定程度，训练收益会降低。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 推理预测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们使用生成的模型应用到分类预测单个或者单组图片数据上，具体步骤如下："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. 将要测试的数据转换成适应LeNet的数据类型。\n",
    "2. 提取出`image`的数据。\n",
    "3. 使用函数`model.predict`预测`image`对应的数字。需要说明的是`predict`返回的是`image`对应0-9的概率值。\n",
    "4. 调用`plot_pie`将预测出的各数字的概率显示出来。负概率的数字会被去掉。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "载入要预测的数据集，并调用`create_dataset`转换成符合格式要求的数据集，并选取其中一组32张图片进行推理预测。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:54:26.317823Z",
     "start_time": "2021-02-03T08:54:25.214530Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Row 1, column 2 is incorrectly identified as 8, the correct value should be 2 \n",
      "\n",
      "Row 3, column 7 is incorrectly identified as 9, the correct value should be 4 \n",
      "\n",
      "[5 8 0 2 7 4 1 7 8 6 6 8 7 9 5 8 7 2 0 4 5 9 9 3 9 1 3 9 7 6 3 4] <--Predicted figures\n",
      "[5 2 0 2 7 4 1 7 8 6 6 8 7 9 5 8 7 2 0 4 5 9 4 3 9 1 3 9 7 6 3 4] <--The right number\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADsCAYAAADXaXXTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9e3xU1bn//14zCQkhIZBAgAAhBAmJRQWMcECRqEdQUYtHpFosSqVRlOpRvP70VFSoFxB+VQSNCIhQW8Bq661gq1guClJEpUQiJOGOJIFg7iQz6/vH3jOZazKZzN4zgfV+veaVzN5r7/nM2ms/s/aznmctIaVEoVAoFOZgCbcAhUKhOJtQRlehUChMRBldhUKhMBFldBUKhcJElNFVKBQKE1FGV6FQKExEGV2FQqEwkYgwukKwQQjqhKBKf+0Jtya/CJGOEB8hxEmEOIYQCxEiKtyyfCEESULwrhBUC8F+IfhluDX5QghihOANXWOlEHwtBFeHW5cvXNqo42UTgpfDras5hGCgfn+tDLcWfwjBDCHYLgT1QrA83Hqao61tIORGVwiCNUAzpCRefw0KqSh/BGcsFwHHgV7AEGAMcHcoZfkiyHp9BTgN9AAmA4uF4GchFeaDILRGAQfR6jIR+D9gtRCkh1iaG8HUqUsbjUer11pgTcjFedCG+wq0dvBVqLS0RJBajwCzgaUhltMs4WgDARtdISgRgseEYLcQnBSCZUIQKwS5QnBICB4RgmPAMr38tUKwUwgqhGCLEJzf2i8XNEKUIMRjCLFb75EuQ4hYhMhFiEMI8QhCOLUixLUIsRMhKhBiC0I0p7U/sBop65DyGPB3CN6QGVWvQtAJuBH4PympkpJNwN+AX0WaVimplpJZUlIiJXYp+QAoBi6MJJ0+mIj2A7wxGJ1maBWCm4EK4J/BajRDq5T8RUreA8rbqtNorR60vg1IKQN6gSwBuQtkX5BJIDeDnA0yF2QjyOdBxoDsCHIYyOMgR4C0grxNPz5GP9cikItczr0BZCnIMv28uYHq8vmCEgm7JPSVkCRhs4TZEnIlNEp4XkKMhI4Shkk4LmGEBKuE2/TjY/RzLZKwyOXcd0lYISFOQm/9c24IVqtR9QpyKMhaj896EOT7kabVx+f0AFkHMivCdX4KclZb2qrB91VnkIX6uWeBXBmpWl0+YzbI5W26/yO8DbT2S9zl8v4akPv0L3EaZKzLvsUgn/E4fg/IMX7OPQJkgl4Jt4GsBDkg6ErXjOZdLu+vkbBPN7qnJcS67Fss4RmP4/dI8KlVQraEf+vGW0pYLkG0sXGEvF5BjgZ5zGPbb0BuiDStHmWiQf4D5GsRrjMNpA1k/6DbqcFaQf4B5CP6/6EyukbXayiNbkS2gdb6dA+6/L8fSNX/L5WSOpd9/YCZele9QggqgL4u5d2Qkq1SUikl9VLyJrAZuKaV2gLWipReWnXXgvbyp1UIC7AO+AvQCegGdAWeN0prG+q1Cujssa0zUBmBWgEQAgvwFpofekak6tSZAmySkuI26jREqxAMAf4bWBACfYZqNZCIbAOtdSL3dfk/Dc35DSA9yh0E5kjJnFae34EERJDHOmiVVqQMRGuSft6FSFkP1CPEMrQBgIfN0hpgvRYCUUIwUEp+0LddAPynDTrBoDYgBAJ4A21g4hopaYhEnS5MAZ4LUpsnRmjNBdKBA0K7k+IBqxCcKyXDIkyrUURmG2hld/07kH3QfCQbQf5e764f8iibA/Kg7jYQIDuBHA8ywcd5u4AcBzIWZBTIySCrQQ4K+vFCcy98J6GP1Hy6GyX8XncvHPIomyPhoO7TFRI6SRgvwUurXr5IwqMSoiR0kfCuhFVtfAwKeb3q5f8E8m293MUgT4H8WYRqfRXklyDjQ/RoaYhO/ZhRehv1WybcWkHGgezp8poHci3I7pGmVS8fpduAZ0G+5bAHkai1rW2gtV/iMZC7QVaAfFO/sF5fQi9/Fciv9LJHQa5xCNRvsFf1/7vr5Sr1sl+CvLJNjVkzuo9J2C2hQsKbUhv48ja6WvmrJHyllz0qYY3T6MKrEl51KTtEwgYJJyWU6WVT2tg4Ql6v+vskkO/pjeMAyF+2pV4NbAP9QEq0wbMql9fkSNLpUv41kG+1qY2apNXluFmExqdrVFudpbcB19esSNTa1jYgtBO0jBCUANOk5B8BHRBOhCgBpiFlxGttT/XaXrS2F52gtBpFJGuNiIw0hUKhOFtQRlehUChMJGD3gkKhUCjajurpKhQKhYkoo6tQKBQm0mxyxJWWmyLK9/CJfY3fhAmlNXj8aW0vOkFpbQtngtb2ohNUT1ehUChMRRldhUKhMJGIXPFAceZy4tcjKbvY/7QKA5c2IjbvNFGRQmEuYTG6tlxtvo3DubEAWOoh7Q87sdfUhEMOAFF9+1B0R1qL5SJBK4AlLo4D9w3BHqO9772hDuuGHWHV1BIVU0YyOG8Xy9L8z/d80abpJG02UdQZRCTeVwpvlHtBoVAoTMTwnq6IiaHipqHYopsG8366qhqAPaMXAVBwuoaZSyeAyb/IlguyASgb1oWqfoKCvEUtHhMJWgEa4wTrpr9An6h4AAZlT6FfwxCAiHo8d73+l87Yyou9Irs33p45PEbr4TracbjaqivW7t0pvfYcr+1dC2uByGqrZmGo0bUkJFCTm83bc+bRPzreyI8KGJEzGFunaAAKb9H+Fl+/OJySWsQ6MIOC6QmAp9amOt0zegUXxNwCQOqpLOy7vjdTok+Cuf4ixIE/1u7daTi3T6uPi959CFtpaWjFnIU0nNuHr+Z431/9P54GQOZZ6EoyzOiKmBhqcrP5/LV8XI2DJ/WygT0NKWA3OMxOCKzdunHZ8i94KGmfsZ8VIqzJSRAVxZ4nEym+PL/F8t8MfxuAq14aj/WmJGzlJ4yW6JdArn+VvY5SW6OhP8hl489h2+9b/6M6dM7d9FrrEmpZX4+t4lQIlYUWS0ICto4RFaoakViTk7BXVSPr67HExQEgEtrW/mSN1mu3Vwa2KIthRrfipqG8PWcezRlcgJdOZvHZZRnYysqMkgKAtVs38rZ8wdVxJ4FoQz8rVPT+qJ4HenxATytAXMDHrc18h/s/uoIDIwyT1iKBXP+JhTdSu6C3bpgji/cfeYHKh5qGPG7eeQc9J0Su0d3z3Ll8ef08/V2nsGqJZHp/VM93Lw8lceWXHLhPc8etvXNeC0c1z/gP7wdg4D1bAypvmNG1RwmvHkx2/t30+azWbZu1ugFZussoGU1YBIOijxMjAjde4aZfx3KyO7Reb7wlljmp63l861gOXxMTlh6vr+sP0P9veZzzthYyFlVeS4fktq7KYwwOX7mDNUOXsPrbphXhN/06B7ndhHYbIDLWRoq1ydjmn0pl7bSxWMojR2M4sSYn0fujeuakrmfD73ZR+HAvhse9DhDUPeaKjLW1qnzIjW7FlJEAjP6tt9WP3y+xfP612zb1QOSNtUsih5enMilxCYH0WjLevZMBWUf4JPt957YUayce6PEJM6MmGKjUP9JHEmTGu3eSlf8T9m8KAKgbl0PvJ/e2eFxb6P6vo1z0+HQa4wTvP/KClzENlMzoTjzRrclPfutLPTn8VA4d1m0PlVQnjuuf+oQMyDdf9NxIZl+y2m3b8YbOiM07z/r76/S4HOTMMmKjGljQ+yPiLZ2YFH8K4sP31BJSo3vi11ocJuBzlLrLrYeoPTmcju9tC+XHtpm5JwawdO04bLGSTZPnufUYzMY6MIM9Tyay/cJX6OpDh0OrK4PePcn39/aAbPeyPa2w58XeDHoqHtsPRUbK9qLbjgqy8+922zbo3ZNOgwtQlRrNyvQNhupoLCohqagES1wc4zo/7Ixrbo4/3Po6Y+Oa74GvTN/ARanZJIVIpyvydAONW7tiOXUAO94JJR2ORZP++BfO972HHWVyQrkBSto/VanRfDX4Pf1dbFi1OAip0S27uKHZwPdPst/n0uk3UMtwgIgxvh8eOY+MNw7wwz19my23rb6Bmz+6n0E1uw3T0pCayL7Ll+Hpw32q9FwA1r6VS9q8LW777EC3zSOZ+rPRAM5r0NUax77Ll9G/Ko+sxTFuBs9o7N8UkPaNxzb9ryOI3xE6aIqemhr6PLul5YLAb6OncbpnA7fnbOHJ7v6v9U9XVZNYPCzkSSkOrY34TihZXxPN/VW/Ie0POzn26yH8MnWd2/GrKpN56/3LSOcLwonlgmxnhFAks6oymSc23eBzX+7P9gA0a9dai0qOUCgUChMxPQ34X+e9y5W/vQ6A07XG+MR8Ul/PzTvvYM3QJWRGd2LuiQF8eOQ8APYfTib2jhgKpyyiOR/q+srzGHjPVmePzUyWbx8FQOY83721pKVfsLtR86fPvK/azb1TfH0+F22dTtI3Pg81HUcQvyM5xsGtJbnEHwn/wJrj0X3tg7l8NjbTub1PfIWbO2TP6BVk776btA0Yxolz4b+7uve2x8Y1sPbOecxcOoEb8z71CoF8ff9oN/dDuCgb1sVnDPzcEwOIL+gQBkXurKpMBmDWu5PIfNS7vuTFQ9hyb3/tjZ+ebjDfxVCje9xWTf5JbcQ3r+u/nb5Sx4BP7swJsM7v4SHFVnGKnhNOMfmDqfw87Vveyb+c1L8eACD2jpgWs9EKG6r564HzSaLQDLlurK+JpsOxlh/TuqzQGs5Gy0j4vfsjb1U/QUrfPjQePGSIxkCxDsygrlejz30/PtafDp+b9CMcAKnztoAeTWTt3p3dN2bD7zaYquGclSd5IWssk/UYbAexwk718HRSopuiE7bVaz9Y+w90I5MSM2X6xF+iy9I147xcZGZqWF2VSGFdL956/zIAMnz8QFkGZ3HkoXr2eNS753n++Oo4Uhe27ruE1OiKOisFp5tSDuf/OJYDIzS/3f6tY1nQ+5/EW8LrzE66tpCNxNIz4Tt+eHwwgN7DbZ68PZNJutZYgytiYjjd2d24Hmqs4v43HiY9QH+kPwryFpEt7ybtqbYbXRETg6VLYssFGxvdwtWsXRJ9JnrYpJ19jbWIxsgba3cE0B+dNJCvH2+5nYQSa3IS9pcq+SbrQ699/aPjveKbb9mcB0D24wegh5ZwZCsrg7N8HcR62UBRQ9MT1Pyn7yJx5Zd+fd7N1buD4oYq5j99FykrW39fhtToDnp0NzOfdglRamwENKN7+JoYJq65kb8380XMJBKDyX0lFFz3/MOkLdsZFpeGPypuGsqy2fNbLDf/xyvdEjQOL09l+4Wv4DlIuK+xlv+9eiqWwl0RF+LkCKBfN/0FWkr0CTW9P6pnQe+PCHTUffuYVwA4pkdr7mlIIX/UyLM+nfmlk1l8dsUA5/suFV83284CqffbZzxAl/XNn8cfITW69spK8JMKZys/QX1jeig/LmgKl+Tw5ysWthga5hrI37G81nDD5yuhILpatnpqvm4f7mVY7HR2/M6YOSXsUSKggPI5qevJd0komJS4xCsMzhnEX7gL2ejb7RAuSuaM5OVfaAH0vuJ7s/PvJmPpAYxS3a9jeaueDLta4/S/ji3HwRLiwOd2RrcP9/LPXSORP7acJOKaQBFv8W0bihuquH3GA8RtKMBeXx+UJtMG0gqX5LBswFKzPq55HZctZXhM8z5Sz0B+M3qavhIDgkkWsJWW0vMdGGq9u00JAa4ceXAUyWOPADA6ObB0xxSre0KBryeKSA3iL3puJLNuWN1svG78fhl2H7kvZh4dxr8WjsDaIOlS8XXLBxhEqBNdgsFWWgoB9PQtg7Owv1TpTKDwxK1O138dtMEFE43uHcM3kdsx/A/JgeoYkHVESzggB4D4gg7awEo7wVZaSq+1wm3+ANpwE1Rln+Y7Z5C5O/3/lkfsMa0pdR95FNCiVFpi7okBvJN/OSlERr1aEhLY89y5yFgbsy9Z3WzCQf+/5ZG1o8LQH+N38i9nVbfL3bbZs6vYM3qF27ZBG6dgKWj6Ye1cJEnSB1Uj7ccMaFM7NAJ58RCOPFSv+3B9P1n8uzyNpGWhqVPDja5jhQNHnjM0BfqXr08l1aRRVl86muOT7PfdMrzmjhjA0vimTLCMNw6EvJfTtbCWC7bd4pwtDLQA/Kp+o+i+U3uIDTShJKpvH364py/JVuNuu5O2GnI+v4fsF8ucGW8HZmmhbZxn2McagqO+vrzef0bitvoGfvHpdAC372wUKT5GxU9MHQmj3bd1/nsnkpZFxg9Xe+RkZke+Gb7c5z4jbJVKjlAoFAoTMbSna+3enaOTBrqtcAAtB/qbpaM1PJS0j4dcYnmv/GwqlhD3dMXmnaREDQWX0MA9o1fAaJdJn30/4bsRlZFO0ZRU72SPEHZ6CxuquenraQyaUYSt4hSnx+VQlRqNPbsq4HM8lLSPfveuYm71L53bUj4/SmNRSeiEBoDf+vJgfeV5ZN6hxRG3bl4p44iUhJJWEQE+D3nxEE5mdgSgPMe3k2juiQGsfSsXIKSuxZAb3ai+2iz9pzO6U5odq8c2Nhm6QAP9Q0nDuX28dEQq1uoG5p4YwL1dvydGNNVT52Qt9M4+Zqhb+agde7FXVmIdmEFDqhY7W3RZx4CWHmoLG2oGkrIgloYLMgDo/eTeZievyT+VSm7cD2RGuxu1SfGnmOSyssDw/286XU0wuq2tr3Amx0DTfVXVz90hGmkJJa7EnrKTfyqVvMQj4ZbixQ+/jqL4av/RPcEmPgRCUEa3ueB4x6QxhVO8v1CoAv3DidGB/HL7Lj67YgDXbv2O7A5NRtfp5/VIkBlzZx6dtpWw50nHRDm+OdRYheV06HTmJR4h723/nwfuQelrp43llYfG8Kchbzj3J1jsbk8exQ1VWExKkCh4NIniq5f43X+osYpKu+sk5tPoOcG8CYM8Kfq1tlK10T+moSTuL1tZUz6uxXZiNtYuiVhi/D+rtCXxIRCCMrrNBcc3Ddx4P6ZFYqB/a4m0QP7lC+dTJy0tri4Rjrp3DUq3lO+i9+2dmBnTlDxzdOI5bllebQk4DzXXPf8wvdY2zfXbu/5IxLgUFG3DX5KOA6PbYVBGN9DgeGgKJu7wUwO9dv+ALYwrk7aFW0ty+fGx/ohGaXggv738BPdNvpuJS9a3+GgW6PpiwSRZuJL93An6N2hppsXXt7y8zq0luZTd28ctKN1zjbFeqyVX7prqfB+3I/iA80A58UEmP0/7lmcSFtPcsk3R1RLbj8cN1dIWnEH6OwradSfGbE58kMma87yTdCA0iQ+BEJTRDSTo2TOYWNbXh62nEL37EEPnBJ4okJ1/N/H73X/n4o80OH1nRvfEZGMjYvNO3nr8Ol5P1B5xm5atX9HcoW4caqziuucfJqpGkvKvo23KnLL9UETWYm0G8Iu2Tm+xfPyRBjpsb97XaCstxfJ5U+C6kcbDsRrDmvOW6H5l/wY3O/9uMv51xLBMs1BQJy102laCLcDFEM92vK+/O6urEpn/9F1tTnwIhKCMrq9VATyJpABtW2kpPZdVB7xygBExuMEQ95etzgegxGJt4u/sgubr3RVLPZpLoaYmJAbEkZ0XKVNEtoqYGP405A2fNxxoA7z3rfwNEDnX35Xen9cBkC2062+ph7TKneGUFDwmJ0cEshrLH18dR8rKLabYquDcCz5WBYh0WrNyQCT2cByrE7R27lb16OkfR+D78u2jtCVwZmnto71c//ZwbaOPnHKGOzrou9PcGg5kNRYjohT8oZIjFAqFwkRMXzlCoQgL+sohXeNqnZvK16cC5iXpnI3Yfigi8w5zF0UNFLOTtBwoo6s4K3CsHOKKWfN+KMKLtbqB2WVZXtvNTtJyoIyuQqE4o5Hbd7HxfO/Zw8K1WrKQZ/lSHgqFQmEmaiBNoVAoTEQZXYVCoTARZXQVCoXCRJTRVSgUChNRRlehUChMJCKMrhCkC8FHQnBSCI4JwUIhIjecTQhuFoICIagWgn1CeK5aFTm0B63t6foLQZXHyyYEL4dbly+EIFsIPhWCU0KwVwhuCLcmfwjBBiGoc6nXPeHW5ItQtNWQG90gb5ZFwHGgFzAEGAMEPrNLkASjVQiuBJ4HpgIJwKWA4Sk37UXrmX79pSTe8QJ6ALXAmpCL86C1WvXyfwU+AJKAPGClEGQaIM/XZwfDDJf6HRRSUT4IV1sN2OgKQYkQPCYEu3Urv0wIYoUgVwgOCcEjQnAMWKaXv1YIdgpBhRBsEYLzmzl9f2C1lNRJyTHg78DPWvNFTNT6FPC0lHwpJXYpOSwlh890rer6+2Qi2g24MQK1ZgGpwAIpsUnJp8Bm4FcRqDWkRHxblVIG9AJZAnIXyL4gk0BuBjkbZC7IRpDPg4wB2RHkMJDHQY4AaQV5m358jH6uRSAXuZz7LpArQMaB7K1/zg2BajNLq77/NMhHQe4FeQjkQpAdz3St6vr7/JxPQc4KVqfB1/88kFUghctnfQLy3UjTqr/fALIUZJl+3twI1dnmttraL3KXy/trQO7Tv8hpkLEu+xaDfMbj+D0gx/g5dzbIf+sVIkEud20sQVZ6yLWCTNX1bQfZC2Q3/YLOOdO1quvv9RlpIG0g+wer0+DrHw2yCOTD+v9j9fOtizSt+r4RIBPQjOFtICtBDohAnW1uq6316R50+X8/2uMLQKmU1Lns6wfM1LvrFUJQAfR1Ke9ECCzAOuAvaAurdQO6ovki20LItaL57wBelpKjUlIGzAeuOUu0nu3X35UpwCYpKW6jTkO0SkkDMAEYDxwDZgKrgbbOzm5IvUrJVimplJJ6KXkTzRVyRrbV1hrdvi7/pwGOBbw8J3A4CMyRki4urzgpPdeyBTQnf19goV7h5Wi+lrYaspBrlZKTaI021BNWtBetZ/X192AK8GYbNRqqVUq+lZIxUpIsJeOADGBbJGr1gaRta0xEblttZZf9O5B90PwkG0H+Xu+yH/IomwPyoP7IIEB2AjkeZIKfcxeh+R6jQHYB+S7IVW18DDJK69MgvwKZArKrfu5nznSt6vq7HTMKZHVzZSJBK8jzQcai+R8fBFmM7quMJK36NR+na40COVmv30GRpDNUbbW1X+QxkLtBVoB8U7+YXl9EL3+VfsNXgDwKco3ji4B8FeSrLmWHoDnST6I50teATGlj4zBKazSac70C5DGQL+HiIzpTtarr71b+NZBvBavPxHqdq9dpFciPQZ4TiVpBdtfLVeplvwR5ZaTpDFVbDXhqRyEoAaZJyT8COiCMKK2hp73oBKXVKNqL1kjXGREZaQqFQnG2oIyuQqFQmIhaOUKhUChMRPV0FQqFwkSU0VUoFAoTaXaWnSstN0WU7+ET+xq/wdJKa/D409pedILS2hbOBK3tRSeoJdhbpGLKSAA63nqU/Qe6kTlte5gVuVO4JId+aWVu2w7v6EXGo+FZXlqhUDSPMrrNcOLXIxmctwuAZWkbmd0zi43EhlmVhiUhgT3Pncufr1jI8Jhot325TAiTKoVC0RKmGd3jM0ZR1817e2wZpCzcYpaMgHBo/a+rvmNZmjZV6qrKZN56/zLSiYwepIjryIfjF5DdIc657anScwEoX59KKiVhUqZQtG8c93/vDU3z4hzO9e5spb97Evs3Ba0+vxpIUygUChMxvKcrYmKouGkoj927iknxp7z2zy7LYuPCyHhk96V1VWUyALPenUTG45HRy7V2787RieeQYLG7bV++fRQAmfPC9+RguSAbgLJhXfyW6VqozTopNu80RZMiMqj5nxHUJQbezzO7nXje/4Oypzj37Rm9yKt8Rvc7GZSf3ererqFG15KQQE1uNm/PmUf/6HgjP6rN+NK6viaap9dOAogog3vsxnP4+vFFQFOdrq+JpsOxaP8HmoB1YAYF0xMAKL5+sd9yF2y7BYDUU1nYd31virb2imVwFo3JHQMqG31E6yjYfjB8yb5mierbB4DTGd3dtv9qzvvkJR7xdYhP+n88DYDMzaHT1hyWLoksmz3f6bLbM3pFs+WLbniN/tY8sl/MAAKvd0ONbuOwc/j8tXxcjcNxWzXltqZoimP1iVh7JDrf2ytOIevrjZTlE19a73vrN6Q/FRn+Zkuc1hCOThqoG9wmDjVWcf8bD5P+bHi1FjyaRPHV+c739bKBooYGADKio4kR2o/CN8O1qUpz508gZqz5OgPFkpCAiPMweHaJrawMTMjktCYnYX+pkk+y/hRQ+QGfTgVg0IxEbBXeT5VmYElI4Id7tKlsC6f4/+FtieO2akSdNVSyWkRERUFyF6zC93WtstdRamv06jwWX59P/2j9x+GOCDC6vrhk1YOcs2Cf833Nhf1YvPUPzvdTn3iAxJVfmi0r4jlw3xAA1k1/AdcfBoDrnn+YtGU7sfs4Lpy8dDKLz64YAMBl/9zHQ0n7Wjgistjz3Ll8OH6B+7aGFPJHjcRWWmr45/f+qJ4FvT+CACNmto95BYDc5XfQc0J4jO6e587ly+vn6e86BX2eS1Y9yKA5WuSQGe3aPmIwL/3xFQZE+X6qmFh4I7ULeusds7ZhutG11gpsPx53vo+u7u02Am+Pastk8cFR8z8j+NWc9922ZeffTcbSAzSarsadwiU53DF8E8PjXgegT5S7wR329HR6vfMDtpqacMhzUrgkhz9fvhjQerO3luRSdm8f5I/ajfPP20fyzUt9WZm+IXwigRMfZPLztG8BeOv9ywBI93AdOco8k7DQrW1qHAeLsW3UmpxE74/qmZO6nnhL4Iarq1XTumboEiZ/MJWkawuNkugXGWsjxdo6Y5t/KpU1d41z2zaw6CCNlZWhlOYXx/2fGe2t2+kKe0LSIbnBa3//v+WR/aIWJ28L8PMMM7qnx+XQ+8m9btsy3r2TQe+d5IRLwkGfePclpqT5Npe6RIuXryl+v6TxYFuXk2o7/dLKeKKbt9/zUGMV1z3/sGZwTeh1NUfhkhyWXbbULV74UFUXYrbvcr6X23dxqCo9DOrc+Xnat8767D9Rq7fXLxrtVmbVoGX6DWi+j9wyOAv7S5Us6P2RX4O7uiqRuc/+EoCHHvuj1wB1ZnQnfp72rekx5UXPjWT2Jau9tjvaalSN70f32FN24j7f6rbNjM6OI/Hpkvu2+vQ1D9o4BXuxdg1q5x+lT/wxt/0Z795JVv5PrfahG2Z0q1KjvcVCczkAACAASURBVHo1A7KO8P29Pcj9WVPCgSv9/5ZH1o4K0x6TaycMB6DLre7G1Wwd/iiZM5Lf9fNuxOtrorn/Dc2lEK4ernVgBgWPJgHw58sXeyVotAcmJ5Rrfwe/57HHt7HT6v1B0ipDO5ruaIcHr7PTObmab7I+xNWlcOl3N1D6RS/n+9gySFmm+e+f7TSZJz1iygHGJnzH0ldmMOjR3dhN6jH2HnbUWacOXNuqPcxPY664Jj692GuHzzKj+hVry1bibasAkrdbgorTNdW98En2+5Dtf/85bzcE9SWCpXSI9vULsjXXwklbDTmf30P2i2Vev16WC7IpuaErlnpI+4OxDcgSF8eB+4bw8i9eZ2yc+yPNqspknl47ifRnt4T1R6EhNZHiq5fo7yLf4Drq1OGmCYZtNQPoY0C9O9ph8dXeYUlXFlwHi7uT9p7vQdKUhVucPTZXhsdE8+H4Bcx8egKYZHR9YVSdtZWyixt8GlJXmtt/wbZb6FFY63d/c6jkCIVCoTCRiJh7ocpex8TCG4kqrzXtF1FePAR7dpXzfWFDNTd9PY1BM4qc4Tby4iGczNRGM8tz7BTdsEjzT/30ML1WG+dLFQnxrL1zno9BHHh9/2ivgR8H/hITun+w1zS/ryOZ5PCOXmREUCqyZ53OPTGAD4+c57f8rAF/I7djU2vcUGvhjW2XkIkBEx75iT67tSSXxhd60HFd86umd7z1aIu9NkUTp8flcE76j0Ed67BVqU9I7LuCczMZZnT9hLu5UdhQzepTF7K/NhnLTbXYyg8bJccNy+AsjjxUz57hTUvbrz51IT0nFDhHIB1lvhm+3O3Y7tYYfjl9Hf/cOhIMMGSWhASqh6cTK9x/frbVa26G/Qe6kelhzBwB9IW3aI/5nokJw2Kmk1ygBayHKoDeWt3A3BMDuLfr9874W4AXCrTA2/SP6xA5g5H6YJrIGUyf+PAPTDpYumYcaX5isEXOYJa8dCm5LmMSs/Zdb/oMcz8+1p8On7f8mfsPdGN2zyyGx2kheZ4uqXCSEv0T9jGa+yNqx17T/Mv+EDmD6f3k3qCiaI7bqnn8yNg226qQG11HEH9DJ99hCK7JETfvnEbPCQVAtf4yh9r5dXzjNXjShCMoXRvQcKeooYHPrhjgDIUKNf4SSm7+6AEAMu/ZioiKwpKc5NzfUgD9jt81GWFnlk+Agdz+kNt38dllGQzc8iNXx530Snzgba036YzTXf6FM073pE3zh5+s6UjPNqkIjJYC3z25ZOl2nxEjkUrmtO1sTujOise1ELi2JCWEmrzEI+S9vQyAMXfm0WlbCbKyyvxBNSGwduvm1g5bS/7JCzkwou22KuRGt7kgfnBPjuhdfyTg2DYzaW1QutF4BorbRwxmwaqmQZe+URbCodVWVkb+qJH88FmRz4Z8b9fvuXbrd4CWkeYYcMv5/B4AzZVjgs6WAt/PBEKVlGAkyxfOp05amPjag/QxOXvS2q0beVu+4Oq4k4R74DdkRrelIH7QEg4GvnGARpfkiEjAEcgPmmHr17GceIv5RsxfksbANw5w5DbN/3hj3qekRK/36e91cGtJLrtXZLv1cA1BSmylpX4TH2JENNkdvBu4vV5L7zQrVVVGCbfA90hJfGmJm15dx/GGzryTf3mz058WLsnhz1csbHVSQqjp+EAsF8y+pelpxwNHCu2CO17nt/HaE5e/8YlQInIGc9nyL9yeyMJJyIyuvyB+B9n5d5Ox4kjYEw4Kl+SwbMBSQA/HARpf6EFsXRV169MBmJS4Cl+9hdVVicx/+i66VHxtiDZfSRrXXPcl/x6Vxi9T1wH4fTQatHEKnf+uaY4/0kDPHXu5qH66doyPAPpQIrfv4vBTOVyU2hQPqA08vmbYZ7aG6N2HGDrnbt5/5AX6RMW3KvHF0UbEi93AoIHB9PdOApCRcqdbnWlt4QgxdzXw4fX+B/2WDVhqepz06XE5lNyEm5/bvut7eswdwkWZWrvz1wbGxjXwu4la/Pks6yTDVzmxdYrW7xvfdTTz6DD+tXCE27ZLZ2z1G7/bVkJidP0F8bvS57NaGotKQvFxbeKO4Zuco9J7S3oAkLluO41jhrJhsMMv6m1w554YwB9fHUfKyi3+BpsN4cVeO8DHxT9uq+aSVQ9irdP843031GHd4NJ4B2ZQdok2oJIeXYbRj1Qd1m0nyeV9tx3Z9LfmUXy9d6767Tlar23tg7mkmjANpa20lJ7LqhnX+WHsMZC+42TAUTKubcQoHLHpg/KzyS69G1usZNPkec6e60NJkTdvRVVqNH++4iV+8YZmYLOfO4HthyLE5p0k6bOCddvh+/tAU2IKN6zmadskU3q8nkw9oGUi7l40mKQVTZ9fMmckw+L3G/a5bTK6zQXxnyk4VmNY+1YuqRG0wkW5TXDOgn3YfjxOxZSR2sz2uaOc++t6NrrM+BX+RypXnuy+G4C42+v5Y904U1YOsdfUOP2IkRao78D+TQFp32gRLP+V8AB/usZ7KaZIYnhMtDNBpn9DHlmLY9ySmzy/j4y1MfuSd92y1iYnlFN83WdsfNxcd97UA6PZlT8YgKQVXxDVtw9Fd6QB+E1KCtXKMSo5QqFQKEwk6J6utXt3jk4ayLrpLzgHzeaeGODc73gcCkfiQ6A4AqRPTB1JVT/vELe5Jwaw9q1cAFMeg+OPNHBrSW5AMYQJFjtHJ55DVM2AgP1Pc08MIL6gQwiUtkzZsC5u8cK3luQyrce/3BIOHkraR33epxGzckikYK+sZOA9W/nFkuleKz03x/hULVLEDFeEZ1stvj6fDNudJG/3Tkm2NkgyH/ia6vFD2DGkn9f8DEYSlZFO0WXeUStffnwe6f/WfOll+v1fkOedhu2guaSkVmsK9sCGc/t4rV6waNMVzv8ful678Acb7VhuacD2Y+TFPX6iz7nAHPft+adSnaPGZroUOqzbTln5YGYvzQIgr+u//Y5I94mK95rMvCWWrhlHmklL+XiGxP74WH9mPd6FDc3ERyvcaW0yxtIntekRH7qzde0iGBxtde7ypgSZohtegxu8yxY3VHH7qQdavXJEKCi9tJdPY1rXq5HvA1jlxAhCGqfra9Ak0thfm0yVvc5nSJhN2tnXWMvaaWMRm3eSgvk+XLl9FxvP17Tt3zqWB3p8AkCyVbNigYYFua7a4MByOoRCA8RRpw2dooiNOjP9/mcr/hJkPOkfHR+Syb+DwdIoKW6o8rniQyAYkcgTEXMvmMnha2KYuOZG/u4j22xfYy3/e/VULIW7TI1Q8Mfha2KYGTUBgL33a66bQLONXFdtcJBWaf7qEo46HfzmtzzX63MiJeFEERpaSpAJN13WfM3tpx4I2ugbkchjqNHNP5XK2mljsZQbkzIbDLbyE1juzeLK5Kle+0Sj1AxuY2SEzdvKTzj/H/iK5ou98mNv3b6wVjcYlqrcGvpFdeCy1f/mtsRvvSbl9kxKUQRPyZyRvPyL4KetDBqXBJn1nS7l+P11fpMjfGF0G5D19XT4qfVPWNn5d9Pns1oGOeYqCWEiT9BGN3r3IS563H/w/cyjw9g25yLiNm+NiF6jK/Zd3/sN24g0rQ4cwfyWAIP6w/09HCuAxAhHYHqTwXVNSumw3dxJZJrD2iWRw8tTmZS4hEhNpfXH6Z4NYQ3blNt3YQF6NDYlRwRC/JEGw9uAZ3JMSzgTuYpKDElTD9ro2kpLSVqmzbLlmL3elc5Fki5/iYxlyxWRwaCNU7AUxNN9p/Yk0dKUhWZiHZjBnicT2X7hK3QNczpte8Y1OSJS8EyOaYmMNw4YmjkbEveCGcHtivZF18Ja54xmDgYubURsjsy20pCayL7LlwFNc1pMPTCabpsjNzmhObbVN3DzR/czqGZ3uKVEBK7JMS1htHNRJUcoFAqFiZx10QsKcxCbd5IZYY+ZrWHm0WFeOfmRTHxBB3L7TnC+33+gG5n3bI24hCSFMroKBaBFe8wuy3K+3zbnonY1JpE6bwvMa3rvubqIInJQRlehwD0pBSCOrWFUoziTEVKGO7hIoVAozh7UQJpCoVCYiDK6CoVCYSLK6CoUCoWJKKOrUCgUJqKMrkKhUJiIMroKhUJhIhFhdIWgyuNlE4KXw63LEyGIEYI3hGC/EFQKwddCcHW4dflDCJKE4F0hqNY1/zLcmlpCCAYKQZ0QrAy3Fl8IwQZdn6Ot7gm3Jn8IQbYQfCoEp4RgrxC+1nWIEITIRohPEeIUQuxFiIjVKgQrheCoEPwkBIVCMK3lo5oIudEVovUJF1IS73gBPYBaYE2otXkShNYo4CAwBkgE/g9YLQTpIZbmRTD1CrwCnEar08nAYiH4WUiF+SBIrQ5eAb4KlZbmaIPOGS5tdlBIRfmhtVr18n8FPgCSgDxgpRBkGiDP88NbV69aeS+tCGG41iDbwLNAupR0Bq4HZgvBhYEeHLDRFYISIXhMCHYLwUkhWCYEsUKQKwSHhOARITgGLNPLXysEO4WgQgi2CMH5AX7UROA4sDFQbWZplZJqKZklJSVSYpeSD4BiCLzCzdIqBJ2AG4H/k5IqKdkE/A34VaRpdTn/zUAF8M9gNZqhM5QYqDULSAUWSIlNSj4FNtOG648QJQjxGELsRoiTCLEMIWIRIhchDiHEIwjh1IoQ1yLEToSoQIgtCNGiVqS0IWWbtRrZBqTkP1JS73irvwb4K+/jBDKgF8gSkLtA9gWZBHIzyNkgc0E2gnweZAzIjiCHgTwOcgRIK8jb9ONj9HMtArnIz+d8CnJWoLrCrLUHyDqQWZGmFeRQkLUen/UgyPcjTav+vjPIQv3cs0CujFCdG0CWgizTz5sbiW0V5Hkgq0AKl8/6BOS7QeuFEgm7JPSVkCRhs4TZEnIlNEp4XkKMhI4Shkk4LmGEBKuE2/TjY/RzLZKwSP//PAlVUs+Q1bd9IiForUbbAH1bDUgJcgfI+IC1tfJL3OXy/hqQ+/QvcRpkrMu+xSCf8Th+D8gxLXxGGkgbyP4haMhGa40G+Q+Qr0WiVpCjQR7z2PYbkBsiTau+7w8gH9H/D4XRNUrnCJAJ+g17G8hKkAMiTavePotAPqz/P1Y/37qg26tmNO9yeX+NhH260T0tIdZl32IJz3gcv0eCd71CtIQiCQ/r/4/Vzxe0VpNsgBXkJSCfABkdqLbW+nQPuvy/H+2RAKBUSupc9vUDZupd9QohqAD6upT3xxRgk5QUt1KXqVqFwAK8heYvnRGhWquAzh7bOgOVkaZVCIYA/w0saKM2Q3UCSMlWKamUknopeRPtMfiaSNMqJQ3ABGA8cAyYCawG2rokgl+tSOmlVXctaC9/9Sql6VpDYa+k5rbZBPQBAl6jqLVO5L4u/6cBjkXspUe5g8AcKZnTyvNPAZ5r5TH+MESrEAjgDbTBqWv0xt1WjNBaCEQJwUAp+UHfdgHwnzYpNUZrLpAOHBDa2mrxgFUIzpWSYRGk0xcSEEEe68AQrVLyLdqgLwBCsAV4sw06oZVakTKwepXSTStCmK61DW0gCgN9ut+B7IPmI9kI8vd6d/2QR9kckAf1RzEBshPI8SATmjn/KJDVzZWJBK0gXwX5Ja3w4YRR659Avq2XuxjkKZA/izStIONA9nR5zQO5FmT3CNPZBeQ4kLEgo0BO1tvsoEirU738+brWODR/fjG6nzKol+Ze+E5CH92nu1HC73X3wiGPsjkSDuo+XSGhk4TxEnzf33C+hFgJcRIelFAsHf7fCKpXkCkgbwYZj+ZeGKe3gZ8HrK2VX+IxkLtBVoB8U7+YXl9CL38VyK/0skdBrnF8Cd1wvepR/jWQbwXdIEzQCrIfSIk2eFbl8pocaVr190kg39MbxQGQv4zEevVx3Cza7tM14vp318tV6mW/BHllpNYpyLkgT+pt9GOQ57Tp3tKM7mMSdkuokPCmbiS9ja5W/ioJX+llj0pY4zS68KqEV13KzpVwUmoDah9LaJNWg9vA53q5n9AM+29aoy3g+XSFoASYJiX/COiAMKK0GkN70dpedEL70ooQJcA0pIx4rZFcrxGRkaZQKBRnC8roKhQKhYmo5XoUCoXCRFRPV6FQKEyk2TjdKy03RVQ3+BP7Gr/xkEpr8PjT2l50gtLaFs4Ere1FJ6ierkKhUJiKMroKhUJhIm2Z97Rdc+LXIym72H8G78CljYjNO01UpFAozgbOSqNbMWUkg/N2sSzN/5S9g+Kn0Dd6GNYNO0xUplAoznSUe0GhUChMxLSebs3/jKAu0d3Gxx9poMO67WZJcNLx1qMsS9vIqspkAF7fPxqA2KgG1ma+Q7wllj2jVzCIKfRrGAIQNlfD6XE5VKVGB3WstUHSZc3XyPr6lgsrFAYhYmKouGkotuimAf2uhbXavrPQhWeY0bV2707DuX2c7381533yEo+4lcndNQHWGaXAP/sPdGN2zyzeev8yANIf/wIAS3IS9390BXNS15Ni7cSe0Su4IOYWAFJPZWHf9b2pOkXOYHo/uZeV6RuCOr64oYrbTz1A3IYC7JVtnUa3ZawDM2hITWx6X92A3L7Lf3m9jYhGiWXrLmRjo+EaASyDs2hM7thiuQ5FpTQebOuUrgpLl0SWzZ5Pdoc457YLtmn3VUrU0BbbSWvxbIctEVVea+q9HVKja0lIQMRpjfnoxHP4+vFFoTx9yMictp2NxJLOF27bbeUnODACHt86lgW9/0m8JZZvhr8NQO78CcSMNVfnJUu380S34BtD/+h4Pn8tnzF35tFx/TeG93gLHk2i+Oolzvezy7LYeH6s3/Jl489h2+8XU3C6hpkjJmD78bih+hzUzq9jw+A/tVguc8V0zllwutkysrIKe01NqKS1iIiJwdLF26CYraOtOO4r3m65nbQWz3bYEld9Px7rLSlgl9jKyrSJBA0kpEZ3z3Pn8uF4bfL/BIsdbS7q9sfha2KYuOZG/p71YbilhITlC+dzy+MPkrjyy3BLaVdsmjyP8pubn5984msP0ufZLSYpgoqbhrJs9vyw6ziTWJv5Dge32tnTkEL+qJHYSksN/byQGd3CJTn8+YqFbo8Q7RVb+Qks92Zxwexbmn6RTcSanETvj+rJ67oZ6NTm8/WPjueB373Ns10mk7Lw7LsxRc5gLlnaNHYwKXEVgdRrirUTKdbmy9g7tFFcgByfMQqAx+5d5fMeW3DH6/w2fhrQ5C6LFOzlJ7hv8t3IKEGPZ4uDdpcZRbwlluwOkBF9kh8+K6Lero2hvJN/uSH3S8iMbr+0MobHuA/4rK5KZO6zvwTgocf+yKT4U6H6OMOx7/qen8pzTP9cy+As7C9VsqD3R8RbAjO4Ge/eSfL2pkHK8hw7RTe85lZmUvwpnkwOqVQnhUty6JdWxux+77ZY9siDo0geq/n2RydvNUaQB7ZO0R5uGq1eryy4DoCKlX2o6icoyItMd9iRB0cx5XZt8MNxDx1qrOK65x/msftWMSn+FGPjGjjdMxQrR3njuL61K3vRZUXrDbps1GLeBXCoKj3k+jwZuLSRiza1vGTZT1dVA7Bn9AoAYkQ0DyXtc+5flXy5IfpCPpA29YAWCfDl388jtgxSlmm/FIX394J2ZHSPPDiK23M2ON//pt9Gnp4zyfBeRGNyRz7J+hMQuI8rebuFpGVNurrtyKa/NY/i6/Pdyv3X1d+x6+BIAJKWhu573DF8k5fvee6JAVpPAfeeQlX2ab4b/F7IPjtYLv3uBljcHYCk974gpW8fsrkbgD/c+jpj41o2YIM2TqHv53UtlmsrVdmn3YzB+ppo7n/jYdKW7eRZMZn9d61z2x9qHNc3O+Nuuhj2KaFDbN5J0uaWyyUWa8vvZRdo190WK9k0eR4pVu1H2aj7JWRGt3x9Kv0PTqPbZq23m7b0zHqMnZxQTvF1n7Hx8dA5/NvCSVsNOZ/fg73eykA9/MaB/ZsCsl/MYED8VLaPeYWuVu1xdFnaRvpfPAiApKXG6vvwyHkR78ooHaI3/yGjiC2DtFma3t9GTwuo16hlLRqbPFMxZSS5P2sa2V9VmczTayeR/uwW7IZ+8pmPI/EpbYP+vkcK5TcLp0vJqPtFJUcoFAqFiYSsp5s6z3evxhEYnRnbNCDlSEo4vKMXGZSESkLQRGWkc3xML7dtseWSTWUDoPtuADbUWnhj2yVkYn4yhy+O2WDQzMN+w6xsPxQxaGYKx7ZC1xYGg85G/nXeu3Be0/vVVYnMrdbGHzrvBfZG021HBfZvCsIjUMeRyOPg9f2jSX/8C+d99di9q0wbK7FnV3Fiqva4Ha7EJqOwdu/O0Ynn6FFXGnNPDCC+IPQjpYZnpPkKjH5i0w0AZD4avlFW1wDqoss6tjiIsuTHSxn0ah1GRvBZu3enNDsw90WssFM9PJ24DbX+Ex8aG5n/45XOZA8z2FavPZbvP9CNTI8fVMvgLDonV5uio7VMij/FpDmL3bb1/1se2S9mANqPWKRgSUigJjebt+fMo3+0eWGZe0avAG3IhltLcikrHxxQUoOIisI+YjAyStAnvthglYHhmUBRmh2r5xU01efSNeNI89OZbAtn5YQ31i6J7HkykX2XL/O5v7ihiu7WKOItTQZwZfoG5i4fwGeXZRgWQF02/hx2/M79xj/UWEUnYXH6ZR04Eh+uvGUqls+/9nk+R7JH/rcXtinJwi9CYO3WjRhLk0G6ZXMeANkP/oA9KgpLcpJzn/2lSr4JQ+yzaJQUNlQzIKojVhG4R634+nz6R2thWJl3hMfonqzpyElbjfP6x0Y1UJObzeev5WNWHPz+2mSq7HW+74crBgBaWJi/jEJLchILVi2KiHBSa3ISREU1e/8bzVlpdA8vT2X7ha8AvhvB7TMeoOP9h72SI+7t+j0Dt/xoSgC1g+uef5ifRtaGrYE0h7VbN/K2fMHVcScBbQB1+5hXAMhdfgc95sawYFXTE0TfKAuticoIFZatu/jfq6fy/3+8jMxoc3r8oaL37UfIWXiP8/qvzXyH0oWNmJl45C9Z6N6u33Pt1u8AuG/y3e1iHoXeH9XzQI8P6GkFf/e/0Zx1RvfEB5msOW8JXT0et/NPpbLmrnEARP3fMV4asBrPAPoYEc2g6ONgaT5LKZREV0sGPXWK/lVaD9IzDKw5Qp1k4YrIGcxlyzWDGyOa4rMdPbI1Q5ewYcnAFns3+adSWTttLJby0OXeeyIbG6G8Apt0v27O/P8F7j8EN726zjlPyJ8v1548frFkOpnTzPdh2ipOYa9vcsrHW2KJN3n425EsdGXyVPbeol3r4uvziRHRZHfQ3k9csp7jDZ19Hh9j2UdGdHCTNoUKx73QGlfbH371OttuHOC2bX9tMoevicFWfiJoLaYb3UEbpzBwqTkTm/ji52nfOns7rsHxsafsxH2uBeuPebnOq8y+71MpuuE1+kZZsL8djeXe0E+AI33Ycik0f2LW4hgALtrqHvTdffdebP5OGBXFAz0+MMSfa9n/I39cPI5bHvmGPlHeN1RmdCcyPSY48mTm0WFsm3MRcZu3Guor98WgjVPo95JmvcRmd/fMW49fx57Hv+LFXjucCT/90spMVthEc8H+ZiUd2Xd9jwXIqsgGtHbomlCi/Ug1d73Da3TtVdV89/JQxkefz6UztvLB3sF0/rv7fdEYJ3j/kRfoE6U9RYyNa2BsnPs9XmWvY+KaG9t0/4fc6NZOGA7Aweu0UUBLjE3vymuM6lfMhl9rsW/xI0f5jXoINZaEBPY8dy7PJCwEor2C41sqk9zVAjdoPY2/Z33IlclTTY23c4yiJ33jvt2fwbUOzGDPk4ludR9KbKWl9FxWzbjOD2OPgV9PDDxA35FAs3vRYLr8JTyDqaP6FbNr0GAAr0D6uL9sZVP8SKbe3anZie7Norlgf7OTjlzboWtCiSvdRx4F9AgRF/r/LY/YY1H811XfmV6vsr7eOffIJttI+h6ox7rBve1Z4uKc7dkTe3YVe0avcN7/F8y+hR5zhwTlUgmZ0a2YMpKfMoSzwovdKrzpEXNZ2kbQKzy37wSYFyoFzSPiOvLh+AXOx93SL3qR9l6TwY/q24cf7unLl9c3ZaS4lom9eAgXbAvPXAytxXJBNgXTEyi+PB/Xup96YLQzeSUU2GtqnJOsrKgax6Js7xm5LDE2twSNqQdGsytfN3ZBpJSGipYC37us+IINY3KcbXV86nf8cca4iE/4MJPGg4dIm+U99aWj49X/umlu27NfLMP2QxEb+jbVazjwl8rs2p49kRcP4YKYpvv/m+Fvc1Hm9IAy3zxRyREKhUJhIiHr6Xa89ShbA8ypD0tyRH09N++8gzVDl2j+WhcnYlRGOkVTUimcsgjHgNPMo8PoXNRUSGzeSUrUUNA7uocu60jGwXQai0zS3wrKhnWh+PrFXtu//Pg8w9Kz/bmJrD3cEzSM1NAaWhv4/lDSPurzPmXjwshIA/fEqED+YOj43jYAMj3Mgd+xh3aA5/3fFkJmdPcf6Ma2gQ3OgYfChmo21Az0Wi1ifU00T6+dBECGiVPQ2SpO0XPCKVbrMat1vRqxjxkKeCdH5J9KZduci7z8jdbqBuaeGMC9Xb+nIG8Rw0um0zWERlf4GE2q6idI6dvH7woGImcwtk7RXsd4sroqkdjykMgMGEtCAtXD04kVETBLgJ4o0q+jVgnv5F9OaitdBSnRPyEv/i9TV7kIFKMC+c0gkuvVFdf7P0ZE+7xfAyFkRjdz2nZufmWGcxLz2/5zOx1f7Urea+4hTve99RvSnwp/4yi+Ph+u971vzV3jnJEMrsjtu/jsigFcu/U7Z6iM0RTkLSIz1v8KBpct/6LFAazihirmP30XKSvNrffGYeeYGsTfHI5EkQN6nLDn7GeBkJd4hNGrFpm6ysXZQLup153f89mkC/n5x9+S2YYQuJBGLwx6dDczn54AgO3n3Vi+cC6RcMO1d5pbwUCLf2y+Adw+4wG6rP/a9LAsheJMwj5iMC/98RUGRLW84wycCQAAFqxJREFUvl5zhNTo2isrQZ8HwHp6gKl54YGy6dc53PpST5+z1xc3VHH7jAeI21Hgd9o811nwuzUXIxsE3T7cy7DY6V6pwIGsYOAL5/fZUIA9AlYEzs6/m4ylB4iEB8iSOSP51XWfNVvmmYTFhDu+1B9GJr4omnCs2HFj3qekRK8PSUajYckRvgL9IwG5fReHn8rhotRsr33WBkmX9V83a6BcZ8EP9cCArbSU5II+LRcMgNVVicx/+q4Wv4+ZxO+XEbO67umeDQHMR+G9Esr8p++iS4XvuS5MxcDEF4WG64odvlx4wdo4UzPSzJppvyU6rNtOkp994X4E71BUSnZ+61YwAG0lhNIvmqanjC2DlJVbwv59IpW+71u4tI82251nEL8/Cut6kbjyy8is0wjt5PgimLo3A0dylIzVulO352zwMrbHbdVcsupBrHWC9B0ng5pI3jCj27Wwlv4fuwdHmzHTfnvHNeA80BUMQGvIrskeiubp+N42amkK4u+cXN1s4suqymTeev8y0omMRR9lTS3jP7yfP12z0GttwkjHs+4BRJ2VQTW7w6bJV3KUJ9vqG7j5owcYNGcX9srKoFfuUMkRCoVCYSKG9XTF5p1kBpEip2gi0pbSbi1R5bXk7prgfB9/xJjVaoPFNYjfMjiL3PkT/JY9vKOXqXHlLWGvrGTgPVtZ/+15DI/5Pvx+sVbiK4EinNHc9sR4UocdZVLBZL9l9h/oRuY9W9us86yb2lFhHvZd3xMz1nVLSZiUtIy3VnciYVkpX/z1wPkApie+nGm0dP0Br5VQgkUZXYWiHZN0bSEbiQ0q2UMRHoQ0YNkZhUKhUPhGDaQpFAqFiSijq1AoFCaijK5CoVCYiDK6CoVCYSLK6CoUCoWJRITRFYJsIfhUCE4JwV4huCHcmvwhBDOEYLsQ1AvB8nDraQ4hWCkER4XgJyEoFIJpLR8VHtpLGxCCKo+XTQheDrcufwjBzUJQIATVQrBPCEaHW5Mv2lNbdSAEA4WgTghWtua4kBtdIVoX+6uX/yvwAZAE5AErhSAz1Nr8fHZrOQLMBnwsZ2gcQWp9FkiXks5oU7bPFoILQ6vMm/bSBoKpUymJd7yAHkAtsCbk4jwIRqsQXAk8D0wFEoBLgaIQS/P1uWdsW/XgFeCr1h4UsNEVghIheEwIdgvBSSFYJgSxQpArBIeE4BEhOAYs08tfKwQ7haBCCLYIwfl+Tp0FpAILpMQmJZ8Cm4FftfbLmKAVKfmLlLwHhCQHyGCt/5ESx7yOUn8NiECtIW0DRtapBxOB40DQS9sarPUp4Gkp+VJK7FJyWEoOR6LWdtRWHee/GagA/tlqcVLKgF4gS0DuAtkXZBLIzSBng8wF2QjyeZAxIDuCHAbyOMgRIK0gb9OPj9HPtQjkIv3/80BWgRQun/UJyHcD1WaWVo/PmA1yebAazdKqb6sBKUHuABkfaVpD3QbMuP76vk9BzorE66/vPw3yUZB7QR4CuRBkx0jT2p7aqv6+M8hC/dyzQK5slbZWfom7XN5fA3Kf/iVOg4x12bcY5DMex+8BOcbHeaNBFoF8WP9/rH6+dW2s8JBr9SgTSqNrtFYryEtAPgEyOtK0hroNmFSnaSBtIPtH4vUHmaobr+0ge4HsphueOZGmtT21VX3fH0A+ov/faqPbWp/uQZf/96M9EgKUSonr7OT9gJl6V71CCCqAvi7lnUhJAzABGA8cA2YCq4G2LjEQcq0GYqhWqT2ybwL6ANMjTatBbcDo6z8F2CQlxW3QaKTWWv3vy1JyVErKgPnANRGo1Umkt1UhGAL8N7AgWFGtdSL3dfk/DW1QCfCaWO4gMEdK5gRyUin5FhjjeC8EW4A3W6nNE0O0GoRZWqNog59Mp720AaPrdArwXJDaPAm5Vik5KQSHfJyjrZztbTUXSAcOCG21jnjAKgTnSsmwgFS1srv+Hcg+aD6SjSB/r3fXD3mUzQF5EM1HIkB2AjkeZIKfc58PMhZkHMgHQRaj+1Pa8GhhlNYoXeuzIN/S/4+KNK0gU0DeDDJef2QbB7Ia5M8jTWuo24CROvVjRul16bdMJGgF+TTIr/S20FU/9zORprU9tVW9ffZ0ec0DuRZk94C1tfJLPAZyN8gKkG/qAry+hF7+Kv2CV4A8CnKN40uAfBXkqy5l54I8iTaY8jHIc0LQkI3SOgu0mnN5zYo0rSC7g/xcL/eT3gB/E8H1GrI2YKROfdtrIN9qS12aVKfRaINAFSCPgXwJF19mpGhtb23V47hZtNKnG/DUjkJQAkyTkn8EdEAYUVqNob1obS86QWk1ikjWGhEZaQqFQnG2oIyuQqFQmIhaOUKhUChMRPV0FQqFwkSU0VUoFAoTaTY54krLTRHle/jEvkb426e0Bo8/re1FJyitbeFM0NpedIJagl2haBccnzEKgBvzPgVgf20yh6+JwVZ+IpyyFEGgjK5C0Q6oS9b+PtHtewAKTtcwM2pCGBUpgkUZXYWiPeD3YVXR3lBG1wfHZ4yirpv2f/edjQB0fG9bGBW5E9W3D0V3pDVbJrYMUhZuMUmRQuEbX22194Y6rBt2hElR+FHRCwqFQmEiqqfrgxvzPnX6zq4suA6A07U5dFi33XQtp8flUJUa7batqp+gIG9Rs8etrkpkfsUtdFnzNbK+vtmyinZARI3NB87pjO5ebTX7/7V39kFNnHkc/24SBEIQ5K0I+AJIMB2sb4gDhRPtiOd1dLBVqqX1iqUo6tlasbWn01pHR6++nZ7aGl9wrNo7xdrW01Y7N+KpqJSz2HNEUF5ExRMILwYSMMnm/lgSSLIhAfYl6n5mGGD32d3fPPvsb599fm/GhRicz488roCgdB3ws+IkACB5WSpwhvvrG5fV45eY73p8XJqsGSPWbsGyn1NheFTLgmTdIw4MhO7FsB4dI7l2B6RazZJETy/iqAi0DdTzLYbTSAZR9/1JRCBUCg+b/W0D9SAnjKbaqrQgb9ziVD6A6lNdiI/FNkJvhOjqDRj19vuakEhAjo+BUULA7eZ9GOrqenxtQek6oNGgoX5rPBHMsyxPAyKpFADwMC0Kv67sfjZuzeQ5GRCd/5VxmQiJBCJ/P0bPSaoaun04maRkhR8qpyo5uVZfEXl74/YiKnd42dwvadtUTldS9X4B/P7WqxDP8uPU9U3s64PSz3xQPinXvK3dqMOPmgFQJsTTKlLC3R0iXx/A3xfbj+yE3M0LcX/OxoADgtJlnNjziwAA0YsrYOBZlqeB6vdHAQDOZH8BKqk+/5DjY7D1cM9eAI54P30hiEvFjJ7zWaB0w4u4Mn1Tx39eDtvnyY9j6elXUD2eXbm68uBACIrG7gQgNW/b3jgc5yZGwFBfT3tM06zRyF27BWLCiEiJZ5+uz5rS1bw2HlNWn6fd9/XJiQCAoSsvs3V5xiDbxQAAQ1Mzz5K4PlXr4vG3N/YAAMIktgpXoVyIsHNai20qhQeufUo/I2IKo4SAop/UccMenpMvLy5lcwjyMlMgUt3gSQJ6yvbG4h+v7ECQ2LGyNSETeWCIpwrVsF2GYIsBUi0GiDvHw1tVyahfEgZjnW1/moJSPllyGIp+UlTqWjBpwQL0e6xDwM07vZqIMap0a3IS4J9ClSGK8//FbIyyJnwmNSVfLU5DxArXV7yuxtEWH6zflo6TH39Bq9z4oGJDPFbPOIoUqc687WiLDzauf9P8f8T5GugrqiyOC74ZiHHtVP3BwF4O4meZig3xWJt41GJbra4/iEvFvNvWRDHDod3SWd8xN3I/4tzdujnCNbnf4gv3IluFW5OTgLnvUIacNFkzZZxeswC+ZynjdG/HKiNKt2pdPJ4E6/BObD4+C7zpsH26t4r6Y8ZRrDGkPRUzXr7QHhoIRcRCi22kmxH6eC28CPsef2c1bli6LweD1dx8AoeOedh5XwFsbIjEka+mICi301eYbgXUUFcHv1zqJSwoXFus+9VVML48CjXL23G9F0bermxsiMRx5SQEwbV8ymtyEjDz7Xws9ys3bytrGwifQ1f6/LJjROm+Pe2c3Vltd6R7qxD4xl4sbXkPg7cVg9RomBCn14ikUlS/Pwpx0j28ytEV34OX4dvlf9FIBW5le6NyUi66rkl15bDaH2vy0jB0fQFITqSkpy0AuP9JgsW91abGoW6U/WE39EQjyOslrMk0snAOHqscf/6uTTzhksrusNofX5+ciKGwP1ERjVSgasYAi21M9ashmSp4ey9bj9K4b/p8vlM1I1wyiMc/pcZiAulMvzuLEBwhICAgwCF9mukS7u5omjUacg/6N16ZrhVLytOQJz8OmYh+oTxFqkPe/E1Ytj8V4HmmS3jLkDd/E+NGl75ifHkUGuWUxVQVS6Jy+m7adofVVFaU1SfSEMHzks1yv3Isz9qF+/oWTHv8ESQa6qPM9637KOnwfaZj3N1s+F1nVhaJSovkG1RymJBVRgTfcBzksudsEtJpPp1byDbMLHud8i9lVkyn2HM3iXY5znqMVMyw9NZgql8fTKCe49IkZr1B+GSsfzUKXxsP6bdXbfax8Uz1SemKfH2Qu3aLhZIqbKcMKVW6ADQZQqDdGoq6HXrIhDl1jyFiY2DwckPt0jZcjzvQbduzGjesyUsDAF4U7t3qABRG6WwMKWESmdP+usrmEHg0M6/KyBu34J7S8bcT7YnYGITJ7ttsrzW0YmVNCkSztDCoHjArZC9xdowQLFnd2o06bG8cjnaSuu8p3v/tkTEtTNaERxNGOxWYwBTWY3XzwGtYu1qDC99SL5Su93/VxRkAADmDBn/GXcbmXMoCAPS/7IlfV+5C1m4luvPXbDfqUKoLAkh+bbGERAL4+0LcZXTWGlpBtIl5EIaAOCAAEw9ctljI747FeZkYtr2j7QtBgF7PqcO5PLMIs3cuxpXpWwCgR25DBiOJcr0WeZkpkF6ynW1whoN+VzaORfX4VgCtnIgj9veDh0RHv7MXY4QJRE+o3yVPqK/SUl0Q5d/aEVCwb182Kqfudfp8h4bmA9/ko0zXig+mZoAsq2Bd8dKNVXeRDuLAQTDU1yNxf1GvbFTOwrjSLZqwEwDQ+jsSzjjHO3JK5gpyfAy2H9lp4ficeDgH0esoVxIuPyXFAQHIKriMqdJGAM7NGi6mb4Jqdqfn6JZHkzl1OAeA6BU3kajOAWA/GomOcr0WH0zNgKjsBq9uUL3pdzYJPd2OraGnARofVr5kHbyN8oZZtr8jly9pZOTZjZR44q8/5mLxm4s4CTqxHqtLBtxCVMEjKBPiAdxj9dqMK12T0/EAB+2A7p2SucYoISB365ydKZQLEbWvGno+cgGICES71cKdcH5tOUjshaAuk3KuHc4BgFSrEbWTGrCTf8yw2DfrqzPI8qmxOcbs6F/Gzadlt/Si39lkiKeK1hZCxMZg4gFK4boT3L4czB5Gduwvig0NmHwwg3afCWopxNIOJCZEkLt5cRZ0QqrVEGs7r+ROuGGqtBG3z1Xgjz6/AfBC+A9ZUGymXihMujPyGgZszynZFZDdNUJ/z3ZdT6B7TH0m6vgt9vXBgwMhSJbehnVY6LKHY1C4bhykl666hKM/uV2NQRJ644NZVvC4/NGBwcutY0nBeYVr5Ch8znC7AqLb3bd5QT8K0UvmAgBKkw5a7ltfiQefc5PRb+h3jQCAiKD5qJixG+6EqV+pcerxUALD7QrGr9sn85ZR3YKZu3NwVmN78zc2REKhXGj+oWsj8GwjjopA6Y4I5I/dZ/EVkVGdhIzqJFzcRm8x5gO9vyd+Gn7KrpfNf1SDOZf1uHISNjZEWmx7b8gF1C5ts3MEPeE/ZCHgWhOTotFSk5OAsn2xaJgXj4Z58XbbEZeKMehLCQZ9KUH0hbkW+w4NzbdJZcoW5PUSkNdLEK18jPAfsiz2RV+Yi9DzPetnZ+nTTJfUaBC2vgB/kmXiSbDlgr+spB8Gb+p0ei58LRIpUmpx+vO6FwEAqrMhCEFVX0RgBNFIBcrmuM5LwajR4tVTS/H3P+x4KsMqAapPS7K9UTlJia5BHBnVSbihjAEA+B10/UhEPsdq0I4CnJo+wsJQlu6tQrqTQQmNBg1izy+CYnM9KzM2oDOgiHQH5s08g+V+5QhHJgDAb7/940yVI/qHxwNJrIjWJ/r/5AVxPjvjU3DkEhAQEOAQRtZ0u8udQBdAcaCIytwj3+Qa4X/1Y3xROZ3dTFc9gVSrEbXoKs7+NgJx7twneO4LopEKAEBpVn+bII5lD8fg5q4Yl5vhSiKGomKibbq+jQ2RyPs6GQAQ4iJj1RnytSKsLp+ORo0nlZKUpQx54sBAPEyLwpnszsRLGxsiISvp5/BY0zhRxfIZqE5h/fy/VZUMWY0dVz0GYN2QRhdA4SqYMty3DLG0MrDlpM8VZbpWfF/9EvxQxul1xVERKMn2BgAbhatsDkHhunHw/dZ1FK7p/lfMDaEtf7T/2BSLJTI+sBd0QocpMCnjXDbkmUUIBntJhMSBgfjf68M6Al8ohXu0xQdHvpqCEKtcCl2rLZgwLefRRVeyFchhjb3n/9En4eh3nj1D3nOdxLxiHlWl1PqBO7ZgCqTn+XfSdxc5vw5Xa2iFykANntnFmQhOZS9pDB2mbPzUGm4nLhP4YIVlhQNbhXtf32IOBOATkyP/qVe3Omw7+/RS6phF7PVzd5VB1m9Lx8C8O1RwTle6VFtwRKWuBSI9N1rX+vk3jVWC5es/10rXVemN43vi4RwM20oZXELbazhPk0iXjR9wncAHaxxVOJj2l48wOLeY1yxtJqJX3MSyNamO22morFhsytxdZZCTH38B9XJbM1FPqi28s/hDKl9tnyXtOVyNVc6VLlsOx88UPXDSH7MmG/4lbYiquAc9xwUoxf5+CD3djiGeKqT57MUAq9Bflwp8sMLoYeg2VNmt1ch7qlETpFoNuEjBTrJjuZYueX5vE+orm0NwbMEUAID0WglInqpXG4wEoGpifaxyrnQXJv4LB+9QHRyyiR03lqcdsqkZGas+xIeffoM0ma0RpGtFhuB/3oGhro42QTibmIIJtoae7vBtdd3AB2voqjF0RaFciIh/13Dep88DESfmw7/Icjbs0Uyal/O4+rKoyUnAvFmd5b3NVSGamC+Mag3nSne5Xzl2KVxgsQwwOz8rCMvKDBEV1bw+cMb2dvgcuoL1vun4LMB2v0c9zBUZ+PhaMFcNGH4K1nkBMqopp8ubu2JcymhmomodVVaILkF5raEViYdzqPBvIRqRFnvPjLNEs5yk3llaFE/M/s/mKieHCjiZIHCudDOqkxBwyTUc/k0O2oPzLbe7ygzHFTPqA0Cj3NMmjeDkkmm4U/WC+d72xS1Mmxpn/tvzu8Jen4eOt6eds1sRQmUgMGxrOefLNE8T9p4ZZ3GFNXJruK5eIQRHCAgICHAI+zPd9nbMLn4Xx0bvhdzNCwV3wzGkVOv4OAGXRVajM1diMEFsDoCcoSQljcM6h6VzNm/n2VeYiMSJZUj2tJxzlelaMbs4E6HttpnQBJ49ZCX9kDyIGsMPrg1EBIch3qwrXUNTM4JTm3H0t7FYFXALpUkHEd5CxWbLL7F9dQE26HemCDhjvbWKsfOzGf0lzyxCxt55eDfuosX276tfQnBqieBR85wQsqkA6PAY5FLhAoKfrsBziDyzCBesDIBcR+8JPL8QRqOrOfQICAgIPLsIhjQBAQEBDhGUroCAgACHCEpXQEBAgEMEpSsgICDAIYLSFRAQEOAQQekKCAgIcMj/ARaqKtLmZA8yAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 32 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds_test = create_dataset(test_data_path).create_dict_iterator()\n",
    "data = next(ds_test)\n",
    "images = data[\"image\"].asnumpy()\n",
    "labels = data[\"label\"].asnumpy()\n",
    "\n",
    "output = model.predict(Tensor(data['image']))\n",
    "pred = np.argmax(output.asnumpy(), axis=1)\n",
    "err_num = []\n",
    "index = 1\n",
    "for i in range(len(labels)):\n",
    "    plt.subplot(4, 8, i+1)\n",
    "    color = 'blue' if pred[i] == labels[i] else 'red'\n",
    "    plt.title(\"pre:{}\".format(pred[i]), color=color)\n",
    "    plt.imshow(np.squeeze(images[i]))\n",
    "    plt.axis(\"off\")\n",
    "    if color == 'red':\n",
    "        index = 0\n",
    "        print(\"Row {}, column {} is incorrectly identified as {}, the correct value should be {}\".format(int(i/8)+1, i%8+1, pred[i], labels[i]), '\\n')\n",
    "if index:\n",
    "    print(\"All the figures in this group are predicted correctly!\")\n",
    "print(pred, \"<--Predicted figures\") \n",
    "print(labels, \"<--The right number\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "构建一个概率分析的饼图函数，本例展示了当前`batch`中的第一张图片的分析饼图。\n",
    "\n",
    "`prb`存储了上面这组32张预测数字和对应的输出结果，取出第一张图片对应[0-9]分类结果`prb[0]`，带入sigmol公式$\\frac{1}{1+e^{-x}}$，得到该图片对应[0-9]的概率，将概率值0.5以上的数字组成饼图分析。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-03T08:54:26.413763Z",
     "start_time": "2021-02-03T08:54:26.318837Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The probability of corresponding numbers [0-9] in Figure 1:\n",
      " [0.16316024637045334, 0.04876983802517727, 0.02261393383191808, 0.9963960715325838, 0.037634749376478496, 0.998856840107891, 0.1612087582052347, 0.08714517716531343, 0.6207903209907534, 0.9653037548477632]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAD3CAYAAAC+eIeLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXzU1b3/8dcnC/suAVlUVmWxiCiguKCAVYuI1qVVexGr4tZeq73V1nvrmNrVn5baRVuvta1rtdbWWnstqBXqgooWZRdkD4QESEJC1pn5/P44X0pEIJnMcmb5PB+PeUyY+X6/886QfHLmfM/3HFFVjDHGpEae7wDGGJNLrOgaY0wKWdE1xpgUsqJrjDEpZEXXGGNSyIquMcakkBVdk1ZE5DURuSZJxz5SRGpEJD/4d18RWSgi1SJyn4jcISIPJ+F1rxCReYk+rslMBb4DmMQSkQ3ANar6su8s6UZVNwFdmj00B9gBdNMEDVgXkUHAeqBQVcPB6z4BPJGI45vMZy1dk8uOAlYkquAa0xpWdLOYiMwWkTdEZK6IVIrIOhGZFDy+WUTKROTKZttPF5F/icju4Pm79jveLBHZKCI7ReTbIrJBRKYFz+WJyDdF5OPg+WdEpNchss0UkSXBa30sIuccYJuhIvJqcLwdIvKEiPRo9vztIlISdA+sFpGpweMTRGRxcOztIvLj4PFBIqIiUiAivwWuBG4LuhymichdIvJ4s+OfKiJvBu/dZhGZ3Yr3aWFwXxkc9+Tg/X692XEnici7IlIV3E9q9txrInJ38P9WLSLzRKT3wf+XTaaxopv9JgIfAocBTwK/B8YDw4AvAT8Xkb0fufcAs4AewHTgBhG5AEBERgEPAFcA/YDuwIBmr/OfwAXAZKA/UAH84kCBRGQC8CjwjeC1Tgc2HGhT4AfB8UYCRwB3Bcc4BvgKMF5VuwJnNzvG/cD9qtoNGAo8s/+BVXU27iP/ParaZf/uGBE5Evg/4GdAETAWWNLS+xR8LwA9guO+td9xewEvAj/F/Z/8GHhRRA5rttnlwFVAH6Ad8F8HeG9MhrKim/3Wq+pvVDUCPI0rXN9R1QZVnQc04gowqvqaqi5V1aiqfgg8hSuiABcDL6jq66raCNwJNP9Yfh3w36q6RVUbcMXxYhE50HmDq4FHVHV+8Folqrpq/41UdW2wTYOqluMK1N48EaA9MEpEClV1g6p+HDzXBAwTkd6qWqOqi9rwvl0BvKyqT6lqk6ruVNUlrXifWjIdWKOqj6lqWFWfAlYBM5pt8xtV/UhV63B/MMa2Ib9JU1Z0s9/2Zl/XAajq/o91ARCRiSLyDxEpF5Eq4Hpg70fb/sDmvTupai2ws9lxjgL+FHwUrwRW4gpj3wNkOgL4+ACPf4KI9BGR3wddCLuBx/fmUdW1wNdwxb0s2K5/sOvVwNHAquDj+3ktvVYsGVt4n1rSH9i432Mb+eSnhtJmX9fyyZN/JsNZ0TXNPQn8BThCVbsDv8R9xAfYBgzcu6GIdMR9PN5rM3CuqvZoduugqiUHeJ3NuI/9LfkBrjU9Jugq+FKzPKjqk6p6Kq7gK/Cj4PE1qnoZ7uP5j4BnRaRzK16vtRkP9T61dFJua5C3uSOBA71PJgtZ0TXNdQV2qWp90O96ebPnngVmBCeB2gHFNCuAuMLzPRE5CkBEikRk5kFe59fAVSIyNTgBN0BERhwkTw3upNQAXB8wwfGPEZEpItIeqMe12CPBc18SkSJVjQKVwS6RmN4J1987TUQuDU68HSYiez/mH+p9KgeiwJCDHPdvwNEicnlw3C8Ao4C/xpjPZCgruqa5G4HviEg1rs/23yegVHU58FXcibhtQDVQBjQEm9yPa/3NC/ZfhDuJ9ymq+g7uRNFcoApYwKdbf+AK+7hgmxeB55o91x74IW6cbSmuVXtH8Nw5wHIRqQlyfVFV61v7JgQZNwGfA74O7MKdRDsuePpQ71Mt8D3gjaCr5aT9jrsTOC847k7gNuA8Vd0RSz6TucSGKJq2CEY8VALDVXW97zzGZApr6ZpWE5EZItIp6B+9F1jKgYd6GWMOwoquicVM3ImgrcBw3Md2+6hkTAyse8EYY1LIWrrGGJNCVnSNMSaFrOgaY0wKWdE1xpgUsqJrjDEpZEXXGGNSyJbrMcakpffee69PQUHBw8CxpGcDMQosC4fD15xwwgllrd3Jiq4xJi0VFBQ8fPjhh48sKiqqyMvLS7sLCqLRqJSXl48qLS19GDi/tful418PY4wBOLaoqGh3OhZcgLy8PC0qKqrCtcRbv1+S8hhjTLzy0rXg7hXki6mOWtE1xpgUsj5dkx7cZOSDcasoHAb0BHrtd98JyAfyR7Ji8ypGDgDCuAnK63Bz65Y3u29+K1WlKZXfkkkwkRMSejzV9w71dG1trUycOHFEY2OjRCIRmTFjRsXcuXO3xvuyVnRNaokMBE4ERuCWw9l7G0gMn7x2021xcJzWCouwFrd224rgthJYperWjjOmuQ4dOujrr7++unv37tGGhgYZP378Ma+88krV1KlT98RzXCu6JnncsuLj97sd7ilNAa7QjwAubPZ4VIQNwDu4FSxeU+VTKxOb3JOXl0f37t2jAI2NjRIOh0VEWtqtRVZ0TeK4tdNOwy2XczbwGb+BWiUPt57ZEOCLACJsBxbiivACYLlqiwtOmiwUDoc59thjR23atKn9lVdeWTZlypS4WrlgRdfEyy1EOQNXaM8AYl11Nx31BS4JbgBlIvwJtxbaAtWYF7k0GaqgoIBVq1at2LFjR/706dOHvvvuux3Gjx8f03p7+7PRCyZ2Ir0RuRGRN4D1wM+A6WRHwT2QPsB1wCvAVhEeFGGKCPmec5kU6d27d+TUU0+tfuGFF7rHeywruqZ1RDohcjkiL+JWA/4FMIlPLsOeC/oA17OvAD8gwnjPmUwSbN26tWDHjh35ADU1NfLaa691GzlyZFytXLDuBdMSkaG4pdevArp5TpNu+gA3ADeI8Dauxf8HVRr9xspSLQzxSrTNmzcXzp49e3AkEkFVZebMmbsuu+yyqniPa0XXHJjINOBm4HPYJ6LWmBjc7hPhV8AvVdmWihcWkZuBa3GfOv5XVX+SitfNdhMnTqxbuXLlikQf136ZzD4ihYhci8hyYD5wHvYzEqu+wJ3ARhGeEiGxA/r3IyLH4gruBOA44DwRGZ7M1zTxsV8oAyL5iFwFfAQ8BIzynCgbFOKGoC0W4Y8ijEzS64wEFqlqraqGcUPcLmxhH+ORFd1cJiKIXIa7OusRYJDfQFnr88AyEX4nkvD3eBlwuogcJiKdcN1BRyT4NUwCWdHNVSLnAx8ATwJHe06TC/KAWcBqEX4hkpgr81R1JfAjXHfQS7j/03Aijm2Sw4purhEZgcg84Hky44qxbNMOuBH4WISQCO3iPaCq/lpVx6nq6cAuYE28xzTJY0U3V4h0RuQe4EPgLN9xDJ2Au4AlIpwWz4FEpE9wfySuK+OpuNOZpLEhY7lAZAbwc9y0iSa9jAQWiPAwcJsqlW04xh/FTS7UBNykqhUJTZgmpDixUztqqOVxvwMGDPhM586dI3l5eRQUFOiyZctWxvu6VnSzmUhP4EHgC76jmEMS3LCvGSJ8TZWnY9lZVeNqKZtDW7BgwUf9+vVLWD+5dS9kK5GpwFKs4GaSw4Hfi/BXEfr5DmOSw4puthFpj8h9uLPZA3zHMW0yHfhAhHN9BzEwderU4aNHjx5577339k7E8ax7IZu4q5OeAMb4jmLiVgS8KMJc4Ju21JAfb7zxxqpBgwY1lZSUFEyZMuXo0aNH15977rk18RzTWrrZQuQK3OoHVnCzhwC3AgtFGOg7TC4aNGhQE8CAAQPC06dPr3zrrbfinr7Uim6mc5fw3gs8DnT0HcckxUnAv0SY5jtILtm9e3deRUVF3t6v//GPf3QbM2ZM3OvpWfdCJnOjE34PfNZ3FJN0vYG/i3CbKvf5DuNDa4Z4JdKWLVsKLrzwwmEAkUhELrroop0XX3zx7niPa0U3U4mMwl1VNsx3FJMyecC9IhwB3KpK1HegbDZq1KjG1atX29SOBhCZAizCCm6uuhl4WoQOvoOY2FnRzTQiFwJ/A7r6jmK8uhiYJ0JP30FMbKzoZhKR2cAfgPaek5j0cBrwuohd3p1JrOhmCpGv4ea8tRVoTXOjgEUijPYdxLSOFd1MIFIMzCX3Vt41rdMPeFkEW6YnA1jRTXci/41bc8uYQzkceEWEo3wHMYdmQ8bSmchXgO/6jmEyxhHAqyKcpspW32ESbsHixC7yOfnEFsf9FhcX93nssceKRIQRI0bUPv300xs6deqk8bystXTTlcgs4Ke+Y5iMMwTX4u3jO0imW79+feFDDz3Ud8mSJSvWrFmzPBKJyMMPP9wr3uNa0U1HbljYI1gfrmmbEcB8EeIuELkuEonInj178pqamqirq8sbOHBg3BMPWdFNNyJn4pZbsVEKJh5jcLOU2fDCNho8eHDTTTfdVDp48OAxffr0Oa5r166Rz3/+83FfBmxFN52IDAWexcbhmsQ4Cfil7xCZqry8PP/FF1/ssXbt2qWlpaUf1tbW5j3wwAPWvZA1RLoCfwH7SGgSarYIX/MdIhO98MIL3Y488siG/v37h9u3b68XXHBB5Ztvvtkl3uNa0U0HInm4ycdH+Y5istK9IrYCdKwGDRrU+P7773eprq7Oi0ajvPrqq11HjhxZH+9xbchYergbmOE7hMla+bgJciaostZ3mDZrxRCvRJoyZcqeGTNmVIwZM2ZkQUEBo0ePrr311lvL4z2uFV3fRC4F7vAdw2S9nsDzIpykSrXvMJli7ty5W+fOnZvQMc/WveCTyGDgf33HMDljFPAb3yFynRVdX0TycUvsdPMdxeSUi0SY5TtELrOi68//AJN8hzA56Wc2HaQ/VnR9EDkZ+LbvGCZndQN+K2JXPPpgRTfV3Hjcx7ErzoxfZ+KW/TEpZkW3FUSkg4i8IyIfiMhycfPbttU9uElJjPHtByI2NjzVbMhY6zQAU1S1RkQKgddF5P9UdVFMRxGZCMxJRkBj2qAD8JgIE1UJ+w7TEhESOrWjKi2O+7377rv7PProo0WqyqxZs8rvvPPOsnhf11q6raBOTfDPwuAW25yabrTCg9h7btLLOOBG3yHS0bvvvtvh0UcfLXr//fdXrly5cvlLL73UY+nSpXHPi2IFoJVEJF9ElgBlwHxVfTvGQ9wEHJ/4ZCYxNuO6OUcCo4H7g8d3AWcBw4P7ioPsnw+MDW7nN3v8CtyEX82vf7kbeD5RwRPhLhEO8x0i3SxdurTjuHHjarp27RotLCzklFNOqX766ad7xHtcK7qtpKoRVR0LDAQmiMixrd5ZpB/uN82krQLgPmAlsAj4BbAC+CEwFVgT3P/wIPt3BJYEt78Ej33Y7P6fQBWwDXgHmJnw7yAOPbGfz08ZO3Zs3dtvv921tLQ0v7q6Om/+/PndN2/e3C7e41rRjZGqVgKvAefEsNuPsYsg0lw/3CdtgK64Fm8JrkV6ZfD4lcCfYzhmIVAHRIFGXGv4TuA7CcibcHNE+IzvEOlk3Lhx9TfffHPplClTjj7zzDOHjxo1qragIP7TYFZ0W0FEikSkR/B1R2AasKqVO58MfDF56UzibQD+BUwEtuMKMsH9wc6j1AMn4qaw3VuYRwJH4or5pcBa3KmAtOxlygd+4jtEurnlllt2rFixYuXixYtX9+rVKzJ8+HCbZSxF+gG/E3cyLA94RlX/2sp9D/Z51KSlGuAiXP2J5cPJJqA/sA6YAnwGGMon69gM4FfA94APcH3E18YfOXGmiPB5VZ7zHSRdlJSUFAwYMCC8Zs2adi+++GKPd955p3WNrUOwotsKqvohbWmeiEwHTk94IJMkTbiCewXw+eCxvrh+2H7B/cHWe+wf3A8BzsC1lIc2e/55XEt4D7AMeAb3o3EF0ClR30Ai3CvCC6rEvRZYorVmiFeinX/++UMrKysLCgoK9Cc/+cmmoqKiSLzHtKKbLCKCLZ+eQRS4GtclcGuzx88Hfgd8M7g/0AmwClzhbA/sAN4Abmv2fBNuNMRfcSfk9l59u7evN62K7mDgS9hsZAC89957qxN9TOvTTZ4LceOHTEZ4A3gMeJV9Q7/+hiu283FDxuYH/wZYDFwTfL0S14o9Djfs7Jt8chGQX+BOwnXCDR9TXPfDKUDcI5CS4XYRqw3JYi3dZHCt3Lt8xzCxOJWDX+/yygEeOxF4OPh6ErD0EMduvkSZ4BZ7TmvH4PpXnvUdJBvZX7PkOAds+I3JaN/yHQCIRqPRtJ4JLcgXjWUfK7rJYauvmkw3ToSzPWdYVl5e3j1dC280GpXy8vLuuDOjrWbdC4kmMgr4rO8YxiTAHcDffb14OBy+prS09OHS0tJjSc8GYhRYFg6Hr2lxy2ZENbZ5W0wLRH6FzSSWdAPYsngrA070nSMHTFLlLd8hskk6/vXIXCKHAf/hO4YxCXSd7wDZxopuYs3BzXxiTLa4WISuvkNkEyu6iXW17wDGJFhn3MQRJkGs6CaKm9hmaIvbGZN5rvIdIJtY0U2cL/kOYEySnCLC0b5DZAsruong1k2zj2Amm1lrN0Gs6CbGOUBv3yGMSaJZIuT7DpENrOgmRkq6FuqBCbhpVUYDof2e/yrQ5SD7PsG+aVzG4v7jl+CWOT4HOBZ4oNn2c3CTExoT6A9M9h0iG1jRjZdIB+C8VLxUe9wcWB/gCuZLuNW8wM15VXmIfa9g3wpejwGDcMX378AJuFW8Hgq2/QB3qU1arm9gfJrhO0A2sKIbv8mkaEJUYV9Ltim4CRABvgHc08rjPAVcFny9dxWvcLPnv02aruJlfDu/5U1MS6zoxu/cVL5YBNdC7YNb7GUi8HPcb0O/Q+zX3NPsK7pnAaXBcW7DrWN7AvvWQTCmmSGD+jWM9B0i09mEN/FLadHNx3URVOJmSV8I/AG3PHFrvI1rlu9dP74AeDL4ugk4G1d4b8Wt+jULa97kNq0e3K9xxRXTdjbNmVE+7Ig+TdPhxJW+U2UyK7rxEBkCfsYv9sCtxPUP3Bqzw4LHa4Ov1x5kv9+zr5W7vwdw6xu8BbTDtYhPxopurmlfGF13+nHVm68/v7z7eZOqRrcr0InNnv4scK+vbNnAim58UtrKLcf1wfbA9cO+DNyO6x7YqwsHL7hRXKt44QGeq8Ct4DUP19LNw/UXx73etMkAWjegd9OyL5y5q+66meWDjx7YMAS3wuaBnMaCxR2YfKL9aLSRFd34pHSS5224lmgEV0Av5dDDJv6CG9Ww96TYQmAgB/5t+g7wP7hCezZuVa/PANcnIrhJOwX50c0njdqz7rrzyztfdHrFsR3b6/hW7toBOA23YFyricgtuEXlFLe20VWqmpOF2+bTjYdIGVDkO0Yusvl0Y6WNfXqEl3/+9IrdN8wsP3LM0LrBcRzsh0w+sdXL+YjIAOB1YJSq1onIM8DfVPW3cWTIWNbSbSuRwVjBNWksP0+3HT+8du0108vbXTZ116hunaOJGno9oQ37FAAdRaQJdy53a4KyZBwrum03seVNjEkljfTsGlk+Y1JlxQ0zy/udNGrP0bR+JGEsTmDBYmHyia36mKyqJSJyL25ATB0wT1XnJSFXRrCi23Zt+WtvTEKJ6I7Rg+pWX3Xuzrwrz9kx8rBukTEpeNnuwHDgo9ZsLCI9gZnAYNxoxz+IyJdU9fHkRUxfVnTbzoqu8UC1a6foynMmVJXfMLO86PTjqkfk53GKhyDjaWXRBaYB61W1HEBEngMmAVZ0TSuJFADjfMcwuUHQqmEDG1bMOntn9Jrp5Ucf3is8yncm3IWRT7Ry203ASSLSCde9MBU3sCYnWdFtm2HYWmgmiTq2j66ZcvzukhsuKO919viq0QX5nOw7036ObXkTR1XfFpFngfdx03z8i33zK+UcK7ptM9x3AJNttOaovo0rLp+2q2HOjPJhgw5vHE56/5y1uugCqGqIT89GmpNswpu2GdbyJsYcSPNZkY9uGlh03cZn7vr4X/Xz32+34emlE75/bclp/XrV9PtC8bcYdvmFTLxhNhu2udFVbyz9gDFfvozx181i7ZbNAFRWV3P2N76Kh/H2A1mwuHuqXzQbWNFtm3RugZi0pfX9erH4+hmPv77y0Sc2Nr78aGH/3m8cdUTRwuPbF2q7vVv9+m/P07NLN9Y++Sduufhybn/oZwDc98zj/PE7P+L7197Eg395FoC7H3uYO66YjYj4+IbSoW8541j3QttYS9e0SkG+bhk/Ys+6OeeVd7zkzIrRnTtE/30VXW19mKZw+FMF8/k3FnLX7GsBuHjyFL5y/z2oKoUFBdQ1NFBbX09hfgEfl2yhpLycyWNPSO03tc9RuPmRTAys6LaNFV1zENrUu3t4+YWnVVbdMLNs4PHD64biprz4t0gkwglz/oO1JVu46cJLmDjqk92jJeVlHFHUF4CCggK6d+nCzqoqvnX5bObc+306tm/PY3cU818P3s/dV3udHWOAzxfPVFZ0Y+WGix3pO4ZJH3l5uv24obVrrpm+o+CKabtGde8SGXuo7fPz81ny6yeprK7mwm9/g2Xr1nLskH1/x5VP98+KwNjhx7Dowd8AsPCD9+nfuwhV5QvF36Iwv4D7bvwafXsdluDv7pAGtryJ2Z8V3dj1BlsVNbdptEeXyPLpJ1ftunFm2eEnjd5zdJ7QN9aj9OjalTPGnsBL77z1iaI7sKgvm8u3M7BPX8LhMFU1NfTqtu+clary3cce4enQ9/nK/fdQPPs6NpRu5afPPc33rrkxMd9i61jRbQMrurHr5TuAST0R3TnyqPrVs8/Zwexzdo4o6hH+TFuOU15ZQWF+AT26dqWuoZ6X33uH2y+b9Yltzp90Gr976UVOHj2GZxe8ypRx4z/R7/u7l/7K9JNOoWfXbtTWN5CXJ+Tl5VFbn9iZElW1LqyRnbXR+qrKcPWe7Y27GrY0bI+sr9+Wt65uS7s1dZsj8yYvavlA5hOs6MYupZ/fjC+qXTpGV312/O6yG2aW9T7z+OqR+XlMiveo23bu4Mof3EUkGiUajXLpmdM4b9Jp3PnILznxmJGcf8pkrv7cTP7j+yGGXX4hvbp14/d3fu/f+9fW1/O7v7/IvHt/DsCtl17ORXfeTrvCQp769ndb8V1pfVgjO+qi9VUV4Zo9ZY07GzY3lEU31G+VdfUlhevrtnbeUL+125aGsl67I3u64VqzB2vR9oj3/chFNp9urEQuAP7kO0auS858ulo1bEDDyi+dtTN8zXk7jh7Qu6lPYo+fHKra4AppQ1VluLqmrGlXw5b6suj6+q2sry9pt84V0q5bGsp6VUVqEjm2doeG1KY3jZG1dGNnLd0s0qFddO0ZY6tLbphZ1v3ciVXHFhZwku9MAKraGCGyozbSUFkVrnGF1LVIWVe3td36+pJO64MWaWW4ujtuJEGqRxP0lGLJ05BGU/y6Gc2Kbuys6GY03XNkn8blX5y6q+G6GeVDhvRvHEaKhgC6QhrdWRdpqKwMV9eUN1XU/7uQ1pe0W1dX0nFj/bZuWxrKDtsV3t0d6B/c0lU+0Bmo9h0kk1jRjV1X3wFMbAoLohsnjd6z8fqZZZ0vOLXy2A7tNGHTcqpqU1BIK6oiNTXljRUNWxq2hzfUl8r6+pJ2H9dt6bixflvXLQ1lh+0MV/XATSqejInFfbGRPDGyohs7+yFLe9pweK+mZRdPrthz/czyo0YPqj8Kd/VU6/ZWDUeI7qiPNlRUhWv2lDdW1G9pLAtvrC+VdXUlhevqt3Tc4FqkvXY2VfZQOBx3y0U2lUCMrOjGzstF7ubQCvK15ISj96y79rwd7b8wZdfoLh2jn7g2VlXDUaI766INFbvDe2rKmyoatjSUNW2s3xYU0pKOG+q3ddvcsL3nzqbKnjleSGNhjZAYWdGNnf1lTwP5Eo306tK05IwJ28uunL6lw4mjKzqVN1XK1oay2sd2lS5eV7+lYH3d1o4b6rd229SwveeOpopeCn0h9osYzCHZ70OMrOjGzlq6nt13Mm+WnzXyqMa82n7PoXnP7camXfHHim6MrOjGzoquJzs7UnnKl1mxuohJsMd3HONY0Y2RFd3YWdH14NmRvP/FS+gXScBVYSahrE83RlZ0Y9foO0AuqSug/nNX8PZrgzgdsT94aSjsO0CmsaIbu92+A+SKN49g1dRZFNYXMtl3FnNQu3wHyDRWdGNnV98kWViIfPkCXn9sDJMQCn3nMQdVqyFN7NRmOcCKbuwqfQfIZqsPY9NJ11BZ2dFatxnAWrltYGceY2c/aEny7TN5fcRX6FXZkTG+s5hW2ek7QCaylm7s7ActwUo7Uz7xWtZv6sGpvrOYmNjvQhtYSzd25b4DZJNHxvLOgK/Dph4kbBIakzJWdNvAWrqx2wJEsT9YcaluR82Zs1nyXn9r3WYwK7ptYEU3VqpNiGzBVgRus78PZemMy+nWlG8FN8OV+g6Qiazots06rOjGrDGPpksu5c2/HMOpiF3JlAVW+w6Qiazots064AzfITLJkr58fOrVNO5pZ0PBssgq3wEykRXdtlnvO0CmiILefC7//PkEJiB08J3HJIxiLd02saLbNut8B8gEm7qxbfwctpV14XTfWUzCbdKQ1vkOkYnsDHzb2MeqFvz4ZN4cdAsdy7owzncWkxT2O9BG1tJtm2W42cba+Q6Sbio6UDXpapavKrIpGLPcSt8BMpW1dNtCtRFY6jtGunl2JO8X3UatFdycYC3dNrKWbtstBk5ocascEMx5+85rgzjN5rzNGR/4DpCprKXbdot9B0gHbx7Bql63U/LaYJtkPIfUAu/5DpGprKXbdjlddCNC9MszWfjocZxic97mnEUa0ibfITKVFd22WwbUQ+6NPf2oF5snXktFZUe7QCRHLfQdIJNZ90JbqYaB133HSLU7z+D1Y75KD5vzNqdZ0Y2DtXTjMw+Y5jtEKmzvzI6J1/LxRpvzNtc1Aot8h8hk1tKNzzzfAVLhkbG80//r6MYeTPSdxXi32K5Ei4+1dOPzIW56u8N9B0mG6nbUTLmSfy0ewGm+s5i0YV0LcbKWbjxUFZjvO0YyzB/C0sNuZ6cVXEs+Cl4AAAnCSURBVLOfv/kOkOms6MYvq7oYGvNomvlFFnz2PxjVlM9RvvOYtLINeMN3iExn3Qvx+z8gTBa8lzbnrWnBcxrSqO8Qmc5auvFS3UmGdzFEQf/zHBYefz3997RjpO88Jm39wXeAbJDxrbM08RRwru8QbbGpG9smzGHrdpvz1hzaduCfvkNkA2vpJsafgYwbRjP3JDfn7fYuNnGPaZF1LSSItXQTQbUakb8Cl/iO0hoVHag65WqWr7QpGE3rPes7QLawlm7iPOU7QGs8N4J/Fd3GHiu4JgbbgQW+Q2QLa+kmzt+AXUAv30EOpK6A+umX8/Y/bApGE7uHNKQR3yGyhbV0E0W1Afi17xgH8tZAVve6nS3/GMJkK7gmRmHgl75DZBNxF1WZhBA5CvgYyPcdBf495+0/Hz2OkxFbz820yR80pJf6DpFNrHshkVQ3IvICcIHvKB/1YvNJ11JR0dEudDBx+ZnvANnGuhcSz/sP6d45bytszlsTnw80pDY2N8GspZtoqq8ishwYneqX3t6ZHSddw8cbetqctyYhfuE7QDaylm5y3J/qF/xNMOfthp42561JiArgCd8hspGdSEsGkXbAR5D8WbpqCtkzZTbvv2tTMJrEulNDerfvENnIWrrJoNoIfDfZLzN/CEt7fZMdVnBNgpUDc32HyFbW0k0WkQJgFTA00YduzKPp0kt48/kRnIqkx/A0k1Vu1ZBa0U0SO5GWLKphRO4GfpvIw37Ql3WnfpmGmvY2FMwkxWbgAd8hspl1LyTX48DqRBxo75y3Y6+nX017m/PWJM13NKQNvkNkM2vpJpNqBJE7gafjOczmbpROuJaS0q42561Jqo9I8Ccz82nW0k021WeA19q6+/0TeeuoW2hf2tXmvDVJF9KQhn2HyHbW0k2NrwBLiOH9tjlvTYq9RZyfyEzrWEs3FVSXE8PlwTbnrUmxMHC9hmwoUyrYkLFUEemGG0LW72Cb1OfTcN4VLHrF5ryNTxR4COgKXAG8DSzCXWP1DaDzQfabB6wBFBiCW/UugpuefjcwHpgQbPuX4N8H/d/MKD/WkH7dd4hcYS3dVFHdjfuVP6C3BrK65zfZ/IrNeRu/RUDvZv8+EpgFdD/EPptwg6VuAG4EtgIbgLVA/+Dx94JtS3GFOTsK7mYg5DtELrGim0qqTwAvN38oIkRnz2TBpKsZXF/IME/JskcVrrU6rtlj/YCeLewnuA/ZkWb3XXAzIzfhWs97vQqcmaC8/s3RkNb4DpFL7ERa6n0ZWAZ0szlvk+Al4Cwg1pGmRwCDgHuDf08AinCLL30APAycgusg6g90S0BW/x7VkL7kO0SusaKbaqqbEbm5eDJX33UGxyEc4TtS1liN66/tD6yPcd+dwA7g1uDfj+G6FwYBFwePRYLHL8MV9yrgOGBEHJn9KQVu8R0iF1n3gg+qv73rTMoQuvqOklU24wrvXNyC4euBP7Zy31XAQKB9cBsGbNlvm3eBscHj+cAlwMK4U/sQBWZpSHf5DpKLrOj6cy2f/rU28ZgGfB3XfrsYGAxc1Mp9u+NatpHgthHXvbBXHe56reNwfbx7T3Vm5qUE39WQzvcdIldZ0fUkaGXM4pOnaEwyLALuww37ehB4Pni8pNnXo3D9tw8Gt77AMc2OsQA4HVdsh+JGNzwIGXid4CtAse8QuczG6XomxXIXNmTHpMY24HgN6XbfQXKZtXT9Kwae8x3CZL0IcJkVXP+s6HoWXHo5Czc3gzHJcqeGdIHvEMaKblrQkO4BZgJlvrOYrPRn4Ae+QxjHim6a0JBuAi4EGn1nMVnlDVy3gp28SRNWdNOIhvRN4DrfOUzWWAnM0JDW+w5i9rGim2Y0pL8Ffug7h8l4JcA5GtIK30HMJ9mQsTQlxfJT4Ku+c5iMVAWcpiFd6juI+TRr6aYpDel/Ar/yncNknAbgAiu46cuKbnq7AfiN7xAmYzQCX9SQvuY7iDk4615Ic1IsecCjuDUQjDmYOuBCDenffQcxh2ZFNwNIseQDTwKX+s5i0tJu4DwN6T99BzEts+6FDKAhjeBauo/6zmLSzi5gqhXczGEt3QwjxVIM3Ok7h0kLpcBZGtJlvoOY1rOim4GkWK7CjWwo9J3FeLMRV3DX+A5iYmNFN0NJsUzDrYuQHat1mVgsAC7RkJb7DmJiZ326GUpD+jJuqcTNvrOYlPoFMM0Kbuaylm6Gk2Lpj5uPd6LvLCapGoAbNaSP+A5i4mMt3QynId0KnArcA9hf0Oy0FZhsBTc7WEs3i0ixnI0bVtbHdxaTMG8CF2lIS30HMYlhLd0sElyNNBa3+KDJbI3AfwOnW8HNLtbSzULBpcPfwq2/lu85jondEuBKDemHvoOYxLOim8WkWCYCDwFjfGcxrRLGLatzt4a0yXcYkxxWdLOcFEsB8DXgLqCz3zTmEFbgWreLfQcxyWVFN0dIsRwB/Ay3AKZJH7XA/wN+aMvq5AYrujlGiuV8XPE90neWHKfAY8AdGtIS32FM6ljRzUFSLJ2Bb+K6Hbp4jpOL/gncal0JucmKbg6TYukN3AbcBHTyHCcXrANu05D+0XcQ448VXYMUS19cy/d6oIPnONloHXAf8LCGtNF3GOOXFV3zb8E8DncA1wLtPMfJBu/hTpI9G0xEb4wVXfNpQfG9BpgDDPAcJxPNA+7RkNqVgeZTrOiagwrWZpuBW5X4LED8JkprdcAfgB9rSD/wHcakLyu6plWkWIYB1wFXAYd5jpNOFgG/AX6vId3tO4xJf1Z0TUykWDoA5wKX4FrBuTjk7CPgKeBJDelHvsOYzGJF17RZUIDPwV3lNh0o8psoaSLAYlxf7Z81pO97zmMymBVdkxDBzGYTgfOA04ETyezhZxtxRXYe8IqGtMJzHpMlrOiapJBiaQeMw63jNim47+s11MGFgZW4KRXfBuZbt4FJFiu6JmWkWIbgWsPHAMOBYcF9zxTGqAI+wBXYvffLNaQNKcxgcpgVXeOdFEsv9hXhwbgi3APovt99D9z0lGGgCbe6wv73dbg1xbYCJfvfa0irkv79iGwAqnF9wWFVPTHZr2kyhxVdYxIsKLonquoO31lM+rE10owxJoWs6BqTeArME5H3RGSO7zAmvRT4DmBMFjpFVbeKSB9gvoisUtWFvkOZ9GAtXWMSTFW3BvdlwJ+ACX4TmXRiRdeYBBKRziLSde/XwGeBZX5TmXRi3QvGJFZf4E8iAu7360lVfclvJJNObMiYMcakkHUvGGNMClnRNcaYFLKia4wxKWRF1xhjUsiKrjHGpJAVXWOMSSErusYYk0JWdI0xJoWs6BpjTApZ0TXGmBSyomuMMSn0/wFxsexm0sqA0gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "# define the pie drawing function of probability analysis\n",
    "\n",
    "prb = output.asnumpy()\n",
    "\n",
    "def plot_pie(prbs):\n",
    "    dict1 = {}\n",
    "    # remove the negative number and build the dictionary dict1. The key is the number and the value is the probability value\n",
    "    for i in range(10):\n",
    "        if prbs[i] > 0:\n",
    "            dict1[str(i)] = prbs[i]\n",
    "\n",
    "    label_list = dict1.keys()\n",
    "    size = dict1.values()\n",
    "    colors = [\"red\", \"green\", \"pink\", \"blue\", \"purple\", \"orange\", \"gray\"] \n",
    "    color = colors[: len(size)]\n",
    "    plt.pie(size, colors=color, labels=label_list, labeldistance=1.1, autopct=\"%1.1f%%\", shadow=False, startangle=90, pctdistance=0.6)\n",
    "    plt.axis(\"equal\")\n",
    "    plt.legend()\n",
    "    plt.title(\"Image classification\")\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "print(\"The probability of corresponding numbers [0-9] in Figure 1:\\n\", list(map(lambda x:1/(1+np.exp(-x)), prb[0])))\n",
    "plot_pie(prb[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上就是这次手写数字分类应用的全部体验过程。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}