{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 量子神经网络在自然语言处理中的应用\n",
"\n",
"[](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r1.7/mindquantum/zh_cn/mindspore_qnn_for_nlp.ipynb) [](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r1.7/mindquantum/zh_cn/mindspore_qnn_for_nlp.py) [](https://gitee.com/mindspore/docs/blob/r1.7/docs/mindquantum/docs/source_zh_cn/qnn_for_nlp.ipynb)\n",
"\n",
"## 概述\n",
"\n",
"在自然语言处理过程中，词嵌入（Word embedding）是其中的重要步骤，它是一个将高维度空间的词向量嵌入到一个维数更低的连续向量空间的过程。当给予神经网络的语料信息不断增加时，网络的训练过程将越来越困难。利用量子力学的态叠加和纠缠等特性，我们可以利用量子神经网络来处理这些经典语料信息，加入其训练过程，并提高收敛精度。下面，我们将简单地搭建一个量子经典混合神经网络来完成一个词嵌入任务。\n",
"\n",
"## 环境准备\n",
"\n",
"设置系统所使用的线程数，当您的服务器CPU较多时，如果不设置，系统默认调用所有CPU，反而会导致模拟变慢甚至卡住。"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ['OMP_NUM_THREADS'] = '1'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"导入本教程所依赖模块"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import time\n",
"from mindquantum.core import QubitOperator\n",
"import mindspore.ops as ops\n",
"import mindspore.dataset as ds\n",
"from mindspore import nn\n",
"from mindspore.train.callback import LossMonitor\n",
"from mindspore import Model\n",
"from mindquantum.framework import MQLayer\n",
"from mindquantum import Hamiltonian, Circuit, RX, RY, X, H, UN"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"本教程实现的是一个[CBOW模型](https://blog.csdn.net/u010665216/article/details/78724856)，即利用某个词所处的环境来预测该词。例如对于“I love natural language processing”这句话，我们可以将其切分为5个词，\\[\"I\", \"love\", \"natural\", \"language\", \"processing”\\]，在所选窗口为2时，我们要处理的问题是利用\\[\"I\", \"love\", \"language\", \"processing\"\\]来预测出目标词汇\"natural\"。这里我们以窗口为2为例，搭建如下的量子神经网络，来完成词嵌入任务。\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"这里，编码线路会将\"I\"、\"love\"、\"language\"和\"processing\"的编码信息编码到量子线路中，待训练的量子线路由四个Ansatz线路构成，最后我们在量子线路末端对量子比特做$\\text{Z}$基矢上的测量，具体所需测量的比特的个数由所需嵌入空间的维数确定。\n",
"\n",
"## 数据预处理\n",
"\n",
"我们对所需要处理的语句进行处理，生成关于该句子的词典，并根据窗口大小来生成样本点。"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"def GenerateWordDictAndSample(corpus, window=2):\n",
" all_words = corpus.split()\n",
" word_set = list(set(all_words))\n",
" word_set.sort()\n",
" word_dict = {w: i for i, w in enumerate(word_set)}\n",
" sampling = []\n",
" for index, _ in enumerate(all_words[window:-window]):\n",
" around = []\n",
" for i in range(index, index + 2*window + 1):\n",
" if i != index + window:\n",
" around.append(all_words[i])\n",
" sampling.append([around, all_words[index + window]])\n",
" return word_dict, sampling"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'I': 0, 'language': 1, 'love': 2, 'natural': 3, 'processing': 4}\n",
"word dict size: 5\n",
"samples: [[['I', 'love', 'language', 'processing'], 'natural']]\n",
"number of samples: 1\n"
]
}
],
"source": [
"word_dict, sample = GenerateWordDictAndSample(\"I love natural language processing\")\n",
"print(word_dict)\n",
"print('word dict size: ', len(word_dict))\n",
"print('samples: ', sample)\n",
"print('number of samples: ', len(sample))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"根据如上信息，我们得到该句子的词典大小为5，能够产生一个样本点。\n",
"\n",
"## 编码线路\n",
"\n",
"为了简单起见，我们使用的编码线路由$\\text{RX}$旋转门构成，结构如下。\n",
"\n",
"\n",
"\n",
"我们对每个量子门都作用一个$\\text{RX}$旋转门。"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"def GenerateEncoderCircuit(n_qubits, prefix=''):\n",
" if prefix and prefix[-1] != '_':\n",
" prefix += '_'\n",
" circ = Circuit()\n",
" for i in range(n_qubits):\n",
" circ += RX(prefix + str(i)).on(i)\n",
" return circ"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"q0: ──RX(e_0)──\n",
" \n",
"q1: ──RX(e_1)──\n",
" \n",
"q2: ──RX(e_2)──\n",
"\n"
],
"text/plain": [
"q0: ──RX(e_0)──\n",
" \n",
"q1: ──RX(e_1)──\n",
" \n",
"q2: ──RX(e_2)──"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"GenerateEncoderCircuit(3, prefix='e')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"我们通常用$\\left|0\\right>$和$\\left|1\\right>$来标记二能级量子比特的两个状态，由态叠加原理，量子比特还可以处于这两个状态的叠加态：\n",
"\n",
"$$\\left|\\psi\\right>=\\alpha\\left|0\\right>+\\beta\\left|1\\right>$$\n",
"\n",
"对于$n$比特的量子态，其将处于$2^n$维的希尔伯特空间中。对于上面由5个词构成的词典，我们只需要$\\lceil \\log_2 5 \\rceil=3$个量子比特即可完成编码，这也体现出量子计算的优越性。\n",
"\n",
"例如对于上面词典中的\"love\"，其对应的标签为2，2的二进制表示为`010`，我们只需将编码线路中的`e_0`、`e_1`和`e_2`分别设为$0$、$\\pi$和$0$即可。下面来验证一下。"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Label is: 2\n",
"Binary label is: 010\n",
"Parameters of encoder is: \n",
" [0. 3.14159 0. ]\n",
"Encoder circuit is: \n",
" q0: ──RX(e_0)──\n",
" \n",
"q1: ──RX(e_1)──\n",
" \n",
"q2: ──RX(e_2)──\n",
"Encoder parameter names are: \n",
" ['e_0', 'e_1', 'e_2']\n",
"Amplitude of quantum state is: \n",
" [0. 0. 1. 0. 0. 0. 0. 0.]\n",
"Label in quantum state is: 2\n"
]
}
],
"source": [
"from mindquantum.simulator import Simulator\n",
"from mindspore import context\n",
"from mindspore import Tensor\n",
"\n",
"n_qubits = 3 # number of qubits of this quantum circuit\n",
"label = 2 # label need to encode\n",
"label_bin = bin(label)[-1: 1: -1].ljust(n_qubits, '0') # binary form of label\n",
"label_array = np.array([int(i)*np.pi for i in label_bin]).astype(np.float32) # parameter value of encoder\n",
"encoder = GenerateEncoderCircuit(n_qubits, prefix='e') # encoder circuit\n",
"encoder_params_names = encoder.params_name # parameter names of encoder\n",
"\n",
"print(\"Label is: \", label)\n",
"print(\"Binary label is: \", label_bin)\n",
"print(\"Parameters of encoder is: \\n\", np.round(label_array, 5))\n",
"print(\"Encoder circuit is: \\n\", encoder)\n",
"print(\"Encoder parameter names are: \\n\", encoder_params_names)\n",
"\n",
"# quantum state evolution operator\n",
"state = encoder.get_qs(pr=dict(zip(encoder_params_names, label_array)))\n",
"amp = np.round(np.abs(state)**2, 3)\n",
"\n",
"print(\"Amplitude of quantum state is: \\n\", amp)\n",
"print(\"Label in quantum state is: \", np.argmax(amp))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"通过上面的验证，我们发现，对于标签为2的数据，最后得到量子态的振幅最大的位置也是2，因此得到的量子态正是对输入标签的编码。我们将对数据编码生成参数数值的过程总结成如下函数。"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"def GenerateTrainData(sample, word_dict):\n",
" n_qubits = np.int(np.ceil(np.log2(1 + max(word_dict.values()))))\n",
" data_x = []\n",
" data_y = []\n",
" for around, center in sample:\n",
" data_x.append([])\n",
" for word in around:\n",
" label = word_dict[word]\n",
" label_bin = bin(label)[-1: 1: -1].ljust(n_qubits, '0')\n",
" label_array = [int(i)*np.pi for i in label_bin]\n",
" data_x[-1].extend(label_array)\n",
" data_y.append(word_dict[center])\n",
" return np.array(data_x).astype(np.float32), np.array(data_y).astype(np.int32)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(array([[0. , 0. , 0. , 0. , 3.1415927, 0. ,\n",
" 3.1415927, 0. , 0. , 0. , 0. , 3.1415927]],\n",
" dtype=float32),\n",
" array([3], dtype=int32))"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"GenerateTrainData(sample, word_dict)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"根据上面的结果，我们将4个输入的词编码的信息合并为一个更长向量，便于后续神经网络调用。\n",
"\n",
"## Ansatz线路\n",
"\n",
"Ansatz线路的选择多种多样，我们选择如下的量子线路作为Ansatz线路，它的一个单元由一层$\\text{RY}$门和一层$\\text{CNOT}$门构成，对此单元重复$p$次构成整个Ansatz线路。\n",
"\n",
"\n",
"\n",
"定义如下函数生成Ansatz线路。"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"def GenerateAnsatzCircuit(n_qubits, layers, prefix=''):\n",
" if prefix and prefix[-1] != '_':\n",
" prefix += '_'\n",
" circ = Circuit()\n",
" for l in range(layers):\n",
" for i in range(n_qubits):\n",
" circ += RY(prefix + str(l) + '_' + str(i)).on(i)\n",
" for i in range(l % 2, n_qubits, 2):\n",
" if i < n_qubits and i + 1 < n_qubits:\n",
" circ += X.on(i + 1, i)\n",
" return circ"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"q0: ──RY(a_0_0)────────●────────RY(a_1_0)───────\n",
" │ \n",
"q1: ──RY(a_0_1)────────X────────RY(a_1_1)────●──\n",
" │ \n",
"q2: ──RY(a_0_2)────────●────────RY(a_1_2)────X──\n",
" │ \n",
"q3: ──RY(a_0_3)────────X────────RY(a_1_3)────●──\n",
" │ \n",
"q4: ──RY(a_0_4)────RY(a_1_4)─────────────────X──\n",
"\n"
],
"text/plain": [
"q0: ──RY(a_0_0)────────●────────RY(a_1_0)───────\n",
" │ \n",
"q1: ──RY(a_0_1)────────X────────RY(a_1_1)────●──\n",
" │ \n",
"q2: ──RY(a_0_2)────────●────────RY(a_1_2)────X──\n",
" │ \n",
"q3: ──RY(a_0_3)────────X────────RY(a_1_3)────●──\n",
" │ \n",
"q4: ──RY(a_0_4)────RY(a_1_4)─────────────────X──"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"GenerateAnsatzCircuit(5, 2, 'a')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 测量\n",
"\n",
"我们把对不同比特位上的测量结果作为降维后的数据。具体过程与比特编码类似，例如当我们想将词向量降维为5维向量时，对于第3维的数据可以如下产生：\n",
"\n",
"- 3对应的二进制为`00011`。\n",
"- 测量量子线路末态对$Z_0Z_1$哈密顿量的期望值。\n",
"\n",
"下面函数将给出产生各个维度上数据所需的哈密顿量（hams），其中`n_qubits`表示线路的比特数，`dims`表示词嵌入的维度："
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"def GenerateEmbeddingHamiltonian(dims, n_qubits):\n",
" hams = []\n",
" for i in range(dims):\n",
" s = ''\n",
" for j, k in enumerate(bin(i + 1)[-1:1:-1]):\n",
" if k == '1':\n",
" s = s + 'Z' + str(j) + ' '\n",
" hams.append(Hamiltonian(QubitOperator(s)))\n",
" return hams"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[1.0 [Z0] , 1.0 [Z1] , 1.0 [Z0 Z1] , 1.0 [Z2] , 1.0 [Z0 Z2] ]"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"GenerateEmbeddingHamiltonian(5, 5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 量子版词向量嵌入层\n",
"\n",
"量子版词向量嵌入层结合前面的编码量子线路和待训练量子线路，以及测量哈密顿量，将`num_embedding`个词嵌入为`embedding_dim`维的词向量。这里我们还在量子线路的最开始加上了Hadamard门，将初态制备为均匀叠加态，用以提高量子神经网络的表达能力。\n",
"\n",
"下面，我们定义量子嵌入层，它将返回一个量子线路模拟算子。"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"def QEmbedding(num_embedding, embedding_dim, window, layers, n_threads):\n",
" n_qubits = int(np.ceil(np.log2(num_embedding)))\n",
" hams = GenerateEmbeddingHamiltonian(embedding_dim, n_qubits)\n",
" circ = Circuit()\n",
" circ = UN(H, n_qubits)\n",
" encoder_param_name = []\n",
" ansatz_param_name = []\n",
" for w in range(2 * window):\n",
" encoder = GenerateEncoderCircuit(n_qubits, 'Encoder_' + str(w))\n",
" ansatz = GenerateAnsatzCircuit(n_qubits, layers, 'Ansatz_' + str(w))\n",
" encoder.no_grad()\n",
" circ += encoder\n",
" circ += ansatz\n",
" encoder_param_name.extend(encoder.params_name)\n",
" ansatz_param_name.extend(ansatz.params_name)\n",
" grad_ops = Simulator('projectq', circ.n_qubits).get_expectation_with_grad(hams,\n",
" circ,\n",
" None,\n",
" None,\n",
" encoder_param_name,\n",
" ansatz_param_name,\n",
" n_threads)\n",
" return MQLayer(grad_ops)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"整个训练模型跟经典网络类似，由一个嵌入层和两个全连通层构成，然而此处的嵌入层是由量子神经网络构成。下面定义量子神经网络CBOW。"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"class CBOW(nn.Cell):\n",
" def __init__(self, num_embedding, embedding_dim, window, layers, n_threads,\n",
" hidden_dim):\n",
" super(CBOW, self).__init__()\n",
" self.embedding = QEmbedding(num_embedding, embedding_dim, window,\n",
" layers, n_threads)\n",
" self.dense1 = nn.Dense(embedding_dim, hidden_dim)\n",
" self.dense2 = nn.Dense(hidden_dim, num_embedding)\n",
" self.relu = ops.ReLU()\n",
"\n",
" def construct(self, x):\n",
" embed = self.embedding(x)\n",
" out = self.dense1(embed)\n",
" out = self.relu(out)\n",
" out = self.dense2(out)\n",
" return out"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"下面我们对一个稍长的句子来进行训练。首先定义`LossMonitorWithCollection`用于监督收敛过程，并搜集收敛过程的损失。"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"class LossMonitorWithCollection(LossMonitor):\n",
" def __init__(self, per_print_times=1):\n",
" super(LossMonitorWithCollection, self).__init__(per_print_times)\n",
" self.loss = []\n",
"\n",
" def begin(self, run_context):\n",
" self.begin_time = time.time()\n",
"\n",
" def end(self, run_context):\n",
" self.end_time = time.time()\n",
" print('Total time used: {}'.format(self.end_time - self.begin_time))\n",
"\n",
" def epoch_begin(self, run_context):\n",
" self.epoch_begin_time = time.time()\n",
"\n",
" def epoch_end(self, run_context):\n",
" cb_params = run_context.original_args()\n",
" self.epoch_end_time = time.time()\n",
" if self._per_print_times != 0 and cb_params.cur_step_num % self._per_print_times == 0:\n",
" print('')\n",
"\n",
" def step_end(self, run_context):\n",
" cb_params = run_context.original_args()\n",
" loss = cb_params.net_outputs\n",
"\n",
" if isinstance(loss, (tuple, list)):\n",
" if isinstance(loss[0], Tensor) and isinstance(loss[0].asnumpy(), np.ndarray):\n",
" loss = loss[0]\n",
"\n",
" if isinstance(loss, Tensor) and isinstance(loss.asnumpy(), np.ndarray):\n",
" loss = np.mean(loss.asnumpy())\n",
"\n",
" cur_step_in_epoch = (cb_params.cur_step_num - 1) % cb_params.batch_num + 1\n",
"\n",
" if isinstance(loss, float) and (np.isnan(loss) or np.isinf(loss)):\n",
" raise ValueError(\"epoch: {} step: {}. Invalid loss, terminating training.\".format(\n",
" cb_params.cur_epoch_num, cur_step_in_epoch))\n",
" self.loss.append(loss)\n",
" if self._per_print_times != 0 and cb_params.cur_step_num % self._per_print_times == 0:\n",
" print(\"\\repoch: %+3s step: %+3s time: %5.5s, loss is %5.5s\" % (cb_params.cur_epoch_num, cur_step_in_epoch, time.time() - self.epoch_begin_time, loss), flush=True, end='')\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"接下来，利用量子版本的`CBOW`来对一个长句进行词嵌入。运行之前请在终端运行`export OMP_NUM_THREADS=4`，将量子模拟器的线程数设置为4个，当所需模拟的量子系统比特数较多时，可设置更多的线程数来提高模拟效率。"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"scrolled": true,
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch: 25 step: 20 time: 0.336, loss is 3.154\n",
"epoch: 50 step: 20 time: 0.449, loss is 2.945\n",
"epoch: 75 step: 20 time: 0.325, loss is 0.226\n",
"epoch: 100 step: 20 time: 0.370, loss is 0.016\n",
"epoch: 125 step: 20 time: 0.377, loss is 0.002\n",
"epoch: 150 step: 20 time: 0.399, loss is 0.006\n",
"epoch: 175 step: 20 time: 0.370, loss is 0.166\n",
"epoch: 200 step: 20 time: 0.345, loss is 0.139\n",
"epoch: 225 step: 20 time: 0.350, loss is 3.355\n",
"epoch: 250 step: 20 time: 0.334, loss is 1.059\n",
"epoch: 275 step: 20 time: 0.339, loss is 0.035\n",
"epoch: 300 step: 20 time: 0.334, loss is 0.024\n",
"epoch: 325 step: 20 time: 0.344, loss is 0.010\n",
"epoch: 350 step: 20 time: 0.344, loss is 0.009\n",
"Total time used: 126.26282787322998\n"
]
}
],
"source": [
"import mindspore as ms\n",
"from mindspore import context\n",
"from mindspore import Tensor\n",
"context.set_context(mode=context.PYNATIVE_MODE, device_target=\"CPU\")\n",
"corpus = \"\"\"We are about to study the idea of a computational process.\n",
"Computational processes are abstract beings that inhabit computers.\n",
"As they evolve, processes manipulate other abstract things called data.\n",
"The evolution of a process is directed by a pattern of rules\n",
"called a program. People create programs to direct processes. In effect,\n",
"we conjure the spirits of the computer with our spells.\"\"\"\n",
"\n",
"ms.set_seed(42)\n",
"window_size = 2\n",
"embedding_dim = 10\n",
"hidden_dim = 128\n",
"word_dict, sample = GenerateWordDictAndSample(corpus, window=window_size)\n",
"train_x, train_y = GenerateTrainData(sample, word_dict)\n",
"\n",
"train_loader = ds.NumpySlicesDataset({\n",
" \"around\": train_x,\n",
" \"center\": train_y\n",
"}, shuffle=False).batch(3)\n",
"net = CBOW(len(word_dict), embedding_dim, window_size, 3, 4, hidden_dim)\n",
"net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')\n",
"net_opt = nn.Momentum(net.trainable_params(), 0.01, 0.9)\n",
"loss_monitor = LossMonitorWithCollection(500)\n",
"model = Model(net, net_loss, net_opt)\n",
"model.train(350, train_loader, callbacks=[loss_monitor], dataset_sink_mode=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"打印收敛过程中的损失函数值："
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA0m0lEQVR4nO3dfXyU9Zno/881kwQEA0SeEROMDxSIVQk1oK0Ptbrq2uJTq+C23d+pRfvr+e36656z9aHL2cVtj/vbbX/unvoS0Xa7PUcsVUS7Ht2KlopWEiBUJAERiCSEhwBxeBCUJDPX+eOeezKZmYRJMg/35L7erxevJHdm5v5mSK77e1/f7/f6iqpijDHGPwL5boAxxpjcssBvjDE+Y4HfGGN8xgK/Mcb4jAV+Y4zxmaJ8NyAd48aN02nTpuW7GcYYU1Dq6+sPq+r4xOMFEfinTZvGxo0b890MY4wpKCLSnOq4pXqMMcZnLPAbY4zPWOA3xhifscBvjDE+Y4HfGGN8xgK/Mcb4jAV+Y4D65hCPr9lJfXMo300xJusKYh6/MdlU3xzi7qdr6eiKUFIU4Jl75lJdUZbvZhmTNdbjN75X29ROR1eEiEJnV4TapvZ8N8mYrLLAb3xvbuVYSooCBAWKiwLMrRyb7yYZk1WW6jG+V11RxjP3zKW2qZ25lWMtzWOGvKwFfhH5OXAzcFBVqxK+91+AfwTGq+rhbLXBmHRVV5RZwDe+kc1Uzy+AGxIPisg5wHVASxbPbYwxphdZC/yquhb4KMW3/n/grwHb5d0YY/Igp4O7IvIVYK+qbs7leY0xxnTL2eCuiIwAHgauT/Pxi4BFAOXl5VlsmTHG+Esue/znAecCm0VkNzAV2CQik1I9WFWXqeocVZ0zfnzSBjLGGGMGKGc9flXdAkxwv44G/zk2q8cYY3Iraz1+EXkWWAdMF5FWEflWts5ljDEmfVnr8avqgtN8f1q2zm2MMaZ3VrLBGGN8xgK/Mcb4jAV+Y4zxGQv8xhjjMxb4jTHGZyzwG2OMz1jgN8YYn7HAb4wxPmOB3xhjfMYCvzHG+IwFfmOM8RkL/MYY4zMW+I0xxmcs8BtjjM9Y4DfGGJ+xwG+MMT5jgd8YY3zGAr8xxviMBX5jjPGZbG62/nMROSgiDXHH/lFE3heR90RklYiMydb5jTHGpJbNHv8vgBsSjq0GqlT1s8AHwINZPL8xxpgUshb4VXUt8FHCsddUtSv6ZS0wNVvnN8YYk1o+c/z/CXi1t2+KyCIR2SgiGw8dOpTDZhljzNCWl8AvIg8DXcAzvT1GVZep6hxVnTN+/PjcNc4YY4a4olyfUES+CdwMXKuqmuvzG2OM3+U08IvIDcD3gatU9WQuz22MMcaRzemczwLrgOki0ioi3wJ+CpQCq0XkXRFZmq3zG2OMSS1rPX5VXZDi8M+ydT5jjDHpsZW7xgD1zSEeX7OT+uZQvptiTNblfHDXGK+pbw5x99O1dHRFKCkK8Mw9c6muKMt3s4zJGuvxG9+rbWqnoytCRKGzK0JtU3u+m2RMVlngN743t3IsJUUBggLFRQHmVo7Nd5OMySpL9Rjfq64o45l75lLb1M7cyrGW5jFDngV+Y3CCvwV84xeW6jHGGJ+xwG+MMT5jgd8YY3zGAr8xxviMBX5jjPEZC/zGGOMzFviNMcZnLPAbY4zPWOA3xhifscBvjDE+Y4HfGGN8xgK/Mcb4TDb33P25iBwUkYa4Y2eJyGoR2RH9aFWxjDEmx7LZ4/8FcEPCsQeAN1T1AuCN6NfGGGNyKGuBX1XXAh8lHJ4P/Fv0838DbsnW+Y1Jl+23a/wm1/X4J6rqfgBV3S8iE3p7oIgsAhYBlJeX56h5xm9sv13jR54d3FXVZao6R1XnjB8/Pt/NMUOU7bdr/CjXgb9NRCYDRD8ezPH5jenB9ts1fpTrVM9vgG8Cj0Y/vpTj8xvTg7vf7spNrUi+G2NMjmRzOuezwDpguoi0isi3cAL+dSKyA7gu+rUxeffCplaW17Vw55PrWF7Xku/mGJNVWevxq+qCXr51bbbOacxA1Da1c6ozggJdEWXxSw1Mn1Rqg7xmyPLs4K4xuTK3ciyBQHeiJ6Jqg7xmSLPAbwwgaOzzQEBskNcMaRb4je/VNrUTjnR/rdr7Y40ZCizwG9+bWzmWuEwPaqkeM8RZ4DcGkLjIH7RUjxniLPAb36ttaicc7s7vRCKW6zFDmwV+43tzK8cSjOvxK1iqxwxpFviN71VXlLFkfhVFASEgUGKlG8wQl+uSDcZ40sKacqZPKqW2qZ25lWNt8ZYZ0izwG4NTntnq9Ri/sMBvfK++OcSCZevoiA7wPlffyrPftrr8ZuiyHL/xvdqmdjrjZvV0WF1+M8RZ4De+58zq6XmsbERJfhpjTA5Y4De+V11Rxp2f697eMyAQOtmRxxYZk10W+I0BZk0ZbdM5jW9Y4De+V98cYsnLjYQjSkCExTfPsoFdM6RZ4De+5264rji1+C3NY4Y6C/zG98pGlOCW54moDeyaoS8vgV9E/l8RaRSRBhF5VkSG56MdxoAzkOsu3BJsYNcMfTkP/CJyNvAXwBxVrQKCwF25bocxrrIRJbH9txTYvOcI9c2hfDZpUOqbQzy+ZmdB/wwmu/K1crcIOENEOoERwL48tcOYWI/fDf6rt7axdschnrmn8Fbv1jeHuPvpWjq6IpQUBQryZzDZl/Mev6ruBf4JaAH2A0dV9bVct8MY19zKsRQHe5Zl/rQzwtI3d+WvUQPkDlRHFDptBbLpRT5SPWXAfOBcYAowUkT+LMXjFonIRhHZeOjQoVw30/iNJJdnW721jeV1LXlozMDNrRxLSVGAoECxrUcwvcjH4O6XgA9V9ZCqdgIvAJcnPkhVl6nqHFWdM378+Jw30vhHbVM7XfG7rcd5tWF/jlszONUVZTxzz1y+d/10S/OYXqWV4xeRkcAnqhoRkQuBzwCvRgN3f7UAc0VkBPAJcC2wcQCvY0xGxE/nTHRj1eTcNiYDqivKThvw65tDtveAj6U7uLsW+EI0TfMGTqC+E7i7vydU1ToReR7YBHQBfwSW9fd1jMmUVNM3ReDeL1SysKY8xTMKmw0Am3RTPaKqJ4HbgP+hqrcCMwd6UlX9b6r6GVWtUtWvq+qpgb6WMYOVasHWgsvKeeCmGXloTfbZALBJO/CLyDycHv7/jh6zTVzMkNC472jSsaopo/PQktwoG1FCQJyCdDYA7E/pBv77gQeBVaraKCKVwJqstcqYHEqV3m9IcTEYCqwgnYE0A7+qvqmqX1HVfxCRAHBYVf8iy20zJidunz2VooS/hOfrW4fkytf4gnRqBel8K63ALyLLRWRUdHbPVmC7iPzX7DbNmNyorihjxb2Xc/HU7vROODw0c982z99A+qmemap6DLgFeAUoB76erUYZk2vVFWUs/vIshhcPjaDYW70em+dvIP0B2mIRKcYJ/D9V1U4R6WXmszGFyQ2KhT6/PXG65uKbZxE62RH7mdKZ52+GtnQD/5PAbmAzsFZEKoBj2WqUMblW3xxi5aZWBLht9lQAHl+zsyAvAPHTNTs6Iyx+qYGIqs3ZNzFpBX5V/RfgX+IONYvINdlpkjG5Vd8cYsGydXSEnZvYFRv3EBChK1yYC5zcPH5nVwQRIaLaY85+If0sJjvSHdwdLSI/cYumiciPgZFZbpsxOVHb1E5nuDtz2RXWgl7gFJ/HXzK/ygZzTZJ0Uz0/BxqAr0W//jrwrzgreY0paHMrxxIIQGKdtkJe4BSfx58+qbTgxy1MZqUb+M9T1dvjvv47EXk3C+0xJue2HzieFPQFuOL8cdz/pQs9HSzTKbZmg7kmUbqB/xMR+byqvg0gIlfgVNY0puClKr2sOJU5vRwwrdiaGah05/HfBzwuIrtFZDfwU+DerLXKmBzqrfTyi39szXFL+seKrZmBSrdkw2ZVvRj4LPBZVb0U+GJWW2ZMjiysKeeWS6YkHd/QHPJ02YbBrsK1Tdn9q18VNqOrd13fAx7LaGuMyZP2Eylq1iienv44mAVnlibyt8FsvZi8SakxBSpVuqcQZvRUV5Tx3WvO73fQtjSRvw0m8FvJBjNkLKwpZ9rYET2OXXXh+CHbC7Zibf7WZ6pHRI6TOsALcEZWWmRMHjz6yjZ2t5/scex37x9keV3LkNx+cajUJTID02fgV9XSXDXEmHz6j8YDScfCEWXxSw1Mn1Q6JAOjze/3r8GkegZMRMaIyPMi8r6IbItu62hM3lxyzpiUxyOqQyL/bTN4TLx87Zv7z8B/qOodIlICjDjdE4zJphHDUv8pFAULL/+9vK6FVxv2c2PVZBbWlNsMHpMk54FfREYBVwJ/DqCqHYDt/2bypr45xPoUvXoB7qieWlBBcnldCw+t2gLAWzsOAxA62RGbwfNpZ4Ql/97I4i/bXrt+lo9UTyVwCPhXEfmjiDwd3dKxBxFZ5FYDPXToUO5baXzB7Q3vPHQi6XvDigPcHq3NXygSy0+82rCfuZVjKQp0z77e3HqUBU/VWtrHx/IR+IuA2cAT0RXAJ4AHEh+kqstUdY6qzhk/fnyu22h8wp3PHi8YgIunjmbxzYXXK05cjzBr8ihqm9q5evqEHsdt7r6/5SPH3wq0qmpd9OvnSRH4jcmF+E1LggHh6ukT+P0Hh9iy9yjb2xoLbkaPO/X01Yb9zJo8il+s201HV4SigFAUFLqi+w7Y3H1/y3ngV9UDIrJHRKar6nbgWmBrrtthDPScz142ooQVG1pidwCFumPVwppypk8q5bHXP4jl9sMR5c7LnIuCu71kof1cJnPyNavn/wGeic7oaQL+rzy1w5hYAIzffhEgWIAzeqB73OJUZwTF2VBGBNY3tVM5/kzuveo8C/o+l5fAr6rvAnPycW5jUqltau8R9KFwSza44xaKM4hXOW4kOw+diP174/2D/PreeQX5s5nMyMsCLmO8Zm7lWAIJZQePnizMWcZzK8dSFAwgQFFRwOnuxwlHlBc2eXuvAZNdFviNiTprZEmPrzd6vB4/9LEiV527l0gkQtOhj5Oe17D3qOd/NpM9FviN7y2va+GrS9/h8Mc9e/iRaD1+r3Jz+T9+bTt3P909L7+2qZ2uiKI4G8hHUpRZ3LL3aI/nGH+xwG88J5d1ZeqbQ/zNi1tSBkeAshElqb/hAfE19U91RlgZTd/0KLkclKQUFmB1+H0uX7N6jEkp13VlapvaCfexs0TDvqNZO/dguStyO8JO7/75+lZuj07TXHzzrFi9npb2Eyxd29TjuQIEA1KQs5bM4FmP33hKrneGmls5lpJg75vJrdiwx7PpkOqKMr4655zYVnjhsPN+1TeHWPJyI3/YeZglLzfSdDi5HIVC0qCv8Q8L/MZTcr0zVHVFGc8umsfCmnLurinn+pkTe3zf6zNgbps9lWHFPd+vxItn27FPUz7XvVAY/7FUj/GUfOwMFb8hSX1ziDe2tfVI/3h5j9FU79f2A8cJiABKcVGAeZVj2dyanLJyUz31zSHbictnLPAbz8nXzlBuAPzyxVN48d19seNVU0bnvC39kXjhWvJyI10RJSDw5/OmcexUV+onirD9wHGWvNxotfp9xgK/MfQscxCf+g7g1LP3Oveite/IJ5zqdGoNRRSefvtDvviZCSmfEw5HeLVhf9KYigX+oc8CvzE4g8pubRuNy+0EAnh+5kv8TKiigBAICOHo/NSIaq8XLhFh1uRRbNj9EZ1dEavY6SMW+I3BCe7BgNCVMKG/KwLbDxz3bC+4vjmUVIXzizMmsub9g4QjSkCE0MnOlM8NR5RfrNvN4ptnETrZkdEcv40beJvN6jEGJ0++ZH4VqSY4rtjQkvP2pMPt6b+94zARdapwFhcFuO+q81gyv4pgtOe/62ByyQZwBq07OiOxXboyGfRTrSg23mGB35io6ZNKUwb+LR6taxOfnhLgivPH8cw9cwFnI5ZwtGxDX7OSIsAfdh7OaIDO9VoM038W+E3e5bJEQ19e2NRKJMXxiBIrh+AlZSNKYkFd6d528e6na3lrx+G0p6FmOkDnei2G6T/L8Zu8ynWJhr7a8dzGPb1+//DxUzlsTXpCJzsIiBO4Bae8ROhkR2xWT7qEzAbofKzFMP1jPX6TV70VGstHOxIHduN5cRGXW6sHiNXqKRtRQjBVVbY+XDdzYsYvuNUVZXz3mvMt6HuUBX6TV6mCV65TPvXNIfYe+YRAHwFzQumwHLYoPdUVZVw9vXuOfjgcIXSygyXzq1JW5OzNxeeMsQDtM3lL9YhIENgI7FXVm/PVjtNZXtfCz99u4tCJU5w8FY5OkXNuryMKQQHEmfsdECd4RaJ32sEAsXnhp/teX68XEOGCCWfyyC0XDbk/ULfQ2PK6Fqd+fDi3i4jiU019xcpZHly9W98c4nfbD8a+dvcIXt14oNcy06ls3nOE+ubQkPvdMr3LZ47/L4FtwKg8toFHX9nGL9ft5tOuSCzQukE3HEl9ix//RxWOmzaR+MfWFZdqTfd7qV4vjLJ1/3Fuf+IdJpSWcP+XprOwpjz9H9Ljbps9lZWbWvOyiCg+1dRX4Pfi6t0XNrXSFVdU6OoLx7P9wPGkEsyn89rWNn7/wSGe/baVa/CLvAR+EZkK/CnwQ+B72TyX22NvjS5lD8T1qFV7Bvb4ANyfHlMuHTzewUOrtrBs7S5+/LVLhsQfqjsYuHJTayz45moBkDsDpbMrgkjyAi6XFzdkSWzpuNJhvNqwf0CvZeUa/CVfPf7HgL8GSnt7gIgsAhYBlJcPrHd7yd/9liOf9CxQFd+jLmS7209yxxPv8MNbLxoyvf8XNrXS0RVxZteI0BXO/kyf+BkoZSNK+JsXt6TcmMWLG7IkFo+rmjKaqimjeWvH4R7HA5Bymmo8m3bpLzkf3BWRm4GDqlrf1+NUdZmqzlHVOePHj+/3eT7/6BtJQX+oUeDhVVtYXufNlaX90WPRT1jpzNMCoKqzU+fyn9vorQ1Z6ptDPXr3gpOOWlhTzi2XTOnx2DnTynpNY00aNYyLp47mb788KysXV6+s0TA95aPHfwXwFRG5CRgOjBKR/6Wqf5bJk+w9knrziaFGgR+8uIXpk0oL+jY9PuUSDAiIEA5nP+cfX5WzrxvBzrDy5Ju7WPaNOVlrS7rcNn8aN19f6U5HXTDRWYGsOD278yeW8t7eoz0e7zpw7BQHjp2icV8DLe0nKD2jOGPpNa+s0TDJch74VfVB4EEAEbka+C+ZDvoAZ48ZTqtPgn9EYembu3jKA0FpoBIX/QA5yfG7dxrpZP/eeP+gJ2a/uG2O5y7gAuciOqy4+yIqOHX5X9/Wxs5DydswAnRFlKVrmwgIGQvSqUo35Pu9M44hu3L37Qeu5fOPvuGb4L96axvL61oKOt+fuAFLLoLE3MqxFAUDSYE0FVX1RPBy7446OiOx3L0Cv964B8GZenrb7KnsbDtOfcuR2FTZdGQySMffxdkYgrfkNfCr6u+B32fr9d9+4FrAmdnz0Kot2TqNZzz8ovMzFnLwzwtNLywK3pjdE3939Ob2g6zf7eTPu8LKM4Mc73ErfGZiS0Yr3eBdQ7bHH29hTTkLa8r53N+v5tDH3puPnSmqTr4fCj/4L69r4dWG/dxYNTmrP0ttUzudqabxpBBW+Nt/b/TEeIp7/n9+/YOMveZl08q4YGIpt82eCpCR/Hy+ttE0ffNVyYYNP7guacbDUBOJBv9Cnunj3qG9teMwD2V51pK7AUu6vFRm+IVNrXTEXbSCARnUH/TG5lCsVpKVVh7afBX4AR6761JWfudyZkzqdQlBwYsoLH6poWCn0CVufJLNjVCqK8q45/Pn9thnty+BgHgiV13fHEp6X6rLx7CgpnzAv9vxQd5KKw9tvkj1JKquKOPV+6+kvjnE0jd3sXprW76blHHhiLJyU2tB3mZPHDUcOJrwdXbUN4d4+g8fxtL8bt2kVAR4ZH6VJ97T2qZ2wgnj0et3h3hv71GuvGA82w4cH9DruvV+LD8/tPmuxx+vuqKMp74xh5XfuZzPTRtav9gKrNiwpyBTPvdedR7FQacLXhwU7r3qvKydK7HeTV+lOs6bcGbW2tFfTooq+fipzgjjSodRNMC/7KsuHB8L8m5pZaDfi7Bs4Za3+bLHn6i6oozn7rs8dgfwxrY2z9bq6Y9wRPnBqsJa3OXOJPm7r1RlfAPwVPrz37zz4Mc8tGoLLe0neOCmGVlrUzqqK8qYXV4Wm9HjUpzSDaM+X9nvYm0Ab35wqMdaheV1LSx+qYGIatqDvPXNIRYsW0dnWCkOCs8umlcwv39+4esefyL3DuC5+y5nYU0518+cmNWxgABOjzbVv0yJAPf9z42e73nVN4d4aNUWFjzlbNK95OXGnKQYbp89td+94yfXNnni/dx35JOkYwKs2X6QZW/1P+hDd1lscP5PFr/UQFdEiSh0pDnIuzI66KxAR1j5/sr3PPF+mW7W408hcQra8roWVmxooaMrwva24xm7G4gAkYSphCJw7xcquW7WJJa+uYv1H7ZzdJA1hw593MHtT7zD9TMncu9V53mu95WqbEJOV3qKW+AgPQp5L99Q3xxiX4rFicGg8Lv3Dw7od1RwZga5c/gfe/0DwnEvFJD0BrYTuy07D37Mgqdqreyzh1jgT4O7DgCcP7iVm1o5fPwUR052sPfIJ+w78mnGCn6qwtK1TWxqCfH9G2fw1DfmsLyuhYdXbRn0OV7b2sbqrW1c57ELQGLZBHcP2LIRJTy+ZmdWe/61Te2E05zHn/i8fJZvqG1qT1lxc1LpsAHXqVKc0g2rGw/wi3W7Yxdi94KwJM2B7VSb1nRYyQZPscDfT6kWpMRfDFxHTnbw0YkOzhpZwpi41Z57PjqZ1oyL9btDsV7SwppyGvcdHfSqTHD+uF/b2sbvth9khUdyr4kF2r465xxmTRnNkpcbs17ga27lWIJBiQ3wBgMkzZZJ5dinXdy1bB2/ytN72NsK4sGWKImok8oS6b4HmjhqGH9x7YVpL6TrbdMaL6x6Ng4L/BnQ39WJbupoWFEgdlFIdUHo6IrEpmTeNnsqKzbs6XND8P7oiuZe/+H2z+Y9+FdXlHHDrEm8vq2NcWc6e9s27juaswJfgbiPk0elX9yvM5y/KbO/j9tyMdPcXr7rwLFT/VqxPLdyLEEhaV8DL+5i5lcW+PMgPnUU79FXtvHk2qYeKZ0VG/ZQNWU0C2vKWTK/qteNQgZi58GP+dqT63hkflVeSzw8+so2Xnx3HwAfnzrJ7vYWggEQEQJoVhcQ1Ta1xy6mEfrfY35u4x5unz0158G/7Vh2iw+eOayIY592jy315+JbXVHGt7/Qc1ZRcdAbC9+Mw2b1eMgDN83g+e9czsVTu3Ok4YjGVuEurCnnrssyG6DDEeXhVVtY9Mv8zfz5j8YDScfCEaIb2wuLb87OJiHg9E77UbEhSVdY81LOYPQZxQN63rjS9NIt8UEfnEHjdAN3fXOIxv3HYncNAnx1zjl5v7M03Szwe0x1RRmLvzyLorhoFNHu4HLb7Kk9vpcJbt7/a0+uy8uCrxtmTer1e6qa9RSBJsxD6c8fhQKb9xzJ6UWzvjnE2zsPn/6BKRT1uaV87wJp1rRwZ2i9teNwLGU0rDjA7dHCb8YbLPB7UHVFGUvmV1EU3UQjIBIbGHO/l43/OLf3n+vg/8BNM7hkavJMkPjphdnywqbWhCmL8Pe3XsRl/VjJneuLZm1T+4CnFB+Im4DQH53R8abTqW1q51TCTl9XXtD/rVNNdlng9yg3px8MCBFVlrzcGOtVLqwp56IUgTITFKeufy6D//K6Ft5tTd7MXOnfytr+qm8OsWLjntjXAYG/v8XZvP77N/ZvZW58Si7b3MHTXFLS23c4sdqp4mwSdPfTtbaIy0Ms8HtY6GQHEdWUpXHv/Fz2BmPduv7ZDv5uPZe+qm92hpUX0uhpDkRinR6A6XErtfv7x9EVLYyXbdUVZVw7Y+KAnpuqvk+6OtMYz3CrncZTrLSz11jg9zB3fnsAZ4ZL/DzohTXl3Hdl5QAztqcXUbI66Ovmgn/82nYa9ib39uNlq9d/MCHtEVFiF5mlb+5KuUDqdNLpFWfCvVedN6BB6TQ3G+vVy5v38fCqLX3+jMdOpVhpLk6JCev1e0POA7+InCMia0Rkm4g0ishf5roNhaK6oozFN88ikCLdA05u/Ie3XpS12/5sDvrGb/TR1/RUgawNDE4oHZZ0zG3Kh4dTb0p+Op1hZcm/N+YkwBUNoPs+2GUg2w4c55m6FhY85aRuUlXh3NmWvEAxHIFn17dYyscj8tHj7wL+SlVnAHOB74rIzDy0oyD0le4Bp+f/62hRuf4MSPZHOKIZT/24dzOnM/qMoqxNA7xt9tSk1EfVlNHUN4fYffjjAb/u5tajWR/srW1qpzONDeKzxR3svfvpWv7pt9u5M+7nTVU8DrDdvDwk54FfVfer6qbo58eBbcDZuW5HoUhnJ6TqijJ+dOtF/Pq+y/nRrRdlJf3jpn4yGcxumz2V809T4/7IJ11ZC6DVFWXcFTdWIkDDvqODmjXjysbFMt7cyrFp7xqWDSKw/sOP+DRaz6crbnD7VB8XJNvNyxvymuMXkWnApUBdPtvhZe5OSHdeVp5WymNhTTnPf+dyrps5sMG/eGeWBHt8rcBDGcj7u/n9X61vSatnnc3ZMrfFlWVW4Pn6VspGlMQutoNZMpHN/Y9XNx4Y0MUpU0tAVJ2V3/Eiqix9cxeHPk697kIgq4vxTPryFvhF5ExgJXC/qh5L8f1FIrJRRDYeOnQo9w30mBc2taadI3X3FRjsxvIfd4SZPCo5D/7a1ja+uvSdAV8AXtjUyqlOJ7+fTraiK9L3bJLB7Pa0/cDxHkXZusIRQic7YhfbwQ6GZmP/4/rmEE8OYJOVGZNKWXBZeUbuCBPfFgFKigJ9jo0ozh2Vyb+81OoRkWKcoP+Mqr6Q6jGqugxYBjBnzpwhsB/WwMUPhPanZspjd13KRyc6WLtjYKs8AfYfS73gJ6LOBeD1bW18aUb6ZZ7rm0M8t3FPjxLM6fznJlZ2dHfqKhtREqviWRSt7HlbmrVz6ptD/M2LPctdS7TmfHVFGS9sas3IjCJ3mmemerorB9iuHYc+5pyzRlBcFKAjg+MDAYGLzh7N6DOKT/u71rj3aF7LWRtHPmb1CPAzYJuq/iTX5y9E6eT5e/PLb9Vw5QXjMtKOVD1F9wLw1aXvpJXSiC+KJsBn0tzhLL5sQ/xU0L95cQufRu8eOsLK8rr0Z47UNrUnzSiaXT4mFpQy2dt4tq4lY73+wylW3waA0SP67sd1hZXVW9uIRDI7KBxRZ0A7nQ7G5taj3PHEwO8WTWbkI9VzBfB14Isi8m703015aEfBcPP837t++oDq0mcq+CtOuiBVnjii6a347XERCwo7DqU3eya+x9/XVFAFOjojPPb6Byyva+GhVVt6nXeeqj78hRO7L0S3z56asamyCtz55DoefWXboF6nvjmUsjJnBDh6Mnn+/Nljhie1I539BrLJnSbsTgk1uZfzVI+qvk3qzqPpQ39r/if65bdquP9Xf4yVPx6obQeOc9+Vlew6fILXt7b16BW7K36BXss8uxex2qZ29h35JK3NZYTuHn99c4i9Rz6hKOhs3BJ//mBA0IgSAd7ecZi34nqgz9W3Jm3915iQbw4GhNviBtCrK8p45JaLMlYKuyuiLF3bxIFjn/LYXZf2+/nuJuYd/WhMYkrHqf3U99qJXMnp9pqmB1u5WyAGM4DpeuyuSwc94AvO1pDnjRvJD2+9KKn3n85MluqKMr57zfmUDku/33H8k07m//RtZ9er9S2gynUzJ8buHoYXB3hkfhVXXDCOgCSnaTq6uu8CHl+zM7YZTrxvf/7cpCDkrpPIxCwp14vv7htQqsPdxLw/EmfYfG6aczHzQs9LsV258sU2YikAbk47E9sQuj3Nwfb8l65tYtrYEfz9LRexZvtBXt/WFpsB4875//32g70O+i6va+mxUUdfNHq+eOGIcvE5Y7j3qvOobWqPDchOn1TKht0f0dEZSSq54N4FBMTp+SbG0OOpSg3gXKguOWcMb2xrG/T8ftdrW9t4bWsbMyeX8sgtF6X1/5mJYH2qy5m1NGNyKVv3n34L0GxbsaEl7Z29TOZYj78ApJrVMxiP3XUpMyenN6jal93tJ3lo1RZaPzrJ5yrKeiwocvO4dy5bl7Jn21dhtnRIQGIrRL97zfmxwOGmkv7qT6bzo1udSpsXTx3d4y6gtzIRH6QoNeCKr5uUSVv3H+eOJ95Jq/c/a8roQc/D37r/GP/02+2eCPrgDPZarj/3LPAXgMHM6unNI7dclIGWObYdOM763SFUk3ul7t6+8X/Y9c2hQc/n7upjBo+bSlpYU86Pbr2IxV+e1SNo9xY73dozqbgXlCsuGJfxNIkC31vxLuBsQ3n1P65JGgSubw6x+KUtg7rjOH/CmYQjmtVS1wNhZRxyT3SwK1RyYM6cObpx48Z8NyOv3HnrbkojEx59ZVva6ZbBCgix+f5PvrmL17a2Zey1L546mqqzR8fm79c3h1i5qRWBHsfcef8N+47y6417kkoyA1w/cyLLvjGn13MNZIA1XcUBiN/D5L4rK3ngJmdfgG//ciOrB/GeuXsNLHm5MWUaLJ+CAeHX986zdE8WiEi9qib9QluOv0C4fxRuzygTfyRuUMlF8Hfn+2cy4Ls2tx5lc+tRfrVhD1ecNza27R/0nM3jvmf1zSGer29FSO79nm4T8+qKMp5dNI+Vm1pZ3Xig1/IEA5GwcRU/e7sp9n+0bZB3SF+5eAoLa8qZPqmU2qZ2jn/SyVNvNXlidk91+Zh8N8F3LPAXiEwO8MZ74KYZXDdrEo++uo0Nuws7zxqOaNIioo6uCI++uo3hxUFurJrM9EmlLPn3xl5Xrs5LI43mXkRunz2VO5a+M+iyDr3pjMC8H73OT++uZnhx8PRP6EP7CecCFX8BLB87kp+/3cTOQwMrQZ0p63eHWPBUbdJ0W5M9FvgLxEDLNqSjuqKM5+67nOV1LTy8aovncsCD5V7Q3kpjZWnizJ5UaSNXdUUZP7zloqy+Z/uPneKOJ96h/KwRg3qdG6sm9/i6vjnEkpcbk/bHzReb059bFvgLhDvA29kVyVppWzcV8INVW9h2wBuzPnItPoAn5vNTLQJz37Ns3jEp0PzRyQE9tyQo/Kcrzk1aUOd2JLxykbc5/blls3oKRH/LMw/mPK/efyX3XVmZtXN4VVFQery3tU3tdMYlwXubfeLeMf3o1szNlMqUjrDyi3W7k2YruR0JLyzkcv233+Rms3pjgb/g9Kc882A8cNMMVmaorn8hEIElX6nq0ZufWzmW4rhiPae703Knj3pNqguW25FYUFNOMFNF+gepM6w8+eaufDfDFyzwF5D4PP+pTmfru2xy6/qv/M7lzEizimahUk2uFe/O4FlYU87dNeVpDT4urCnPy91Sb7tYBtLYue2R+VUUeST4v7a1bdCF7Mzp2Tz+ApKYcw4GhEfmVwHwasN+bqya3GtxtExYXtfCD//3Vk50hLN2jnyqOGsEb/71NRl5LbcWUEdXhM5whMrxZ/Je6xEO9LK/wWCdP+HMpB2xLptWxlXTJySt/Ui1JqS+OcT3V76X9Br5MnXMcP7vay7I6u+zH/Q2j98Cf4F5eNWWPitaXjatjO/fOCOrsyO+8bO6QW3u4mUrv3P5aYPkQNU3h7j9iXcG28QkInDvFyp5+g8fxhalFQeFXy1KXhTV17TghzK8p3ImlASFM4cX87XqqbE1DV6SjYWVmWSBf4iobw5x55PrYpuZpBK/SjZbv4zL61r4yertHM7gAiYv+K9/Mp3vXnM+kJ21E6e7cA/U8OIAi2+eRcO+o0lTT+ODU21TOz9+bTsRhaDAnZeVc/aYM2KpoAXL1tEZVkTIWEG6TAkKnDWyhHPHjeSCiaVp77SWLcvrWlj8UgMR1YyurckkC/xDyPK6Fn7w4unrtripoGynfx5fs4O9R/pe8VoIAgLP3dfd4398zc5eg+RA/8DTuXAPREDgr67vedFK3JqypMi5OCx5uZHOrogzqCtCV7j7e+6FY9aU0Sz+TUPKshZeUhIURpQEGTm8mLNHD8/JBcFd27Fiwx7CcbvJLfDg4L4F/iFmeV1LWhuECHBvXM2XbKlvDvG9Fe8OeL55PAE+O3U077Uezek88/PHj+T1v7o69rXb408Mkv3d2zdRuv93/VEUEFbcOw9w6vY/X99KVzhCQISIauzi9b3rp8d6/vuOfMKz61uIqDPLIxAQuiJKQGDRFyq5btYkVm5qZWebU4SvkIwoDoAII0uCnDtuJGNGlDC+dNigLgpuwH++vjVpEyDoPb2WTxb4h6D4VaWzpoxmzfaDvdaMz0XuHwY/ACzAsGLntnl144GcFZEDYmWc47k95817jrA6YccxN0AO5KJa3xxi6Zu7knYxGwg3x3/drEnc/XQtpzq7g5Ib0FWV4oR0RPyFTUSS7kLc96O+OcQdT7zjmcVemRAAEOf/sKQoSMVZIygdXsRHJzo4a2QJYxIWk+356CTvtx0/bXmOqWOG888LZnsm+Fvg9wk3oKSq5BgQmFNRlpHez+kMtPLnFy4Yx/1fujDWrkxsF5muVIEfTl+Rc8akUs6JK6lw5GQHp7oi3Pk5Z1Xvyk2tsQ3SE9/35XUtg66XIzi9zRmTR7Fl79HYhd+9iC6+eRahkx0pU1TxKaHE9OEXLhjH//xWzYDHJQIeHCfIlaBASTBAWOGskcWcOayoxwUl239/Lk8FfhG5AfhnIAg8raqP9vV4C/z9l27gTQxa4ASuj050UDn+zEENELt3JDvbjrOxOXTaIOCWDnZ7mW46YnldS056mxdPHc1L//nzScfjc/39JSRvAwnO+146vIj6liOxPHGmuBeC/qajEn9nrp85kaunT+Bvf9PQ7zLUZ48Zzpc/O4Un1zYNqTuFfCgKwM4f/emAnuuZwC8iQeAD4DqgFdgALFDVrb09xwL/wDz6yrZB/+EJzj6tY0aUxC4IqW6FT/e9vUc+4VS4uyBYZzhCZ5cSAD6OSwsVBYXZ54yJXSjcZUW5+C3trRZ/bHaPx+rYuxIvLvEX0P5y1x9s3X+McER7jBH0V9Ajm7oPBUGBXf+9/8HfS/X4LwN2qmoTgIj8CpgP9Br4zcC4JZeXvrlrwPvFKiQP7PWVlhhkid+usPY4Xy7jxtXTJ6Q87pY3cFMioZMd7Gg7zkvv7st7b/a6mRMZXzqsxwwTgNDJgU2zXVhTTuhkR3fKSJVAQBA0NsAdjl7ATxfULehnTlidDkimUkP5CPxnA3vivm4FahIfJCKLgEUA5eW2em+g3LILbtrl8PFT7PnopG+rb/alr2AZX8fe9fV503rk712J7687tnL8064+33cBPhNNAaWTGispCnBfNBVXNWV0jznlg6nemlgJNn6MAOgxTTR+INn9GYqCQiTiXCgiQDissT2P/Zrzz4RMlq3OR+BPVRQk6ddBVZcBy8BJ9WS7UUNdYuByb+mHFQV6Tc3sO/Jpznq0bkpp054jeZk7PpBgmepi4Oqtjn/i+z6+dBizpoxOGnyNv1DHj7lcPX1CykVa8btrDXYVafwdTqrXcr92B66fr2+lqytCICAsmV/Vox3Q80KReDGJv4AATCwdxqGPT9ndQgqZLMWejxz/POBvVfVPol8/CKCq/72351iOPz/igw+cPo/f3++lGkROPGcqp3u9/rYvVzMshqp0yxb09rhMPD/V74x7UV2z/SAfHvq437+bieNSw4qCjBpWRGc4QnEwwLFPOzmjpIiqKaPYsPsjjn3aRXFAOHKykwjdU0aFwaW9RpYE+eW3agb0++mlwd0inMHda4G9OIO7C1W1sbfnWOA3xpj+88zgrqp2ich/Bn6LM53z530FfWOMMZmVl60XVfUV4JV8nNsYY/zONmIxxhifscBvjDE+Y4HfGGN8xgK/Mcb4TEFU5xSRQ0DzAJ8+DiikfQILqb2F1FYorPYWUluhsNpbSG2FwbW3QlXHJx4siMA/GCKyMdU8Vq8qpPYWUluhsNpbSG2FwmpvIbUVstNeS/UYY4zPWOA3xhif8UPgX5bvBvRTIbW3kNoKhdXeQmorFFZ7C6mtkIX2DvkcvzHGmJ780OM3xhgTxwK/Mcb4zJAO/CJyg4hsF5GdIvJAntrwcxE5KCINccfOEpHVIrIj+rEs7nsPRtu7XUT+JO54tYhsiX7vX0Qk1YY2g23rOSKyRkS2iUijiPylx9s7XETWi8jmaHv/zsvtjZ4nKCJ/FJGXC6Ctu6PneVdENnq5vSIyRkSeF5H3o7+/8zzc1unR99T9d0xE7s9pe1V1SP7DKfm8C6gESoDNwMw8tONKYDbQEHfs/wMeiH7+APAP0c9nRts5DDg32v5g9HvrgXk4+zq8CtyYhbZOBmZHPy/F2TdhpofbK8CZ0c+LgTpgrlfbGz3P94DlwMte/l2Inmc3MC7hmCfbC/wbcE/08xJgjFfbmtDuIHAAqMhle7P2A+X7X/TN+G3c1w8CD+apLdPoGfi3A5Ojn08GtqdqI86eBfOij3k/7vgC4MkctPsl4LpCaC8wAtiEs3+zJ9sLTAXeAL5Id+D3ZFujr72b5MDvufYCo4APiU5W8XJbU7T9euAPuW7vUE71pNrU/ew8tSXRRFXdDxD9OCF6vLc2nx39PPF41ojINOBSnF60Z9sbTZ28CxwEVquql9v7GPDXQCTumFfbCs5e2K+JSL2ILPJweyuBQ8C/RtNoT4vISI+2NdFdwLPRz3PW3qEc+NPa1N1jemtzTn8WETkTWAncr6rH+npoimM5ba+qhlX1Epze9GUiUtXHw/PWXhG5GTioqvXpPiXFsVz/LlyhqrOBG4HvisiVfTw2n+0twkmnPqGqlwIncFIlvfHCe4uIlABfAZ473UNTHBtUe4dy4G8Fzon7eiqwL09tSdQmIpMBoh8PRo/31ubW6OeJxzNORIpxgv4zqvqC19vrUtUjwO+BGzza3iuAr4jIbuBXwBdF5H95tK0AqOq+6MeDwCrgMo+2txVojd7tATyPcyHwYlvj3QhsUtW26Nc5a+9QDvwbgAtE5NzolfUu4Dd5bpPrN8A3o59/EyeX7h6/S0SGici5wAXA+uht33ERmRsdtf9G3HMyJvraPwO2qepPCqC940VkTPTzM4AvAe97sb2q+qCqTlXVaTi/i79T1T/zYlsBRGSkiJS6n+Pkohu82F5VPQDsEZHp0UPXAlu92NYEC+hO87jtyk17szlwke9/wE04M1N2AQ/nqQ3PAvuBTpwr9LeAsTiDfDuiH8+Ke/zD0fZuJ26EHpiD84e3C/gpCQNZGWrr53FuFd8D3o3+u8nD7f0s8MdoexuAxdHjnmxv3Lmupntw15Ntxcmbb47+a3T/fjzc3kuAjdHfhReBMq+2NXqeEUA7MDruWM7aayUbjDHGZ4ZyqscYY0wKFviNMcZnLPAbY4zPWOA3xhifscBvjDE+Y4HfmDgi8rA4lT7fi1ZOrIlWThyR77YZkyk2ndOYKBGZB/wEuFpVT4nIOJxKj+8Ac1T1cF4baEyGWI/fmG6TgcOqegogGujvAKYAa0RkDYCIXC8i60Rkk4g8F61t5Nav/wdx9ghYLyLnR49/VUQaxNk3YG1+fjRjulmP35ioaAB/G2dV5evAClV9M1pfZ46qHo7eBbyAs3ryhIh8Hximqkuij3tKVX8oIt8AvqaqN4vIFuAGVd0rImPUqStkTN5Yj9+YKFX9GKgGFuGU+V0hIn+e8LC5OBtj/CFaDvqbOJtouJ6N+zgv+vkfgF+IyLdxNt4wJq+K8t0AY7xEVcM4VT5/H+2pfzPhIYJT939Bby+R+Lmq3iciNcCfAu+KyCWq2p7ZlhuTPuvxGxMlzl6oF8QdugRoBo7jbEUJUAtcEZe/HyEiF8Y95864j+uijzlPVetUdTFwmJ4ldo3JOevxG9PtTOB/REs9dwE7cdI+C4BXRWS/ql4TTf88KyLDos/7AU4VWIBhIlKH06ly7wr+MXpBEZyqi5tz8cMY0xsb3DUmQ+IHgfPdFmP6YqkeY4zxGevxG2OMz1iP3xhjfMYCvzHG+IwFfmOM8RkL/MYY4zMW+I0xxmf+D/tgfj3fRW11AAAAAElFTkSuQmCC",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"plt.plot(loss_monitor.loss, '.')\n",
"plt.xlabel('Steps')\n",
"plt.ylabel('Loss')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"通过如下方法打印量子嵌入层的量子线路中的参数："
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/plain": [
"array([ 1.5175925e+00, -5.5282825e-01, 1.9824509e-01, -2.3327057e+00,\n",
" 8.4526891e-01, -1.3019586e+00, 9.3813318e-01, -1.0318477e-01,\n",
" 4.4351882e-01, 1.8607093e+00, 6.0036021e-01, -3.0638957e-01,\n",
" 9.3188483e-01, 6.0410827e-01, -1.8905094e-01, 6.5970606e-01,\n",
" -1.2129487e+00, -3.1650740e-01, -2.5501034e+00, 3.6324959e-02,\n",
" 4.0066850e-01, 7.5752664e-01, -5.6982380e-01, -5.6846058e-01,\n",
" -9.0591955e-01, 3.3477244e-01, -6.1832809e-01, 2.1618415e-01,\n",
" 1.0225463e-01, 4.0966314e-01, -9.0604734e-01, 1.3528558e+00,\n",
" -5.3387892e-01, -3.2625124e-02, 6.8196923e-02, 4.1799426e-01,\n",
" 2.6094767e-01, -3.3765252e+00, -1.9021339e+00, -1.1502613e+00,\n",
" -2.0344164e+00, 8.0160522e-01, -2.8717926e-01, 3.3720109e-01,\n",
" -2.1616800e+00, 1.1822585e+00, -7.0481867e-01, 4.0014455e-01,\n",
" -2.8856799e-01, 8.4199363e-01, -5.8137196e-01, -1.9842222e+00,\n",
" 1.7555025e-01, 4.1823694e-01, -3.1270559e+00, 2.6714945e+00,\n",
" 2.3251233e+00, 3.0707479e-01, -5.3547442e-01, 3.0258337e-01,\n",
" -1.5764916e+00, 3.0099937e-01, -2.9257689e+00, -1.1786047e+00,\n",
" -5.7270378e-01, 2.0587114e-03, -1.5863895e+00, -2.1442556e+00,\n",
" -1.7923084e-01, -1.2772868e+00, 4.1606693e-04, -9.2881303e-03],\n",
" dtype=float32)"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"net.embedding.weight.asnumpy()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 经典版词向量嵌入层\n",
"\n",
"这里我们利用经典的词向量嵌入层来搭建一个经典的CBOW神经网络，并与量子版本进行对比。\n",
"\n",
"首先，搭建经典的CBOW神经网络，其中的参数跟量子版本的类似。"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"class CBOWClassical(nn.Cell):\n",
" def __init__(self, num_embedding, embedding_dim, window, hidden_dim):\n",
" super(CBOWClassical, self).__init__()\n",
" self.dim = 2 * window * embedding_dim\n",
" self.embedding = nn.Embedding(num_embedding, embedding_dim, True)\n",
" self.dense1 = nn.Dense(self.dim, hidden_dim)\n",
" self.dense2 = nn.Dense(hidden_dim, num_embedding)\n",
" self.relu = ops.ReLU()\n",
" self.reshape = ops.Reshape()\n",
"\n",
" def construct(self, x):\n",
" embed = self.embedding(x)\n",
" embed = self.reshape(embed, (-1, self.dim))\n",
" out = self.dense1(embed)\n",
" out = self.relu(out)\n",
" out = self.dense2(out)\n",
" return out"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"生成适用于经典CBOW神经网络的数据集。"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"train_x shape: (58, 4)\n",
"train_y shape: (58,)\n"
]
}
],
"source": [
"train_x = []\n",
"train_y = []\n",
"for i in sample:\n",
" around, center = i\n",
" train_y.append(word_dict[center])\n",
" train_x.append([])\n",
" for j in around:\n",
" train_x[-1].append(word_dict[j])\n",
"train_x = np.array(train_x).astype(np.int32)\n",
"train_y = np.array(train_y).astype(np.int32)\n",
"print(\"train_x shape: \", train_x.shape)\n",
"print(\"train_y shape: \", train_y.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"我们对经典CBOW网络进行训练。"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch: 25 step: 20 time: 0.031, loss is 3.155\n",
"epoch: 50 step: 20 time: 0.033, loss is 3.027\n",
"epoch: 75 step: 20 time: 0.033, loss is 3.010\n",
"epoch: 100 step: 20 time: 0.033, loss is 2.955\n",
"epoch: 125 step: 20 time: 0.032, loss is 0.630\n",
"epoch: 150 step: 20 time: 0.034, loss is 0.059\n",
"epoch: 175 step: 20 time: 0.033, loss is 0.008\n",
"epoch: 200 step: 20 time: 0.031, loss is 0.003\n",
"epoch: 225 step: 20 time: 0.032, loss is 0.001\n",
"epoch: 250 step: 20 time: 0.030, loss is 0.001\n",
"epoch: 275 step: 20 time: 0.032, loss is 0.000\n",
"epoch: 300 step: 20 time: 0.030, loss is 0.000\n",
"epoch: 325 step: 20 time: 0.030, loss is 0.000\n",
"epoch: 350 step: 20 time: 0.029, loss is 0.000\n",
"Total time used: 11.819875240325928\n"
]
}
],
"source": [
"context.set_context(mode=context.GRAPH_MODE, device_target=\"CPU\")\n",
"\n",
"train_loader = ds.NumpySlicesDataset({\n",
" \"around\": train_x,\n",
" \"center\": train_y\n",
"}, shuffle=False).batch(3)\n",
"net = CBOWClassical(len(word_dict), embedding_dim, window_size, hidden_dim)\n",
"net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')\n",
"net_opt = nn.Momentum(net.trainable_params(), 0.01, 0.9)\n",
"loss_monitor = LossMonitorWithCollection(500)\n",
"model = Model(net, net_loss, net_opt)\n",
"model.train(350, train_loader, callbacks=[loss_monitor], dataset_sink_mode=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"打印收敛过程中的损失函数值："
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEGCAYAAABvtY4XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA580lEQVR4nO3dfXyU5b3g/881M0l4MMAYHgLGJEZtCoQeTZAEbVF7xCMetyrWVtM9bbdVpNt9/Y6vnv5W69O62HY9u9u+ume3L/Ghbk/PAYqKYOvq1mfxiQBBMUCMQiQhPCSAAwQCeZj72j/u+57cM3PPZBJmknn4vl8vXiQzc89cCeE7V77X9/peSmuNEEKI7OMZ6wEIIYRIDQnwQgiRpSTACyFElpIAL4QQWUoCvBBCZCnfWA/AaerUqbq8vHyshyGEEBmjsbHxiNZ6mtt9aRXgy8vL2bp161gPQwghMoZSqi3WfZKiEUKILCUBXgghspQEeCGEyFIS4IUQIktJgBdCiCwlAV4IIbJUWpVJZovVDe08/W4rx870A3C6L8iZviBKgQa0Bo/1sWGY13g9ybnP0OYfrwJU8p8PYEKel4J8LwAFPi/nTR4HwBen+jh3Yj5TJuSHfT+mFRawtLqEmjJ/0r7HQoihSYAfhkdfauYPH+zlzIARCoyRQTJu82XHnUbEAweM5N4X1IOvl+zn6+4N0t0bDN23P3B68IGHT+FmdUM7dy2q4N7rZ7veL4RIPgnwLuwZ+OFTvfT0BgkaGq3Dg7cRJ1iLaBpYubGVQyfO8JvbLh3r4QiREyTAW1Y3tPPbNz/j0IkzBI2hHy9GZsNHB/ho3zGuuGiqpG2ESLGcDvCPvtTMqoY2TvUFR20W7rXSOZmYg9ck57eVvUd72Hu0nTWb27lm9gzuuvJCCfRCpEDKA7xSygtsBfZrrW9I9eslYnVDOyv+vJMzAyOfqscLoAU+DzMmjSPPq0KLjtmy0NjYFmDdtg52d3az/9hpel1+3TndF+SUI0cfi6HhlV2dvNbcyc9vmkd9bWkqhixEzhqNGfzfA83ApFF4rZjOJgWjMGe2+T4vVbMmcc+S2RkfqEeqpsyf0Nfe2BZg5dt7eL25c8hZv6HhvvVNbPiwg4tnFGbFG6EQ6UCl8tBtpVQJ8M/AL4CfDDWDnz9/vk52N8nVDe38+tUWjpzsS/gar4KCPAnmyWAH+teaO0n0R83rUTxyY5XM6IVIgFKqUWs93/W+FAf454D/AhQCP3UL8EqpZcAygNLS0pq2tpidL4elsS3Aj/+1kUPdvQk9Pt+rmF5YwL+/+mIJLCmwuqGd+9Y3Jfx4BSyeI/l5IYYSL8CnLEWjlLoB6NJaNyqlror1OK31E8ATYM7gz/Z1hztjLy+awK++dYkEkRSrry3lrZYuXtnVmdDjNWZ+/o2WLtYuWyj/PkKMQCpz8FcA31BKXQ+MAyYppf5Va/1vk/1CjW0BHn25mY/3HaM3mNh7xJyZhTxy0zzXwGEvJB6xZv/ZskA61u668kJe/6SLoEtSXgFfLi7kk87usFTOQFCzbluHfO+FGIGUpmhCL2LO4F1TNE4jycE3tgW45bH3Rz64DObzgH9CPpeW+sc8leF8UzzW0xdqWxC5aLq6oZ0HNjS5LrzW15ZSNWsy969vCttUdu2cGTzxXdffQIXIeWOWg3cM4CpSFOAvuPf/xG8PINJCYYGXn10/h/ra0phB3l5cBcLul0VXIWKLF+BHpZuk1vqtVNXAS3DPDN29Qe5b38R3f9dAfW0pP79pnrmXwCFoaB56YQeVxYXcvqA06vbGtsAoj1qIzJbx7YIz/gvIMRs/OxIK8rctiJ6RBw3NptajLK0uwedRYbev29YxmkMVIuNlfHxsffRvx3oIYpg2fnaErzz8F+bOmhwWxMFcbK2rKKKmzM+KG6tCs3wNPNfYIbN4IYYhK3rRXDtnRsLld7bZxYUUjvPxxak+KqadM+aLlLGsbmhn7ZZ2Cnweus8M0BHoQWvoDRr0J1gxlI5OnBngvvVNnDshj0BPfyjVZgAth7qpKfNTX1vKzgPHWd3Qjgb6BwypqBFiGEZlkTVRI93JmoxKGgVcVu4PHVbhrARxHmBh357Obwrp4tGXmlm5sXXY1100bSKv/cNVgPlve/sTH9BnvZnl+zysubNOvu9CWMZko9NoqinzU1xYkPCuVTca2LzX5df/GAdY7D58ild3dSb0phDvvpFcM1qvdbb1//deP5vFc4v5/9ZsY/+xMwlft/vwKVY3tFNfW0pNmZ9b558vs3ghRiArZvAA969vYlVDe9zH+DyD7W9FYjwKKmcU0h80ot4YEn0DaGwL8O3HP2BgGN/4r108lX/5YW3oepnFC+FuzMskR8PS6pIhHzNgWL3NPWaLggXlfs6bMg415JW5y9DQfKib3YdPsXlvgFd2dYb+rGpo55bH3udbK9+Pu/hpL5h6hvGN3r7vWOg57Vm8fXkwaLCp9ehZfFVC5IasSNGAGQQWlPvd0ywRgoZ56ETb0R7uWlTB4rnFYa0JIH6aY/+x0xw4dkZq8C2b9wb45mPvx20OZm9SenBDE4msDZ84M8CtK9/n2eWXU1PmZ2l1Ceu2ddDXb6CUwh+RYhJCRMuaFA0kr22BV0FlcexeNfZrJfqmEO++dM3BH+vpY2tbYNjpLI8i7ilNiaTSnJxtClY3tPPQCzswtCbf52HVHZKmESLrF1ltw5nFxxPUsOtg98jfLGIszMa9byTXjPZrJcA+pen1T7pc2wssrS5h7ZZ9CefjX2vupLEtQE2Zn0BPH4bWGNpcbN3UelQCvBBxZE0O3nbPktljPQSBufP0gQ1NrI6YrdeU+bnjqxck/DyGJrSDta6iiHyfBw9ImkaIBGRdgK8p87N8UcVYD0NgBucHNzRFLcAWjs8LLbja+w/i2d3ZDZj/tg/dMBePR2FozYoXd8rOViHiyLoAD2b9dcmUcTHvV5g7WSfme0dvUDkqqOEnaz8Ku62uoijUokAD2zuOs+jiqTGfY2tbIBTI3dI0Qgh3WRngAf791RfHvE9jlv6dGTD45c3z2Pvo37LuR5ezeM4Mpp6TT15km0NxVtq+6GHxr94KfR5Z9tg/YFBy7gTKzp3ger2h4fG39wCDaRqvgjyfh7qKohSPXojMlVVVNJG++7sGNn52JO5jFPCLm+dJr/E47IOzdx04bvXAMTh5ZoCgMbznWXTxVP4QZ/PSw/9mbsxzWz2KUMnk6oZ2Xt5xkCVVM+XfTeS8MT/wI1HJDvC/fXM3/+0vLQk9dkG5P+r0IRGffVTilmFULS1fVMG915sL4fevbwq1IPAq+Mm1lWzfdyxm47hrrTr77zy1ib4BQ0olhSBHdrK6qasocv0C811SMJv3BljV0M43H3ufa379dlT1h4hWU+bn2eWXs+5HlzO7uDCha1ZubA3l05dWl1CQF14Vc9eVF8a89vVPunh+Wwd9A4bk4IVIQFYH+JoyP8tcKmr6gjpmewIN7O46yX3rm1jym40s+8NW7l8fXQkiBtWU+Xn57kXMmZlYkH/05ebQdZFVMQDFkwpcrzMMjQYplRQiQVm10clN4fg819s15jmhJ3uDMVsONB/qpvmQWaK3ZnN7VNOts+22mG0euWket658f8jdr1v2BuJuXrrpkvNc2wxrYFKBj4dumBva0brixZ1UFhfKv4EQLrI+wNdVFJHvVaHFPKfu3iBg5t/BDDyxYpPddAsI2+25ZnM7863gMlR7gGzP8deU+bl9QWlCrQjslr92VYyzx8yPr76I37+/lzMD0au4T777Obdddr7saBUiAVkf4O2SvHhBZ/PeAL+8eR73LJk97EVDQ0f0kY/THmDz3kDoDcHu9zKW/d5H8nxD/dZitiJoxyU2h7F7+NhpmsgZeeE4H2dO9kVdF3SkafoHDCmVFCKOrA/wkFj/k/vWN/HLm+fx7PLLwxqJjbTpVixRbwiQPr1mEnw+t3SVbVphAV//8tBHKB7rGQzebmmaS0v9MZ9jUoGPVXfUsW5bh7R6FiKOnAjwdj/yoVrV3re+iQ0fdnDPktn88uZ5odvtgL+7szs0owWSGvgzSax0lU1Zf+J9a5x5eDtN45yR+yfkxwzwT737OaVFE0MVNeu2dUi5pBAuciLAA6EDnIfKD2/ea7YcnnZOPpeW+kNtb92CR+RMP16aI8/roaWzOyfeEBL5EjWw8u09PPnd+dSU+aNm5IGe6PSMLWhoXt5xMKpcUgK8EOFyJsDD8FrVHj7ZFzq5qLxoAldcNDUq9xwr8McS2UM+03LwyU5XfX74ZNjnzhn5QzfMxaPcj1fUQNHEfMnDCzGEnArwiaZqIu092sPeo+2samhnyngfWoFhmA3L7lkyO+EgP9w3hHTklq5yHjoerxIpUuuRU6E0zabWo2Ez8kBPHz+/aV7M1gUvfnyQFTdWsePAccnDCxFDTgV4cE/VKKD03Am0fdEz5PXHTg+EPrbTOVPG+/D5PPQHDc70meUj4/M8+HyD+8j6gwaGASVTxnN+RFOtyJlyOtfXD/UmtbqhnQc2NCU0yzf0YJrGLQ8PxJzFG1qz48BxycMLEUfOBXiITtVooCPQwy9vnsebLV1s/vwoxx2BfCjHXB7bG6NO0Ll5KopjwXJVQzvn5Hvpt8bo9oYR781kuPf1Bw36BzQT871cMHXiiGv262tLqSwuZOXbe3i9uXPIQG+nadzy8Jtaj8a83uf1oEDy8ELEkZMB3k7V3L++KZROCGp4s6WLJx3nf/761RaOuNRij5aTfcHQx7HeMJJ9X09fkMMn+0K9eeyUlB38nQvPsdSU+Xnyu/NpbAtwz3Pb2R2nNLPti55QmgZc8vCA21cwEDQoLPBJHl6IOLK6m+RQlv1ha1gpnlLwi5vCWwc7c877j52m12qVe6Z/mL1ys8zs4kKqy/xDzvIb2wJ887H3Y+blFfDTv6nkx1dfxG/f3M2vXmnB0IPdJf0T8mPm4X0exYobqwj09FFXUSSzd5GTcubQ7eG668oLef2TLoJ2qkbDAxvMYGIH+Vg559UN7Tz9bivHzvSHbouXGsm2NwU71bS6oZ3FVhtft+9TTZmfxXNib3zSQPdp83sYKw/v9eDae97QOhTc7a6SEuSFGJTTM3hwXxRUEDdonc1rrd3SToHPE7dUEWD/sdOc7BtIap493n2n+4Kc6h1MCQ2X1wPP3HV5zP0C8WbxX7t4Kv/iOAhkU+vR0Iw8Xk9/+5CQFS/ulP7wImfJDD4Oe6buDPIaeGVXJ69/0sUjN1Yl7dSg+trStD6BKHLjlp2SSiT4Bw14YH0TL9+9KOq+mjI/XymZzPaO467XFk3MD3usM0DHawf8VyWTCfT0yUKrEDHkfIAH9yAP5o7J+9eHp2yyWbwSSDv4f9gWiFkF1Hyom0dfag6d2OS0sKIoZoB/8eOD/N3CcmrK/FEz+EBPX8xSyW3tx7j50hJZaBUiBgnwFjuAR26C0pg9at5q6Up6yiaTOIO/fUbrqy559cffaWXx3OKo71OsvvxgvpHaOfTI4/jsvLzb+oVhmDn4VXfUhb0pCCFMWX2i03DV15byzPLLuWj6OVH3vbKrk1tXvs+3Vr6f8yc82WWQ6350OdPOCU+haG32eo9UV1GEL8ZPm8ZMxUTuZrXTLd9fWO5+oSIU1O2F1lz+dxEikgT4CDVlfv7xlq/gczm31W71u6qhnVseM4N9LgeUmjI/K/9uPp6Ib9XaLe1R35eaMj/fvix2mmvHgeOh2bpXEZZu2XnwhOs1WsOrOw/R2BbgO09t4levtPCdpzbl9L+JEE4S4F3UlPlZu2whi+fMiApeTpv3mtUh31r5fs6e3Wqf4uQUNMwWBJGWVpfE/IF7dus+AFbdUcdPrq0Mq4aZO3NSzNd/4p1WOYhbiBgkBx+DczdmvG33mvADPFY3tDPLP57zJo+LKoVM5x4zZ2NpdQl/3NwetnbxenNn2A5VML+nFdPPYXfXyajnGAiaefgfX32Ra/4+Vn95rZETnoSIIWUBXik1DtgIFFiv85zW+j+l6vVSxRnoh6oiATPY7A+cZn/gtOv9qxvauazcn5I2v2P1BlJT5ueRiM6Phh48d9XpB1dc4Loz1edVocAcWUlTV1FEQZ77QqvPq7iluoRbqktkoVWICKmcwfcCX9dan1RK5QHvKqVe1lpvSuFrpoyzisTexbrn8KmEW+PaImf8YZJw1F684/Qg9hvG2b451NeW8lZLV9iOVbvvfeTjnn7v86hZfNDQtFhvnJGVNPZC68qNrVHPNxA0r7OroGRHqxCDUhbgtblF1v5fnGf9SZ9ts2fB3rAUuTFoOL3QU2Wo4/RCXO5zppdG0k3yrisv5I2WLgasXM0bLV1RaRpwn8UbGh56YQffvux810qamAutwIMv7ACQHa1CREhpDl4p5QUagYuA32qtG1weswxYBlBamlmbiSI3Btn5+s8PnzyrgzDGijO9tHlvgDWb25lf5k84/VNT5ufrldNDs/iBoHZN09TXlvJ/dxxk42dHwm4PGjpmPn3uzEm8E/F4myFH+AnhKqUBXmsdBC5RSk0B1iulqrTWOyIe8wTwBJi9aFI5nlSz8/VuIo/rc0pGDh6Sfwi4XRZqW7O5nZ9HdNuMNK2wIOxzt6+3sS3Ae7ujg7VSxMynx1tozfN5WFI1ky17v5CFViEcRqWKRmt9TCn1FnAdsGOIh2el0TiuL95xeja3N4xEf8MwNNy/IX7rhqXVJazdui9ummZT61HXIxPtvndu36tYC63nTsznp9dWhg4akYVWIQalsopmGtBvBffxwDXAP6bq9cTZvYk400t5Xg8tnd3uZaEuLZUjxzBUmqauooh8r6IvIsprzAM/YrUdfuiGuWGHtAB8caqPB63xVBYXDvOrFiK7pXIGPxP4ZysP7wGe0Vq/mMLXE2chMr3k/G0gcnZvDBHkI9M0kXvFasr8rFm2kEdfbmZLREXRp53dodePnI2/1dLl+ltG0BqPz+thICiLrELYUllF8zFwaaqeX6SWW3Ox15o7Q2mUeEHemabxeRVLq0tcn/+qyulRAb6xLcDqhnbXipjOE2dijtdeXNXIIqsQNmlVIIZkz+5/cdO8sNYNdpBf3dAedY1HmQ/UmlB9e6S6iqKo2b2hzV42bq0H4vWy8XkVeS59bITIZdKqQCTMrW++Xb9eWVwYmjFvaj1Kv3Wgd9DQUfc7zS/3R83idx48gc/rIRgMr4ipry2l/eipqA1PheN8LKwo4qrK6XI+qxAOEuDFsLgF+aARvpBaV1GE16MYsB5gaB2VMrE7QLq1H9CG5psLzue8KeOjgrVbuWT3mQFe2dXJW58eZs2dknsXwiYpGjFs9bWl/PymedgdlTXwXGNHqJNmTZmfFTdW4VXmAqvPo6JSJnbvdzcaqJo12bXxmH9Cfsxyzv4Bg3XbOvjtm7tzrqunEG4kwIsRqa8t5TZHm+CBiDa9lcWFKI9CA25h3O797sbQZtsBtyAd6OmLytvblDLfaKQvvBAmCfBixObOmhz62CD8gOx12zpCm53sWninmjI/q+6oo7621DVg9/YbridDxZrBK+Ca2TMYCEpfeCFskoMXI+Y8ENujzM9tkUHbLYjbpZhVsyZHNR/TwDNb96EgrAeO8zWcvlxcyNTCAtfFWSFylczgxYjZaRYPZlmkcwa/tLqEfJ8Hhdk8zK0W3vZmS5fr7QNBzaqG9rB0S11FES6nKdJ8qJs1De2gNbctKJWNTkIgAV6cBbt9gMejMLQOy5vXlPlZc2cd18yZweziwpi18ACfH4nT1hjoc6Rbasr8/PXsGa6P08CAoZk1ZbwEdyGQAC/OUqCnD0Nr17x3y6FuXt3VyfaO49y33n1DVGNbYLDLWAweFV6Fc9eVF5LvNo13eawQuUwCvDgrzjSNikjTvLzjYNhjIz+3a+F3Rxw+sujiqYzLM9M7HgV3fPWCqLNd1yxbyEXTJoZdpxSsuLEKQEolhUACvDhL8dI0S6pmhj028vNYtfDv7D7CdXOL8Vp9EX7/wd6oYN1yqJs9EW8Ml5X52XHgOLc/uUlKJYVAArxIglhpmvraUn558zwumjaRi6afE3Wdc/bvpDW8sP1AzNRPY1uABzc0RZVLbm0LsKbBvY+NELlIyiTFWbMDdazTlOwUjF0Kabc7sGvhN7Ue5cXtBwbPksUM8l6PAq2jUj+xDgxx9q9XSNMxIWQGL86aHah/cm1lVHniUHn4mjI/P776IkrOnRB2u517V1ZPm4f/vDOsVDLWIiuAV5lvIlIqKXKdBHiRFDVlfuoqitjUejQs7z1UHt42PeKQkGtmz6C7d4CgNS3vGzB43trZai+yLp4zI2oDlUfBIzfN4xc3z5PgLnKepGhEUtgVMZGHdNjpmKffbTXLXGJYWl3Cs40doUXXYy47Vp1ZmZoyP5ecP4VXraMBnbcHevpY3dAurYNFzpMAL5LCrohxLm46A2usPDwMHs93fVUxGz46AMDmvQG8HvB6IGiAzwO3ROyG9UccKG4/11briEGPQo7vEzlNArxIingLrW55eDvAO2f+kfudgoa50KrQeDzR2US3vjTOhdZYbzZC5AoJ8CIpnBUxkWmRJVUzeeezI2Gf25wz/0gK0FqHzll1HirS2Bbgo33H4o7JI8f3iRwnAV4kjfOgbqd4eXjnzB8IK3+cX+5n+75j9AV16FARO03znac20etyGpRt8ZwZXHL+FMnBi5wmAV6MGrc8vHPm75+Qz8N/2kGfFeXPmzKeL80oZJXVw8Z5qEjfgBHzZCefR7H8ygslsIucJ2WSIqka2wKufWDi1cPbtfD1taVcP28wfbPhowPs+6In9Ll9qEisHbBg/kB//cvTk/GlCJHxJMCLpLEXTN36wCRaDx+ZV/94/3GsljShQ0XsWf8VF08N3WczgFd3dUofGiGQAC+SyK1U0mb3pfnaxVP55c3zwsoknUojdrSWnzuBfJ8Hr1XyaC+Y1pT5ufuaL+GLjPCY9fJ9cgC3EJKDF8kzVE8aO6jb6Rm3Wvjjp/vDrikcn8eqO+pY+fYeuk6coeVQdyi3XlPm59b554dy9JGea+xgIGhILbzIWRLgRdLEK5UEWN3QHlpgtcsm62tLw2rhIyfkc2dOCh0cArC9I3yjVOQOWNusyeM4cPyM1MKLnCYpGpFU9oKpWzCNtdDqTO1oDZeVD1771Hufs3ZLu+t1jW0B1m3rwHA5EWpcnhef10ztSC28yFUygxejJtaGp8jUjnMSPxDUUbPzJVUzQ7P+3n73csnWI6fweRS3LShlaXWJzN5FTpIAL0aNMwe/pGqma1/4uooiVvx5Z9h1+T4Pv7x5XthGKXvWH6sW3tAQlAO4RY6TAC+Szl4wdcvDx1pode6CXVhRxPaO46FrFlrpFedGqeWLKkKzfqXMvjWRwd7rldSMyG0JBXil1ETgtNbaUEp9Cfgy8LLWun+IS0WOidU22BZrodXpRO9A2Od7jpxi58ETYbftPHgibAfsgxuaok55uvJL0wDzAG5pWSByUaKLrBuBcUqp84DXgX8H/D5VgxKZK14tPMTf0Wrvgj3S3Rv2mDc+6WLuzElhty2pmhla0A309Lke4ff54ZNyALfIaYmmaJTWukcp9UPgf2qt/6tS6sNUDkxkpqFq4WMttDpn/j6PwqMGW/8GDc2J3gHyvIr+oCbPq6gsLgx7TefjbXsOnwqlbaRUUuSihAO8Umoh8B3gh8O8VuSQoWrhYy20Omf+QUNTOaMw7BDu3Z3doeP7BoI6rHVwTZmfZV+rYOXG1tDjFYM5eTmAW+SqRIP03cDPgPVa651KqQrgzZSNSmS0WG2DbZXFhew4cJydB47T2BYInefqnPn3R0zHvzjVh8+jQq2Dn9m6L9Q6eFPrURbPLQZgw0f7QUPnyV6Uhjyv4tb550uppMhJSrtsEol7gVIe4Byt9YkhHzxM8+fP11u3bk3204o00tgW4PYnPgi1BM73eVhzp7kQ66y+Wfn2nrDzVu0Dtl+JuO2dzw6H0joG5uzeafmiCu69fvZofGlCjAmlVKPWer7bfQktsiqlViulJlnVNLuAFqXU/5/MQYrcsKn1KP2OIOxciHXugl1+5YX4rJ9Oj4KrK6czrbAg7Lm6TpwZXNAN6qjgDkRV3wiRSxKtopljzdhvAl4CSoG/S9WgRPaqqygizzu4VzVWbrymzM8dX63Ao8z2BSte3MncWZPJt6K+UnDB1ImhTpN5XhXVxwZgfJ5XqmdEzko0B5+nlMrDDPD/S2vdr5SKm9tRSp0P/AEoxmzT/YTW+n+czWBF5qsp87Nm2ULWbetAQczceGNbgKfe/TxUGdPbbxDo6eMHl5ezcmMrWpsHgixfVEHh+DzqKop4deehsIVWj4LXmjt5q6VL8vAiJyUa4B8H9gLbgY1KqTJgqN99B4B/0FpvU0oVAo1KqVe11rtGPFqRFWItwjpz8JtajzLgWGjVmKc5RdbV7zx4gn/5YS1A1H325X1BzeqGdtZt65C2wSKnJBTgtdb/BPyT46Y2pdTVQ1xzEDhofdytlGoGzsPM4YssF69dAZg7Wp2lkpE7YL+/sDzqmh0HjkfV0Ts3QPkn5Mccj0Zq4UXuSbRVwWTgPwGLrJveBlYAx2NeFH59OXAp0OBy3zJgGUBpqfspPyKzjKRdQaCnL2wHrNvi6JHuXuprS2k/eorH3zHTNE+/v5fFc4upKfMT6OlzHY/XSuRLLbzINYkusj4NdAPfsv6cAP53Ihcqpc4B1gF3u5VWaq2f0FrP11rPnzZtWoLDEelsJO0K7Dp4u3/7kqqZ5HvDV03f+vQwjW0BunsHsKt7+wYMHn97D799czf+CfmMy4v+ka6aNYmfXFsp6RmRcxLNwV+otb7F8fl/Vkp9NNRF1sLsOmCV1vr5EYxPZKCRtCtw2wFbWVzIij/vDHWWDAbNN4vI1f3Xmjt5rbkzlNr588cH2H/sTOj+GZPGSbMxkZMSDfCnlVJf1Vq/C6CUugI4He8CpZQCfgc0a61/fXbDFJlkpO0KIhdfa8r8LKwoomn/cbQOL6l8Zks79jkgocXUfoMn32mNajz2WnMnGz87LDN4kXMSDfDLgT9YuXiAAPC9Ia65ArNWvskx279Pa/3SsEcpMs5Q7Qrqa0uj2gRD+OJsy6HusLLH7y8sDz3nty8rZXVDe1i/GY9HhVXe2ORcVpGrEq2i2Q78lVJqkvX5CaXU3cDHca55F3DZeiKEKbLSJnJxtsQ/IezxHzhy+c7Dtj0Krpk9g6sqp7v2hVeYC60Hjp0O9b4RIhcM69BtrfUJx0LpT1IwHpEj7J40//0vLdz+xAehYO9cnCWiT1Lk2ayGYX5uaHMBtrK4kEdumkfE2iwaCGpYs7ld+sKLnDKsAB9BZudixNZt6wh1huyz2v9GVtL84KsVYT+gzYe6Wd3QDpiVOkFHvLdTMPW1pTyz/HLOnZgX9npBQ8es6hEiW51NgB9eG0ohHCJnB4rBxVm7pLG+tpR5JZPDHmeXWEb2tPF4VNhGp8Cp8NMkFWbrAqmFF7kkbg5eKdWNeyBXwPiUjEjkBDuHbpdSLrV6u0cuzkYewG3vXK0p8/PwN6p4+t1WWo+cwjA0D72wAzB3vLr90HqU4qEb5koOXuSMuAFea10Y734hRqqmzM+aO91LKZ2Lr5EHcNufN7YFWPHiTnr7jVAwH7CCfHXplKjX04DWOuZuVyGykRy7J8aMHdSd/eAjK2kWXRy+u9k+kNtekI2cqQ8Y2nURVY7tE7lIArxImaEajtmVNPZB2muWLYyqpJlaWIDPq0KHedjtCpy7ZYGw0kiXUnhQSHpG5JyzWWQVIiZ7Jv6rV1piliYmUklTNWtyWMfIgeDghiV7Qfa2BaWhRVt7MTWS1uYCrZRIilwiM3iREm4NxyJnz/EqaTa1HsU/IZ8VL+7kTP9gPaShB9sC28+3blsHXq8iaP0mcFXl9LCzW23vfHaEhs+/CJ0BK0S2kxm8SInImbhb7ntpdQn5Pg8K8/BtZyXNj6++KNRCONKbLV3A4G8JaxraGbB+E0CZAT6yE6Wtb8Dg+W0dyfoyhUhrMoMXKTFUwzH7MWvurAsd3+fU2BZg/7HT+LyeqCD/+eGTgPtCa/+AwY4Dx7l1/vmssjZFRZINHCJXSIAXKTNUwzHb89vMnjL2kXpAqJLG51EsKPezee9g7rzti56whda+fgP7LUADz2zdx4pvVOHzQOQvAPlexS3WbwpCZDtJ0Ygx5Zard94WNDRXVk7n2jkzQtcYhg5baP2Hv6lkQfngG8lAULP+w46oahqvR/HwN6ok/y5yhszgxZiqqyjC51H0BzVejwrl6iMPDKmrKOKtTw/TP2Dg9Q7m9O3fEg4cOx02y9/aFojsVUbQkI1OIrdIgBdjTylAW3+75+8b2wIEDTPfHjTC8y6NbQE0kOc13yiUimpEGdJ9ut/9DiGykKRoxJja1HqUgaAZuPutPDwMVtLYwf2edR+HukcGDXj87T3AYCXNHze3o7W2Dtg2H6eILsV84p3WUEdKIbKdBHgxpuwUDZhx+bnGjtBmpMa2APetb+L2Jzexu+tk2HU7D5gNyMLz9WZ+XmP+YH+lZDKeiJ9wQ8NDL+yQDU8iJ0iAF2OqpszPrfPPD8207YO1nTXubrXw+4+dYXVDe3i9vVeR5/PgwWwfXODzhPWMtxlaS094kRMkwIsxt7S6hIK88E1RkTXubumWl3ccDGtZsGbZQn5weTkoc0E1VtOxfGk6JnKEBHgx5mrK/Dx0w1yqzpsc6h7pnJnnexX1taXctagi7Dpnb/gfX30RAE+9+zmGNtM9hnY5dkyajokcIlU0ImWG6ibpfNzDf9pBX1ADx3nr08OsubPOdSfsoRNn2PDRAQCefu9zFs8tDi3E/ua1TwlGFL9PPSefwycdpZEaKZUUOUMCvEiJyL7uq+6I3eBrU+tR+h39fu0NT/as3Jkv/5MV3MHsQLny7T0sv/JCvvPUprDDP8CcxYcFd6KP9hMim0mAFymRSDdJm32+ap8V5O08vPNNwudRlJ47gcg10zc+6WJ6YUEoX+8BSosm0Ha0x7XnTNDQrHhxJ5XFhZKmEVlPcvAiJRLpJmmrKfOzZtlC6mtLuXbODG6tMXvFON8k+oKa3YdPRV1rl0WG8vV5HpYtujDsQO5IZ/oH6+2FyGYygxcpkUg3ycjHw2CTsXXbOnjohrnk+zxRqRcnDUwq8Lm+1oMbmsJOenJ6dus+bqkukVm8yGoygxcp49yNmojnt3XQ2z+Y1gn09LHqjjpury0NzdB9HiieVBB23VPvfg4Q9lqBnr64bYEHglILL7KfzOBFWmhsC/Ds1n2Dde9W4zHnm8OR7l7e+vQwh070hl07YOhQysXuLT931uSYZ7aCHMAtcoMEeJEWIitp7GZhzoVWj1IYMbqI/XFzO2u3tId2rub7PDz8b+ay48BxNrceDcvfnzsxn59eWynpGZH1JMCLtFBXUYTXoxiw6tgNa1Z+3pTxoYVWHatFJOamJmdOxj7Z6bmt+0LVObYvTvXx8J+lkkZkP8nBi7RQU+ZnxY1V2MUvduMx/4R88q3+MhExPK48n4cj3b1Rwd0mZ7OKXCABXqSN+tpSbltQGmovYM/CV91Rx7ySyQk/z0XTz2HNnXVMKyyI+7iu7t649wuR6STAi7Qyd9ZkPBGz+JZD3TQfPBHzmshGZJ8fOUXLoW7mzor/pvD2p4elbbDIahLgRdpobAuw4sWdYRUv/QMGa7e0hxZgFXBZuR/nPqY8n4fFjjNbg4bm/vVNbPiwI/Rm4cZuTSxEtpIAL9KGvXPVSQM7DhwP5d41EOjpDztQ+69KJnPXlReGDg6xH7d5bwCvR4Xq550UUiopsp8EeJFSjW0Bfvvm7oRSIc72Bs4ZeuShHbu7ToYttm7ZG6DlUDd3fPWCqOccl+fhr2fP4OtfnsHs4sJQKkcD11mdKIXIVlImKVJmOB0lYbC9wbptHRzp7uWNli4GYvUaiLB2SzvXzi2Our37TJBXdnW6XrPhowMsuKCI+trSxL4gITKMzOBFyrh1lEzE89s6eK25E20kWhRppnH8E/LJj9NkzM3aLXIAt8heEuBFygyno6TN+aagNTEXSRUwZfzgL6BBA95s6WLNsoUsnjMj7uKq08cdx1ndIEFeZCcJ8CJlnOelDpWesYUd1ZfnYdnXKqKP3cPMoR87PRB22xufdAGw/MoLuW1BKQvK/a7XRj7P/RuaJMiLrJSyHLxS6mngBqBLa12VqtcR6a2mzD+shUz7fNaXdxxkSdVM6mtLaT1yKmYe3SlomCc8vfPZYXr7DbwexV2LKmhsD7B1byB2y2END72wQ1oXiKyTyhn874HrUvj8IgvZtfDv7T7Cihd30tgWMEsgE8ytv97cGeofP2Bonnr3c740oxA1xOVBR0dKIbJFygK81noj8EWqnl9kp1hH/a21cutDhXlDExbMDW2e+OQbIilv75qVna0im4x5Dl4ptUwptVUptfXw4cNjPRwxxuwcvAdQavCA7JoyP09+dz7P/ehyiofoMfPVi6bi8yg8ymwbfEt1CVdVTh/ytfsH5Cg/kV3GPMBrrZ/QWs/XWs+fNm3aWA9HjDE7B+/xmL3f7TSN8/6/u7w87nO8t+coK26s4h+sxV2ANz6JncP3Wv8LZBYvss2YB3iR/YazmxXM4/YMrTE09LockF1XURS33j1oaNZuacc/IZ9NrUd5fltH1G7YMI7VV+lPI7KJ7GQVKTXc3axgBnCfR9EX1KFZtfOA7JoyPw9/o4r71jfFfI7tHcfZ3tGER5n5d69XhXbFehRhvWwMbbZGMDR4raMChcgGKZvBK6XWAB8AlUqpDqXUD1P1WiJ9jWQ3a02Zn1vnnx/WFz5yFr/zwPGEXt/Q5oz+W/PPp762lGvnzODcc/LDHqMBj70IO1S5jRAZJJVVNLdrrWdqrfO01iVa69+l6rVE+hrJblaApdUl5FlpGLfceOJNDMzXvaW6hFuqS3ijpYsj3X1Rjwka5m8LkqIR2URy8CKlRrKb1b4u3iz+luqSqBbAbpSC7y8sp6bMz+Nv74nZvMyj1LDfhIRId5KDFyk33N2stqXVJTxrHZqtgbVb9lE1a3Ko+6PH4wEj3uqpuUv18Y2tHDpxhtebY1fS3PCVmUwo8A1ZZy9EJpEAL9KWPYtfZfWJCRo61FJgU+tR+gfiB3ebxmwNHM+fth/A5/UwEDR/UxjObxtCpCtJ0YhRMdxSSdvS6pKwXah2S4G6iiK8ibaMTIC9CDzc1sZCpDMJ8CLl7FLJX73Swnee2jSsIF9T5mfFjVWhE5408MzWfQCuJzidDWWdJCV5eJEtJMCLlBvpwR+2+tpS/nr24KHaA0Gza2Th+Ly4OXOPYsi2Bk6GNt9QbqkuGdb4hEhXEuBFysXqLzMc0yIC9WvNnXSf7g+VUgJRwd7Q0Nndi1dBYYE3oQXUzXsDrNncPuzfNIRIRxLgRcoN1V8mEUurS8Jy7nZ1zOTxeYO3uVyngaCG7t5g6H4FlBdNYHZxoetrSR5eZAsJ8GJUDNVfZig1ZX4eubEq7Cg+DRw+Gb1pKR4FFOR5uG5uMS2HumM+zuuVPLzIfFImKUZFZH+ZyJr2RNiPfWBDE8M4jxuFuYDq8yhunX8+c2dN5sENTcQrsrzyS9OkTFJkPJnBi1Fh17Tbgobm/vXDPwu1vraUn980j3gHPDkP4wZzpq+AqyqnM3fWZNZuaSfGhtaQNz7pknNaRcaTAC9GzdLqkrDAPNIDr+trS3lm+eUsnjMDt1L4XpcNUEENr+zq5L71TWzvGLpRmb2pShZaRSaTAC9GTU2ZP6zcEczF0gdGEOTtE56eXX45504Mr8o53W8kpeVA0NCy0CoymgR4MarcDtA2NCOeLdeU+fnptZVRt0dmYJwzfQUJNSrTMKKSTiHShQR4MarCDtB2BF27BcFI1NeWsujiqXEf41yU9SiYMXn8kM/rwaz+ESJTSYAXo85Or/zCsVh6tueh/v01X0r4sUEN+wOnh3ycxzOyTVlCpAsJ8GLM1NeWctuC0rgnNyUq2bly81i/kW3KEiJdSIAXY2qok5sSZdfZJ2qoRxra/NMnO1pFBpMAL8ZUIuevJvo8w+kuOWNSYk3IDC0LrSJzSYAXYy5Zs/hCR1+aeBRw+GRvQo+VhVaRySTAizEXOYsf6cHXdRVF5Mfb4mrRQDCBw6A8CvLzpCeNyFwS4EVaWFpdQkHe2bUUrinzc1Xl9KSNad55k+XoPpHRJMCLtJCMlsIAU4dxwMdQ2o6e4vltIy/dFGKsSYAXaSPQ00fQMFsK9/WPLE1zS3VJUtoUABw7PcCqhnZuf1IO/xCZSQK8SBv+CfmhFgMGI69eUcmK8BY5/ENkKgnwIm0EevpCPWMUsOPA0F0fI21qPYoeRq/4RHg8ShZaRUaSAC/ShnOz0kjLJesqisLOaU0G/4Q8WWgVGUkCvEgbySiXjDxYJBmOnOzj7j9+mNTnFGI0SIAXaSUZ5ZJLq0sSqocfjg0fHZATnkTGkQAv0oqzXNI+VWkkh4EkexYP8PS7rUl/TiFSSQK8SDt2uaQGBgw9ohOfIo8HTIbdh0/JLF5kFAnwIu3UVRThdXSGNEZwrF9NmZ+q8yYnfWz3rW+SmniRMSTAi7RTU+ZnxY1VYcfsGRruX9/Esj9sTTjAfvuy0pSM7/YnPpAgLzKCBHiRluprS/n5TfPCgrwGXtnVya0r309oNl9fW8pNl8xK+tj6gppbHntfKmtE2pMAL9KWW5AHazafYMrmN7ddyuziwpSMb8NHB7j4/pckLy/SltLJ3vZ3FubPn6+3bt061sMQaWZ1Qzv3r28i8idVAYvnzOCuKy+MuxGpsS3ALY+9n9IxApQXTeBX37pENkWJUaWUatRaz3e9TwK8yASrG9p5YEMThsuPqwIunH4OP7jiAupr3fPuj77UzMqNo1fm6PVA5YxCHrlpngR8kVIS4EVWaGwLsPLtPbzW3Bmz38zs4kKqy/wsrS6JCqx3//FDNnx0YBRGGpsEfpFsEuBFVlnd0M6DG5oIxvnR9SiYbwXQL071UTHtHO668kL+5YO9Yx7khyPfq8j3elAexeziQu5ZMlveGESYMQvwSqnrgP8BeIGntNaPxnu8BHiRqNBsfldnVG4+FgVcVu5nd9dJvujpT+XwRBwK8w0YBRPyvHi8iv4BTb5X4fN5KPB5OW/yOKZMyGdaYYHrb2Ni0JgEeKWUF/gUWAx0AFuA27XWu2JdIwFeDFdjW4BHX25my16pSxeZzQO0Pvq3w74uXoBPZZnkAmC31rpVa90H/BG4MYWvJ3JQTZmfZ5dfzrofXU59bSkLyv1RZZVCZAIDqLj3/yT1OX1JfbZw5wH7HJ93ALWRD1JKLQOWAZSWpmbnoch+NWX+0K/xjW0B1m3r4Eh3L8d6+th/7DQHjp1JOJUjxFgxkvx8qQzwbvOoqP9jWusngCfATNGkcDwiRziDvc0Z9G3Hevr44lQf507MZ4qjLfGxnj5aj5zi+Ol++uOt5AqRZMlOqaQywHcAzp6tJUDmlC+IrOIW9BMR+cYQ600h1n3HevrYeeAEp/qCKMwySeU4NNY+ZNy+TwOG4TITEllvpDn4eFIZ4LcAFyulLgD2A7cB9Sl8PSGSbqRvDEKkg5QFeK31gFLqPwB/wSyTfFprvTNVryeEECJcKmfwaK1fAl5K5WsIIYRwJ90khRAiS0mAF0KILCUBXgghspQEeCGEyFJp1U1SKXUYaBvh5VOBI0kcTipl0lghs8abSWOFzBpvJo0VMmu8ZzPWMq31NLc70irAnw2l1NZYDXfSTSaNFTJrvJk0Vsis8WbSWCGzxpuqsUqKRgghspQEeCGEyFLZFOCfGOsBDEMmjRUya7yZNFbIrPFm0lghs8abkrFmTQ5eCCFEuGyawQshhHCQAC+EEFkq4wO8Uuo6pVSLUmq3UureMRzH00qpLqXUDsdt5yqlXlVKfWb97Xfc9zNrzC1Kqb9x3F6jlGqy7vsn5Wwenryxnq+UelMp1ayU2qmU+vt0Ha9SapxSarNSars11v+crmN1vI5XKfWhUurFDBjrXut1PlJKbc2A8U5RSj2nlPrE+vldmI7jVUpVWt9T+88JpdTdoz5WrXXG/sFsQ7wHqADyge3AnDEayyKgGtjhuO2/AvdaH98L/KP18RxrrAXABdbX4LXu2wwsxDwD4mVgSQrGOhOotj4uxDwcfU46jtd63nOsj/OABqAuHcfqGPNPgNXAi+n8c2C9zl5gasRt6TzefwbusD7OB6ak83it1/ICh4Cy0R5rSr6g0fpjfdF/cXz+M+BnYziecsIDfAsw0/p4JtDiNk7MnvkLrcd84rj9duDxURj3C8DidB8vMAHYhnm2b1qOFfPksteBrzMY4NNyrNZz7yU6wKfleIFJwOdYxSHpPl7H818LvDcWY830FI3bwd7njdFY3MzQWh8EsP6ebt0ea9znWR9H3p4ySqly4FLMmXFajtdKeXwEdAGvaq3TdqzAb4D/SPj5yek6VjBPB3xFKdWolFqW5uOtAA4D/9tKgT2llJqYxuO13QassT4e1bFmeoBP6GDvNBRr3KP69SilzgHWAXdrrU/Ee6jLbaM2Xq11UGt9CebseIFSqirOw8dsrEqpG4AurXVjope43DbaPwdXaK2rgSXAj5VSi+I8dqzH68NMgz6mtb4UOIWZ5ohlrMeLUiof+Abw7FAPdbntrMea6QE+3Q/27lRKzQSw/u6ybo817g7r48jbk04plYcZ3FdprZ9P9/ECaK2PAW8B16XpWK8AvqGU2gv8Efi6Uupf03SsAGitD1h/dwHrgQVpPN4OoMP6DQ7gOcyAn67jBfONc5vWutP6fFTHmukBPnSwt/VOeRvwpzEek9OfgO9ZH38PM9dt336bUqpAmYeSXwxstn5l61ZK1Vkr5d91XJM01nP/DmjWWv86ncerlJqmlJpifTweuAb4JB3HqrX+mda6RGtdjvmz+IbW+t+m41gBlFITlVKF9seYueId6TperfUhYJ9SqtK66a+BXek6XsvtDKZn7DGN3lhTtbAwWn+A6zGrQPYA94/hONYAB4F+zHfdHwJFmAtun1l/n+t4/P3WmFtwrIoD8zH/k+0B/hcRC0pJGutXMX/N+xj4yPpzfTqOF/gK8KE11h3AQ9btaTfWiHFfxeAia1qOFTOnvd36s9P+/5Ou47Ve5xJgq/XzsAHwp+t4MYsCjgKTHbeN6lilVYEQQmSpTE/RCCGEiEECvBBCZCkJ8EIIkaUkwAshRJaSAC+EEFlKArzISUqp+5XZnfJjq9tfrdXtb8JYj02IZJEySZFzlFILgV8DV2mte5VSUzE7E74PzNdaHxnTAQqRJDKDF7loJnBEa90LYAX0bwKzgDeVUm8CKKWuVUp9oJTappR61urdY/dQ/0dl9qnfrJS6yLr9VqXUDmX2rt84Nl+aEINkBi9yjhWo38XcafgasFZr/bbVQ2a+1vqINat/HnNH4Sml1D1AgdZ6hfW4J7XWv1BKfRf4ltb6BqVUE3Cd1nq/UmqKNnvnCDFmZAYvco7W+iRQAyzDbD+7Vin1/YiH1WEewvCe1ar4e5gHNtjWOP5eaH38HvB7pdSdmIc8CDGmfGM9ACHGgtY6iNmZ8i1r5v29iIcozN7zt8d6isiPtdbLlVK1wN8CHymlLtFaH03uyIVInMzgRc5R5nmZFztuugRoA7oxjzAE2ARc4civT1BKfclxzbcdf39gPeZCrXWD1voh4Ajh7V+FGHUygxe56Bzgf1ptiAeA3ZjpmtuBl5VSB7XWV1tpmzVKqQLrugcwO5cCFCilGjAnSfYs/79ZbxwKs1Pg9tH4YoSIRRZZhRgm52LsWI9FiHgkRSOEEFlKZvBCCJGlZAYvhBBZSgK8EEJkKQnwQgiRpSTACyFElpIAL4QQWer/AX8/wv6QJp6jAAAAAElFTkSuQmCC",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"plt.plot(loss_monitor.loss, '.')\n",
"plt.xlabel('Steps')\n",
"plt.ylabel('Loss')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"由上可知，通过量子模拟得到的量子版词嵌入模型也能很好的完成嵌入任务。当数据集大到经典计算机算力难以承受时，量子计算机将能够轻松处理这类问题。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 参考文献\n",
"\n",
"[1] Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean. [Efficient Estimation of Word Representations in\n",
"Vector Space](https://arxiv.org/pdf/1301.3781.pdf)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "MindSpore",
"language": "python",
"name": "mindspore"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}