seminars/02_pytorch/seminar02_pytorch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TDm6dCbbPfGy"
   },
   "source": [
    "# 02. Библиотека PyTorch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MlRU0ZTkPs7_"
   },
   "source": [
    "## План\n",
    "1. `numpy` -> `pytorch`\n",
    "2. Автоматическое дифференцирование (`torch.autograd`)\n",
    "3. Пример: задача регрессии\n",
    "4. Переопределение `backward()`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[PyTorch](https://pytorch.org) - это фреймворк для машинного обучения:\n",
    "* В первую очередь - для нейросетевых моделей\n",
    "* Большие возможности для работы с тензорами\n",
    "* Поддержка выполнения на CPU / GPU / TPU ([!](https://github.com/pytorch/xla))\n",
    "* Автоматическое дифференцирование вычислительных графов (`torch.autograd`)\n",
    "* Широкий набор строительных блоков для DL-моделей (`torch.nn`) и готовых архитектур (`torchvision.models`+)\n",
    "* Удобная поддержка реализации своих операций / слоев / функций потерь / ..."
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "!pip install torch  \n",
    "# or goto https://pytorch.org/get-started/locally/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 1. `numpy` <-> `pytorch`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import numpy as np\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [],
    "toc-hr-collapsed": true
   },
   "source": [
    "### 1.1. Работа с тензорами"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "~Все, что можно делать с многомерными массивами в `numpy`, можно делать и в `pytorch`:\n",
    "* `np.ndarray` -> `torch.Tensor`\n",
    "    * [broadcasting](https://numpy.org/doc/stable/user/basics.broadcasting.html)\n",
    "* `np.zeros()` -> `torch.zeros()`\n",
    "* `np.stack()` -> `torch.stack()`\n",
    "* `np.concatenate` -> `torch.cat()` \n",
    "* `np.random.normal()` -> `torch.randn()`\n",
    "* ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### `np.ndarray` <-> `torch.Tensor`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 3)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_np = np.random.uniform(size=(3, 3))\n",
    "data_np.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.83869386, 0.95509596, 0.99434133],\n",
       "       [0.06305396, 0.66546129, 0.45132992],\n",
       "       [0.68912379, 0.91360254, 0.9169107 ]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Создание тензора из данных в форме `numpy.ndarray` через [`torch.from_numpy()`](https://pytorch.org/docs/stable/generated/torch.from_numpy.html):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([3, 3])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_pt = torch.from_numpy(data_np)\n",
    "data_pt.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_np == data_pt.numpy()  # <- get data from Tensor as numpy array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('float64')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_np.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.float64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_pt.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Важно: при создании тензора через `torch.from_numpy()` данные шарятся между объектами:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-999.1613, -999.0449, -999.0057],\n",
       "        [-999.9369, -999.3345, -999.5487],\n",
       "        [-999.3109, -999.0864, -999.0831]], dtype=torch.float64)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = data_pt.numpy() \n",
    "t -= 1000\n",
    "data_pt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-999.16130614, -999.04490404, -999.00565867],\n",
       "       [-999.93694604, -999.33453871, -999.54867008],\n",
       "       [-999.31087621, -999.08639746, -999.0830893 ]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Создание тензора через конструктор `torch.tensor()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 3)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_np = np.random.uniform(size=(3, 3))\n",
    "data_np.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([3, 3])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_pt = torch.tensor(data_np)\n",
    "data_pt.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_np == data_pt.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "При создании тензора через `torch.tensor()` данные копируются:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-999.4625, -999.3375, -999.0617],\n",
       "        [-999.6645, -999.3929, -999.3243],\n",
       "        [-999.2786, -999.1055, -999.3943]], dtype=torch.float64)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = data_pt.numpy()\n",
    "t -= 1000\n",
    "data_pt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.53751275, 0.66251523, 0.93834524],\n",
       "       [0.3355004 , 0.60709476, 0.67570673],\n",
       "       [0.72138201, 0.89449512, 0.6057461 ]])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NB**: [`torch.Tensor()`](https://glaringlee.github.io/tensors.html?highlight=torch%20tensor#torch.Tensor) != [`torch.tensor()`](https://glaringlee.github.io/generated/torch.tensor.html?highlight=torch%20tensor#torch.tensor)!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Типы данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('float64')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_np.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "floats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.float64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_pt = torch.tensor(data_np)\n",
    "data_pt.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.float32"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_pt = torch.tensor(data_np, dtype=torch.float)\n",
    "data_pt.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.float16"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_pt = torch.tensor(data_np, dtype=torch.float16)\n",
    "data_pt.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.float16"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_pt = torch.tensor(data_np, dtype=torch.half)\n",
    "data_pt.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.float64"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_pt = torch.tensor(data_np, dtype=torch.double)\n",
    "data_pt.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ints"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.int32"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_pt = torch.tensor(data_np, dtype=torch.int)\n",
    "data_pt.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.int64"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_pt = torch.tensor(data_np, dtype=torch.long)\n",
    "data_pt.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.int8"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_pt = torch.tensor(data_np, dtype=torch.int8)\n",
    "data_pt.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Больше - [тут](https://pytorch.org/docs/stable/tensors.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Можно и без `numpy`, разумеется:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(2)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.tensor(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1, 2, 3, 4, 5])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.tensor([1, 2, 3, 4, 5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 1.],\n",
       "        [2., 3.]])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.tensor([[0., 1], [2, 3]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Если в тензоре один-единственный элемент, извлечь его численное значение можно через `.item()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.tensor(10)\n",
    "x.item()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "ename": "RuntimeError",
     "evalue": "a Tensor with 2 elements cannot be converted to Scalar",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[28], line 2\u001b[0m\n\u001b[1;32m      1\u001b[0m x \u001b[38;5;241m=\u001b[39m torch\u001b[38;5;241m.\u001b[39mtensor([\u001b[38;5;241m10\u001b[39m, \u001b[38;5;241m11\u001b[39m])\n\u001b[0;32m----> 2\u001b[0m \u001b[43mx\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mitem\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n",
      "\u001b[0;31mRuntimeError\u001b[0m: a Tensor with 2 elements cannot be converted to Scalar"
     ]
    }
   ],
   "source": [
    "x = torch.tensor([10, 11])\n",
    "x.item()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NB**: поведение конструкторов `torch.tensor()` и `torch.Tensor()` - разное:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.Tensor([1, 2, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.Tensor(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.Tensor(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Broadcasting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_np = np.random.uniform(size=(3, 3))\n",
    "data_pt = torch.from_numpy(data_np)\n",
    "data_pt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt -= 100\n",
    "data_pt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Обратите внимание на особенности broadcasting в \"неочевидных\" случаях (часто лучше перепроверить):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_np = np.random.uniform(size=(3, 3)).astype(np.float32)\n",
    "data_pt = torch.from_numpy(data_np)\n",
    "data_pt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "b = torch.Tensor([100, 50, 0])\n",
    "b.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt + b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "b = torch.Tensor([100, 50, 0]).view(1, 3)\n",
    "b.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt + b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "b = torch.Tensor([100, 50, 0]).view(3, 1)\n",
    "b.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt + b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Векторные и матричные операции"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_np = np.random.uniform(size=(16, 16))\n",
    "data_pt = torch.from_numpy(data_np)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%timeit data_np @ data_np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%timeit data_pt @ data_pt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.randn(4, 1)\n",
    "A = torch.randn(8, 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Умножение матрицы на вектор:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = A @ x\n",
    "y.shape, y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = A.matmul(x)\n",
    "y.shape, y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Тоже повнимательнее с размерностями:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.randn(4)\n",
    "A = torch.randn(8, 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = A @ x\n",
    "y.shape, y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Задание:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Даны два набора векторов одинаковой размерности, `X` (`m x dim`) и `Y` (`n x dim`)\n",
    "* Требуется посчитать (без циклов!) попарные значения косинуса углов для всех пар (`x`, `y`)\n",
    "    * На выходе ожидается матрица размером (`m x n`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dim = 8\n",
    "m = 11\n",
    "n = 17\n",
    "X = torch.randn(m, dim)\n",
    "Y = torch.randn(n, dim)\n",
    "\n",
    "# YOUR CODE HERE\n",
    "X_n = X / torch.norm( X, dim = 1, keepdim=True)\n",
    "Y_n = Y / torch.norm( Y, dim = 1, keepdim=True)\n",
    "cos_mat = X_n @ Y_n.T\n",
    "\n",
    "# END OF YOUR CODE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.spatial.distance import cdist\n",
    "cos_mat_expected = 1 - cdist(X, Y, metric=\"cosine\")\n",
    "\n",
    "torch.testing.assert_allclose(cos_mat, cos_mat_expected)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Работа с размерностями"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.randn(3, 1)\n",
    "x.shape, x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Транспонирование:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xT = x.T\n",
    "xT.shape, xT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Если не указано явно, то данные НЕ копируются:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xT *= 100\n",
    "xT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "С копированием:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xT_clone = x.T.clone()\n",
    "xT_clone.shape, xT_clone"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xT_clone == xT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xT_clone *= 100\n",
    "xT_clone"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Доступны и более сложные операции над размерностями.\n",
    "\n",
    "`permute()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.randn(3, 64, 256)\n",
    "x.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.permute(1, 2, 0).shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.transpose(1, 2).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`view()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.randn(3, 64, 256)\n",
    "x.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.view(3, 256, 64).shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.view(3, -1).shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.view(-1).shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = x.view(-1)\n",
    "z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z *= 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.randn(3, 64, 256)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = x[:,:3,:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(id(y.data))\n",
    "print(y.view(-1))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = y.reshape(-1)\n",
    "print(id(z.data))\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Задание:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Создать тензор размером `1x3x224x224` (имитируем батч из одной картинки с 3 каналами RGB размером `224х224`)\n",
    "    * Все значения в тензоре = `0.5`\n",
    "* Выполнить нормализацию через mean / std из датасета Imagenet:\n",
    "    * mean = `[0.485, 0.456, 0.406]`\n",
    "    * std = `[0.229, 0.224, 0.225]`\n",
    "* Посчитать среднее значение по каждому из 3-х каналов полученного тензора\n",
    "    * Ответ (`output`) должен иметь размерность (3,)\n",
    "\n",
    "\n",
    "NB: агрегирующие операции типа `min`/`max`/`mean`/... устроены аналогично в `numpy` и `torch`, но отличаютеся ключевым словом для уточнения размерностей:\n",
    "* в `numpy` - `axis`\n",
    "* в `torch` - `dim`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "\n",
    "x    = torch.ones(1,3,224,224)*0.5\n",
    "mean = torch.tensor([0.485, 0.456, 0.406]).view(1,-1,1,1)\n",
    "std  = torch.tensor([0.229, 0.224, 0.225]).view(1,-1,1,1)\n",
    "\n",
    "x = (x - mean)/std\n",
    "output = torch.mean(x, dim=(2,3)).view(-1)\n",
    "# END OF YOUR CODE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "expected_output = torch.Tensor([0.0655, 0.1964, 0.4178])\n",
    "torch.testing.assert_allclose(output, expected_output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [],
    "toc-hr-collapsed": true
   },
   "source": [
    "### 1.2. CPU <-> GPU"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`CUDA` - это библиотека для вычислений на графических ускорителях (от NVidia).\n",
    "`torch` умеет работать с GPU, но для этого в системе должны быть установлены совместимые версии `torch` / `cudatoolkit`. \n",
    "Подробнее можно почитать на [официальном сайте](https://pytorch.org/get-started/locally/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Получение информации о доступных \"девайсах\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.cuda.is_available()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.cuda.device_count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "device_alias = \"cuda:0\"\n",
    "# device_alias = \"cpu\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.cuda.get_device_name(device_alias)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "device = torch.device(device_alias)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "device"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Перемещение тензоров между девайсами: `.to(...)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt = torch.randn(8, 8)\n",
    "data_pt.device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt = data_pt.to(device)\n",
    "data_pt.device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = data_pt.to(torch.device(\"cpu\"))\n",
    "b = data_pt.to(\"cpu\")\n",
    "c = data_pt.cpu()\n",
    "\n",
    "a.device, b.device, c.device"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Скорость вычислений"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`8х8` @ `8x8`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt = torch.randn(8, 8)\n",
    "data_pt.device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%timeit data_pt @ data_pt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt = data_pt.to(device)\n",
    "data_pt.device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt @ data_pt;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%timeit data_pt @ data_pt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`64х64` @ `64x64`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt = torch.randn(64, 64)\n",
    "data_pt.device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%timeit data_pt @ data_pt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt = data_pt.to(device)\n",
    "data_pt.device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%timeit data_pt @ data_pt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`256х256` @ `256x256`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt = torch.randn(256, 256)\n",
    "data_pt.device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%timeit data_pt @ data_pt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt = data_pt.to(device)\n",
    "data_pt.device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%timeit data_pt @ data_pt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Объекты с разных девайсов"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt = torch.randn(3, 3)\n",
    "data_pt = data_pt.to(device)\n",
    "data_pt.device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt.numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt.to(torch.device(\"cpu\")).numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Взаимодействовать друг с другом могут только объекты, лежащие на одном девайсе"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt_1 = torch.randn(3, 3)\n",
    "data_pt_1 = data_pt_1.to(device)\n",
    "data_pt_1.device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt_2 = torch.randn(3, 3)\n",
    "data_pt_2 = data_pt_2#.to(\"cpu\")\n",
    "data_pt_2.device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt_1 + data_pt_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt_1 + data_pt_2.to(data_pt_1.device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt_2.device"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Поддержка \"особенных\" типов на GPU"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://docs.nvidia.com/deeplearning/tensorrt/support-matrix/index.html#hardware-precision-matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt = torch.randn(3, 3).type(torch.float16).to(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt = torch.randn(3, 3).type(torch.int8).to(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pt + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Прекрасно, что `pytorch` умеет делать все то же, что и `numpy`.\n",
    "Но зачем он нужен, если *уже есть* `numpy`?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 2. Автоматическое дифференцирование"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Вспомним, что алгебраические выражения можно интерпретировать как вычислительные графы:\n",
    "\n",
    "* $f(x) = w1 \\times x + w2 \\times y + w3 \\rightarrow $\n",
    "\n",
    "![comp_graph_03](https://i.ibb.co/f8yv1Kc/aim-seminar01-compgraphs-11-1.png)\n",
    "\n",
    "На прошлом семинаре мы считали градиенты руками, а теперь у нас есть `torch`:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### `.backward()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(x, y, w1, w2, w3):\n",
    "    return w1 * x + w2 * y + w3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.tensor([1.])\n",
    "x.requires_grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = torch.tensor([2.])\n",
    "y.requires_grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Для \"включения\" градиентов у переменной, нужно об этом прямо заявить: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w1 = torch.tensor([0.33])\n",
    "w1.requires_grad_(True)\n",
    "w1.requires_grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w2 = torch.tensor([-1.5]).requires_grad_(True)\n",
    "w2.requires_grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w3 = torch.tensor([0.01], requires_grad=True)\n",
    "w3.requires_grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "variables = [x, y, w1, w2, w3]\n",
    "names = [\"x\", \"y\", \"w1\", \"w2\", \"w3\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"name\\tval\\tgrad\")\n",
    "for n, v in zip(names, variables):\n",
    "    print(f\"{n}\\t{v.item():2.3f}\\t{v.grad}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Еще раз насладимся магией автоматического дифференцирования:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output = f(x, y, w1, w2, w3)\n",
    "output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"name\\tval\\tgrad\")\n",
    "for n, v in zip(names, variables):\n",
    "    print(f\"{n}\\t{v.item():2.3f}\\t{v.grad}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output.backward()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"name\\tval\\tgrad\")\n",
    "for n, v in zip(names, variables):\n",
    "    print(f\"{n}\\t{v.item():2.3f}\\t{v.grad}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Попробуем вызвать `backward()` еще раз:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output.backward()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Вопрос:** зачем может быть нужно делать `backward()` больше одного раза?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "del output\n",
    "for var in variables:\n",
    "    var.grad = None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"name\\tval\\tgrad\")\n",
    "for n, v in zip(names, variables):\n",
    "    print(f\"{n}\\t{v.item():2.3f}\\t{v.grad}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output = f(x, y, w1, w2, w3)\n",
    "output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output.backward(retain_graph=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"name\\tval\\tgrad\")\n",
    "for n, v in zip(names, variables):\n",
    "    print(f\"{n}\\t{v.item():2.3f}\\t{v.grad}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output.backward()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"name\\tval\\tgrad\")\n",
    "for n, v in zip(names, variables):\n",
    "    print(f\"{n}\\t{v.item():2.3f}\\t{v.grad}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output.backward()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### `torch.no_grad()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Данный контекст гарантирует, что во всех вычислениях внутри него будут отключены градиенты."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.randn(64, 512, 16, 16)\n",
    "x.requires_grad_(True);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = (10 * x).sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y.requires_grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with torch.no_grad():\n",
    "    z = (100 * x).sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z.requires_grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.requires_grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Зачем это может быть нужно?\n",
    "\n",
    "Посмотрим на потребление памяти при использовании функции активации сигмоида:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "\n",
    "device = torch.device(\"cuda:0\")\n",
    "\n",
    "from torch.nn import Sigmoid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bytes_in_kilobyte = 1024\n",
    "\n",
    "def get_allocated_memory():\n",
    "    mem_MB = torch.cuda.memory_allocated(device=device) / bytes_in_kilobyte\n",
    "    return round(mem_MB, 3)\n",
    "\n",
    "def get_tensor_memory(t):\n",
    "    mem_MB = t.nelement() * t.element_size() / bytes_in_kilobyte\n",
    "    return round(mem_MB, 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.cuda.empty_cache()\n",
    "get_allocated_memory()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = Sigmoid().to(device)\n",
    "get_allocated_memory()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.randn(64, 512, 16, 16).to(device)\n",
    "x.requires_grad_(True);\n",
    "get_tensor_memory(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "64 * 512 * 16 * 16 * 4 / 1024"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_allocated_memory()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = s(x).mean()\n",
    "get_tensor_memory(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_allocated_memory()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y.backward()\n",
    "get_allocated_memory()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Сделаем то же самое, но с `torch.no_grad()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# torch.cuda.empty_cache()\n",
    "get_allocated_memory()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = Sigmoid().to(device)\n",
    "get_allocated_memory()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.randn(64, 512, 16, 16).to(device)\n",
    "x.requires_grad_(True);\n",
    "get_tensor_memory(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_allocated_memory()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with torch.no_grad():\n",
    "    y = s(x).mean()\n",
    "get_tensor_memory(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_allocated_memory()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Почему потребление памяти уменьшилось?\n",
    "Подробнее [тут](https://medium.com/deep-learning-for-protein-design/a-comprehensive-guide-to-memory-usage-in-pytorch-b9b7c78031d3) или [тут](https://pytorch.org/tutorials/recipes/recipes/tuning_guide.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Пример задачи регрессии"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import tqdm\n",
    "import torch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtkAAAE9CAYAAADecH6XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnCUlEQVR4nO3de5Bc5Xnn8d9vdEcjg9CICUZYwmvWcdYbGzTLZb3rmoltQijHbFzORtiV4LJhYm/wOrXr9SXe9TUxceW22ZCKIwNFXAFN4jhUCME2F3sgZBFG4xI2N68VCoURBKRBEI2EJEb97B/TjUZN99z6XN7u/n6qptR9+kyfd/qZaT39nuc8ryNCAAAAALLTU/YAAAAAgE5Dkg0AAABkjCQbAAAAyBhJNgAAAJAxkmwAAAAgYyTZAAAAQMaWlj2APPT19cWmTZtyP87Bgwe1evXq3I+DhSM26SI26SI26SI26SI26SoiNmNjY/siYn2jxzoyyd60aZN27NiR+3FGR0c1ODiY+3GwcMQmXcQmXcQmXcQmXcQmXUXExvbuZo9RLgIAAABkjCQbAAAAyBhJNgAAAJCxjqzJbuSll17S+Pi4Dh8+nNlznnzyyXr00Ucze756K1eu1IYNG7Rs2bLcjgEAAIDsdU2SPT4+rjVr1mjTpk2ynclzHjhwQGvWrMnkuepFhCYmJjQ+Pq6zzjorl2MAAAAgH11TLnL48GGtW7cuswQ7b7a1bt26TGfeAQAAUIyuSbIltU2CXdNu4wUAAMC0rkqyU3TDDTfoqaeeKnsYAAAAyBBJdslIsgEAAOZWqYT2HjiiiCh7KPPSNRc+puCLX/yi/vzP/1zr16/XmWeeqc2bN2vHjh163/vep1WrVum+++7TqlWryh4mAABAUiqV0GVf3a6x3fu1eeNabbvyAvX0pF1Wm/tMtu3rbT9r+6EZ2/7C9s7q1xO2dzb53ids/7C6X/7rpNfJ8hPTAw88oG984xt68MEH9c1vfvPlZd8HBgZ04403aufOnSTYAAAADUwcPKqx3fs1VQmN7d6viYNHyx7SnIqYyb5B0jWSvlbbEBG/VLtt+/ckvTDL9w9FxL7cRtdE1p+Y/uEf/kGXXnqpVq5cqZUrV+rnf/7nMxwtAABA5+rrXa7NG9e+nJf19S4ve0hzyj3Jjoh7bG9q9Jin22f8Z0k/k/c4FqrRJ6b1a1aUPSwAAICuY1vbrrxAEwePqq93eVt0YCv7wsf/KOmZiPhxk8dD0u22x2wPFziulz8xLe1xJp+Y3vKWt+hv//ZvdfjwYU1OTurWW2+VJK1Zs0YHDhzIYsgAAAAdq6fHWr9mRVsk2JLkIq7QrM5k3xoRb6zb/ieSdkXE7zX5vjMiYo/t0yTdIekjEXFPk32HJQ1LUn9//+aRkZETHj/55JP1ute9bkHjrkTouYMvad3qZQ0DeuzYMS1ZsmTez/elL31JX//613Xaaadp/fr1evvb3661a9fq85//vFatWqU777zzFXXZu3bt0gsvzFZNg0YmJyfV29tb9jDQALFJF7FJF7FJF7FJVxGxGRoaGouIgUaPlZZk214qaY+kzRExPo/n+JykyYj43bn2HRgYiNqFhTWPPvqo3vCGNyxw5LNb6LLqtWAfOnRIb33rW7V161ade+65s35PHuPuBqOjoxocHCx7GGiA2KSL2KSL2KSL2KSriNjYbppkl9nC7+2SHmuWYNteLaknIg5Ub18k6QtFDjBrw8PDeuSRR3T48GFdfvnlcybYAAAAaE+5J9m2t0kalNRne1zSZyPiOklbJG2r2/fVkq6NiEsk9Uu6uVqmsVTSTRHxrbzHm6ebbrqp7CEAAACgAEV0F7msyfb3N9j2lKRLqrcfl/SmXAcHAAAA5KDs7iKFapdlOGvabbwAAACY1jVJ9sqVKzUxMdE2iWtEaGJiQitXrix7KAAAAFigMi98LNSGDRs0Pj6uvXv3Zvachw8fzjUJXrlypTZs2JDb8wMAACAfXZNkL1u2TGeddVamzzk6Oqpzzjkn0+cEAABA++uachEAAACgKCTZAAAAQMZIsgEAAICMkWQDAAAAGSPJBgAAADJGkg0AAABkjCQbAAAAyBhJNgAAAJAxkmwAAAAgYyTZAAAAQMZIsgEAANC2KpXQ3gNHFBFlD+UES8seAAAAALAYlUrosq9u19ju/dq8ca22XXmBenpc9rAkMZMNAACANjVx8KjGdu/XVCU0tnu/Jg4eLXtILyPJBgAAQFvq612uzRvXammPtXnjWvX1Li97SC+jXAQAAABtyba2XXmBJg4eVV/vctlplIpIJNkAAABoYz091vo1K8oexitQLgIAAIBCpNoJJA/MZAMAACB3KXcCyUPuM9m2r7f9rO2HZmz7nO09tndWvy5p8r0X2/6R7V22P5n3WAEAAJCPlDuB5KGIcpEbJF3cYPsfRMSbq1+31T9oe4mkP5b0c5J+StJltn8q15ECAAAgFyl3AslD7uUiEXGP7U2L+NbzJO2KiMclyfaIpEslPZLh8AAAAFCAlDuB5KHMCx+vsv2DajnJ2gaPnyHpyRn3x6vbAAAA0IZqnUA6PcGWJBdxdWd1JvvWiHhj9X6/pH2SQtIXJZ0eER+o+573SLo4Iq6o3v9lSedHxFVNjjEsaViS+vv7N4+MjOT00xw3OTmp3t7e3I+DhSM26SI26SI26SI26SI26SoiNkNDQ2MRMdDosVK6i0TEM7Xbtr8q6dYGu+2RdOaM+xuq25o951ZJWyVpYGAgBgcHMxnrbEZHR1XEcbBwxCZdxCZdxCZdxCZdxCZdZcemlHIR26fPuPsLkh5qsNsDks62fZbt5ZK2SLqliPEBAACgdd3UF7te7jPZtrdJGpTUZ3tc0mclDdp+s6bLRZ6Q9KvVfV8t6dqIuCQipmxfJenbkpZIuj4iHs57vAAAAGhdt/XFrldEd5HLGmy+rsm+T0m6ZMb92yS9or0fAAAA0taoL3aKy5/nhWXVAQAAkLlu64tdj2XVAQAAkLlu64tdjyQbAAAAuaj1xe5GlIsAAAAAGSPJBgAAADJGkg0AAABkjCQ7I93cbB0AAGA+uilf4sLHDHR7s3UAAIC5dFu+xEx2Bho1WwcAAMBx3ZYvkWRnoNubrQMAAMyl2/IlykUy0O3N1gEAAObSbfkSSXZGurnZOgAAwHx0U75EuQgAAACQMZJsAAAAIGMk2QAAAEDGSLIBAACAjJFkAwAA4BW6aXXGPNBdBAAAACdotjpjpRJd04KvVSTZAAAAOEGj1RnXrV7eVcuit4pykYxwSgUAAHSKRqszdtuy6K1iJjsDzU6pAAAAlGmx5R2NVmesJd61fKfTl0VvFUl2Bhp9suuW1YwAAECaWp0ErF+dsduWRW9V7uUitq+3/azth2Zs+x3bj9n+ge2bbZ/S5HufsP1D2ztt78h7rIvV6JQKAABAmfIo76gl3iTYcytiJvsGSddI+tqMbXdI+lRETNn+sqRPSfpEk+8fioh9+Q6xNXyyAwAAqaG8o1y5J9kRcY/tTXXbbp9xd7uk9+Q9jrzVn1IBAAAoE5OA5Uqhu8gHJH2zyWMh6XbbY7aHCxwTAABA26O8ozwuouVcdSb71oh4Y932T0sakPTuaDAQ22dExB7bp2m6xOQjEXFPk2MMSxqWpP7+/s0jIyMZ/xSvNDk5qd7e3tyPg4UjNukiNukiNukiNulq99hMVUJLO7QjWhGxGRoaGouIgUaPldZdxPb7Jb1T0tsaJdiSFBF7qv8+a/tmSedJaphkR8RWSVslaWBgIAYHB3MY9YlGR0dVxHGwcMQmXcQmXcQmXcQmXe0am25oP1x2bEopF7F9saSPS3pXRBxqss9q22tqtyVdJOmhRvsCAABg/lhYJn9FtPDbJuk+Sa+3PW77g5ruNrJG0h3V9nxfqe77atu3Vb+1X9K9th+U9D1JfxcR38p7vAAAAJ2O9sP5K6K7yGUNNl/XZN+nJF1Svf24pDflODQAAICuROeR/LHiIwAAQBei/XC+UmjhBwAAAHQUkmwAAAAgYyTZAAAAQMZIsgEAAICMkWS3gUoltPfAERWxOicAAABaR3eRxHXDikwAAACdhpnsxLEiEwAAQPshyU4cKzIBAAC0H8pFEjfbikyVSrBSEwAAQIJIsttAoxWZqNUGAABIF+UibYpabQAAgHSRZOcoz9Z71GoDAIAs0TI4W5SL5CTvco5GtdrUaAMAgPmozxkoQ80eSXZOGpVz1NdVt2pmrTZ/HAAAYD4a5QxF5C3dhnKRnBRdzkGNNgAAmI9GOQNlqNljJjsns7Xey0Ptj6P2qZQ/DgAA0EijnKHovKUbkGTnqFHrvbzwxwEAAOajWc5QZN7SDUiyOwh/HAAAYD7IGfJHTTYAAACQMZLsNkL/SgAAgPZAuUiboEUfAABA+yhkJtv29baftf3QjG2n2r7D9o+r/65t8r2XV/f5se3LixhvimjRBwAAJM5st4uiykVukHRx3bZPSrorIs6WdFf1/glsnyrps5LOl3SepM82S8Y7Hf0rAQDoTjOT6tqZ7Quvvktbtm5XpUKinapCykUi4h7bm+o2XyppsHr7zySNSvpE3T4/K+mOiHhOkmzfoelkfVteY00VLfoAAOg+9eWi/2fLOazM2CbKvPCxPyKert7+Z0n9DfY5Q9KTM+6PV7d1pVq7HRJsAAC6Q325qC3ObLcJF1XPU53JvjUi3li9/3xEnDLj8f0Rsbbuez4maWVE/Gb1/v+S9GJE/G6D5x+WNCxJ/f39m0dGRvL6UV42OTmp3t7e3I+DhSM26SI26SI26SI26SoiNo/vO6hDR47ppBVL9Nq+1ZKkqUpoKQ0QZlVEbIaGhsYiYqDRY2V2F3nG9ukR8bTt0yU922CfPTpeUiJJGzRdVvIKEbFV0lZJGhgYiMHBwUa7ZWp0dFRFHAcLR2zSRWzSRWzSRWzSVURs3loJykUXoey/mzLLRW6RVOsWcrmkv2mwz7clXWR7bfWCx4uq2wAAALoC5aLtqagWftsk3Sfp9bbHbX9Q0m9LeoftH0t6e/W+bA/YvlaSqhc8flHSA9WvL9QuggQAAABSVVR3kcuaPPS2BvvukHTFjPvXS7o+p6EBAAAAmWNZ9S5EE3sAAIB8sax6l2F5dgAAgPwxk91lWJ4dAAAgfyTZXYbl2QEAAPJHuUiXYXl2AACA/JFkd6Fav00AAADkg3IRAACAhNAFrDMwkw0AAJAIuoB1DmayAQAAEkEXsM5Bkg0AAJAIuoB1DspFAAAAEkEXsM5Bkg0AAJAQuoB1BspFAAAAgIyRZAMAAAAZI8kGAAAAMkaSDQAAsAAsFoP54MJHAACAeapfLObDry97REgVM9kdgE/UAAAUo36xmKkK//eiMZLsNlf7RH3h1Xdpy9btqvDHDgBAbuoXi1nKkudognKRNtdo+VV6awIAkI/6xWLuvvvusoeERM05k237I7bXFjEYLBzLrwIAUKzaYjGsxojZzGcmu1/SA7a/L+l6Sd8Oin+TwfKrAAAA6ZlzJjsi/qeksyVdJ+n9kn5s+0u2/1UrB7b9ets7Z3z9i+1fr9tn0PYLM/b5TCvH7FR8ogYAAEjLvGqyIyJs/7Okf5Y0JWmtpL+yfUdEfHwxB46IH0l6syTZXiJpj6SbG+z69xHxzsUcA8WqVIIZdQAAAM0jybb9UUm/ImmfpGsl/Y+IeMl2j6QfS1pUkl3nbZL+MSJ2Z/BcKFAtsT71pGV677X3v9w3dNuVF6iHK64BAECXms9M9qmS3l2fAEdExXZWM8xbJG1r8tiFth+U9JSkj0XEwxkdEy2a2ZD/3244WT948nkdC9HlBAAAdD2XfQ2j7eWaTqD/TUQ8U/fYqyRVImLS9iWS/jAizm7yPMOShiWpv79/88jISM4jlyYnJ9Xb25v7cVI1VQk99vQBhUKWtWr5Er149JhOWrFEr+1bXerYuj02KSM26SI26SI26SI26SoiNkNDQ2MRMdDosRSS7Esl/VpEXDSPfZ+QNBAR+2bbb2BgIHbs2JHRCJsbHR3V4OBg7sdJVURoy9bjS8vedMX5eu7QS0nUZHd7bFJGbNJFbNJFbNJFbNJVRGxsN02yU1iM5jI1KRWx/ROSnqleeHmepruhTBQ5ODTXqH0gJSIAAAAlJ9m2V0t6h6RfnbHtQ5IUEV+R9B5JH7Y9JelFSVvo0Z2WWvtAAAAAHFdqkh0RByWtq9v2lRm3r5F0TdHjAgAAAFox52I0QFYqldDeA0fEyQgAANDpUqjJRheY2e6PPtoAAKDTMZONQkwcPKqx3fs1VYmX+2gDAFAkzqiiSCTZyFXtDW3d6mXavHGtlvZYmzeuVV/v8rKHBgDoIrUzqhdefZe2bN2uSmX2RJuEHK2iXAS5qS8RufGD52v/i2n00QYAdKZKJU5oLVu7HxHasXu/jlWm/51tZWJKHJEFkmzkpr5EZP+LL9HuDwCQuVoifepJy/Tea+8/YXLnfddN3z/3Nado1bIeTR45ppOWL9GpJy1r+nyNShz5/wsLRZKN3PT1LtfmjWtffrOjRAQAkLWZs84/veFkPTj+go5Vk+NdeyePJ8v/9LyiWiJy6MiUnjvUfOKH/7+QBZJs5KbRipAAAGRp5qzzg08+r58+8xT9cPwFbd64Vv+6v/flZPncjWulCH3/n56fM3Hm/y9kgSQbuWJFSABAq+rrrGeqn3W+6Yrz9dyh49f/zEyWIzTvxJn/v9Aqkmy0jdneZAEAnWmuixAbzTrPTI5nJsu2SJxRGJJstAWu9AaA7jSfixCZdUaK6JONtsBiNgDQHrLuL10rB1nIOgv0uEYKmMlGW+BKbwBIXx5nHRd6ESJnPpEKkmy0Ba70BoD05dVfulE5SLPrdOhxjVRQLoK2UXuTJcEGgDQtprRjMWZbIr2oMQBzYSYbAABkoqizjrPNVnPmE6lgJhsAAGSmiLOOc81Wc+YTKWAmGwAAtBVmq9EOmMkGAAC5yqOlHrPVSB0z2QAAIDdFttRjZWCkhJlsJKvRzAcLDABAeylqMbHZOo4AZSDJRpIavVnyBgoA7aeolnqsDIzUlF4uYvsJSQckHZM0FREDdY9b0h9KukTSIUnvj4jvFz1OFKvZmyULDABAeynqIkVWBkZqSk+yq4YiYl+Tx35O0tnVr/Ml/Un1X3SwZm+WvIECQPtptGJj1ug4gtSkkmTP5lJJX4vpItzttk+xfXpEPF32wJCfZm+WvIECAJopIpkH5iuFmuyQdLvtMdvDDR4/Q9KTM+6PV7ehwzVqz0TLJgAA0A5cdpcG22dExB7bp0m6Q9JHIuKeGY/fKum3I+Le6v27JH0iInbUPc+wpGFJ6u/v3zwyMpL72CcnJ9Xb25v7cbBwC4nNVCW0NKd2UikcLzX83aSL2KSL2KSL2KSriNgMDQ2N1V9PWFN6uUhE7Kn++6ztmyWdJ+meGbvskXTmjPsbqtvqn2erpK2SNDAwEIODg3kN+WWjo6Mq4jhYuPnEZq7erVn3Wy2yV2zK+LtJF7FJF7FJF7FJV9mxKbVcxPZq22tqtyVdJOmhut1ukfQrnnaBpBeox0YWZmv3lEe7QNpLAQDQPcquye6XdK/tByV9T9LfRcS3bH/I9oeq+9wm6XFJuyR9VdJ/KWeo6DSz9W7NIyEuqlcsAAAoX6nlIhHxuKQ3Ndj+lRm3Q9KvFTkudIfZ2j3l0W+V9lIAAHSP0muygTI1a/eUV0JMeykAKWl07UnW16MA3YokG2iChBhAJ2t0MbYkLtAGMlJ2TTYAAChBo2tPuEAbyA5JNgAAXajRxdhcoA1kh3IRAAC6ULNrT7hAG8gGSTYAAF2q0bUnXI8CZINyEQAAACBjJNkAAABAxkiyAQAAgIyRZAMAAAAZI8kGAAAAMkaSDQAAAGSMJBsAAADIGEk2AAAAkDGSbAAAulSlEtp74IiOHato74EjioiyhwR0DFZ8BACgC1Uqocu+ul07nnhOJ61YqkNHj2lg41ptu/IC9fSwnDrQKmayAQDoQhMHj2ps934dC+nA4Skdq4TGdu/XxMGjZQ8N6Agk2QAAtJlamUcr5R19vct17mtOOWHbua9Zq77e5S2ODoBEkg0AQFuplXlcePVd2rJ1uyqVxSXatvVHl52rJdXSkCWWrnnvObIpFQGyQJINAEAbqZV5TGVQ3nHaq1ZoYONaLe2xBjadqvVrVmQ4UqC7ceEjAAAlqFRCEwePqq93+YJmj/t6l2vzxrUa271fmze2Vt5hW9uuvGBR4wAwO5JsAAAKViv5qCXKC+nokXVi3NPjJGewF/shBEhFaeUits+0/V3bj9h+2PZHG+wzaPsF2zurX58pY6wAAGSp1ZKPWmLcqclnVnXnQJnKnMmekvTfI+L7ttdIGrN9R0Q8Urff30fEO0sYHwAAuciy5KMTNfoQkuJsOzCb0pLsiHha0tPV2wdsPyrpDEn1STbQVjjFCWAu1ELPjg8h6ARJ1GTb3iTpHEn3N3j4QtsPSnpK0sci4uEixwYsRCt1lgC6S6q10CngQwg6gVtpZJ/JAOxeSXdL+q2I+Ou6x14lqRIRk7YvkfSHEXF2k+cZljQsSf39/ZtHRkZyHrk0OTmp3t7e3I+DhSsrNlOV0GNPH1AoZFk/efoaLSXJPgF/N+kiNukiNukiNukqIjZDQ0NjETHQ6LFSk2zbyyTdKunbEfH789j/CUkDEbFvtv0GBgZix44d2QxyFqOjoxocHMz9OFi4smITEdqy9fhM9sjwBczA1OHvJl3E5kQplX4Rm3QRm3QVERvbTZPs0spFPP2OdZ2kR5sl2LZ/QtIzERG2z9N0N5SJAocJLAinOIHOQOkXgFaVWZP9Fkm/LOmHtndWt/2GpNdIUkR8RdJ7JH3Y9pSkFyVtibLrW4A5UGcJtD+6WwBoVZndRe6VNOu0QERcI+maYkYEdKZGp7xTOg0OpIjuFgBalUR3EQD5aHTKWxKnwYE5UPoFoFWlrfgIIH+NTnm3utIc0C06fVVFAPkiyQY6WO2U99Iev3zKu9E2AACQLcpFgA7W7JQ3p8EBAMgXM9lAh2t0ypvT4EA+KpXQ3gNHlEcjrDyfG0D2mMkGALSF1Lvi5N1bmwuWgfbCTDYAIHm1BPbCq+/Slq3bVamkN5ub50XFtefkgmWgfZBkAwCSl1UCm2fJRZ4XFdeekwuWgfZBuQgAIFdZlHlksThM3uUceffWnuu5Uy+nAboNSTYAdKBUEq6sEtssEtg8lkqvf51rFxXnYbbnzvsDBICFo1wEADpMSvXLWdYpt9oVJ+tyjk59nQFkg5lsALmbOdtXxDHa+VR5Fj9HHjO2i5VFmUdW5jMbvpDXn9cZwGxIsoEOlFLCWX8a+8Ovz/8Y7XqqPKufI6WEK+865YXKsuSC1xnAbEiygQ6TWsJZP9s3dfaq3I9R5oxiK7L4OWofsG664nw9d+ilJBKuPOuUszTb69/og2tqiW27vM5At6AmG+gwqdVm1tfBLs0h4c+zdVqRWv05ZtYIv/fa+7Vu9cITv25eUbDZ6z9b7TWrpwJohplsoMOkdApbeuVs39133537Mdo14Wn152hlJrxSCT2+76CuuPquJM6AlKHZ698pZ0oAFIskG+gwWSacWdV2F3Eae65a23ZJwFt5rVr5gDVx8KgOHTmmqcqSrk4kG73+qX1wBdAeSLKBDpRFUptabfdidcrPMR+tfMDq612uk1YsafuSmzx0ypkSAMUiyQbQUKecIu+Un2O+FvsBy7Ze27da933qQhLJBrioEMBCceEjgIYWchFepRLJXjDXKRdFFqXTLuLL4nez2XOk/HsPoHzMZANoaL6nyFMvx+BUf3trpZ4+i9/NZs+R+u89gPIxkw2gqfm0J0utZWAjtFlrT60uW57F72az52iH33sA5So1ybZ9se0f2d5l+5MNHl9h+y+qj99ve1MJwwQwC8oxkJdWE9ksfjebPQe/9wDmUlq5iO0lkv5Y0jskjUt6wPYtEfHIjN0+KGl/RLzO9hZJX5b0S8WPFkAznVKO0U5t/tpBFq9nq63zsvjdbPYcnfJ7DyA/ZdZknydpV0Q8Lkm2RyRdKmlmkn2ppM9Vb/+VpGtsO7jKBEhK2Z0XWk3oiqqv7ZZEPqvXM4tENovfzWbPUfbvPYC0lVkucoakJ2fcH69ua7hPRExJekHSukJGB6AttFq3KxVTX5vFOIuQRceMLF9P6ukBtCuXNSls+z2SLo6IK6r3f1nS+RFx1Yx9HqruM169/4/VffY1eL5hScOS1N/fv3lkZCT3n2FyclK9vb25HwcLR2zSlXVspiqhx54+oFDIsn7y9DVauohZ08f3HdShI8d00oolem3f6szGl/U48zQ5OalnX7QOHZ3SScuX6rXrF/865P16dhve09JFbNJVRGyGhobGImKg0WNllovskXTmjPsbqtsa7TNue6mkkyVNNHqyiNgqaaskDQwMxODgYNbjfYXR0VEVcRwsHLFJV9axiQh9devx0oRfG7pgUbOeb825lCOrcebpzu98V1+495Bq/zXc/xsXqP9VKxf1XLXX89STlum5Qy91fIlM3nhPSxexSVfZsSkzyX5A0tm2z9J0Mr1F0nvr9rlF0uWS7pP0HknfoR4bwExZXYCWd31tO14o18oIe3qsdauXZ17r3i117QDaX2k12dUa66skfVvSo5L+MiIetv0F2++q7nadpHW2d0n6b5Je0eYPANqlbjf1cS7tsc7bdKqW9FjnnXVqyx86sq51r1RCW7bepwu+dKd+6U/vS7auHQCkkld8jIjbJN1Wt+0zM24flvSLRY8LALrVyPDiZ9vrZ5lbbcFXb+/kEX3vif2SpO89sV97J48supwFAPLGsuoAUIB2KXNYbNlMs7Z9WZbI1H93uq8iALCsOgDkrl3a97WiWWlIliUy69esyLScBQDyxEw2gLbULjPDUuMENKsEMZXXIevSkEZst1TOAgBFIskG0HaKWqExK3kloCm9DkV1T2GVRQDtgiQbQNvJc2Y4D3kloKm9DiTAAHAcNdkA2k5tZnhpj3MrTchaHu372vF1AIBuwUw2gLbTjgu75IHXAQDSxUw2gLZQqYT2Hjii2qKvqS/sUpSsX4eZrzEAYPFIsgEkrxta4JWtUgk9vu9gcq9x/YcrAGgXJNkAkpf18tx4pYmDR3XoyLGkXmM+XAFoZyTZAJLHBX756+tdrpNWLEnqNebDFYB2xoWPAJLHBX75s63X9q3WfZ+6MJnXuIgFbgAgLyTZANpCp/VgTmWlxnopvcZ8uALQzkiyAaBgKa3UmLpO+3AFoHtQkw0Ai7TYzhfUGgNA5yPJBoBFaKXzBRdyAkDno1wEABah0Wz0fMsaqDUGgM7HTDYALEKrs9GsWAkAnY2ZbABYBGajAQCzYSYbABZpMbPRLBMOAN2BmWwAKAit+wCgezCTDQAFoXUfAHSPUpJs279j+zHbP7B9s+1Tmuz3hO0f2t5pe0fBwwSATNG6DwC6R1nlIndI+lRETNn+sqRPSfpEk32HImJfcUMDgHxwsSQAdI9SZrIj4vaImKre3S5pQxnjAICi0boPALpDCjXZH5D0zSaPhaTbbY/ZHi5wTAAAAMCiOa82UrbvlPQTDR76dET8TXWfT0sakPTuaDAQ22dExB7bp2m6xOQjEXFPk+MNSxqWpP7+/s0jIyMZ/STNTU5Oqre3N/fjYOGITbqITbqITbqITbqITbqKiM3Q0NBYRAw0eiy3JHsutt8v6VclvS0iDs1j/89JmoyI351r34GBgdixI//rJEdHRzU4OJj7cbBwxCZdxCZdxCZdxCZdxCZdRcTGdtMku6zuIhdL+rikdzVLsG2vtr2mdlvSRZIeKm6UAAAAwOKUVZN9jaQ1ku6otuf7iiTZfrXt26r79Eu61/aDkr4n6e8i4lvlDBcAAACYv1Ja+EXE65psf0rSJdXbj0t6U5HjAgAAALKQQncRAAAAoKOQZAMAAAAZI8kGAAAAMlZaC7882d4raXcBh+qTxJLvaSI26SI26SI26SI26SI26SoiNhsjYn2jBzoyyS6K7R3NeiOiXMQmXcQmXcQmXcQmXcQmXWXHhnIRAAAAIGMk2QAAAEDGSLJbs7XsAaApYpMuYpMuYpMuYpMuYpOuUmNDTTYAAACQMWayAQAAgIyRZLfI9hdt/8D2Ttu323512WPCNNu/Y/uxanxutn1K2WPCNNu/aPth2xXbXJVfMtsX2/6R7V22P1n2eHCc7ettP2v7obLHguNsn2n7u7Yfqb6XfbTsMWGa7ZW2v2f7wWpsPl/aWCgXaY3tV0XEv1Rv/1dJPxURHyp5WJBk+yJJ34mIKdtflqSI+ETJw4Ik22+QVJH0p5I+FhE7Sh5S17K9RNL/k/QOSeOSHpB0WUQ8UurAIEmy/VZJk5K+FhFvLHs8mGb7dEmnR8T3ba+RNCbpP/F3Uz7blrQ6IiZtL5N0r6SPRsT2osfCTHaLagl21WpJfGpJRETcHhFT1bvbJW0oczw4LiIejYgflT0OSJLOk7QrIh6PiKOSRiRdWvKYUBUR90h6ruxx4EQR8XREfL96+4CkRyWdUe6oIEkxbbJ6d1n1q5TcjCQ7A7Z/y/aTkt4n6TNljwcNfUDSN8seBJCgMyQ9OeP+uEgWgHmzvUnSOZLuL3koqLK9xPZOSc9KuiMiSokNSfY82L7T9kMNvi6VpIj4dEScKelGSVeVO9ruMldsqvt8WtKUpuODgswnNgDQzmz3SvqGpF+vO7ONEkXEsYh4s6bPYJ9nu5RSq6VlHLTdRMTb57nrjZJuk/TZHIeDGeaKje33S3qnpLcFFyAUagF/NyjXHklnzri/oboNwCyq9b7fkHRjRPx12ePBK0XE87a/K+liSYVfPMxMdotsnz3j7qWSHitrLDiR7YslfVzSuyLiUNnjARL1gKSzbZ9le7mkLZJuKXlMQNKqF9ddJ+nRiPj9sseD42yvr3UTs71K0xd1l5Kb0V2kRba/Ien1mu6UsFvShyKCWaAE2N4laYWkieqm7XR+SYPtX5D0R5LWS3pe0s6I+NlSB9XFbF8i6X9LWiLp+oj4rXJHhBrb2yQNSuqT9Iykz0bEdaUOCrL9HyT9vaQfavr/f0n6jYi4rbxRQZJs/7SkP9P0+1mPpL+MiC+UMhaSbAAAACBblIsAAAAAGSPJBgAAADJGkg0AAABkjCQbAAAAyBhJNgAAAJAxkmwAAAAgYyTZAAAAQMZIsgGgi9n+d7Z/YHul7dW2H7b9xrLHBQDtjsVoAKDL2f5NSSslrZI0HhFXlzwkAGh7JNkA0OVsL5f0gKTDkv59RBwreUgA0PYoFwEArJPUK2mNpme0AQAtYiYbALqc7VskjUg6S9LpEXFVyUMCgLa3tOwBAADKY/tXJL0UETfZXiLp/9r+mYj4TtljA4B2xkw2AAAAkDFqsgEAAICMkWQDAAAAGSPJBgAAADJGkg0AAABkjCQbAAAAyBhJNgAAAJAxkmwAAAAgYyTZAAAAQMb+Pw4Avxw4xp8dAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 864x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "np.random.seed(1234)\n",
    "_a = np.random.uniform(1, 5)\n",
    "_b = np.random.uniform(-3, 3)\n",
    "_c = np.random.uniform(-3, 3)\n",
    "\n",
    "num_samples = 100\n",
    "\n",
    "xs = np.random.uniform(-3, 3, size=num_samples)\n",
    "ys_clean = _a * xs ** 2 + _b * xs + _c\n",
    "ys_noise = np.random.normal(0, 1, size=len(ys_clean))\n",
    "ys = ys_clean + ys_noise\n",
    "\n",
    "plt.figure(figsize=(12, 5))\n",
    "plt.scatter(xs, ys, label=\"gt\", s=5)\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.legend()\n",
    "plt.grid(True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def model(x, a, b, c):\n",
    "    return a * x ** 2 + b * x + c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# device = torch.device(\"cuda:0\")\n",
    "device = torch.device(\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Создадим переменные `xs_device` и `ys_device`, которые будут содержать обучающие данные сразу на нужном девайсе в виде тензоров.\n",
    "\n",
    "**Вопрос:** `requires_grad=True` или `requires_grad=False`?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "xs_device = torch.tensor(xs, device=device, requires_grad=False)\n",
    "ys_device = torch.tensor(ys, device=device, requires_grad=False)\n",
    "# END OF YOUR CODE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Теперь инициализируем веса `a`, `b` и `c` нормальным распределением:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "a = torch.randn(1, device=device, requires_grad=True)\n",
    "b = torch.randn(1, device=device, requires_grad=True)\n",
    "c = torch.randn(1, device=device, requires_grad=True)\n",
    "# END OF YOUR CODE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Допишем рутину обучения, чтобы получить значения весов модели."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [00:06<00:00, 45.43it/s]\n"
     ]
    }
   ],
   "source": [
    "num_epochs = 300\n",
    "lr = 1e-4\n",
    "indices = np.arange(len(xs))\n",
    "\n",
    "loss_list = []\n",
    "a_list = []\n",
    "b_list = []\n",
    "c_list = []\n",
    "\n",
    "\n",
    "for epoch in tqdm.trange(num_epochs):\n",
    "    np.random.shuffle(indices)\n",
    "\n",
    "    loss_epoch = []\n",
    "    for i in indices:\n",
    "        # YOUR CODE HERE\n",
    "        x      = xs_device[i]\n",
    "        y_true = ys_device[i]\n",
    "        y_pred = model(x,a,b,c)\n",
    "        loss = (y_true - y_pred)**2\n",
    "        loss.backward()\n",
    "        # END OF YOUR CODE\n",
    "        \n",
    "        with torch.no_grad():\n",
    "            #a.data -= lr * a.grad.data\n",
    "            #b.data -= lr * b.grad.data\n",
    "            #c.data -= lr * c.grad.data\n",
    "            a -= lr * a.grad\n",
    "            b -= lr * b.grad\n",
    "            c -= lr * c.grad\n",
    "            a.grad.zero_()\n",
    "            b.grad.zero_()\n",
    "            c.grad.zero_()\n",
    "            \n",
    "        loss_epoch.append(loss.item())\n",
    "        \n",
    "    loss_list.append(np.mean(loss_epoch))\n",
    "    a_list.append(a.item())\n",
    "    b_list.append(b.item())\n",
    "    c_list.append(c.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "А теперь посмотрим на результаты:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "xs_sorted = np.sort(xs)\n",
    "\n",
    "ys_pred_per_epoch = []\n",
    "for epoch in range(num_epochs):\n",
    "    a_epoch = a_list[epoch]\n",
    "    b_epoch = b_list[epoch]\n",
    "    c_epoch = c_list[epoch]\n",
    "    with torch.no_grad():\n",
    "        ys_pred_epoch = model(xs_sorted, a_epoch, b_epoch, c_epoch)\n",
    "    ys_pred_per_epoch.append(ys_pred_epoch)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtkAAAE9CAYAAADecH6XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAABO4klEQVR4nO3dd3RU1d7G8e9OAgRIpElClSJdpAYUEE0AAekqShdEREH0Wq7tXt+r144drxVEEaUoIL23IFUp0jtKCU2aQJAAyez3jwkaQgKBzMw5SZ7PWlnJzJw55zezSXhmn332NtZaRERERETEd4KcLkBEREREJLtRyBYRERER8TGFbBERERERH1PIFhERERHxMYVsEREREREfU8gWEREREfGxEKcL8Idrr73Wli1b1u/HOXXqFPnz5/f7ceTKqW3cS23jXmob91LbuJfaxr0C0TYrV648bK0tmtZj2TJkly1blhUrVvj9OLGxsURHR/v9OHLl1DbupbZxL7WNe6lt3Ett416BaBtjzK70HtNwERERERERH1PIFhERERHxMYVsEREREREfy5ZjstNy7tw54uLiSEhI8Nk+CxQowKZNm3y2v8wKDQ2lVKlS5MqVy+lSRERERHK0HBOy4+LiCA8Pp2zZshhjfLLPkydPEh4e7pN9ZZa1liNHjhAXF0e5cuWcLkdEREQkR8sxw0USEhIoUqSIzwK22xhjKFKkiE976kVERETk6uSYkA1k24B9XnZ/fSIiIiJZRY4K2W40bNgw9u3b53QZIiIiIuJDCtkOU8gWERERyQCPBw4eBGudriRDcsyFj27wyiuv8O2331K0aFFKly5N3bp1WbFiBd26dSNv3rwsXbqUvHnzOl2miIiIiLt4PBATA0uWQMOGMH8+BLm7r9jv1RljvjTG/G6MWZ/ivu+MMauTv3YaY1an89ydxph1ydv5f5301Hz4iWn58uWMGzeONWvWMH369L+WfY+KimLEiBGsXr1aAVtEREQkLYcOeQN2YqL3+6FDTld0WYH4CDAMaJnyDmttJ2ttLWttLWAc8MMlnh+TvG2U/0pMw/lPTKVKQXS093YmLF68mPbt2xMaGkp4eDht27b1TZ0iIiIi2V1EhLcHOyTE+z0iwumKLsvvw0WstT8aY8qm9ZjxTodxL9DE33VcsbQ+MUVGOl2ViIiISM5jjHeIyKFD3oCdBWZUc3owS2PgoLV2WzqPW2CWMWalMaZvAOvy+SemRo0aMXnyZBISEoiPj2fKlCkAhIeHc/LkSV9ULCIiIpJ9BQV5OzyzQMAGMDYAV2gm92RPsdZWT3X/p8B2a+276TyvpLV2rzEmApgNPGqt/TGdbfsCfQEiIyPrjh49+oLHCxQoQIUKFa6scI8Hc/gwtmjRNBs0KSmJ4ODgDO/u9ddfZ8yYMURERFC0aFGaNWtGoUKF+O9//0vevHmZM2dOpsdlb9++nePHj2dqH9lBfHw8YWFhTpchaVDbuJfaxr3UNu6ltnGvQLRNTEzMyvSGNDsWso0xIcBeoK61Ni4D+3gJiLfWvnO5baOiouz5CwvP27RpE1WrVr3Cyi/tSpdVP9/Yf/75J7feeiuDBw+mTp06Pq3JH68zK4qNjSU6OtrpMiQNahv3Utu4l9rGvdQ27hWItjHGpBuynZzCrxmwOb2AbYzJDwRZa08m/9wceDmQBfpa37592bhxIwkJCfTs2dPnAVtERERE3MHvIdsYMwqIBq41xsQBL1prhwKdgVGpti0BfGGtbQVEAuOTlwoPAUZaa2f4u15/GjlypNMliIiIiEgABGJ2kS7p3N8rjfv2Aa2Sf/4VqOnX4kRERERE/MDp2UVERERERLIdhWwRERERybISEhOcLiFNCtkiIiIikiX9kfAHNT+rycc/f+x0KRdRyM6CYmNjadOmjdNliIiIiDjGWkuvCb349div1C5e2+lyLqKQ7SJJSUlOlyAiIiKSJby95G0mbpnIO7e/Q8PSDZ0u5yIK2QGyc+dOqlSpQrdu3ahatSodO3bkzz//pGzZsjz77LPUqVOHMWPGMGvWLBo0aECdOnW45557iI+PB2DGjBlUqVKFOnXq8MMPPzj8akREREScE7szlufnPs+9N9zLYzc95nQ5aVLIDqAtW7bQv39/Nm3axDXXXMMnn3wCQJEiRVi1ahXNmjXj1VdfZc6cOaxatYqoqCjee+89EhISePDBB5k8eTIrV67kwIEDDr8SEREREWfsO7mPTmM7UalIJb5o+wXJa6q4jpMrPjrm8RmPs/rA6kzvJykpieDgYABqFavFBy0/uOT2pUuXplGjRgB0796dDz/8EIBOnToBsGzZMjZu3PjXNmfPnqVBgwZs3ryZcuXKUbFixb+eO3jw4EzXLyIiIpKVnEs6x71j7uXU2VPM7zmf8DzhTpeUrhwZsp2S+pPW+dv58+cHvAP4b7/9dkaNumAhTFavXh2Q+kRERETc7Nk5z7J4z2JG3T2KakWrOV3OJeXIkH25HueMOnnyJOHhGf8EtXv3bpYuXUqDBg0YOXIkt9xyC7/88stfj99888088sgjbN++nQoVKnDq1Cn27t1LlSpV2LlzJzt27OD666+/KISLiIiIZHdjNozh/WXv81j9x+hcvbPT5VyWxmQHUOXKlfn444+pWrUqx44do1+/fhc8XrRoUYYNG0aXLl2oUaPGX0NFQkNDGTx4MK1bt6ZOnTpEREQ49ApEREREAm/z4c30ntSbBqUa8Hbzt50uJ0NyZE+2U0JCQvj2228vuG/nzp0X3G7SpAnLly+/6LktW7Zk8+bN/ixPRERExHWOJxynw+gO5MuVj+/v+Z7cwbmdLilDFLJFRERExJU81kOP8T3YcWwHc++bS6lrSjldUoYpZAdI2bJlWb9+vdNliIiIiGQZryx4hclbJ/Nhyw+5tcytTpdzRTQmW0RERERcZ9KWSby04CV61uzJgPoD0t/Q44GDB8HawBWXATkqZFuXvfm+lt1fn4iIiOQMWw5vocf4HtQtXpdPW3+a/oIzHg/ExECpUhAd7b3tEjkmZIeGhnLkyJFsG0SttRw5coTQ0FCnSxERERG5aifOnKDDdx3IE5yHHzr9QN5cedPf+NAhWLIEEhO93w8dClyhl5FjxmSXKlWKuLg4DvnwzU9ISHBVqA0NDaVUqaxzQYCIiIhISh7roeeEnmw7so05983hugLXXfoJERHQsKE3YDds6L3tEjkmZOfKlYty5cr5dJ+xsbHUrl3bp/sUERERyale+/E1JmyewActPiC6bPTln2AMzJ/v7cGOiPDedokcE7JFRERExL2mbJ3Ci7Ev0r1Gdx676bGMPzEoCCIj/VfYVcoxY7JFRERExGHpzASy9chWuv3QjVrFavF5m8/Tv9AxC1HIFhERERH/S2cmkJNnTnLnd3eSKygX4zuNJ1+ufM7W6SN+D9nGmC+NMb8bY9anuO8lY8xeY8zq5K9W6Ty3pTFmizFmuzHmOX/XKiIiIiJ+ksZMIOcvdNx8eDPf3/M9ZQqWcbpKnwlET/YwoGUa979vra2V/DUt9YPGmGDgY+AOoBrQxRhTza+VioiIiIh/nJ8JJCTkr5lA3lz0JuM3j+ft29+mSbkmTlfoU36/8NFa+6MxpuxVPLU+sN1a+yuAMWY00B7Y6MPyRERERCQQUs0EMm37dF6Y9wJdb+zKEzc/4XR1PufkmOwBxpi1ycNJCqXxeElgT4rbccn3iYiIiEhWlDwTyPZjO+g6ris1ImswpO2QbHGhY2omECsgJvdkT7HWVk++HQkcBizwClDcWts71XM6Ai2ttX2Sb/cAbrLWprl4vTGmL9AXIDIysu7o0aP99Gr+Fh8fT1hYmN+PI1dObeNeahv3Utu4l9rGvdQ2V+500mn6r+rP0bNH+azOZxTPW9wvxwlE28TExKy01kal9Zgj82Rbaw+e/9kYMwSYksZme4HSKW6XSr4vvX0OBgYDREVF2ejoaJ/UeimxsbEE4jhy5dQ27qW2cS+1jXupbdxLbXNlrLXcO/Zedp/ezczuM2lWvpnfjuV02zgyXMQYk/Ijy53A+jQ2Ww5UNMaUM8bkBjoDkwJRn4iIiIj4QKp5sd9c9CZjN45lYLOBfg3YbhCIKfxGAUuBysaYOGPMA8Bbxph1xpi1QAzwRPK2JYwx0wCstYnAAGAmsAn43lq7wd/1ioiIiIgPpJoXe/Lmifx73r/pXL0zTzV4yunq/C4Qs4t0SePuoelsuw9oleL2NOCi6f1ERERExOVSzIu9Yetiuv6wkjrF6zC03dBseaFjalrxUURERER8L3le7CPhwbTrmYuwPOFM7Dwx26zoeDkK2SIiIiLie8Zwbs4sOr57E3vDLBM6TaDkNTlnNmZHZhcRERERkezvH7OeIHbfEoZ3GM5NpW5yupyAUk+2iIiIiPjcRz9/xKcrPuXphk/To2YPp8sJOIVsEREREfGpGdtn8I8Z/6Bd5Xa80fQNp8txhEK2iIiIiPjMht830GlsJ26MuJERd40gOCjY6ZIcoZDtK6kmWxcRERHJaX4/9TttRrUhX658TO4ymbDcqZY1z0F5SSHbF1JNto7H43RFIiIiIgGVkJjAnd/dycH4g0zqPInSBUpfuEEOy0sK2b6QYrJ1lizx3hYRERHJIay1PDDpAZbsWcLwO4dTr2S9izfKYXlJIdsXkidbJyTE+z0iwumKRERERALm1R9fZeS6kbzW5DU6VuuY9kY5LC9pnmxfMAbmz/d+IouI8N4WERERyQG+W/8d/4n9D/fVvI/nb3k+/Q1zWF5SyPaVoCCIjHS6ChEREZGA+SnuJ3pN7MUt193C4DaDMZcLzjkoL2m4iIiIiIhcsV1/7KLd6HaUCC/B+E7jyROSx+mSXEU92SIiIiJyRU6eOUnbUW05k3iG2J6xXJvvWqdLch2FbBERERHJsCRPEl3GdWHjoY1M7zadqkWrOl2SKylki4iIiEiG/XPWP5m6bSqftv6U26+/3elyXEtjskVERETkYmmszvjp8k/54KcPePymx3k46mEHi3M/hWwRERERuVAaqzNO2TqFAdMH0KbM7bxz+9tOV+h6CtkiIiIicqFUqzOu3DiHTmM7Uft4Pkb3n0dwk6bZfln0zFLI9pU0TqmIiIiIZEkpVmfc1aQObWb2pGiewkwZepr8p5NyxLLomaWQ7QtpnFIRERERcdzVdgImr874x44NtGofz+lzp5nafTrFajbKMcuiZ5ZCti+kOqWiT3YiIiLiuEx2Ap61idw1/2G2HdnG+E7juSGyundZ9Lg4iI3N9suiZ5bfQ7Yx5ktjzO/GmPUp7nvbGLPZGLPWGDPeGFMwnefuNMasM8asNsas8HetVy0ighF3Xs/QqCB9shMRERF3yEQnoLWWPpP6MH/nfIa2G0pMuRjvA+eXRVfAvqxA9GQPA1qmum82UN1aWwPYCjx/iefHWGtrWWuj/FRfpllgZPvyPNTWMHXw0/qHJyIiIs5LMa76SjsB/2/+//HN2m94OfpletTs4ccisy+/h2xr7Y/A0VT3zbLWJibfXAaU8ncd/mSM4bt7vqdmsZrcO64Ty/cud7okERERyemSx1Vf6fCOz1d8zmsLX+PBOg/ywq0v+LfGbMwNY7J7A9PTecwCs4wxK40xfQNY0xULyx3G1K5TicgfQeuRrdlxdIfTJYmIiEhOd4XDOyZvmUz/af1pVbEVn7T+BKOz81fN2ABMOWeMKQtMsdZWT3X/v4Eo4C6bRiHGmJLW2r3GmAi8Q0weTe4ZT+sYfYG+AJGRkXVHjx7t41dxsfj4eMLCwi64b/efu3n0l0cJzxXOR7U+omDugn6vQy6WVtuIO6ht3Ett415qG/fK8m2TmOgdTgJsOrGJJ9Y8Qdn8ZXm/5vvkDc7rcHGZE4i2iYmJWZnekGbHQrYxphfwENDUWvtnBvbxEhBvrX3ncttGRUXZFSv8f51kbGws0dHRF92/dM9SmgxvQs3ImszrOY98ufL5vRa5UHptI85T27iX2sa91DbulWXb5vzMI0uWQMOGbBs7mIbDbiE8dzhLH1hKZFik0xVmWiDaxhiTbsh2ZLiIMaYl8AzQLr2AbYzJb4wJP/8z0BxYn9a2btOgdANG3T2K5fuWc8+YeziXdM7pkkRERET+lmLmkQNrFtPim+YAzOw+M1sEbDcIxBR+o4ClQGVjTJwx5gHgIyAcmJ08Pd9nyduWMMZMS35qJLDIGLMG+BmYaq2d4e96faVDlQ581vozpm2bRq+JvfBYLVAjIiIiLpE888jJfMG0fiAvBxMOM7XrVCoWqeh0ZdlGiL8PYK3tksbdQ9PZdh/QKvnnX4GafizN7x6s+yBHTh/h+bnPUzi0MB/e8aEuIBARERHnGcPZOTO5e3hL1uxdxOR7JlO/ZH2nq8pW/B6yc7pnGz3L4T8P8+7Sd7k237W8GP2i0yWJiIhIDuexHnpP7sPsuAV81f4r7qh4h9MlZTsK2X5mjOHt29/myOkjvLTgJYrkK8KA+gOcLktERERyKGstT896mhHrRvB6k9fpVauX0yVlSwrZAWCMYUjbIRw7fYxHpz9K4byF6XpjV6fLEhERkRzorcVv8d6y93is/mM8d8tzTpeTbblhMZocISQohNEdRxNdNpqeE3oybdu0yz9JRERExIeGrhrKc3Ofo0v1Lrzf8n1dK+ZHCtkBFBoSysTOE6kRWYOO33dk0e5FTpckIiIiOcT4TePpO6UvLa5vwbAOwwgyioH+pHc3wK7Jcw3Tu02ndIHStBnZhrUH117+SR4PHDwIAVg4SERERLKf2J2xdBnXhfol6zPu3nHkDs7tdEnZnkK2AyLyRzC7x2zC84TT4tsW7Di6I/2Nz6/IVKoUREd7b4uIiIhk0Kr9q2g3qh3XF76eqV2nkj93fqdLyhEUsh1yXYHrmNV9FueSztH82+bsP7k/7Q1TrMjEkiXe2yIiIiIZsO3INlp+25JCeQsxs/tMCuct7HRJOYZCtoOqFq3K9G7TORh/kBbftuDY6WMXb5S8IhMhId7vERGBL1RERESynLgTcTT/tjkWy6zusyh1TSmnS8pRFLIdVq9kPSZ2nsiWI1toM6oNp86eunADY2D+fIiLg9hY7+3zNFZbRERE0vD7qd9pNrwZR/48wvRu06l8bWWnS8pxFLJdoGn5poy6exTL4pbRcUxHziadvXCDoCCIjLw4YGustoiIiKRy7PQxmn/TnN3HdzO161SiSkQ5XVKOpJDtEndVvYvP23zOjO0z6DWhFx57mdCssdoiIiKSSvzZeFqPbM3GQxsZ32k8jcs0drqkHEsh25+ucDhHnzp9GNhsIKPWj+Kx6Y9hL/U8jdUWERGRFBISE+gwugM/7f2J0R1H06JCiyvbgYah+pRCtr9c5XCOZxo9w9MNn+bj5R/zUuxL6W+Y1lht/XKIiIjkSOeSztFpbCfm/jaXr9p/xV1V77r0E1JnBg1D9TmFbH/JxHCOgc0G8kDtB3j5x5f58KcP098w5Vht/XKIiIjkSEmeJHpO6MmkLZP4uNXH3Ffzvks/Ia3MoGGoPqeQ7S+ZGM5hjOGzNp9xZ5U7+ceMfzBi7YjLP0m/HCIiIjmOtZZ+U/sxav0o3mz6Jv3r9b/8k9LKDBqG6nMK2f5yqan3MiAkKISRd4+kSbkm9JzQkylbp1z6CfrlEBERyVGstfxz1j8ZsmoI/7rlXzx7y7MZe2JamSGTuUUuppDtT2lNvXcFQkNCmdBpArWL16bj9x2Z++vc9DfWL4eIiEiO8sqPr/DesvcYUG8ArzZ5NeNPTC8zZDK3yIUUsl0uPE84M7rNoGKRirQb3Y5Fuxelv7F+OURERHKE95e+z4uxL9KrVi8G3TEIc6X/9ysz+J1CdhZQJF8R5vSYQ6lrStFqRCt+3vuz0yWJiIiIQ4asHMKTs56kY7WODGk7hCCjOOdGapUsIjIskrndZ1M0tDAtvm3Bqv2rnC5JREREAuzr1V/z0JSHuKPCHYy4awQhQSFOlyTpUMjOKjweSrXvwbzX4rjmj9Pc/s3trDu4zumqREREJEBGrRtF70m9aVq+KePuHUfu4NxOlySXEJCQbYz50hjzuzFmfYr7ChtjZhtjtiV/L5TOc3smb7PNGNMzEPW6UvJ0O2WOJDFvaCJ5g/LQdHhTNh3a5HRlIiIi4mfjNo6jx/geNL6uMRPvHU/eoye0+JzLBaonexjQMtV9zwFzrbUVgbnJty9gjCkMvAjcBNQHXkwvjGd7Kabbub5qI+b2nEeQCaLp8KZsO7LN6epERETETyZtmkDncZ25udTNTOk8iXzNW2vxuSwgICHbWvsjcDTV3e2Br5N//hrokMZTWwCzrbVHrbXHgNlcHNZzhlTT7VQuWoW5980l0ZNIzNcx7Di6w+kKRURExMembJ5Ex9F3UWdPItO+tYQdO6XF57IIJ8dkR1pr9yf/fACITGObksCeFLfjku/LmVJNt3NDxA3MvW8uCYkJxHwdw6/HfnW4QBEREfGVadumcfeYe6h5wDLzG7hm4c/eDKDF57IEYwM0nscYUxaYYq2tnnz7D2ttwRSPH7PWFkr1nH8CodbaV5Nv/x9w2lr7Thr77wv0BYiMjKw7evRof72Uv8THxxMWFub341zO9vjtPLXmKfIG5+X9mu9TPG9xp0tynFvaRi6mtnEvtY17qW3cy19t8/PRn3lh/QuUzV+Wd/M9TPhpIH9+qFzZu0FiojdoS7oC8XsTExOz0lobldZjTrbOQWNMcWvtfmNMceD3NLbZC0SnuF0KiE1rZ9bawcBggKioKBsdHZ3WZj4VGxtLII5zOdFEU7duXZoOb8pzW55jfs/5lC9U3umyHOWWtpGLqW3cS23jXmob9/JH28zaMYv/LPoPN0R6z1gXzlPQOyzk/PLnkiFO/944OVxkEnB+tpCewMQ0tpkJNDfGFEq+4LF58n2SSu3itZl731ziz8YTPSxaY7RFRESyoKlbp9JuVDuqXFuFOT3mUDhvYa3OmEUFagq/UcBSoLIxJs4Y8wDwJnC7MWYb0Cz5NsaYKGPMFwDW2qPAK8Dy5K+Xk++TNJwP2n+e+5Pbht2mWUdERESykDEbxtDhuw5Uj6jO3PvmUiRfEadLkkwI1OwiXay1xa21uay1pay1Q621R6y1Ta21Fa21zc6HZ2vtCmttnxTP/dJaWyH566tA1JuV1SpWi/k953M26Sy3DbuNLYe3OF2SiIiIXMbwNcPpPK4zN5W8SQE7m9CKj9nQjZE3Mr/nfJJsErcNu+3iBWs8Hjh4UJPYi4iIuMCnyz+l54SeNCnXhJndZ1IgtIDTJYkPKGRnUzdE3EBsz1iMMUR/Hc2G3zd4H/B4ICZGk9iLiIi4wDtL3qH/tP60rdSWyV0mkz93fqdLEh9RyM7GqhatSmzPWIJNMNFfR7P24Nq/lmfXJPYiIiLOsdbyUuxLPD37aTrd0Ilx944jNCTU6bLEhxSys7nK11ZmQa8F5AnOQ5Ovm7Das0+T2IuIiDjIWsszs5/hvwv+S69avRhx1whyBedyuizxMYXsHKBikYrE9oolb668NBnelFUj3/1reXZNByQiIhI4HuvhkWmP8M7Sd3ik3iMMbTeU4KBgp8sSP1DIziEqFK7Agl4LCM8TTtNvb+fnxF0K2CIiIgGU6Enk/on38+mKT3mm4TP8747/EWQUxbIrtWwOUr5QeRb0WkCh0EI0Hd6Uub/OdbokERGRHOFs0lm6jOvC8DXDeSXmFd5s9iYmvc4uzQKWLShk5zBlC5ZlUe9FlClQhlYjWzF+03inSxIREcnWTp87zZ3f3cnYjWN5r/l7vHDrC5cO2JoFLFtQyM6BSoSX4Mf7f6RO8Tp0HNORL3/50umSREREsqX4s/G0GdWG6dum83mbz3miwROXfoJmAcs2FLJzqMJ5CzO7x2yalmvKA5Me4N0l7zpdkoiISLbyR8IftPi2BbE7Yxl+53D61u17+SdFRGgWsGwixOkCxDlhucOY3GUy3cd355+z/8nR00d5tcmr6Z/CEhERkQw5/Odhmn/TnPW/r+f7jt9zd7W7M/ZEY2D+fG8PdkSEJinIwhSyc7g8IXkYffdoHs7zMK8vep0jp4/wcauPNZ2QiIjIVdp/cj/NvmnGr8d+ZWLnidxR8Y4r20FQEERG+qc4CRiFbCE4KJjBbQdTJF8RBi4eyB8JfzD8zuHkDs7tdGkiIiJZyq4/dtF0eFMOxB9gerfpRJeNdrokcYhCtgBgjOHNZm9SOG9hnp3zLMfPHGfsPWPJnzu/06WJiIhkCduObKPp8KacOHOCOffN4eZSNztdkjhIFz7KBZ5p9AxD2g5h1o5ZNP+2OcdOH3O6JBEREdfb8PsGbh12K6cTTzO/53wFbFHIlov1qdOH7zt+z4p9K7ht2G3sP7nf6ZJERETcI9ViMSv3reS2YbdhMCzotYDaxWs7XKC4gUK2pOnuancztetUfj32K7d8dQu/HvvV6ZJEREScl2qxmHXH19FkeBPCcoex8P6FVCtazekKxSUUsrMDPy2/2qx8M+beN5djp49xy5e3sO7gOp/uX0REJMtJsVjM3H2LeGbtM0Tmj2Th/Qu5vvD1TlcnLqKQndX5efnVm0rdxML7F2KM4dZht7J0z1Kf7l9ERCRLSV4sZmqVIFp3sRTPW5wf7/+R0gVKO12ZuIxCdlYXgOVXb4i4gUX3L6JI3iI0+6YZs3bM8vkxREREsgRjGP2/h+jQNYjqperwfs33KRZWzOmqxIUuG7KNMY8aYwoFohi5CgFafrVcoXIs6r2ICoUr0GZkG8ZsGOOX44iIiLjZ+0vfp8v4bjQo1YC5982lQK4CTpckLpWRnuxIYLkx5ntjTEujNbfd5fzyq3FxEBvr1+VXi4UVY0GvBdQvWZ9OYzsxZOUQvx1LRETETTzWw1Mzn+LJWU9yd9W7mdVjFgVCFbAlfZcN2dbaF4CKwFCgF7DNGPO6MSZTo/uNMZWNMatTfJ0wxjyeaptoY8zxFNv8JzPHzLbOL78agM8/BUMLMqvHLFpWaEnfKX0ZuGig348pIiLipDOJZ+j2QzfeW/YeA+oN4LuO3xEaEup0WeJyGVrx0VprjTEHgANAIlAIGGuMmW2tfeZqDmyt3QLUAjDGBAN7gfFpbLrQWtvmao4h/pEvVz4mdJ5Arwm9eG7ucxw5fYSBzQZirPWOCY+ICEjgFxER8bfjCce587s7mb9zPgObDeTphk+jk/qSEZcN2caYfwD3AYeBL4CnrbXnjDFBwDbgqkJ2Kk2BHdbaXT7YlwRA7uDcfHvXtxQKLcjbS95m/4l9DP3fLnIvXuYdGz5/vreHXUREJIvad3Ifd4y4g42HNjK8w3B61OzhdEmShWSkJ7swcFfqAGyt9RhjfNXD3BkYlc5jDYwxa4B9wD+ttRt8dEzJpCALH721npImiH8zgv1lYNwyKHB+lpPISKdLFBERuSqbDm2i5YiWHD19lKldp9L8+uZOlyRZjLE+XsDkigswJjfeAH2DtfZgqseuATzW2nhjTCtgkLW2Yjr76Qv0BYiMjKw7evRoP1cO8fHxhIWF+f04rpWYCGvXgrXMTFzJ2wnfUSYokjeKPEpEtQaOlpbj28bF1DbupbZxL7VNYP1y7Bf+s/E/5DK5ePPGN6kUXindbdU27hWItomJiVlprY1K6zE3hOz2wCPW2st+RDTG7ASirLWHL7VdVFSUXbFihY8qTF9sbCzR0dF+P45rWetdAGfJEmjYkDlf/Iu7x95DWJ5wpnadSq1itRwrLce3jYupbdxLbeNeapvAGbZ6GA9OfpCKhSsytetUyhUqd8nt1TbuFYi2McakG7LdMGi2C+kMFTHGFDs/ZaAxpj7eeo8EsDa5lFTTBzar2IJFvRcTbIJp/FVjpm2b5nSFIiIiGeKxHl6Y9wL3T7yf28rcxpIHllw2YItciqMh2xiTH7gd+CHFfQ8bYx5OvtkRWJ88JvtDoLN1uutdLpRq+sAbI29kWZ9lVCpSibaj2vLZis8cLlBEROTSEhIT6DquK68tfI0+tfswvdt0CoYWdLosyeIyNIWfv1hrTwFFUt33WYqfPwI+CnRdkjklwkuwoNcCOo/tTL+p/dhxdAcDbx9IkHHDiRMREZG/HTp1iPaj27M0bqmm6BOfUuoRvwjLHcaEzhN4pN4jvLP0He7+/m5OJZyEgwe9Y7lFREQctvnwZm4eejO/HPiFMfeM4ZlGzyhgi88oZIvfhASF8FGrj/iw5YdM2jKJ2/5dgn1VSnovlvR4nC5PRERysPm/zafB0AbEn40ntmcsHat1dLokyWYUssXvHr3pUSa1/JoteeKp1zuJ5b8t9s6jLSIiEkgeDxw8yLBfvqL5t80pEV6Cn/r8xE2lbnK6MsmGFLLFv5L/oLWO6sqS1XXJ7YHGvTyMODjb6cpERCQn8XjwxETzQvfi3D+pN9FlbmNx78WULVg23e01xFEyQyFb/MfjgZgYKFUKYmK4cdIylv9jAzeXv5Xu43vw3JznSPIkOV2liIhkJ6nDcfLtU3t3cm/xRbx2i6XPL4ZpzYalP4NIyv+/NMRRrpJCtvjPoUPehWoSE73fjxzh2rLVmN1jNg/XfZiBiwfS4bsOnDhzwulKRUQkKzsfrJOSLgzHiYkQE8OeaiVp/EENfqhieWcmDI4NJ1dk8fT3l/r/Lw1xlKugkC3+ExEBDRtCSIj3e0QEALmCc/Fpm0/5pNUnTN82nZu/uJntR7c7XKyIiGRJKXudb7nlwnC8eTPLdi2mXu8ktuc5xZRRhqeWgok/BYcvsXh0Ov9/iVwJhWzxn1QrQpJqWqR+9foxu8dsDp46SP0h9Zn761xn6hQRkawrZa/z8uVQr95f4XjYuZ+5raeH/Odg2S91aRV5i/exRo0uHZwv8/+XSEYoZIt/pVoRMrWYcjEsf3A5JcJL0OLbFnz080doUU8REbnApS5CTNnr3KgRLFxI4u6dPPF8be6f9ACNr2/Cz09spNq05d7AnNHgfJn/v0QuRyFbHFe+UHmWPrCU1pVa8+j0R3loykOcTTp78Ya60ltEJOe53EWIqXqdj549Tqs59/PBT4N4rP5jzOg+gyJlqnq3U3CWAFLIFlcIzxPO+E7j+dct/2LIqiE0G96MQ6dSXGiiK71FRHKmjFyEmBye1/2+nnpD6rFg1wKGthvKoDsGERIUEviaRVDIFhcJMkG81vQ1Rt41kuX7llNvSD3WHFjjfVBXeouIZA2+PuuYwYsQx2wYQ4OhDTh97jQL7ptP7xKtdeZTHKWQLa7T5cYuLLx/IYmeRBoMbcDIdSN1pbeISFbgj7OOl7kIMdGTyHNznuPesfdSq1gtVvZZzs09nteZT3GcQra4UlSJKFb0XUFUiSi6/dCNx2c+wbk5s3Slt4iIm/nrrGNaY6k9Hg7t3EDLb1sycPFAHq77MPN6zqN4QojOfIorKGSLaxULK8bc++byj5v+waCfBtFsRHMO5LcK2CIibhWos44eDyvaRRH1YXUWbZ/HV+2G8mmbT8kdnFtnPsU1FLLF1XIF5+KDlh8w4q4RLN+7nDqf12Hx7sVOlyUiImkJwPzS1loGL3iPRrV/wVhY/FUQvUq0DmgNIhmhkC1ZQtcbu7KszzLy5cpH9NfRvLX4LTxW4+xERFzHj9Pk/XnuT+6feD8P/fg0MccLsWJoMHXLp7GwjKbqExdQyJYso0ZkDVb0XUG7yu14ds6ztB7Zmt9P/e50WSIiEgDbj26nwdAGDF8znJdue4mp7x3k2m171VstrqWQLVlKwdCCjL1nLJ+0+oT5v82n5mc1tRy7iIjbZXJav/GbxlN3cF3iTsQxrds0Xox+keCQXOqtFldTyJYsxxhDv3r9+PnBnykYWpDbv7mdF+a9QKIn0enSREQktUxM63c26SyPz3icu76/i8pFKrOq7ypaVmh56WNpZWBxCYVsca+0/limuK9GZA1WPLiCXrV68drC14geFs3u47udq1dERC52ldP67fpjF42/asyg5OXRF/VeRJmCZdJ/glYGFpdRyBZ3SuuPZRr35c+dny/bf8m3d37LmoNrqPVZLSZsnuBw8SIi8permFJv3MZx1Pq8FpsPb2bMPWMYdMcg7/R8l6KVgcVlHA/Zxpidxph1xpjVxpgVaTxujDEfGmO2G2PWGmPqOFGnBFhafywv8Qe0W41urOq7inKFynHnd3fy4fYPSUhMcPAFiIgIcEVT6p06e4q+k/vScUxHKhauyMq+K+lYrWPGjqP5scVlHA/ZyWKstbWstVFpPHYHUDH5qy/waUArE2ek9cfyMn9AKxapyJLeS3j8pscZv3c8DYY2YOuRrQ69ABER+UsGptRbc2ANUUOi+GLVFzzb6FkW9V5EhcIVMn4MzY8tLuOWkH0p7YHh1msZUNAYU9zposTP0vpjmYE/oHlC8vB+y/d57YbX2H18N3U+r8M3a74JePkiIpIx1loGLRtE/S/qczzhOLN7zObNZm9efnhIWjQ/triIG0K2BWYZY1YaY/qm8XhJYE+K23HJ90l2l9Yfywz+AW14bUPWPLyGOsXrcN+E++g5oSfxZ+P9XLCIiFyJ30/9TptRbXh85uO0uL4Fax5eQ9PyTZ0uS8QnjHV4mhtjTElr7V5jTAQwG3jUWvtjisenAG9aaxcl354LPGutXZFqP33xDichMjKy7ujRo/1ee3x8PGFhYX4/jly5822TZJP4Ztc3DN81nFJ5S/Gfav+hQliq04+Jid4hKIES6OO5jH5v3Ett417ZsW1WHF3BG1ve4OS5k/S7vh8dSnTAZMEe6OzYNtlFINomJiZmZTrDnb2nadzyBbwE/DPVfZ8DXVLc3gIUv9R+6tatawNh/vz5ATmOXLnUbTP/t/m2xLslbJ5X8tiPfvrIejwea5OSrL31VmtDQrzfk5Iu3ElSkrUHDljr8fimqMsdL4fQ7417qW3cKzu1zZnEM/bpWU9bXsJW+7iaXXNgjdMlZUp2apvsJhBtA6yw6eRRR4eLGGPyG2PCz/8MNAfWp9psEnBf8iwjNwPHrbX7A1yqZHHRZaNZ/dBqmpZvyoDpA7jr+7s4umdr+tM9+WO+VU0vJSI53LYj22g4tCFvL3mbflH9WPHgCmpE1nC6LBG/cHpMdiSwyBizBvgZmGqtnWGMedgY83DyNtOAX4HtwBCgvzOlSlZXNH9RJneZzLvN32Xq1qnUHt+CxXfckPZsJf4IxJpeSkRyKGstw1YPo/bntfntj98Y32k8n7T+hLy58jpdmojfODow1Fr7K1Azjfs/S/GzBR4JZF2SfQWZIJ5s8CSNr2tM53GduS1qPS8+/BTPt3iVkJRjAc8H4iVLfBeIz8+OcuiQd39ZcOyhiMiVOnb6GP2n9Wf0+tFEl43mmzu/odQ1pZwuS8TvnO7JFnFEvZL1WNV3FffecC//Wf4Wjb66hc2HN/+9gb/mW9X0UiLiJh4PHDwIKSdBSOu+qzRrxyyqf1qdsRvH8lqT15jTY44CtuQYCtmSYxUILcDIu0cy+u7RbD+6ndqf12bQskF4bPL4awViEcnO0rr2xEfXo5w6e4r+U/vT4tsWFAwtyE99fuJfjf9FcFCwT1+CiJspZEuO16l6J9b3W0/Tck15fObjNB3elJ1/7HS6LBER/0rr2hMfXI+yePdian1ei89WfMZTDZ5iZd+V1Clexw8vQMTdFLJFgOLhxZncZTJD2w1l5b6V1Pi0BkNXDT0/baSISPaT1sXYmbhA++SZkzw67VEaf9WYRE8i83vO553m7xAaEurHFyHiXgrZIsmMMfSu3Zu1/dZSt0Rd+kzuQ+uRrdn1xy6nSxMR8b20rj25yutRpm+bzg2f3MDHyz/m0fqPsq7fOm4re5tfyxdxO4VskVTKFizL3PvmMqjlIH7c9SM3fHIDH/70IUmeJKdLExHxrbSuPbmC61EO/3mY7j90p9XIVoTnCWdx78UMumMQYbm1AqKIQrZIGoJMEI/d9Bjr+6+ncZnG/GPGP2j0ZSPW/556rSQRkZzHWsuodaOo+nFVvt/wPS/e9iKr+q6iQekGTpcm4hoK2SKXULZgWaZ1ncaIu0aw49gOan9em/+b938kJCY4XZqIiCP2HN9Du9Ht6PpDV8oXKs+qh1bxUvRL5AnJ43RpIq6ikC1yGcYYut7YlU2PbKJL9S68uvBVan1Wi4W7FjpdmohIwHish0+Xf8oNn9zAvN/m8V7z91jSewnVI6o7XZqIKylki2TQtfmuZfidw5nZfSZnks5w67BbeXjKwxxPOO50aSIifrXl8Baih0XTf1p/bip1E+v6reOJBk9o3muRS1DIFrlCza9vzvp+63ny5icZsmoI1T6pxoTNE5wuS0TE584lneONhW9Q87OarPt9HV+2+5JZ3WdRvlB5p0sTcT2FbJGrkD93ft5t8S7LHljGtfmu5c7v7uTu7+9m38l9TpcmIuITq/avov4X9fnXvH/RplIbNj2yiftr34/RKrgiGaKQLZIJ9UrWY8WDK3ij6RtM3TqVah9XY8jKIX8vzS4iksWcPneaZ2c/S/0h9TkQf4Bx945j7L1jKRZWzOnSRLIUhWyRTMoVnIvnbnmOdf3WUbt4bfpO6UvM1zFs+H2D06WJiFyRadumccMnN/DWkrfoVasXG/tv5K6qdzldlkiWpJAt4iMVi1Rk3n3zGNJ2COsOrqPmZzV5auZTnDhzwunSRETS5vHAwYPsPvobdw1vTeuRrckTkod5983ji3ZfUChvIacrFMmyFLJFfMgYQ586fdj66FZ61+7N+8vep/JHlRmxdgTWWqfLExH5m8fDmSa38Vqn4lR5rzwzt0zjze3lWNP3F2LKxThdnUiWp5At4gfX5ruWwW0H81Ofnyh9TWm6j+9O9NfRrDu4zunSRESw1jLup6+oVmsRL8RY7tgGmz6GZ0fvIfdRTUsq4gsK2SJ+VK9kPZb1WcbgNoPZ8PsGan9em35T+mkWEhHJnORhHlzFGbJV+1cR/XU0HWf1IV+ufMwaDuO+h+tOGGjYECIifF+vSA6kkC3iZ0EmiAfrPsjWR7fSL6ofX/zyBRU+rMDzc57n2OljTpcnIlmNxwMxMVCqFERHe29nwL6T+7h/4v1EDY5i06FNfNb6M355fDO37w7xbhAUBN99B5qiT8QnFLJFAqRw3sL8r9X/2DJgC3dVvYuBiwdS/sPyDFw0kD/P/el0eSKSVRw6BEuWQGKi9/uhQ5fc/M9zf/LKgleo9L9KjFw3kn82/CfbHt3GQ1EPEVKilLf3OiQEGjWCyMgAvQiR7E8hWyTAyhcqz7d3fcsvD/1Co9KNeG7uc1T4sAKfr/icc0nnnC5PRALlaod8RET8HYwvMbzDWsvIdSOp8lEV/hP7H1pWaMnG/ht56/a3KBBawLuRMTB/PsTFQWyserFFfEghW8QhNYvVZErXKfzY60fKFyrPw1Mfpton1fhu/XdazEYku7vKIR9AhoLx0j1LaTC0Ad1+6EbR/EWJ7RnL2HvHcn3h6y/eX1CQtwfbbQE7E+PORdzAsZBtjCltjJlvjNlojNlgjPlHGttEG2OOG2NWJ3/9x4laRfypcZnGLLx/IZO7TCY0JJTO4zoTNTiKmdtnato/kezqCod8XCSdYLzrj110GdeFhl82ZPfx3XzV/iuWP7ic28re5sPiAyAzH0JEXMLJnuxE4ClrbTXgZuARY0y1NLZbaK2tlfz1cmBLFAkMYwxtKrVh9UOr+ebObziWcIyWI1rSZHgTlsUtc7o8EfG1DA75yKhjp4/x/JznqfJxFSZsnsD/3fp/bH10K71q9SLIZMGT1pn9ECLiAo795llr91trVyX/fBLYBJR0qh4Rn8nEKc7goGC61+jOlgFb+N8d/2PjoY00GNqAO7+7k42HNvqhWBFxhI/GQv957k8GLkq+iHrxQO6uejdbBmzh5ZiXCcsd5tuaA8nHH0JEnOCKj7fGmLJAbeCnNB5uYIxZY4yZboy5IbCViVwhH53izB2cmwH1B7DjsR28EvMK836bx42f3sj9E+9n1x+7fFuziDgjE2OhzyWd47MVn1Hhwwo8N/c5GpVuxC8P/cK3d33LdQWu80OxAaYLMiUbME6P+TTGhAELgNestT+keuwawGOtjTfGtAIGWWsrprOfvkBfgMjIyLqjR4/2c+UQHx9PWFgW7inIxhxrm8REWLvW24ttDNSo4e2JyaTj544zcvdIxu8dD0D7Eu3pdl03CuYumOl9B5p+b9xLbeNe59smySYx9/e5fL3za/Yl7KP6NdV5sNyD1ChYw+kScyz93rhXINomJiZmpbU2Kq3HHA3ZxphcwBRgprX2vQxsvxOIstYevtR2UVFRdsWKFb4p8hJiY2OJjo72+3HkyjnWNtZ6e7CXLPGe4vRxD8ye43v474L/8tXqr8iXKx//bPBPnmzwJOF5wn12DH/T7417qW1S8Xi8Y4EjIhzvSZ03fx6HIw7zUuxLbDq8iZqRNXm1yau0rtgao15eR+n3xr0C0TbGmHRDtpOzixhgKLApvYBtjCmWvB3GmPp46z0SuCpFrpCfT3GWLlCaL9p9wYb+G2hxfQteWvAS5T8sz6BlgziTeManxxLJ0Vwyu0WiJ5ERa0fQe0VvOo3thDGGMfeMYdVDq2hTqY0CtoiLOTkmuxHQA2iSYoq+VsaYh40xDydv0xFYb4xZA3wIdLZOj28RuZwAzDlb5doqjL13LD/1+YkakTV4fObjVPqoEl+v/pokT5LfjiuSYzg8u8XZpLMMXTWUKh9Vofv47gSZIEbdPYq1D6+lY7WOWXPGEJEcJvODRa+StXYRcMkUYq39CPgoMBWJZD31S9Zn7n1zmfPrHJ6b8xy9JvbirSVv8XqT12lXuZ23lyutU94uOg0u4krnZ7c4P/QrQLNbnD53mi9/+ZKBiwey58QeokpEMaH5BML3h9OkepOA1CAivqGPwiLZQLPyzVj+4HLG3DOGRE8iHb7rQMMvGzJtyxRsTPSFp7xdchpcxNUCPLtF/Nl43l3yLuU/LM+A6QO4rsB1zOg2g5/7/Ez7Ku3Vcy2SBTnWky0ivmWMoWO1jnSo0oFhq4fx8oKXaT26LTWrwbNH4Z6liwk5f8o79WnwyEhnixdxo/NDv/zoj4Q/+Pjnj3l/2fscOX2EpuWaMuruUdxW5jaNtxbJ4vTRWCSbCQkKoU+dPmx/bDtftfuSM+F56doRKjwZwqDfRhFfMJ8WeRBx2O7ju3lm9jOU/aAsL8x/gZtL3cyS3kuYc98costGK2CLZAMK2SLZVO7g3PSqfT8b3jjBxJZfU7pSPR6f+QTXfVCGf/9fIw5sXaVFHkQCbFncMjqN7UT5QeV5d+m7NL++Oav6rmJK1yk0KN3A6fJExIcUskWyuaDgENrddB8Ley9k6QNLiSkXwxuL36TMiCh6TuzFin3+n1NeJMfweODgQe+c+ckSPYl8t/47bv7iZhoMbcDM7TN54uYn+PWxX/n+nu+pXbz2Ve9bRNxLIVskB7m51M2Mu3ccWwZs4cE6D/LDph+oN6QeDYY2YOS6kZxNOut0iSLpc3vITHVR8bFTR3hr8VuUH1SezuM6c+T0Ef53x/+IezKOt5u/TZmCZa5s/7pgWSRLUcgWyYEqFqnIR60+Yu+TexnUchBH/jxCtx+6UeaDMvw39r8ciD/gdIkiF8oKs+Ikz629tUAij1yzkFIflObZOc9SoXAFJnWexJYBWxhQfwBhua9imefzFyo7NG+3iFw5hWyRHOyaPNfw2E2PsXnAZqZ3m07tYrV5acFLXPf+dXT7oRs/xf3kdIkiXr5aHMZPveGJnkQmHF1Ci37hVH4UvqgD997QiV8e+oV5PefRtnLbzE3Dd/5CZV2wLJJlaAo/ESHIBNGyQktaVmjJ1iNb+fjnj/lq9VeMXDeS2sVq80DtB+h6Y1cK5S3kdKmSFfli8SNfLA5zvjf8/D7mz/dO05cJe0/sZciqIQxZNYR9J/dRsmxJXqr8AA/d+iTFwotnat8XmT//0u+jFpkScRX1ZIvIBSoVqcSgOwax98m9fNzqYyyWAdMHUOK9EnT/oTvzf5uPx7rwVL1cyC3jl301zMMXi8P4qDc80ZPIpC2TaD+6vXeI1YL/cmPEjUzoNIGdj+/kxdZv+z5gw9/zdqcXsN0+nEYkh1HIFpE0hecJp3+9/vzy0C+s7LuS3rV6M2XrFJoMb0LF/1Xk9YWvs/fEXqfLlLS4KXD5apgHXDpkZsT53vCrHHKx9chWnpvzHKXfL0370e35Ke4nntpZnO0fBzNjyGnaV2pLSJBDJ4h9+T6LiE8oZIvIZdUpXoePW3/M/qf2882d33Bdgev497x/c90H19FmZBsmbJ7AuaRz6e8gEL2qbum5zSxfvA43Ba5MBlufykhveKr3/3jCcb5Y9QWNv2pM5Y8q886Sd6hXoh4TOk1gT9cVDPzmANcfStL7LCIXUcgWyY78FDjz5spL9xrdmd9zPtse3cazjZ5l1f5V3PndnRR/tzj9pvRj4a6FFw4nSd2r6g9u6rnNDF+9DjcFLl8M8/ClDAy5OHtdSSbffSOdxtxLsXeL8eDkBzn852HeaPoGe57Yw6Quk2hfpT25ipfU+ywi6dKFjyLZjR8u7kpLhcIVeL3p67wc8zIzts9g5LqRfL3maz5b+RnXFbiOLtW70PXGrtxoIzApe1UTE31eS5o9t5GRvj+Ov/nidZy/+G3ePDh82B0XwZ0Pti6W5Enix9UT+L7AQsY8bjmSbwNFft3PA7UfoGfNnkQVq4M5fBjCUgTp88HWLRcbZoH3WSQnUcgWyW4CHDhDgkJoU6kNbSq1If5sPBM3T2Tk+pG8s+QdBi4eyA1Fb6Brl1LcPXMPlask9/r5mi9mnnCDzL6OtD5gXWnwO3jQHYExAJI8SSzes5jvN3zP2I1jOXjqIPlqBdF2M3RPrEaL71eRKyT3pT+4KtiKSDoUskWyGwcDZ1juMLrV6Ea3Gt04dOoQYzeOZeT6kfz7+g38uz9ULLyfWjs+gbLQqHQjcgXn8s2B3dajeLUy+zoy8wHL44EtW+D22/16BsRpHuth6Z6lfLfhO8ZuHMv++P3kDclL60qtubfavbS6viX5j/954fufXc6UiEhAKWSLZDe+DJyZmHe3aP6i9KvXj371+rHn+B4mb53M5K2TmbBjAmO+HkPB0ILcUeEO2lZqS8sKLTM/B/elehSz0vzBmekZzcwHrEOH4NSpbBkkExITmPfbPCZunsjELRM5eOogoSGhtKrYinur3UvrSq0vXIUxNPzCHWSXMyUiElAK2SLZkS9OYftwbHfpAqXpX68//ev1Z9qcaSSUTGDy1slM3TqVUetHEWyCaVymMW0rtaVtpbZULFIxc7X76XW4XmY+YEVEQP787riIzwfiTsQxc/tMpm+fzswdM4k/G09Y7jBaVWxFh8odaFOpDeF5wi+/I8g+Z0pEJKAUskUkbX46RZ4vJB+tqrbirqp3keRJ4ue9P//Vy/3UrKd4atZTVC5S2Ru4K7elYemGmZt7OKed6r/aD1jGQOXK3tkpsmCQTEhMYOGuhczcMZMZ22ew4dAGAEqGl6Rr9a50qNKBJuWakCckz9UdQGOvReQKKWSLSNqu5BT5VQ7HCA4KpkHpBjQo3YDXm77Ozj92MnmLN3AP+mkQ7yx9h8J5C/81rKRFhRYUDC3ov9chWSZIWmvZdnQbM7bPYMb2GcTujOV04mlyB+fm1jK3cn+t+2lZoSXVilTxzgqSmQ8O6f37zkrDkEQk4BSyRSRtGT1F7sPhGGULluXRmx7l0Zse5cSZE8zaMeuvYSUj1o0gyAQRVSKKJmWb0KRcExpd14h8ufL55nWIO6UIsodPH2HBzgXM+XUOM3bMYOcfOwGoVKQSfer0oWWFltxW5jby587/93Mz+28zvX3kpGFIInJVFLJFJH0ZOUXup+EY1+S5ho7VOtKxWkeSPEksjVvK7B2zmbdzHu8sfYc3F79JrqBc1C9Zn1vL3MqtZW7l5lI3p93TrVP9WY61lt+O7uDnh9uyMGELC6rkZUPYn4B3Fpum5ZryTMNnaFGhBeULlU97J774t5nePnLaMCQRuWKOhmxjTEtgEBAMfGGtfTPV43mA4UBd4AjQyVq7M9B1isglBGA4RnBQMLdcdwu3XHcL/+W/xJ+NZ/Huxcz7bR4/7v6Rt5e8zRuL3gCg6rVVaVCqATeXupkGpRtQ9dqqBAcF+7wm8a0D8QdYvnc5P+/9meX7lrNi3wqOnD4C1SHsDDSKO03XJs8TXb0NUSWiyB2c+/I79cW/zfT2oWFIInIZjoVsY0ww8DFwOxAHLDfGTLLWbkyx2QPAMWttBWNMZ2Ag0Cnw1YpIuhwYjhGWO4wWFVrQokILAE6dPcWyuGUsi1vG0rilTNwykS9XfwlAeO5w6pesz00lb6J28drUKlaL8oXKE2RSndrX+FrfusT7eTzhOCv2rWD5vuUs3+cN1nEn4gAIMkFUj6hOhyodqFciinovDubGOWvJdXMjaPnalbWNL/5tprcPDUMSkctwsie7PrDdWvsrgDFmNNAeSBmy2wMvJf88FvjIGGOstTaQhYrIZTg8HCN/SF6a5q9O08ZNwBistWw/uv2v0L0sbhkDFw8kySYB3pBeI7IGtSJrUatYLWpF1qB6j6fIu+gn/46vzSlBPsV45ZON67N+2Fss37/SG6r3LmfLkS1/bXp9oetpfF1j6pWoR72S9ahdrPbfY6oBJvfN3Hvmi3+b6e1Dw5BE5BKcDNklgT0pbscBN6W3jbU20RhzHCgCHA5IhSLifmlcgGaCgqhYpCIVi1SkR80egHeKtw2/b2D1gdXer4Or+WbtN3yy4hMAgm6FyjdApaMLqThpABVL16JiYe8+SoSXuLjn2wd1uvJCuav4IJDoSWTvib3s/GMn249uZ9PuFWwqvZANAyy7Ci6Br24BoHhYceqVrEePGj2oV7IeUSWiKJy38KV3riArIlmUcapT2BjTEWhpre2TfLsHcJO1dkCKbdYnbxOXfHtH8jYXhWxjTF+gL0BkZGTd0aNH+/01xMfHExYWdvkNJeDUNu7l87ZJTIS1a8FabyisUcO7oEoGeKyHAwkH2B6/ne27fuLXM7uI4yj7kg5zzp77a7s8QXkokbcEpfKWomTekn99L5m3JIVzFybYZGDMdybqDJT4+HjC9u6F+HgIC/POmw0k2SQOnTnEgYQDHEg4wMGEg3/9fODMAX5P+B0Pnr/2k8vk4rrgSMpwLeVCy1CuTD0qhVeiaJ6iTr20LE9/09xLbeNegWibmJiYldbaqLQec/Iv/F6gdIrbpZLvS2ubOGNMCFAA7wWQF7HWDgYGA0RFRdno6Ghf13uR2NhYAnEcuXJqG/fyedtYC6+88ncP8RNPXN2wghQ9uEnWQ9yJOLYd3ca2I9u835N/XrZ3Gec8fwfwIBNE0XxFKR5enGJhxSge9vf3C+7LH0l+X9TpQ2cSz3As4Rh/JPzBHwl/sGzJYiI/e4E9BWBnQfjNNGTn6X3sOb7nr6E2AAZDifASlC1YlqYFm1K2YNm/vsoVLEfZgmUJxnjfz2uvhczOUy36m+Ziahv3crptnAzZy4GKxphyeMN0Z6Brqm0mAT2BpUBHYJ7GY4vIBXx1AVqKYQnBJpgyBctQpmAZmpVvdsFmSZ4kdh/fzbaj29hxdAf74/ez/+R+9sfv50D8AdYeXMvB+IMXhNLzwlqGUaRdMQqFnaDg8CZck+cawnKHkT9X/gu+5wnJQ7AJJsgE/fUVHPT37dSPGWM4m3T2r68ziWc4nXiaE2dO/BWgU4bp818JiQkXvw8dvN9KnICynnM0LN2QcjeWuyBIl76mdMZWTixa1PdDZHLKuHYRyfIcC9nJY6wHADPxTuH3pbV2gzHmZWCFtXYSMBT4xhizHTiKN4iLiFwogON2g4OCKVeoHOUKlYPr097GYz0c/vMwB+IPsP+kN3yfD+Epw+6uP3Zx6twp4s/GE382nlNnT2HxXT9CSFAIhUILUTC04F9fpa8pTcHQghfdXzC0IL9u/JU7PvmakgtWkeemRvBObOaCrK/nkvZ4IDr679AeG+vOce0iIjg8T7a1dhowLdV9/0nxcwJwT6DrEhHJjCATRET+CCLyR1AjskaGn2et5XTiac4knsFjPX99Jdmkv3/2JF30mLWW3MG5yROSh9zBuckdnJu8IXkJDQnFXEFIjt0bS/npy66+pzh1L7Ov55I+eBAWLvT+vHCh93bx4pnbp4iIn7jrqhsRkewqA8McjDHky5Xv8kvF+9PVnhVIb/YUX84lbYz36/zFoxouIiIupvNsIiL+dj6AlirlHe7g8Vz2KVlOWkND4O/Q7otAHBkJjRt7Z2Vp3FhT+4mIqylki0jW5PF4hwtkhWuh0wugvuCW9+H80JCQEP8tM37+Ite4OO94bPVki4iLKWSLSNaT1XqG/RVA3fQ+BCoA+7JnXETEjzQmW0SyHl/PWuFvvppmMDW3vQ9anVFE5C/qyRaRrCcQQxN8zR89sFnxfRARySHUky0iWY+/eoazGr0PIiKupZ5sEckaUl/gp7G5Xr5+H9xwEaWISDagkC0i7uemC/yyK48Htmxx33vsltlTRESukEK2iLifP6fAE69Dh+DUKXe9x/pwJSJZmEK2iLifLvDzv4gIyJ/fXe+xPlyJSBamCx9FxP10gZ//GQOVK3vnuXbLe3z+w9X5pdrdEPxFRDJIIVtEsobsNgezx+PODw1ueo/14UpEsjANFxERCTSNNc44zSIjIlmUQraIyNW62pkvNNZYRCTbU8gWEbkamemN1oWcIiLZnsZki4hcjbR6ozM6nlljjUVEsj31ZIuIXI3M9kZrrLGISLamnmwRkauh3mgREbkE9WSLiFytq+mN1jLhIiI5gkK2iEigaOo+EZEcQyFbRCRQNHWfiEiO4UjINsa8bYzZbIxZa4wZb4wpmM52O40x64wxq40xKwJcpoiIb2nqPhGRHMOpnuzZQHVrbQ1gK/D8JbaNsdbWstZGBaY0ERE/OX+xZFwcxMbqYkkRkWzMkZBtrZ1lrU1MvrkMKOVEHSIiAaep+0REcgQ3jMnuDUxP5zELzDLGrDTG9A1gTSIiIiIiV81YP00jZYyZAxRL46F/W2snJm/zbyAKuMumUYgxpqS1dq8xJgLvEJNHrbU/pnO8vkBfgMjIyLqjR4/20StJX3x8PGFhYX4/jlw5tY17qW3cS23jXmob91LbuFcg2iYmJmZlekOa/RayL8cY0wt4CGhqrf0zA9u/BMRba9+53LZRUVF2xQr/XycZGxtLdHS0348jV05t415qG/dS27iX2sa91DbuFYi2McakG7Kdml2kJfAM0C69gG2MyW+MCT//M9AcWB+4KkVEREREro5TY7I/AsKB2cnT830GYIwpYYyZlrxNJLDIGLMG+BmYaq2d4Uy5IiIiIiIZF+LEQa21FdK5fx/QKvnnX4GagaxLRERERMQX3DC7iIiIiIhItqKQLSIiIiLiYwrZIiIiIiI+5tgUfv5kjDkE7ArAoa4FDgfgOHLl1DbupbZxL7WNe6lt3Ett416BaJsy1tqiaT2QLUN2oBhjVqQ3N6I4S23jXmob91LbuJfaxr3UNu7ldNtouIiIiIiIiI8pZIuIiIiI+JhCduYMdroASZfaxr3UNu6ltnEvtY17qW3cy9G20ZhsEREREREfU0+2iIiIiIiPKWRnkjHmFWPMWmPMamPMLGNMCadrEi9jzNvGmM3J7TPeGFPQ6ZrEyxhzjzFmgzHGY4zRVfkOM8a0NMZsMcZsN8Y853Q98jdjzJfGmN+NMeudrkX+ZowpbYyZb4zZmPy37B9O1yRexphQY8zPxpg1yW3zX8dq0XCRzDHGXGOtPZH882NANWvtww6XJYAxpjkwz1qbaIwZCGCtfdbhsgQwxlQFPMDnwD+ttSscLinHMsYEA1uB24E4YDnQxVq70dHCBABjzK1APDDcWlvd6XrEyxhTHChurV1ljAkHVgId9HvjPGOMAfJba+ONMbmARcA/rLXLAl2LerIz6XzATpYf0KcWl7DWzrLWJibfXAaUcrIe+Zu1dpO1dovTdQgA9YHt1tpfrbVngdFAe4drkmTW2h+Bo07XIRey1u631q5K/vkksAko6WxVAmC94pNv5kr+ciSbKWT7gDHmNWPMHqAb8B+n65E09QamO12EiAuVBPakuB2HwoJIhhljygK1gZ8cLkWSGWOCjTGrgd+B2dZaR9pGITsDjDFzjDHr0/hqD2Ct/be1tjQwAhjgbLU5y+XaJnmbfwOJeNtHAiQjbSMikpUZY8KAccDjqc5si4OstUnW2lp4z2DXN8Y4MtQqxImDZjXW2mYZ3HQEMA140Y/lSAqXaxtjTC+gDdDU6gKEgLqC3xtx1l6gdIrbpZLvE5FLSB7vOw4YYa39wel65GLW2j+MMfOBlkDALx5WT3YmGWMqprjZHtjsVC1yIWNMS+AZoJ219k+n6xFxqeVARWNMOWNMbqAzMMnhmkRcLfniuqHAJmvte07XI38zxhQ9P5uYMSYv3ou6Hclmml0kk4wx44DKeGdK2AU8bK1VL5ALGGO2A3mAI8l3LdPML+5gjLkT+B9QFPgDWG2tbeFoUTmYMaYV8AEQDHxprX3N2YrkPGPMKCAauBY4CLxorR3qaFGCMeYWYCGwDu///wD/stZOc64qATDG1AC+xvv3LAj43lr7siO1KGSLiIiIiPiWhouIiIiIiPiYQraIiIiIiI8pZIuIiIiI+JhCtoiIiIiIjylki4iIiIj4mEK2iIiIiIiPKWSLiIiIiPiYQraISA5mjKlnjFlrjAk1xuQ3xmwwxlR3ui4RkaxOi9GIiORwxphXgVAgLxBnrX3D4ZJERLI8hWwRkRzOGJMbWA4kAA2ttUkOlyQikuVpuIiIiBQBwoBwvD3aIiKSSerJFhHJ4Ywxk4DRQDmguLV2gMMliYhkeSFOFyAiIs4xxtwHnLPWjjTGBANLjDFNrLXznK5NRCQrU0+2iIiIiIiPaUy2iIiIiIiPKWSLiIiIiPiYQraIiIiIiI8pZIuIiIiI+JhCtoiIiIiIjylki4iIiIj4mEK2iIiIiIiPKWSLiIiIiPjY/wP46NWSKbgLXwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 864x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(12, 5))\n",
    "plt.scatter(xs, ys, label=\"gt\", s=5, c=\"r\")\n",
    "plt.plot(xs_sorted, ys_pred_per_epoch[-1], label=\"pred\", c=\"g\")\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.legend()\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Если есть нужные зависимости у `jupyterlab` (**осторожно, может все сломать**), можно сделать интерактивно:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install ipympl"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6e8ff86e73b14f70af56245067c6975f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(IntSlider(value=0, description='i', max=299), Output()), _dom_classes=('widget-interact'…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "10bfb53501af4f908002231321a0673a",
       "version_major": 2,
       "version_minor": 0
      },
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABLAAAAH0CAYAAAAt0NQSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB4+klEQVR4nO3deVxUZf//8fewL4K4A4r7vptbKJmmuJSW7WmlmVmZlmV3mfzKULuxXSstW9VvZlmmtlqiuUSaO+4buKAibikIKAwwvz/mhiJxG4FzZng9Hw8eZ84yZz6jF8PMe67rOhabzWYTAAAAAAAAYFJuRhcAAAAAAAAAXAoBFgAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAUyPAAgAAAAAAgKkRYAEAAAAAAMDUCLAAAAAAAABgagRYAAAAAAAAMDUCLAAAAAAAAJgaARYAAAAAAABMjQALAAAAAAAApkaABQAAAAAAAFMjwAIAAAAAAICpEWABAAAAAADA1AiwAAAAAAAAYGoEWAAAAAAAADA1AiwAAAAAAACYGgEWAAAAAAAATI0ACwAAAAAAAKZGgAUAAAAAAABTI8ACAAAAAACAqRFgAQAAAAAAwNQIsAAAAAAAAGBqBFgAAAAAAAAwNQIsAAAAAAAAmBoBFgAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAUyPAAgAAAAAAgKkRYAEAAAAAAMDUCLAAAAAAAABgagRYAAAAAAAAMDUCLAAAAAAAAJgaARYAAAAAAABMjQALAAAAAAAApkaABQAAAAAAAFMjwAIAAAAAAICpEWABAAAAAADA1AiwAAAAAAAAYGoEWAAAAAAAADA1AiwAAAAAAACYGgEWAAAAAAAATI0ACwAAAAAAAKZGgAUAAAAAAABTI8ACAAAAAACAqRFgAQAAAAAAwNQIsAAAAAAAAGBqBFgAAAAAAAAwNQIsAAAAAAAAmBoBFgAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAUyPAAgAAAAAAgKkRYAEAAAAAAMDUCLAAAAAAAABgagRYAAAAAAAAMDUCLAAAAAAAAJgaARYAAAAAAABMzcPoAuB68vLylJycrICAAFksFqPLAQAAAAC4OJvNprNnzyo0NFRubvTVcUUEWCh2ycnJCgsLM7oMAAAAAEAZc+jQIdWoUcPoMlACCLBQ7AICAiTZXzgCAwMNrsbOarVq8eLF6tmzpzw9PY0uB06EtgNH0XbgKNoOHEXbgaNoO3CUmdpOWlqawsLCCj6PwvUQYKHY5Q8bDAwMNFWA5efnp8DAQMNfWOFcaDtwFG0HjqLtwFG0HTiKtgNHmbHtMI2N62JgKAAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAU2MOLBgmNzdXVqu1VB7LarXKw8ND58+fV25ubqk8Zknx9PSUu7u70WUAAAAAAFBqCLBQ6mw2m1JSUnTmzJlSfczg4GAdOnTIJSb1CwoKUnBwsEs8FwAAAAAALocAC6UuP7yqWrWq/Pz8SiWEycvLU3p6usqVKyc3N+cdOWuz2ZSZmanjx49LkkJCQgyuCAAAAACAkkeAhVKVm5tbEF5VqlSp1B43Ly9P2dnZ8vHxceoAS5J8fX0lScePH1fVqlUZTggAAAAAcHnO/UkeTid/zis/Pz+DK3Fu+f9+pTWHGAAAAAAARiLAgiGYu+na8O8HAAAAAChLCLAAAAAAAABgagRYQDGpXbu2pkyZYnQZAAAAAAC4HAIsAAAAAAAAmBoBFnCFzp49q/vvv1/+/v4KCQnR5MmT1bVrVz399NPq2rWrDh48qGeeeUYWi4U5qgAAAACYW06ONGGC1LOnfZmTY3RFwCURYAFXaPTo0frjjz/0/fffKzY2Vr///rs2btwoSZo/f75q1KihCRMm6OjRozp69KjB1QIAAADAJcTESNHRUmysfRkTY3RFwCURYDmxlStXql+/fgoNDZXFYtHChQsL7c/vCfTvnzfeeOOi54yOjr7g+MaNG5fwMzG/s2fPatasWXrzzTfVvXt3NW/eXDNmzFBubq4kqWLFinJ3d1dAQICCg4MVHBxscMUAAAAAcAlxcZLNZr9ts9nXARMjwHJiGRkZatWqlaZNm1bk/vyeQPk/n332mSwWi+68885LnrdZs2aF7hdnxheyUu7uum/fPlmtVnXo0KFgW/ny5dWoUaMSfVwAAAAAKBEREVL+1CcWi30dMDEPowuA4/r06aM+ffpcdP+/ewF999136tatm+rWrXvJ83p4eJi/B1F+d1ebTVqyxL5t3DhDSwIAAAAApxEVZV/GxdnDq/x1wKTogVVGHDt2TD/99JOGDh162WP37t2r0NBQ1a1bV/fff7+SkpJKocKrVMrdXevWrStPT0+tW7euYFtqaqr27NlTsO7l5VUwpBAAAAAATM3Dw94JYPFi+9KD/i0wN1poGTFr1iwFBATojjvuuORxHTt21MyZM9WoUSMdPXpU48eP1w033KBt27YpICCgyPtkZWUpKyurYD0tLU2SZLVaZbVaCx1rtVpls9mUl5envLw8x59Q586yLFkii80mm8UiW+fO0iXOZ/tf2JX/2FfL399fgwYN0nPPPaegoCBVrVpV0dHRcnOzZ8B5eXmqVauWVqxYoXvuuUfe3t6qXLmyY8/tCuTl5clms8lqtcrd3b3EHgcqaMP/bsvA5dB24CjaDhxF24GjaDtwlJnajhlqQMmy2PI/2cOpWSwWLViwQP379y9yf+PGjRUZGan33nvvqs575swZ1apVS2+//fZFe29FR0dr/PjxF2yfM2eO/Pz8Cm3LH54YFhYmLy+vq6qlkJwceb/9tjxWr1ZOeLiyRo8u8W8Mzp49q9GjR+vnn39WQECAnnrqKc2fP1833HCDXn75Za1bt07PPPOMEhISlJWVpdOnT5dYLdnZ2Tp06JBSUlKUw+VuAQAAAJRxmZmZGjhwoFJTUxUYGGh0OSgBBFgu4lIB1u+//64uXbooPj5erVq1uupzt2/fXj169NCkSZOK3F9UD6ywsDCdPHnygheO8+fP69ChQ6pdu7Z8fHyuuhZH2Ww2nT17VgEBAbLkT1R4jTIyMhQWFqY33njjioZmFqfz58/rwIEDCgsLK9V/x7LIarUqNjZWkZGR8vT0NLocOBHaDhxF24GjaDtwFG0HjjJT20lLS1PlypUJsFwYQwjLgE8//VRt27Z1KLxKT09XYmKiHnzwwYse4+3tLW9v7wu2e3p6XvAilpubK4vFIjc3t4Lhd6Uhf9hg/mM7YtOmTdq1a5c6dOig1NRUTZgwQZJ0++23l+pzkSQ3NzdZLJYi/41RMvi3hqNoO3AUbQeOou3AUbQdOMoMbcfox0fJYxJ3J5aenq74+HjFx8dLkvbv36/4+PhCk66npaXpm2++0SOPPFLkObp3766pU6cWrP/nP//RihUrdODAAa1atUq333673N3dNWDAgBJ9Ls7izTffVKtWrdSjRw9lZGTo999/L9G5rgAAAAAAAD2wnNr69evVrVu3gvXRo0dLkgYPHqyZM2dKkr766ivZbLaLBlCJiYk6efJkwfrhw4c1YMAAnTp1SlWqVFFERIT+/PNPValSpeSeiJNo06aNNmzYYHQZAAAAAACUOQRYTqxr16663BRmjz76qB599NGL7j9w4ECh9a+++qo4SgMAAAAAACg2DCEEAAAAAACAqRFgAQAAAAAAwNQIsAAAAAAAAGBqBFgAAAAAAAAwNQIsAAAAAAAAmBoBFmBCtWvX1pQpU4wuAwAAAAAAUyDAAgAAAAAAgKkRYAElJDs72+gSAAAAAABwCQRYwBXq2rWrRo4cqZEjR6p8+fKqXLmyXnrpJdlsNkn2YX8TJ07UoEGDFBgYqEcffVSSFBcXpxtuuEG+vr4KCwvTU089pYyMjILzHj9+XP369ZOvr6/q1KmjL774wpDnBwAAAACAWRFgAVdh1qxZ8vDw0Nq1a/XOO+/o7bff1ieffFKw/80331SrVq20adMmvfTSS0pMTFTv3r115513asuWLZo7d67i4uI0cuTIgvs89NBDOnTokJYtW6Z58+bp/fff1/Hjx414egAAAAAAmJKH0QUANptNmdbMEn2MvLw8ZVgz5J7tLjc3e27r5+kni8VyVecJCwvT5MmTZbFY1KhRI23dulWTJ0/WsGHDJEk33XSTnn322YLjH3nkEd1///16+umnJUkNGjTQu+++qxtvvFEffPCBkpKStGjRIq1du1bt27eXJH366adq0qRJMTxrAAAAAABcAwEWDJdpzVS5SeVK/XHTx6bL38v/qu5z/fXXFwq9wsPD9dZbbyk3N1eS1K5du0LHb968WVu2bCk0LNBmsykvL0/79+/Xnj175OHhobZt2xbsb9y4sYKCghx4RgAAAAAAuCYCLKAY+fsXDsTS09P12GOP6amnnrrg2Jo1a2rPnj2lVRoAAAAAAE6LAAuG8/P0U/rY9BJ9jLy8PKWdTVNgQGChIYRXa82aNYXW//zzTzVo0EDu7u5FHn/ddddpx44dql+/fpH7GzdurJycHG3YsKFgCOHu3bt15syZq64NAAAAAABXRYAFw1kslqseyne18vLylOuZK38v/4IAyxFJSUkaPXq0HnvsMW3cuFHvvfee3nrrrYseP2bMGF1//fUaOXKkHnnkEfn7+2vHjh2KjY3V1KlT1ahRI/Xu3VuPPfaYPvjgA3l4eOjpp5+Wr6+vwzUCAAAAAOBquAohcBUGDRqkc+fOqUOHDhoxYoRGjRqlRx999KLHt2zZUitWrNCePXt0ww03qE2bNho3bpxCQ0MLjpkxY4ZCQ0N144036o477tCjjz6qqlWrlsbTAQAAAADAKdADC7gKnp6emjJlij744IML9h04cKDI+7Rv316LFy++6DmDg4P1448/Ftr24IMPXlOdAAAAAAC4EnpgAQAAAAAAwNQIsAAAAAAAAGBqDCEErtDy5cuNLgEAAAAAgDKJHlgAAAAAAAAwNQIsAAAAAAAAmBoBFgxhs9mMLsGp8e8HAAAAAChLCLBQqjw9PSVJmZmZBlfi3PL//fL/PQEAAACg2OXkSBMmSD172pc5OUZXhDKMSdxRqtzd3RUUFKTjx49Lkvz8/GSxWEr8cfPy8pSdna3z58/Lzc15c1ubzabMzEwdP35cQUFBcnd3N7okAAAAAK4qJkaKjpZsNmnJEvu2ceMMLQllFwEWSl1wcLAkFYRYpcFms+ncuXPy9fUtlcCspAUFBRX8OwIAAABAiYiLs4dXkn0ZF2dsPSjTCLBQ6iwWi0JCQlS1alVZrdZSeUyr1aqVK1eqS5cuTj/sztPTk55XAAAAAEpeRIS955XNJlks9nXAIARYMIy7u3upBTHu7u7KycmRj4+P0wdYAAAAAFAqoqLsy7g4e3iVvw4YgAALAAAAAABcyMODOa9gGs47mzUAAAAAAADKBAIsAAAAAADMLidHmjBB6tnTvszJMboioFQxhBAAAAAAALOLiZGio+0Tqi9ZYt/G8D6UIfTAAgAAAADA7OLi7OGVZF/GxRlbD1DKCLAAAAAAADC7iAjJYrHftljs60AZQoDlxFauXKl+/fopNDRUFotFCxcuLLT/oYceksViKfTTu3fvy5532rRpql27tnx8fNSxY0etXbu2hJ4BAAAAAOCKREXZhxBGRtqXUVFGVwSUKubAcmIZGRlq1aqVHn74Yd1xxx1FHtO7d2/NmDGjYN3b2/uS55w7d65Gjx6t6dOnq2PHjpoyZYp69eql3bt3q2rVqsVaPwAAAADgCnl4MOcVyjQCLCfWp08f9enT55LHeHt7Kzg4+IrP+fbbb2vYsGEaMmSIJGn69On66aef9Nlnn+mFF164pnoBAAAAAAAcQYDl4pYvX66qVauqQoUKuummm/TKK6+oUqVKRR6bnZ2tDRs2aOzYsQXb3Nzc1KNHD61evfqij5GVlaWsrKyC9bS0NEmS1WqV1WotpmdybfLrMEs9cB60HTiKtgNH0XbgKNoOHEXbgaPM1HbMUANKFgGWC+vdu7fuuOMO1alTR4mJiYqKilKfPn20evVqubu7X3D8yZMnlZubq2rVqhXaXq1aNe3ateuijzNp0iSNHz/+gu2LFy+Wn5/ftT+RYhQbG2t0CXBStB04irYDR9F24CjaDhxF24GjzNB2MjMzjS4BJYwAy4Xdd999BbdbtGihli1bql69elq+fLm6d+9ebI8zduxYjR49umA9LS1NYWFh6tmzpwIDA4vtca6F1WpVbGysIiMj5enpaXQ5cCK0HTiKtgNH0XbgKNoOHEXbgaPM1HbyRwLBdRFglSF169ZV5cqVlZCQUGSAVblyZbm7u+vYsWOFth87duyS82h5e3sXOTm8p6en4S9i/2bGmuAcaDtwFG0HjqLtwFG0HTiKtgNHmaHtGP34KHluRheA0nP48GGdOnVKISEhRe738vJS27ZttXTp0oJteXl5Wrp0qcLDw0urTAAAAAAAgEIIsJxYenq64uPjFR8fL0nav3+/4uPjlZSUpPT0dD333HP6888/deDAAS1dulS33Xab6tevr169ehWco3v37po6dWrB+ujRo/Xxxx9r1qxZ2rlzp4YPH66MjIyCqxICAAAAAACUNoYQOrH169erW7duBev581ANHjxYH3zwgbZs2aJZs2bpzJkzCg0NVc+ePTVx4sRCw/0SExN18uTJgvV7771XJ06c0Lhx45SSkqLWrVvrl19+uWBidwAAAABACcjJkWJipLg4KSJCioqSPPjoDvBb4MS6du0qm8120f2//vrrZc9x4MCBC7aNHDlSI0eOvJbSAAAAAACOiImRoqMlm01assS+bdw4Q0sCzIAhhAAAAAAAmEVcnD28kuzLuDhj6wFMggALAAAAAACziIiQLBb7bYvFvg6AIYQAAAAAAJhGVJR9+c85sAAQYAEAAAAAYBoeHsx5BRSBIYQAAAAAAAAwNQIsAAAAAAAAmBoBFgAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAUyPAAgAAAAAAgKkRYKHsysmRJkyQeva0L3NyjK4IAAAAAIrG5xeUcR5GFwAYJiZGio6WbDZpyRL7tnHjDC0JAAAAAIrE5xeUcfTAQtkVF2d/8Zfsy7g4Y+sBAAAAgIvh8wvKOAIslF0REZLFYr9tsdjXAQAAAMCM+PyCMo4hhCi7oqLsy7g4+4t//joAAAAAmA2fX1DGEWCh7PLwYMw4AAAAAOfA5xeUcQwhBAAAAAAAgKkRYAEAAAAAAMDUCLAAAAAAAABgagRYAAAAAAAAMDUCLAAAAAAAAJgaARYAAAAAAABMjQALAAAAAAAApkaABQAAAAAAAFMjwAIAAAAA4Grk5EgTJkg9e9qXOTlGVwS4PA+jCwAAAAAAwKnExEjR0ZLNJi1ZYt82bpx9mZNj3x8XJ0VESFFRkgcfvYFrxW8RAAAAAABXIy7OHl5J9mVc3N/7LhVuAXAYQwgBAAAAALgaERGSxWK/bbHY1/NdKtwC4DB6YAEAAAAAcDWiouzLfw4TzBcRYe95ZbNdGG4BcBgBFsomxqUDAAAAcJSHx8WHBV4q3ALgMD6xo2xiXDoAAABQNpT2l9eXCrcAOIwAC2UT49IBAACAsoEvrwGXwCTuKJsuNekiAAAAANfBl9eASyDAcmIrV65Uv379FBoaKovFooULFxbss1qtGjNmjFq0aCF/f3+FhoZq0KBBSk5OvuQ5o6OjZbFYCv00bty4hJ+JAaKi7N/CREbal4xLBwAAAFwTX14DLoEhhE4sIyNDrVq10sMPP6w77rij0L7MzExt3LhRL730klq1aqXTp09r1KhRuvXWW7V+/fpLnrdZs2Zakt+1VpKHK05uzrh0AAAAoGxgUnXAJbhgMlF29OnTR3369ClyX/ny5RUbG1to29SpU9WhQwclJSWpZs2aFz2vh4eHgoODi7VWAAAAADAEX14DLoEAqwxJTU2VxWJRUFDQJY/bu3evQkND5ePjo/DwcE2aNOmSgVdWVpaysrIK1tPS0iTZhzFardZiqf1a5ddhlnrgPGg7cBRtB46i7cBRtB04irYDR5mp7ZihBpQsi82WP5sdnJnFYtGCBQvUv3//IvefP39enTt3VuPGjfXFF19c9DyLFi1Senq6GjVqpKNHj2r8+PE6cuSItm3bpoCAgCLvEx0drfHjx1+wfc6cOfLz83Po+QAAAAAAcKUyMzM1cOBApaamKjAw0OhyUAIIsFzEpQIsq9WqO++8U4cPH9by5cuv6pf5zJkzqlWrlt5++20NHTq0yGOK6oEVFhamkydPmuaFw2q1KjY2VpGRkfL09DS6HDgR2g4cRduBo2g7cBRtB46i7fxPTo701lvS6tVSeLj07LP24Ye4KDO1nbS0NFWuXJkAy4Xx2+jirFar7rnnHh08eFC//fbbVf8iBwUFqWHDhkpISLjoMd7e3vL29r5gu6enp+EvYv9mxprgHGg7cBRtB46i7cBRtB04qsy3nUmT7Fcot9mkRYukvDzmzrpCZmg7Rj8+Sp6b0QWg5OSHV3v37tWSJUtUqVKlqz5Henq6EhMTFRISUgIVAgAAAIBJxMXZwyvJvoyLM7YeAIUQYDmx9PR0xcfHKz4+XpK0f/9+xcfHKykpSVarVXfddZfWr1+vL774Qrm5uUpJSVFKSoqys7MLztG9e3dNnTq1YP0///mPVqxYoQMHDmjVqlW6/fbb5e7urgEDBpT20wMAAACA0hMRIVks9tsWi30dgGkwhNCJrV+/Xt26dStYHz16tCRp8ODBio6O1vfffy9Jat26daH7LVu2TF27dpUkJSYm6uTJkwX7Dh8+rAEDBujUqVOqUqWKIiIi9Oeff6pKlSol+2QAAAAAwEhRUfZlXJw9vMpfB2AKBFhOrGvXrrrUHPxXMj//gQMHCq1/9dVX11oWAAAAADgfDw/mvAJMjCGEAAAAAAAAMDUCLAAAAAAAAJgaARYAAAAAAABMjQALAAAAAAAApkaABQAAAAAAAFMjwAIAAAAAAICpEWABAAAAAADA1AiwAAAAAAAAYGoEWIARcnKkCROknj3ty5wcoysCAAAAAMC0PIwuACiTYmKk6GjJZpOWLLFvGzfO0JIAAAAAADAremABRoiLs4dXkn0ZF2dsPQAAAAAAmBgBFmCEiAjJYrHftljs6wAAAAAAoEgMIQSMEBVlX8bF2cOr/HUAAAAAAHABAizACB4ezHkFAAAAAMAVYgghYFZcqRAAAAAAAEn0wEIZZrPZlJOXI093T6NLKRpXKgQAAAAAQBI9sFCGvb/ufXX+rLP2n95vdClF40qFAAAAAABIIsBCGZVpzdR/f/+v1iWvU5sP22jejnlGl3QhrlQIAAAAAIAkAiyUUX6efvrzkT8VXiNcqVmpuvubuzX8x+E6Zz1ndGl/i4qyDyGMjLQvuVIhAAAAULqYlxYwDebAQplVs3xNrXhohV5e/rJejXtV0zdM16rDqzS3/xdq/NF8+5C9iAh7cORhwK8KVyoEAAAASl5Ojn3+2aLe/zMvLWAaBFgo0zzdPRXTPUZda3fVgwse1JZjW9T2w+v0/kKrBsfLfH+kLvXHFQAAAMDVu1RIxby0gGkwhBCQ1LNeT21+fLO61+muTFn1UH/pwdulNC+T/ZHK/+MaG2tfxsQYXREAAADg3C4VUjEvLWAaBFjA/wSXC9avD/yq/6q73POk2a2k1o9LqzvXMrq0v/ENEAAAAFC8LhVSMS8tYBqMPQL+wd3NXVH/7xd1/e/juv/8F9pf4bxusMzQuBVhirohSh5uBv/KRETYuzXbbHwDBAAAABSH/FDqn9N05GNeWsA0CLCAf/PwUKeXP1H8+bc04ucR+mLrF3p5+ctanLhYs++YrdpBtY2r7VJ/XAEAAABcPUIqwCkwhBC4iPI+5TX7jtmafftsBXgF6I9Df6jV9Fb6cuuXxhWV/8d18WL7kgncAQAAAABlAAEWcBn3t7xfmx/frPAa4UrLStPA+QM1aMEgpWWlGV0aAAAAAABlAgEWcAXqVKijlUNW6uUbX5abxU2fb/lcbT5soz8P/2l0aQAAAAAAuDwCLOAKebh5KLprtFY+tFK1ytfSvtP7FPFZhCaumKjcvFyjywMAAAAAwGURYAFXqXPNztr8+GYNbDFQubZcjVs+Tl1nddXBMwcdP2lOjjRhgtSzp32Zk1Ns9QIAAAAA4OwIsAAHlPcpry/u+EKf3/65ArwCFJcUp1bTW+mrbV85dsKYGCk6WoqNtS9jYoqzXAAAAAAAnBoBFnANHmj5gOIfj9f1Na5XalaqBnw7QA8tfEhns85e3Yni4iSbzX7bZrOvAwAAAAAASQRYwDWrW6Gufh/yu8Z1GSc3i5tmbZ6lNh+20ZrDa678JBERksViv22x2NcBAAAA2DHlBlDmEWA5sZUrV6pfv34KDQ2VxWLRwoULC+232WwaN26cQkJC5Ovrqx49emjv3r2XPe+0adNUu3Zt+fj4qGPHjlq7dm0JPQPX4eHmofHdxmvFQytUs3xNJZ5OVOfPOuu/K/97ZRO8R0XZhw5GRtqXUVElXTIAAADgPJhyAyjzCLCcWEZGhlq1aqVp06YVuf/111/Xu+++q+nTp2vNmjXy9/dXr169dP78+Yuec+7cuRo9erRefvllbdy4Ua1atVKvXr10/PjxknoaLiWiZoQ2P75Z9zW/T7m2XL247EXd9H83KSk16dJ39PCQxo2TFi+2Lz08SqdgAAAAwEwu1tOKKTeAMo8Ay4n16dNHr7zyim6//fYL9tlsNk2ZMkUvvviibrvtNrVs2VL/93//p+Tk5At6av3T22+/rWHDhmnIkCFq2rSppk+fLj8/P3322Wcl+ExcS5BPkObcMUez+s9SOa9yWnlwpVpNb6Wvt39tdGkAAACAuV2spxVTbgBlHt08XNT+/fuVkpKiHj16FGwrX768OnbsqNWrV+u+++674D7Z2dnasGGDxo4dW7DNzc1NPXr00OrVqy/6WFlZWcrKyipYT0tLkyRZrVZZrdbieDrXLL+O0qxnQNMB6hDSQYO/G6y1yWt177x79dOenzQ5crICvANKrQ5cGyPaDlwDbQeOou3AUbQdOMpUbWfNGsnHp/C61So995zk5iatXi2Fh0vPPmvfDkOZqe2YoQaULIvNlt8PE87MYrFowYIF6t+/vyRp1apV6ty5s5KTkxUSElJw3D333COLxaK5c+decI7k5GRVr15dq1atUnh4eMH2559/XitWrNCaNUVPSh4dHa3x48dfsH3OnDny8/O7xmfm/HJsOZqbMlfzjs2TTTYFewVrdK3Raujf0OjSAAAAAMAlZGZmauDAgUpNTVVgYKDR5aAE0AML12zs2LEaPXp0wXpaWprCwsLUs2dP07xwWK1WxcbGKjIyUp6enqX++LfqVj2e9Lge+v4hHUo7pKjEKI27YZyeC39O7m7upV4PrpzRbQfOi7YDR9F24CjaDhxlqraTkyO99VbhnlbMD2taZmo7+SOB4Lp4JXBRwcHBkqRjx44V6oF17NgxtW7dusj7VK5cWe7u7jp27Fih7ceOHSs4X1G8vb3l7e19wXZPT0/DX8T+zciabqp3kzY/vlmP//S4vt7+tcatGKclB5bo//r/n2oF1TKkJlw5M7ZnOAfaDhxF24GjaDtwlCnajqen9OKLxtaAq2aGtmP046PkMYm7i6pTp46Cg4O1dOnSgm1paWlas2ZNoeGB/+Tl5aW2bdsWuk9eXp6WLl160fvg6lTwraCv7vxKM26bIX9Pf608uFItp7fU/23+PzGaFwAAAACAohFgObH09HTFx8crPj5ekn3i9vj4eCUlJclisejpp5/WK6+8ou+//15bt27VoEGDFBoaWjBPliR1795dU6dOLVgfPXq0Pv74Y82aNUs7d+7U8OHDlZGRoSFDhpTys3NdFotFD7V+SJsf36xOYZ2UlpWmwQsH665v7tLJzJNGlwcAAAAAgOkwhNCJrV+/Xt26dStYz5+HavDgwZo5c6aef/55ZWRk6NFHH9WZM2cUERGhX375RT7/uKpHYmKiTp78OzS59957deLECY0bN04pKSlq3bq1fvnlF1WrVq30nlgZUa9iPa18aKVe/+N1jVs+TvN3ztcfSX/o434fq1+jfkaXBwAAAACAaRBgObGuXbtectiZxWLRhAkTNGHChIsec+DAgQu2jRw5UiNHjiyOEnEZ7m7uGnvDWPWu31sPLnhQ209s161f3aqhbYbq7V5vK9DbHJPgAwAAAABgJIYQAibQJqSN1j+6Xv8J/48ssujTTZ+q1fRWWnFghdGlAQAAAABgOAIswCR8PHz0Rs83tPyh5aodVFsHzhxQt1nd9Oyvz+p8zvlrO3lOjjRhgtSzp32Zk1M8RQMAAAAAUAoIsACT6VKri7Y8vkWPtHlENtn09p9v67oPr9P65PWOnzQmRoqOlmJj7cuYmOIqFwAAAACAEkeABZhQgHeAPr71Y/0w4AcFlwvWzpM7df0n1+vlZS8rOzf76k8YFyflz5dms9nXAQAAAABwEgRYgIn1bdhX24Zv073N7lWuLVcTVk5Qx086auuxrVd3oogIyWKx37ZY7OsAAAAAADgJAizA5Cr5VdJXd32luXfNVSXfSopPiVfbj9rqlZWvyJprvbKTREXZhw5GRtqXUVElWTIAAAAAAMWKAAtwEvc0u0fbntim2xrdJmueVS8te0nXf3r9lfXG8vCQxo2TFi+2Lz08Sr5gAAAAAACKCQEW4ESCywVrwb0LNPv22argU0Ebj268+t5YAAAAAAA4GQIswMlYLBbd3/J+bX9iu25tdGuh3lhbjm0xujwAAAAAAIodARbgpEICQrTw3oWFemO1+6idJq6YSG8sAAAAAIBLIcACnFh+b6wdI3YUzI01bvk4dfykozanbDa6PAAAAKBoOTnShAlSz572ZU6O0RUBMDkCLMAF5M+NNeeOOaroW1GbUjap3cftNGHFBHpjAQAAwHxiYuxXx46NtS9jYoyuCIDJEWABLsJisWhAiwHa/sR29W/cXzl5OXp5+cvq8EkHemMBAADAXOLiJJvNfttms68DwCUQYAEuJrhcsObfM7+gN1Z8SrzafdxO45ePV3ZuttHlAQAAAFJEhGSx2G9bLPZ1ALgEAizABeX3xtrxxA7d3vh25eTlKHpFtNp+1FZrDq8xujwAAACUdVFR9qGDkZH2ZVSU0RUBMDkCLMCFVStXTd/e862+vPNLVfarrG3Htyn803A9tegpnc06a3R5AAAAKKs8PKRx46TFi+1LDw+jKwJgcgRYgIuzWCy6r/l92jlipwa1GiSbbHpv7Xtq9n4z/bjnR6PLAwAAAADgsgiwgDKisl9lzeo/S4sfWKw6QXV0KO2Q+n3ZT/d8c49S0lOMLg8AAAAAgIsiwALKmMh6kdr2xDY91+k5uVvc9c2Ob9RkWhN9vOFj5dnyjC4PAAAAAIALEGABZZCfp59ej3xd64atU9uQtjpz/owe/fFRdZvVTbtP7ja6PAAAAAAACiHAAsqwNiFt9Ocjf+qtnm/Jz9NPKw+uVMvpLTVxxURl52YbXR4AAAAAAJIIsIAyz8PNQ6PDR2v7E9vVu35vZedma9zycbruw+u0+tBqo8sDAAAAAIAAC4Bd7aDa+nngz5pzxxxV8aui7Se2q/NnnTXy55FKy0ozujwAAAAAQBlGgAWggMVi0YAWA7RzxE491Poh2WTTtHXT1HRaU3236zujywMAAEBxycmRJkyQevaUXnvN6GoA4LIIsABcoJJfJc24bYaWPLhE9SrU05GzR9R/bn/d9fVdSj6bbHR5AAAAuFYxMVJ0tBQbK02aZHQ1AHBZBFiAM/jnN2QTJtjXS0H3ut21dfhWvdD5Bblb3PXtzm/VdFpTfbj+Q+XZ8kqlBgAAAJSAuDjJZrPfzl8CgIkRYAHO4J/fkEVH29dLia+nryb1mKQNj25Q+9D2Ss1K1eM/Pa4bZ96onSd2llodAAAAKEYREZLFYr+dvwQAEyPAApzBv78hi4sr9RJaBbfS6qGrNaXXFPl7+isuKU6tP2yt8cvHKysnq9TrAQAAwDWIirJ/MRoZKY0da3Q1AHBZBFiAM/j3N2QREYaU4e7mrlHXj9L2J7br5gY3Kzs3W9ErotXmwzb6I+kPQ2oCAACAAzw8pHHjpMWLpTFjjK4GAC6LAKuUDB48WCtXrjS6DDirf35DFh1tXzdQraBa+nHAj/rqzq9U1b+qdp7cqYgZERr+43Clnk81tDYAAAAAgOshwColqamp6tGjhxo0aKCYmBgdOXLE6JLgTP75Ddm4cfZ1g1ksFt3b/F7tHLFTD7d+WJI0fcN0NX2/qRbsXGBwdQAAAAAAV0KAVUoWLlyoI0eOaPjw4Zo7d65q166tPn36aN68ebJarUaXBzisom9FfXrbp/pt0G9qULGBks8m646v71D/r/orKTXJ6PIAAAAAAC6AAKsUValSRaNHj9bmzZu1Zs0a1a9fXw8++KBCQ0P1zDPPaO/evUaXCDisW51u2vz4ZkVFRMnDzUPf7f5OTaY10et/vC5rLiEtAAAAAMBxBFgGOHr0qGJjYxUbGyt3d3fdfPPN2rp1q5o2barJkycX62PVrl1bFovlgp8RI0YUefzMmTMvONbHx6dYa4Lr8vX01X+7/1fxj8Xrhpo3KNOaqTFLxqj1h6218iBzwAEAAAAAHEOAVUqsVqu+/fZb9e3bV7Vq1dI333yjp59+WsnJyZo1a5aWLFmir7/+WhMmTCjWx123bp2OHj1a8BMbGytJuvvuuy96n8DAwEL3OXjwYLHWBNfXrGozrXhohWbeNlOV/Sprx4kdunHmjXpwfCulnDlsdHkAAAAAACdj/EzQZURISIjy8vI0YMAArV27Vq1bt77gmG7duikoKKhYH7dKlSqF1l999VXVq1dPN95440XvY7FYFBwcXKx1oAzJyZFiYmSJi9PgiAj1yx2iqI1v6KO20mzLFn3/Tn290vsNDW8/XB5uvAQBAAAAAC6PT4+lZPLkybr77rsvORwvKChI+/fvL7EasrOzNXv2bI0ePVoWi+Wix6Wnp6tWrVrKy8vTddddp5iYGDVr1uyix2dlZSkrK6tgPS0tTZK915lZJqjPr8Ms9bi0116z/9hs0h9/KKBWLb13wFeDduTpychsbQzJ0lO/PKVPN32q93q9p+trXG90xZdE24GjaDtwFG0HjqLtwFG0HTjKTG3HDDWgZFlsNpvN6CJQOr7++msNHDhQSUlJCg0NLfKY1atXa+/evWrZsqVSU1P15ptvauXKldq+fbtq1KhR5H2io6M1fvz4C7bPmTNHfn5+xfoc4NxybbmKPRWr2UdnKz03XZLUvWJ3DQodpPIe5Q2uDgAAAICzyszM1MCBA5WamqrAwECjy0EJIMAqQ3r16iUvLy/98MMPV3wfq9WqJk2aaMCAAZo4cWKRxxTVAyssLEwnT540zQuH1WpVbGysIiMj5enpaXQ5ru2116RJk+w9sCwWacwYyc1NWr1aCg+Xnn1WJ7JOK2pZlGZtmSVJCvIJ0oQbJ2hYm2Fyd3M3+AkURtuBo2g7cBRtB46i7cBRtB04ykxtJy0tTZUrVybAcmEMISwjDh48qCVLlmj+/PlXdT9PT0+1adNGCQkJFz3G29tb3t7eRd7X6BexfzNjTS7nhRekvDwpLk6KiJCioiSPwi81ob6+mnn7TD3a7lGN+HmE4lPi9dSvT2nG5hmaevNUdQrrZFDxF0fbgaNoO3AUbQeOou3AUbQdOMoMbcfox0fJ4yqEZcSMGTNUtWpV3XLLLVd1v9zcXG3dulUhISElVBlcjoeHNG6ctHixfelx8Zy8U1gnrR+2XlP7TFWQT5A2pWxS5886a9CCQUo+m1yKRQMAAAAAzIwAqwzIy8vTjBkzNHjwYHn8K0wYNGiQxo4dW7A+YcIELV68WPv27dPGjRv1wAMP6ODBg3rkkUdKu2yUEe5u7hrRYYT2jNyjoW2GyiKLPt/yuRq+11CTfp+k8znnjS4RAAAAAGAwAqwyYMmSJUpKStLDDz98wb6kpCQdPXq0YP306dMaNmyYmjRpoptvvllpaWlatWqVmjZtWpolowyq4l9Fn9z6idYOW6vwGuHKsGYo6rcoNXu/mb7b9Z2Yrg8AAAAAyi7mwCoDevbsedEP/8uXLy+0PnnyZE2ePLkUqgKK1i60nf54+A/N2TpHzy95XvtO71P/uf0VWTdSU3pPUdMqhKkAAAAAUNbQAwuA6VgsFt3f8n7tHrlbURFR8nL3Uuy+WLX8oKVGLRql0+dOG10iAAAAAKAUEWABMIecHGnCBKlnT/syJ0flvMrpv93/q50jdqp/4/7KteXq3bXvqsF7DfTh+g+Vm5drdNUAAAAAgFJAgAXAHGJipOhoKTbWvoyJKdhVt0JdLbh3gWIfjFXTKk116twpPf7T42r7UVutPLjSsJIBAAAAAKWDAAuAOcTFSflztdls9vV/6VG3hzY/vlnv9n5XQT5B2nxss26ceaPunXevklKTSrlgAAAAAEBpIcACYA4REZLFYr9tsdjXi+Dh5qEnOz6pvU/u1eNtH5ebxU1fb/9ajac21vjl45VpzSzFogEAAEpAEVMrAEBZx1UIARgjJ8c+TDAuzh5WPf+8fXv+elTUJe9e2a+yPuj7gR5r95hG/TJKKw+uVPSKaH0W/5neiHxDdze9W5b8QAwAAMCZ5E+tYLNJS5bYt40bZ2hJAGA0AiwAxiimN2atg1tr+eDl+mbHN/rP4v8oKTVJ9867V++GvavJvSarffX2xVs3AACAo/79BV5UlORRxEeyK5ha4ZrODwBOiCGEAIzh6BuzIlgsFt3T7B7tGrlLL9/4snw9fPXHoT/U4ZMOenDBgzqcdriYigYAALgGRV20pqjhgp06Fb7fv9ev5vwA4CIIsAAY4wrnvLoafp5+iu4arT1P7tGgVoMkSbO3zFbD9xpq3LJxSs9Ov+bHAAAAuCJFBVNFfYFXVOiUf0y+f69fTDF+QQgAZkN/UgDGyJ/j6grnvLoaNQJraFb/WXqyw5Ma/eto/Z70uyaunKiPN36sV7q9oodaPyR3N/diezwAAIALFDVdQkSE/bbN9vcXeFcSOq1efWWPWdT5AcBFEGABMIaHR4lPRtoutJ1WPLRC83fO1/NLnte+0/v0yA+PaMqaKXq1+6u6ucHNTPQOAABKRlHB1M8//70v/wu8mJiiQydHgqgS/IIQAIxGgAXApVksFt3Z9E71bdhX09ZN08SVE7Xt+Db1/bKvbqh5g17r8ZrCw8KNLhMAALiaonpDFfUF3qVCp6sNokrhC0IAMAoBFoAywdvDW6PDR+uh1g/p1bhX9e6ad/V70u/q9Fkn9W/cXzE3xahJlSZGlwkAAMzuSq/0d6W9oS4WOhFEAUAhTOIOoEyp6FtRr0e+rr1P7tXDrR+Wm8VNC3ctVPMPmuuR7x/hioUAAODSrvRKf/nB1OLF9mVRIRcA4IoRYAFwbUVdAUhSWPkwfXrbp9o6fKtua3Sb8mx5+nTTp2rwXgO9sOQFnT532uDCAQCAKXGlPwAwBAEWANd2mW9Jm1ZpqoX3LdQfD/+hiJoROp9zXq/98ZrqvVtPb/zxhs5ZzxlSNgAAMKmICPucVhJX+gOAUkSABcC1XeG3pJ3COmnlQyv1/X3fq1mVZjp9/rSeX/K8mk1vpiWnlignL6cUiwYAAKYVFWX/Uiwy0r7kSn8AUCoIsAC4tqv4ltRisahfo37a/PhmzbxtpsICw3T47GFNPTRVbT9pq+92fSdbfhgGAACMdZFpAkrctc5tZVTdAODkmEkQgGu70isA/YO7m7sGtx6se5vfq3f/fFevLH9FO0/uVP+5/dUprJNe6/GaImoyXAAAAEPlTxNgs0lLlti3OcOV+5y1bgAwGAEWANd2sUtTXwEfDx890/EZVT9WXVsDt+rdte9q1aFVumHGDerXsJ9iuseoedXmxVwwAAC4ImabTD0nxx5O/fNLs6J6Z5mtbgBwEgwhBIDLKOdRThO7TlTCUwl6rO1jcre464c9P6jV9FYa8t0QJaUmGV0iAABlj9kmU7/MhWMKmK1uAHAS9MACgCsUGhCq6X2n65nrn9H/++3/6dud32pm/Ex9ufVLjewwUmMjxqqSXyWjywQAoGxwYJqAEnWlPavMVjcAOAkCLAC4So0qN9K8e+ZpzeE1emHpC1p+YLneWv2WPtn4icZ0HqNR14+Sn6ef0WUCAODarmGagBIREWGf08pmu3TPKrPVDQBOgiGEAOCgjjU66rdBv2nR/YvUqlorpWalKuq3KDV4r4E+3vCxcvK4qhAAAGVGVJR96GBkpH1JzyoAKFYEWABwDSwWi3rX762Nj23U7Ntnq3ZQbSWfTdajPz6q5u8317c7vlWeLc/oMgEAQEnL71m1eLF9WdQE7gAAhxFgAUAxcLO46f6W92vXiF16p/c7quxXWbtP7dZd39ylth+11Q+7f5Atf14MAAAAAMBVIcACgGLk7eGtpzo+pcSnEjWuyzgFeAUoPiVet351qzp80kGL9i4iyAIAAIDTsNlssuZajS4DIMACgJIQ6B2o8d3Ga/+o/Xqh8wvy8/TT+uT1unnOzer8WWct2beEIAsAgJKSkyNNmCD17Glf5jAvJXC1bDabfk34VeGfhuuVla8YXQ5AgAUAJamSXyVN6jFJ+0ft17Phz8rHw0erD69W5OeR6jqrq1YcWGF0iQAAuJ6YGPtE6rGx9mVMjNEVAU7DZrNpceJidf6ss3p/0VtrjqzRRxs/ohcWDEeABQCloKp/Vb3Z803te2qfnurwlLzcvbTy4Ep1ndVVPf6vh/5I+sPoEgEAcB1xcVJ+T2ebzb5uJHqEwQnYbDYt2bdEETMi1Gt2L60+vFo+Hj565vpntOmxTfJ09zS6RJRxBFgAXMfl3hya4M1jSECI3unzjhKfStTwdsPl6eappfuXKmJGhLrN6qbf9v/G0EIAAK5VRIRksdhvWyz2dSPRIwwmZrPZtHTfUnWZ2UWRn0dq1aFV8nb31qiOo7TvqX16u9fbCi4XbHSZgLi2KwDXkf/m0GaTliyxbxs37sr3l6IagTX0/i3v6/nOzyvm9xjNjJ+p5QeWa/mB5eoU1kkvdXlJver1kiX/zTcAALhyUVH2ZVycPbzKXzeK2XqEAbIHV8sOLFP08mj9nvS7JMnb3VuPtX1MYyLGKDQg1OAKgcLogQXAdVzuzaEJ3zzWDqqtj/p9pISnEjSi/Qh5u3tr1aFV6vNFH3X8pKO+3/09PbIAALhaHh72L6kWL7YvPQz+3t5sPcJQptlsNv205yd1/qyzuv9fd/2e9Lu83L00sv1IJT6VqHf6vEN4BVMiwHJh0dHRslgshX4aN258yft88803aty4sXx8fNSiRQv9/PPPpVQtUAwu9+bQxG8ea5avqak3T9W+Ufv0zPXPyNfDV+uS1+m2r25Tmw/baN6Oecqz5RldJgAAcERUlL0XeGSkfWl0jzCUSXm2PM3bMU/XfXSd+n7ZV6sPr5a3u7dGtB+hxKcS9d7N76l6YHWjywQuiiGELq5Zs2Zakj9USpLHJb59WrVqlQYMGKBJkyapb9++mjNnjvr376+NGzeqefPmpVEucG0uN1zAbMMJihAaEKq3e72tFyJe0Nur39a0ddO0+dhm3f3N3Wpapan+3w3/T/c2u1fubu5GlwoAAK5Ufo8wwAA5eTn6cuuXiomL0a6TuyRJ/p7+Gt5uuEaHj1ZIQIjBFQJXhgDLxXl4eCg4+Mom3HvnnXfUu3dvPffcc5KkiRMnKjY2VlOnTtX06dNLskygeFzuzaETvXms6l9Vr/Z4Vc91ek7vrHlH7655VztO7ND98+9X9PJoRd0Qpftb3M/VYAAAAFCkrJwszYyfqdf+eE37z+yXJAX5BOnJDk9qVMdRquRXyeAKgatDgOXi9u7dq9DQUPn4+Cg8PFyTJk1SzZo1izx29erVGj16dKFtvXr10sKFCy/5GFlZWcrKyipYT0tLkyRZrVZZrdZrewLFJL8Os9QD52F02wn0DNRLES/pqXZPadr6aXp33bva+9deDfluiMYvH6/nOz2vB1s8KG8Pb0Pqw8UZ3XbgvGg7cBRtB46i7biWjOwMfRr/qSavmawjZ49Ikqr4VdGoDqP0eNvHFegdKKl4/r/N1HbMUANKlsXG7MAua9GiRUpPT1ejRo109OhRjR8/XkeOHNG2bdsUEBBwwfFeXl6aNWuWBgwYULDt/fff1/jx43Xs2LGLPk50dLTGjx9/wfY5c+bIz8+veJ4MAEnSudxz+uXUL1p4fKFSc1IlSZU8K+mOqneoR6Ue8nYjyAIAACiLMnIztOjkIn1//Hul5do7FVTyrKT+VfurZ6WeLv8+MTMzUwMHDlRqaqoCAwONLgclgACrDDlz5oxq1aqlt99+W0OHDr1gv6MBVlE9sMLCwnTy5EnTvHBYrVbFxsYqMjJSnp4MucKVc7jt5ORIb70lrV4thYdLzz5brFdAyrRm6tNNn+qtP99ScnqyJCnYP1jPXP+MHm3zqPy9/A2rDXa87sBRtB04irYDR9F2nNupzFN6b917mrZ+mlKz7F9w1gmqo+fCnyvxnvpmajtpaWmqXLkyAZYL4xNLGRIUFKSGDRsqISGhyP3BwcEXBFXHjh277Bxa3t7e8va+8EXR09PT8BexfzNjTXAOV912Jk2yX2XIZpMWLZLy8oqefysnR4qJKTyx/BWESeU9y2t059F6ouMTmrFphl7941UlpSZpzNIxemP1G3q649N6ov0TquBbwfHaUCx43YGjaDtwFG0HjqLtOJejZ4/qrdVvafr66cqwZkiSmlRuoqgbonRf8/vk4VZ6H/fN0HaMfnyUPDejC0DpSU9PV2JiokJCir7KRHh4uJYuXVpoW2xsrMLDw0ujPMC1xMXZAyLJvoyLK/q4mBh7mBQba1/GxFzVw/h4+Gh4++Ha++RefdLvE9WtUFcnM0/qxWUvquaUmvrP4v/oSNoRx2oDAACA6Rw8c1AjfhqhOu/U0Vur31KGNUNtgtto3t3ztO2JbXqg5QOlGl4BpYUAy4X95z//0YoVK3TgwAGtWrVKt99+u9zd3QuGCA4aNEhjx44tOH7UqFH65Zdf9NZbb2nXrl2Kjo7W+vXrNXLkSKOeAuC8IiIki8V+22KxrxelmMIkL3cvDb1uqHaP3K3Zt89Wi6otlJ6drrdWv6U679TR0O+GFlw2+YprAwAAgGnsOLFDQ74bovrv1df7699XVm6WOoV10k8Df9KGRzfozqZ3ys3CR3y4LmJZF3b48GENGDBAp06dUpUqVRQREaE///xTVapUkSQlJSXJze3vF7hOnTppzpw5evHFFxUVFaUGDRpo4cKFat68uVFPAXBeUVH25T+HBhYlIkJassQeXhVDmOTh5qH7W96vgS0G6peEX/TqH69q5cGV+iz+M82In6H+jftrzOBn1fFKagMAAIDh4pLi9Pofr+uHPT8UbOtep7te7PKibqx1oyz5X0wCLo4Ay4V99dVXl9y/fPnyC7bdfffduvvuu0uoIqAM8fC4snmlrjToukoWi0V9GvRRnwZ9tPrQar32x2v6bvd3WrBrgRbsWqCutbvq2fue1c0NbuabOgAAAJPJs+Xph90/6PVVr2vVoVWSJIssur3J7Xq+0/PqWKOjwRUCpY8ACwCMdKVB1zUIDwvXwvsWaueJnXp91euavWW2lh9YruUHlqtehXp6ssOTGtJmiAK9uVoLAKAEOHjBEqAsOmc9p8+3fK7Jf04umP7By91Lg1sN1rPhz6pR5UYGVwgYh78cAFBGNKnSRDNum6GJ3Sbq3TXv6uONHyvxdKKe/vVpvbTsJQ1pPUQjO4xUg0oNjC4VAOBK8i9YYrPZh81LXP0W+JeU9BS9v+59fbD+A53MPClJKu9dXk+0f0JPdXxKweUufWV4oCxg3AgAlDE1Amvo9cjXdfiZw/rglg/UpHITnc0+q3fXvqtGUxup75y+ik2MlS1/cnkAAC4nJ0eaMEHq2dO+zMn5ex9XvwUuatvxbXr4u4dVa0otTVw5USczT6p2UG1N7jVZh545pJjuMYRXwP/QAwsAyih/L3893u5xPdb2MS3Zt0TvrHlHP+39qeCnSeUmeqrjU3qw5YPy9/I3ulwAgJldqpdVMV+wBHB2NptNixMX6+0/39bixMUF28NrhGt0+Gj1b9xfHm58VAf+jd8KACjjLBaLIutFKrJepPae2qupa6dqRvwM7Ty5U8N/Gq6xS8dq2HXDNKL9CNUKqmV0uQAAM7pUL6sSumAJ4GzO55zXF1u+0OQ/J2v7ie2SJDeLm+5scqeeuf4ZhYeFG1whYG4EWACAAg0qNdA7fd7RxJsmasamGXpv7XtKPJ2oN1a9obdWv6X+jftrVMdRuqHmDVyyGQDwt0v1siqFC5YAZnYi44Q+WP+Bpq2bpuMZxyVJ5bzK6ZE2j+ipjk+pToU6BlcIOAcCLADABQK9AzXq+lEa2WGkFiUs0jtr3tGSfUs0f+d8zd85X62DW2tUx1G6r/l98vHwMbpcAIDR6GUFXGDHiR2a8ucU/d/m/1NWbpYkKSwwTKM6jtIj1z2i8j7lDa4QcC4EWACAi3J3c1ffhn3Vt2FfbT++Xe+ueVefb/lc8SnxGvLdED0f+7wea/uYhrcfrtCAUKPLBQAYhV5WgCQpz5anXxN+1btr39UvCb8UbG8X2k7Phj+rO5vcKU93TwMrBJwXVyEEAFyRZlWb6cN+H+rw6MN6rcdrCgsM04nME3rl91dUa0otDfx2oNYcXmN0mQAAAKUuLStN7615T42nNtbNc27WLwm/yCKLbm98u34f8rvWPrJW9zW/j/AKuAb0wAIAXJWKvhX1fOfnNTp8tBbuWqh317yr35N+15fbvtSX275Ux+odNarjKN3Z9E55uXsZXS4AAECJ+ecFcM5mn5Vkn4phaJuhGtF+hOpVrGdwhYDrIMACADjEw81DdzW9S3c1vUsbj27Uu2ve1ZfbvtSaI2s0cP5AhSwO0RPtn9BjbR9TFf8qRpcLAABQLPJsefol4RdNXTtVixIWFWxvXLmxnuzwpAa1GqRyXuUMrBBwTQwhBABcs+tCrtPM/jOV9HSSJnSdoOBywTqaflQvLXtJYZPDNOS7IYpPiTe6TAAAAIf9de4vvbXqLTV4r4FumXOLFiUskkUW9W3YV78+8Ku2P7FdT7R/gvAKKCH0wAIAFJtq5arppRtf0piIMfpm+zd6Z807Wpe8TjPjZ2pm/Ex1qdVFozqO0q2NbpWHG3+CAACA+W06uknT1k3TnK1zdC7nnCQpyCdID7d+WMPbD1f9ivUNrhAoG/j0AAAodl7uXrq/5f0a2GKg/jz8p95d+67m7ZinlQdXauXBlapVvpZGtB+hR657RBV8KxhdLgAAQCHnrOf09fav9f7697X2yNqC7a2DW2tE+xEa2GKg/Dz9DKwQKHsIsAAAJcZisSg8LFzhYeF6M/JNvb/ufX244UMdTD2o55c8r+gV0RrUcpCe7PikmlZpanS5AACgjNt7aq+mr5+uGfEzdPr8aUmSp5un7mp6l0a0H6FOYZ1ksVgMrhIomwiwAAClonpgdf23+3/1YpcX9eW2L/XOmne05dgWTd8wXdM3TFdk3UiNaD9CtzS8heGFAACg1Fhzrfp+9/eavmG6luxbUrC9VvlaeqztYxp63VBV9a9qYIUAJAIsAEAp8/X01cNtHtaQ1kO04uAKvbPmHX2/+3vF7otV7L5YVQ+orqFthuqR6x5RWPkwo8sFAAAuKvGvRH2y8RPNiJ+hYxnHJEkWWdSnQR8Nbzdcfer3kbubu8FVAshHgAUAMITFYlHX2l3VtXZX7T+9Xx+s/0Az4mfoyNkjmrBygl75/RX1rt9bw64bplsa3CJPd0+jSwYAAE4uOzdb3+36Th9t/KhQb6tq/tX0cJuH9WjbR1U7qLZxBQK4KAIsAIDh6lSoo9cjX9fEbhM1f+d8fbjhQ604uEI/7/1ZP+/9WcHlgjWk9RA9ct0jqluhrtHlAgAAJ7P31F59vPFjzYyfqROZJyTZe1v1qt9Lw64bpn4N+/FlGWByBFgAANPw9vDWgBYDNKDFAO05tUefbPxEM+NnKiU9RZPiJmlS3CR1r9Ndw64bpv6N+8vbw9vokgEAgEmlZ6dr3o55+mzTZ/o96feC7SHlQjS0zVANvW4ova0AJ0KABQAwpYaVGur1yNf1yk2v6IfdP+jjjR9rceJiLd2/VEv3L1VF34oa2HyghrQZojbBbbgiEAAAkM1m06pDqzQjfobmbp+r9Ox0SZKbxU296vXSY20f44IxgJPitxYAYGpe7l66s+mdurPpnTpw5oA+2/SZPtv0mY6cPaKp66Zq6rqpalmtpYa0HqL7W9yvKv5VjC4ZAMqenBwpJkaKi5PCwyWLRVq1SoqIkKKiJA8+dqBkJZ9N1uebP9dn8Z9pz6k9BdvrV6yvIa2HaFCrQaoRWMPACgFcK/6SAACcRu2g2prQbYJevvFlLdm3RDPiZ2jhroXacmyLnvn1GT0f+7z6NuyrIa2HqE+DPny7CgClJSZGio6WbDYpNvbv7Uv+N0n2uHGGlAXXlp2brR/3/KjPNn2mRQmLlGfLkyT5e/rr7mZ36+HWDyuiZgS9tAEXwTt7AIDTcXdzV6/6vdSrfi+dPndaX277UjPiZ2h98not2LVAC3YtUDX/anqw5YMa0maImlZpanTJAODa4uLs4dW/2Wz2fUAx2npsqz7b9Jlmb52tk5knC7ZH1IzQkNZDdHfTuxXgHWBghQBKAgEWAMCpVfCtoCfaP6En2j+hrce2amb8TH2+5XMdyzimN1e/qTdXv6kO1TtoSOshuq/5fQryCTK6ZABwPRERhXte5bNY7PuAa3Qq85Tmbp9b8IVVvpByIRrcarCGtBmihpUaGlghgJJGgAUAcBktqrXQW73e0qs9XtVPe3/SjPgZ+mnPT1p7ZK3WHlmrUb+MUr+G/fRAywfUp34frmIIAMUlKkpavlxatuzvbXXrSoMH2/cBDsjKydLPe3/W51s+1497fpQ1zypJ8nTz1K2NbtXDbR5Wz3o9mTIAKCP4TQcAuBxPd0/1b9xf/Rv317H0Y5q9ZbZmbp6pbce36dud3+rbnd+qgk8F3dPsHt3f4n51rtlZbhY3o8sGgJLxzwnWS2pSdQ+PC89Zrx5zX+Gq2Ww2rTmyRp9v/lxfbf9Kf537q2Bfm+A2GtRqkB5o+YAq+1U2sEoARiDAAgC4tGrlqunZTs9qdPhobTm2RbO3zNacbXOUfDZZH274UB9u+FC1ytfS/S3u1wMtH1CTKk2MLhkAitc/J1gvyUnVIyLs57fZGDqIq7b75G59sfULzdk6R4mnEwu2hwaE6oEWD+jBVg+qedXmBlYIwGgEWACAMsFisahVcCu1Cm6lV3u8quUHlmv21tn6dse3Oph6UDFxMYqJi9F1IdfpgRYP6L7m9ykkIMTosgHg2v1zgvWSnFQ9f6jgP3t6AZdw9OxRfb39a83eOrvQvFb+nv66vcntGtRykG6qc5Pc3dwNrBKAWRBgAQDKHHc3d3Wv213d63bX+ze/rx/2/KDZW2ZrUcIibTy6URuPbtR/Yv+jHnV76IEWD6h/4/5czQhA6SnuIX+l1TPKw4Mhg7isw2mH9e2ObzVv5zz9kfSHbLKHq+4W+xWG729xv25rdJv8vfwNrhSA2RBgAQDKNF9PX93T7B7d0+wencw8af8meMtsrT68WosTF2tx4mL5ePjo5gY3695m96pvw77y8/QzumwArqy4h/zRMwoGO3jmoL7d+a3m7Zin1YdXF9p3fY3rNbD5QN3b/F5V9a9qUIUAnAEBFgAA/1PZr7KeaP+Enmj/hBL/StQXW7/QF1u/0J5TezR/53zN3zlf/p7+6teon+5tdq961+8tHw8fo8sG4GqKe8gfPaMcVxoT4Luofaf3FfS0WntkbcF2iyzqXLOz7mpyl+5ocofCyocZWCUAZ8KrLwAARahXsZ7G3ThOL3V5SfEp8Zq7fa7mbp+rA2cO6KttX+mrbV8p0DtQtzW6Tfc1v0896vaQl7uX0WUDcAVMhm4epTUBvotI+CtB83bM0zc7vtHGoxsLtltkUZdaXXRXU3toFRoQamCVAJwVAZYLmzRpkubPn69du3bJ19dXnTp10muvvaZGjRpd9D4zZ87UkCFDCm3z9vbW+fPnS7pcADAli8WiNiFt1CakjSZ1n6R1yes0d9tcfb3jax1OO6zPt3yuz7d8rgo+FXRHkzt0V9O71K12N7nJzejSATgrhvyZR2lNgO/Edp3cpXk75mnejnnafGxzwXY3i5u61u6qu5rcpdub3K7gcsEGVgnAFRBgubAVK1ZoxIgRat++vXJychQVFaWePXtqx44d8ve/+KSIgYGB2r17d8G6xWIpjXIBwPQsFos6VO+gDtU76I2eb2jVoVWau22uvtnxjY5lHNOnmz7Vp5s+VaB3oHrX662aGTUVkRWhSp6VjC4dgDNhyJ950BvuAjabTTtO7CjoabX9xPaCfe4Wd91U5ybd3fRu9W/cX1X8qxhYKQBXQ4Dlwn755ZdC6zNnzlTVqlW1YcMGdenS5aL3s1gsCg7mGxLAKTFXR6lxs7gpomaEImpGaErvKVp5cKW+3v61vtv9nY6mH9XXO76WJL075V11r9Ndtze+Xbc2ulXVylUzuHIAwBWjN5wke2i19fhWfbP9G83bOU+7Tu4q2Ofp5qkedXvorqZ36bZGt6mSH1/aACgZfKopQ1JTUyVJFStWvORx6enpqlWrlvLy8nTdddcpJiZGzZo1u+jxWVlZysrKKlhPS0uTJFmtVlmt1mKo/Nrl12GWeuA8nK7tvPaa/cdmk/74Q3Jzk8aMMbqqMiGiRoQiakRoSs8pWpe8TvN3ztfczXOVnJWsRQmLtChhkR778TGF1wjXrQ1v1W2NblO9CvWMLhsm5HSvOzAN2k4JGTv279s2m+SC/75FtR2bzab4Y/H6dte3mr9rvhL+SijY5+XupR51eujOJneqb/2+quBb4YJzoWww0+uOGWpAybLYbPmDuuHK8vLydOutt+rMmTOKu8TY/dWrV2vv3r1q2bKlUlNT9eabb2rlypXavn27atSoUeR9oqOjNX78+Au2z5kzR35+XGoeQNlls9l0OOuw/kz9U2vOrFHCuYRC+2v61NT15a9Xx/IdVde3LkO2AQCGstlsSjiXoFVnVmnVmVU6ln2sYJ+nxVPXBV6nTuU7qV35dvJ3v/iUJIARMjMzNXDgQKWmpiowMNDoclACCLDKiOHDh2vRokWKi4u7aBBVFKvVqiZNmmjAgAGaOHFikccU1QMrLCxMJ0+eNM0Lh9VqVWxsrCIjI+Xp6Wl0OXAiTtd2XntNmjTp77k6xo6lB5ZBimo7h9IO6Yc9P+j7Pd9rxcEVyrXlFhxfq3wt3drwVt3a8FZ1DussDzc6SZdVTve6A9Og7cARebY8rU5arSmLp2hT1iYlpSUV7PP18FXver11R+M7dHP9mxXgHWBgpTAjM73upKWlqXLlygRYLox3x2XAyJEj9eOPP2rlypVXFV5Jkqenp9q0aaOEhISLHuPt7S1vb+8i72v0i9i/mbEmOAenaTsvvCDl5f09V8cLLzAHlsH+2XbqVqqrUeGjNCp8lP4695d+2vOTFuxaoF8SftHB1IN6b917em/de6rkW0n9GvXT7Y1vV2TdSPl6+hr8LGAEp3ndgenQdnA52bnZikuK0/e7v9e3O7/V4bTDBfv8Pf3Vt2Ff3dX0LvWp30f+XvS0wuWZ4XXH6MdHyeNTjQuz2Wx68skntWDBAi1fvlx16tS56nPk5uZq69atuvnmm0ugQgDFjitXOY2KvhX1YKsH9WCrB5VpzVRsYqwW7FqgH/b8oFPnTmlm/EzNjJ8pP08/9a7fW/0b9VffhoXnGQFQzLgQBlzYiYwTWpSwSD/u+VG/Jv6qtKy0gn0BXgFq499GI28aqVsa3SI/T6YBAWA+/EV2YSNGjNCcOXP03XffKSAgQCkpKZKk8uXLy9fX/m3+oEGDVL16dU2aNEmSNGHCBF1//fWqX7++zpw5ozfeeEMHDx7UI488YtjzAABX5+fpp9sa36bbGt+mnLwc/X7wdy3ctVALdi3QobRDmr9zvubvnC93i7u61OqiWxrcopsb3KzGlRszbxZQnGJipOho+zDsJUvs2/hSAE4q/8qBP+75UT/u+VF/Hv5TNv09e0xV/6q6ucHNur3x7epWs5t+W/ybbm50M71YAJgWAZYL++CDDyRJXbt2LbR9xowZeuihhyRJSUlJcnNzK9h3+vRpDRs2TCkpKapQoYLatm2rVatWqWnTpqVVNgCUaR5uHupWp5u61emmKb2naFPKJi3YuUALdy/UtuPbtOzAMi07sEz/if2P6laoq5vr36xbGt6irrW7ysfDx+jyAecWF2cPryT78hIXvgHMKD07Xcv2LyvoaXUo7VCh/a2DW6tfw37q27Cv2oW2k5vF/jmAq7cBcAYEWC7sSubnX758eaH1yZMna/LkySVUEQBcAYbwFLBYLLou5DpdF3KdJt40UQl/JeinPT/p54SftfzAcu07vU9T103V1HVT5evhq251uqlP/T7qU7+P6lWsZ3T5gPOJiLD3vMq/EEZEhNEVAZdks9m05dgW/ZLwi35N/FVxSXGy5v0dRvl4+KhH3R7q26Cvbml4i2oEXt18uABgJmXzEwEAwLwuN4SnDAdc9SvW16jrR2nU9aOUnp2upfuW6qe9P+nnvT/ryNkj+nnvz/p578+SpAYVG6hnvZ7qWa+nutXuxpWjgCsRFWVf/vP1BTCZk5knFZsYq18Tf9Wvib8qJT2l0P46QXXUq14v9W3YV93qdGM+KwAuo2y84wcAOI/LDeFhjhpJUjmvcgXzZuXPc7Jo7yItSlikPw79ob1/7dXev/Zq2rpp8nDzUHiN8IJAq21IW7m7uRv9FADz4UIYMKGcvBytObymoJfV+uT1heay8vP0U7fa3dS7fm/1qtdL9SvWZ35EAC6JAAsAYC6XG8LDHDUXsFgsalmtpVpWa6kxEWOUlpWmZfuXaXHiYi3et1gJfyXo96Tf9XvS73pp2Uuq4FNB3et2V8+69kCrVlAto58CAOAf9p3ep6X7lurXxF+1ZN8SpWalFtrfslpL9arXS73r91bnsM7y9vA2qFIAKD0EWAAAc7ncEB7mqLmsQO/Agt5Zkv2DUGxirGL3xWrJviU6ff605u2Yp3k75kmyD028qfZN9snja3dTtXLVjCwfAMqcExkntOzAMi3Zt0RL9i3R/jP7C+2v6FtRPev1VK96vdSzXk+FBoQaVCkAGIcACwBgLpcbwsMcNVetboW6eqzdY3qs3WPKycvR+uT19t5ZiYv15+E/lfBXghL+StBHGz+SJDWt0lQ31b5JN9W5STfWvlEVfSsa/AwAwLWknk/VyoMr9dv+3/Tbgd+05diWQvs93DzUsXpH9azXU73r92boNwCIAAsA4GyYo+aaeLh56Poa1+v6Gtdr3I3jlHo+Vb8n/a7f9v+mZQeWKT4lXjtO7NCOEzs0dd1UWWRR6+DW6la7m26qc5NuqHWDAr0DjX4aAOBU0rLS9EfSH1p2YJmWHVimjUc3Ks+WV+iYFlVbqEfdHupep7u61OrCxTcA4F8IsAAAKMPK+5RX34Z91bdhX0n2q1utOLBCyw4s02/7f9POkzu1KWWTNqVs0tt/vi13i7vahbZTl1pdFFEzQp3COqmyX2WDnwVgMmX4aqmwO33utOKS4rTy4EqtOLhCG45uuCCwql+xvrrX6a6b6tykrrW7qqp/VYOqBQDnwF9SAABQoLJfZd3Z9E7d2fROSdLRs0e1/MDygkAr8XSi1hxZozVH1uiNVW9IkppUbqKImhEFP3WC6nAFLJRtRl8t9bXXpJUrCc9KUfLZZMUlxen3g/YLZmw5tqXQlQIlqV6Ferqx1o3qVqebbqx1o8LKhxlULQA4J/6aAQCAiwoJCNGAFgM0oMUASVJSapKWH1iuuKQ4xSXFaefJnQU/H2/82H6fciGFAq2W1VrKw423HChDjL5a6qRJUmamMeFZGZBny9Puk7vtr4OH7KHVvyddl6TGlRurS80u6lKri26sfaNqBNYwoFoAcB28mwQAAFesZvmaGtRqkAa1GiTJPuRw1aFVBYHW+uT1Opp+VN/s+Ebf7PhGklTOq5zCa4Src1hnRdSMUMcaHVXOq5yRTwNGKStD64y+WqqR4ZkLysjO0LrkdVp1aJVWHVql1YdX669zfxU6xs3iptbBrRURZg/uu9TqwhVdAaCYueA7BgAAUFoq+1XWrY1u1a2NbpUknbOe0/rk9QU9E/5I+kOpWamK3Rer2H2xkiR3i7taB7dWx+od1aF6B7Wv3l6NKzeWm8XNyKeC0mD00LrSYvTVUvOH8BoRnjm5nLwcbT++XWuPrLX/JK/V9uPblWvLLXScr4evOtboqBtq3qCImhG6vsb1XOACAEoYARYAACg2vp6+uqHWDbqh1g2S7ENtth/fXhBoxSXFKSk1SRuObtCGoxv0/vr3JUkBXgFqF9pO7UPbq0P1DupQvYNqBNZgLi1XUxpD68zQy8voq6WOHVt4DiwUyWazaf+Z/X+HVUfWauPRjTqXc+6CY6sHVFfnmp3VqUYnda7ZWa2qtZKnu6cBVQNA2UWABQAASoybxU0tqrVQi2otNLz9cEnSodRDWnVoldYlr9O65HVan7xeZ7PPFlxePl81/2oFYVb70PZqX729KvpWNOqpuL7SCH5KY2hdWenldSljxkgvvnhlx5oh8CslxzOOa92RdQU9q9YdWadT505dcFygd2BBmJ6/rB5Y3YCKAQD/5Jp/nQAAgGmFlQ/TveXv1b3N75VkH7Kz88ROrUteV9ALYuvxrTqWcUw/7PlBP+z5oeC+9SrUKwi12oW2U6tqrRTgHWDUU3EtpRH8lMbQutKcQN0Vwh8XDfxSz6cqPiW+0OvKwdSDFxzn5e6l1sGt1SG0Q8GQ5oaVGjKkGQBMyMn+wgIAAFfj4eZR0Evr4TYPS7LPpRWfEq+1R9YWfADd+9deJZ5OVOLpRH257UtJkkUWNajUQG2C29h/QtqoVbVWxkye7OxhRmkEP6UxtK40J1B3hfDH6CsmFoOU9BRtOrpJm1LsPxuPbtS+0/suOM4iixpXblyoZ2fLai3l7eFtQNUAgKvlRO+qAABAWeHr6avwsHCFh4UXbDt97rTWJ68vGP6zIXmDjpw9oj2n9mjPqT2au31uwbHV/KupVXArtar2v5/gVmpUqVHJzlnj7GGG0VfOKy6lOYG6C4Q/zvT/fub8GW0/vl3bjm/T9hP25bbj23Qi80SRx9csX1PtQtupQ6i9Z1XbkLYq71O+lKsGABQXAiwAAFyJs/cCuoQKvhUUWS9SkfUiC7YdSz+m+JT4gp4X8Snx2ntqr45lHNPixMVanLi44Fgvdy81rdJUzao0U9MqTQt+6lWoJ3c392sv0NnDDKOvnFdcSnMCdScKfy7KhP/vGdkZ2nFiR6GQavuJ7TqcdrjI4y2yqFHlRoV6YrYJbqNKfpVKuXIAQElyjXe0AAAY7d/B0XPPGVOHs/cCukrVylVTr/q91Kt+r4JtGdkZ2nZ8mzYf26zNKZu1+dhmbTm2RWezzyo+JV7xKfGFzuHt7q1GlRvZA63K9lCrWdVmqleh3tX12HL2MMPoK+cZ4VoDXxOGP1fNwP/3rJws7T61u1BIte34Nu0/vV822Yq8T43AGmpetbmaV2muZlWbqXnV5mpSuYn8vfxLuXoAQGkjwAIAGM8Veg39Ozhyc5NatCj9Opy9F1Ax8PfyV8caHdWxRseCbXnjo3XgnfHaUk3aWUXa0b2ldoR4aOeJnTqXc05bjm3RlmNbCp3H081TDSs1LNRbq1mVZmpQqYG83L0ufGBXCDPKmmsNfMti6OeArJwsJfyVoJ0nd9qHAJ6wB1Z7T+1Vri23yPtU9a96QVDVtEpTBfkElW7xAADTcLJPBwAAl+QKvYb+HRytXm1MgOXsvYBKiNsfq1T3tFT3tNR/lySfatLixcrNy9XB1IPacWLHBT8Z1gxtP7Fd209sL3Qud4u76lSoo4aVGqphxYZqWKGeGv6yXg3WJapG++5y+/ln5wtgy6rLBb6uEK6Xkpy8HB1KPaSEvxK059Qe7f1rr/ac2qPdp3brwJkDyrPlFXm/IJ+gC4KqZlWaqYp/lVJ+BgAAs+MvMADAeK7Qa+jfwVF4+OXvUxLoBVS0iwR77m7uqluhrupWqKu+DfsWHJ5ny9Oh1EOFQ62T9mVaVpoS/kpQwl8J+lk///0Y7SXvnDjV/u801anfTnWC6qhOUB3VDqqtOhXstyv6VpTFYintZ4+LuVzg6wrhejGx2Ww6kXlC+0/v177T+7T/zN/L/af3Kyk16aK9qSQp0DtQjSs3LhRUNa/aXCHlQvidAABcEQIsAIDxXKHX0L+Do2eflRYvvvR9SgJDmop2lcGem8VNtYJqqVZQLfVp0Kdgu81mU/LZ5ILeJXtO7dGen2ZpT95J7asgZXlIu3VSuxN+KfK8AV4BBYFW7fJ/B1t1KtiDrkDvwL8Pfu01aeVKev6UpMu1C1cI169CRnbG38HUP4Kq/JAqw5pxyft7uXupboW6Bb0TG1RqoEaVGqlR5Uaq5l+NoAoAcE14JwQAMJ4r9Br6d3BktRpXCy5UTMGexWJR9cDqqh5YXV1rd7Vv/DNQio5WjsWmpPLS/lGDtb9fhPaftn/wP3DmgPaf2a+U9BSdzT6rrce3auvxrUWev6JvRdUuX1ve57y17M91qnU2RyHfxCrEdkDBI8YoJCBEAV4BBAHF5XLtwhXC9f+x5lqVfDZZh9MO61DaIfsy9VDB7YOpB3U84/glz2GRRaEBoapboa7qVKijukF1C0LYuhXqKiQgRG4Wt1J6RgCAsoYACwBgvJLuNcQ8NihJ/wtcPeLiVDciQnUv0r7OWc/pYOrBC4Kt/aftt0+dO6W/zv2lv879JUla3f6f954hTZshSfLz9FNwuWCFlAspWIYEhBTeFhCiKn5V5O7mXtLP3rWZPFy32WxKzUrV8YzjOpZ+TMlpyfrt5G9av3K9jmceV3J6so6kHVHy2WQdzzh+0Sv7/VOQT5A9oAr6O5iqU8G+rFW+lrw9vEvhmQEAcCHevQMAXB/z2BSNYK94XGEA6+vpq8aVG6tx5cZF7k/LStOBMweUcDJBP6/+WYELZinZL0cp5aSjtSspxduqtKw0ZVozte/0Pu07ve+Sj+dmcVNV/6pFBl3V/Kupkl8lVfarrEq+lVTJr5J8PHwcevourZSH5NpsNqVnp+v0+dM6fe60jmcct4dTGccKlsfSjxXalp2bfeGJDhd9fk83T1UPrK6wwDCFlQ9TjYAa9mVgDdUsX1N1guqogm+Fkn2SAAA4iHepAADXV8bmsbliBHumEugdqJbVWqpJxSZyT3DXzeG15blypdTi73AxIztDKekpSklP0dH0o/bl2aN/3/7f8njGceXZ8gqOvRJ+nn6q6FtRFX0rKtA7sOAnwCvgousB3hfuK8u9vvJseUrPTtfZrLP2ZfZZnT53WqfPn9aZ82cK3/5fSPXv25eaCP1iArwCVK1cNVXxqyLbWZta1G2h0MBQVQ+ortCAUIUGhCokIERV/asyxA8A4LQIsAAArs+F5rEpVgR75jZmjPTii4U2+Xv5q17FeqpXsd4l75qTl6MTGSeKDLqOph/VycyTBT+nMk8p15arTGumMq2ZOpx2ke47V8jP0++CoKucVzn5efrJ19NXvh72H28Pb3m7e8vT3VMebh7ydPvf8n/rl9uWv92iC+cDs8mmPFuecvNylWvLVW5ernLycgpu/3uZnZut7NxsWfOsys7NVlZOls7nnNf5nPPKys3SOes5ncs5V/Bv9M/bGdkZBWFVpjXzmv7t8nm6eaqCbwVV9a+qqv5VVc2/WuFluWoFt6v6V5Wvp68kyWq16ueff9bNfW6Wp6dnsdQCAIBZEGABAFyfyeexMQzBnsvycPNQSIB9yGAbtbnksXm2PKVlpemvc3/pVOYpnT5/WmezziotK01pWWk6m/337aLW07LSdDbrrLJysySpINhJ0ZX1/HJFbhY3BXgFKMA7QEE+QargU0EVfCv8fdvnf7d9K1y4z7eCfD18magfAIB/IcACALg+R+excfU5ogj2IHvYEuQTVDB5t6OycrIKwq1/B2Bns87qXM65gp5M56znlJVr7+WUk5dT8GPNs9qXudYLtv1zuzXPqpzcHFn/OiGdPy/5+EhBQRc8L3c3d7lb3OXu5i4PN4+C2/9eerl7Ffx4unnK28NbPm5e8t64Rb6HU+Rdo7b8buolP+9y8vXwlZ+nn/zcvOU7b6H8t+xSuZbtFPDE0yrnF6QArwD5ePgQQAEAUMxc6F04AADFzNXniCrlCarh2rw9vOXt4a3KfpVL5wEnTJDGR/+vB+F5Kfrp4m3PEyZI4z/+3/kPSpZu0riR/9o/275/wXrJEszvEwAAJYhZHAEAuBjmiAKuTU6OPejp2dO+zMkpvnOX9O/n5c7P6wMAAKWKAAsAgIuJiLDPDSUxRxTgiPxejLGx9mVMTPGdu6R/Py93fl4fAAAoVQwhBADgYpgjCrg2JdlLqaR/Py93fl4fAAAoVQRYZcC0adP0xhtvKCUlRa1atdJ7772nDh06XPT4b775Ri+99JIOHDigBg0a6LXXXtPNN99cihUDgEkwR9SFXH1iexSvkrzSZUn/fl7u/Lw+AABQqhhC6OLmzp2r0aNH6+WXX9bGjRvVqlUr9erVS8ePHy/y+FWrVmnAgAEaOnSoNm3apP79+6t///7atm1bKVcOADClkhwShtJXknNUSfaAMzpaioy0L+mlBAAAHESA5eLefvttDRs2TEOGDFHTpk01ffp0+fn56bPPPivy+HfeeUe9e/fWc889pyZNmmjixIm67rrrNHXq1FKuHABw1Uo6jJCYuNrVlHQgmd9LafFi+5LeegAAwEG8i3Bh2dnZ2rBhg8aOHVuwzc3NTT169NDq1auLvM/q1as1evToQtt69eqlhQsXXvRxsrKylJWVVbCelpYmSbJarbJardfwDIpPfh1mqQfOg7YDRxnSdl57zf5js0l//CG5uUljxhTvY3TpYj93/pCwLl0kI34/cnKkt96SVq+WwsOlZ591mXCkVNvOmjWSj0/hdV7vnBZ/s+Ao2g4cZaa2Y4YaULIsNlv+16hwNcnJyapevbpWrVql8PDwgu3PP/+8VqxYoTVr1lxwHy8vL82aNUsDBgwo2Pb+++9r/PjxOnbsWJGPEx0drfHjx1+wfc6cOfLz8yuGZwIAAAAAwMVlZmZq4MCBSk1NVWBgoNHloAS4xleVMNTYsWML9dpKS0tTWFiYevbsaZoXDqvVqtjYWEVGRsrT09PocuBEaDtwlCFt57XXpEmT/u4dNXZs8ffAMov+/aVly/5e79ZNukRvYWdS0HYeflie587ZN44ZUzLzR7lwT7ayiL9ZcBRtB44yU9vJHwkE18U7FBdWuXJlubu7X9Bz6tixYwoODi7yPsHBwVd1vCR5e3vL29v7gu2enp6Gv4j9mxlrgnOg7cBRpdp2XnhBysv7+wqBL7zgumFEx47SokV/h3UdO0ou9jvqee7c3wHW//2f9PLLJfAgntKLL/59dcnbbuPqki6Av1lwFG0HjjJD2zH68VHyeGfiwry8vNS2bVstXbpU/fv3lyTl5eVp6dKlGjlyZJH3CQ8P19KlS/X0008XbIuNjS00BBEAYFL5E2aXBfm9kfLDOq5ud23yJ3O32aQlS+zbykpbAgAAToEAy8WNHj1agwcPVrt27dShQwdNmTJFGRkZGjJkiCRp0KBBql69uiZNmiRJGjVqlG688Ua99dZbuuWWW/TVV19p/fr1+uijj4x8GgAAFFaWwjpJevDBkj1/aV9dMr/H1z8DSHp8AQCAS+Cdgou79957deLECY0bN04pKSlq3bq1fvnlF1WrVk2SlJSUJDc3t4LjO3XqpDlz5ujFF19UVFSUGjRooIULF6p58+ZGPQUAAMqmqChp5crS6WEWEWHveZU/JDMiomQfb+JEacIE++3YWCk3VyrigjAAAAD5CLDKgJEjR150yODy5csv2Hb33Xfr7rvvLuGqAADAJY0ZY5+fqjhdrOdTaQ/JnD37wnUCLAAAcAkEWAAAAGXFxea6KmtDMgEAgNNxu/whAAAAJpKTYx9+1rOnfZmTY3RFzqO057q6mH/P6VXSc3wBAACnRw8sAADgXLhinuNKe66ri3nxRcnNjatIAgCAK0aABQAAnItZehE5o9Ke6+piGLIIAACuEgEWAADO7GKTcrsyo3oRucK/NcERAABwUk72rgsAABRSFofTGdWLqCz+WwMAAJgEARYAAM6sLA6nM6oXUVn8twYAADAJrkIIAIAzi4iwD6OTjJ2Uuyzg3xoAAMAw9MACAMCZmWVS7rKAf2sAAADDEGABAODMmJS79PBvDQAAYBiGEAIA4AxycqQJE6SePe3LnByjKwIAAABKDT2wAABwBlwBr+zIDyf795c6drQPVfTgLRsAACjbeDcEAIAz4Ap4Zcdbb0ktWkjLlkmLFtm3EVYCAIAyjiGEAAA4A66AV3asXv33bWcPKxn6CgAAigk9sAAAcAZcAa/sCA//+7azh5UMfQUAAMWEAAsAAGfAFfDKjmeflRYvlrp1+3sOLGfF0FcAAFBMCLAAAADMJH/C9oULJU9PQ0u5ZhER9p5XNpvz9yYDAACGIsACAABAyWDoKwAAKCYEWAAAwPnl5NjnW/pnUOLB2xzDMfQVAAAUE97ZAQAA58dk4QAAAC7NzegCAAAArhmThQMAALg0AiwAAFB6cnKkCROknj3ty5yc4jlvRIR9knCJycIBAABcEEMIAQBA6SmpoX5MFg4AAODSCLAAAEDpKamhfkwWDgAA4NIYQggAAEoPQ/0AAADgAHpgAQCA0sNQPwAAADiAAAsAAJQehvoBAADAAQwhBAAAAAAAgKkRYAEAAAAAAMDUCLAAAIBzysmRJkyQeva0L3NyjK4IAAAAJYQ5sAAAgHOKiZGioyWbTVqyxL6N+bUAAABcEj2wAACAc4qLs4dXkn0ZF2dsPQAAACgxBFgAAMA5RURIFov9tsViXwcAAIBLIsByUQcOHNDQoUNVp04d+fr6ql69enr55ZeVnZ19yft17dpVFoul0M/jjz9eSlUDAHAVoqLsQwgjI+3LqCijKwIAAEAJYQ4sF7Vr1y7l5eXpww8/VP369bVt2zYNGzZMGRkZevPNNy9532HDhmnChAkF635+fiVdLgAAV8/DgzmvAAAAyggCLBfVu3dv9e7du2C9bt262r17tz744IPLBlh+fn4KDg4u6RIBAAAAAACuCAFWGZKamqqKFSte9rgvvvhCs2fPVnBwsPr166eXXnrpkr2wsrKylJWVVbCelpYmSbJarbJarddeeDHIr8Ms9cB50HbgKNoOHEXbgaNoO3AUbQeOMlPbMUMNKFkWmy3/8j1wZQkJCWrbtq3efPNNDRs27KLHffTRR6pVq5ZCQ0O1ZcsWjRkzRh06dND8+fMvep/o6GiNHz/+gu1z5sxh+CEAAAAAoMRlZmZq4MCBSk1NVWBgoNHloAQQYDmZF154Qa+99tolj9m5c6caN25csH7kyBHdeOON6tq1qz755JOrerzffvtN3bt3V0JCgurVq1fkMUX1wAoLC9PJkydN88JhtVoVGxuryMhIeXp6Gl0OnAhtB46i7cBRtB04irYDR9F24CgztZ20tDRVrlyZAMuFMYTQyTz77LN66KGHLnlM3bp1C24nJyerW7du6tSpkz766KOrfryOHTtK0iUDLG9vb3l7e1+w3dPT0/AXsX8zY01wDrQdOIq2A0fRduAo2g4cRduBo8zQdox+fJQ8AiwnU6VKFVWpUuWKjj1y5Ii6deumtm3basaMGXJzc7vqx4uPj5ckhYSEXPV9AQAAAAAAisPVJxpwCkeOHFHXrl1Vs2ZNvfnmmzpx4oRSUlKUkpJS6JjGjRtr7dq1kqTExERNnDhRGzZs0IEDB/T9999r0KBB6tKli1q2bGnUUwEAAAAAAGUcPbBcVGxsrBISEpSQkKAaNWoU2pc/7ZnVatXu3buVmZkpSfLy8tKSJUs0ZcoUZWRkKCwsTHfeeadefPHFUq8fAAAAAAAgHwGWi3rooYcuO1dW7dq19c85/MPCwrRixYoSrgwAAAAAAODqMIQQAAAAAAAApkaABQAAAAAAAFMjwAIAAAAAAICpMQcWil3+vFppaWkGV/I3q9WqzMxMpaWlydPT0+hy4ERoO3AUbQeOou3AUbQdOIq2A0eZqe3kf/785zzPcC0EWCh2Z8+elWSfFB4AAAAAgNJy9uxZlS9f3ugyUAIsNuJJFLO8vDwlJycrICBAFovF6HIk2dP4sLAwHTp0SIGBgUaXAydC24GjaDtwFG0HjqLtwFG0HTjKTG3HZrPp7NmzCg0NlZsbsyW5Inpgodi5ubmpRo0aRpdRpMDAQMNfWOGcaDtwFG0HjqLtwFG0HTiKtgNHmaXt0PPKtRFLAgAAAAAAwNQIsAAAAAAAAGBqBFgoE7y9vfXyyy/L29vb6FLgZGg7cBRtB46i7cBRtB04irYDR9F2UJqYxB0AAAAAAACmRg8sAAAAAAAAmBoBFgAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAUyPAQplz6623qmbNmvLx8VFISIgefPBBJScnG10WTO7AgQMaOnSo6tSpI19fX9WrV08vv/yysrOzjS4NTuC///2vOnXqJD8/PwUFBRldDkxs2rRpql27tnx8fNSxY0etXbvW6JLgBFauXKl+/fopNDRUFotFCxcuNLokOIFJkyapffv2CggIUNWqVdW/f3/t3r3b6LLgBD744AO1bNlSgYGBCgwMVHh4uBYtWmR0WSgDCLBQ5nTr1k1ff/21du/erW+//VaJiYm66667jC4LJrdr1y7l5eXpww8/1Pbt2zV58mRNnz5dUVFRRpcGJ5Cdna27775bw4cPN7oUmNjcuXM1evRovfzyy9q4caNatWqlXr166fjx40aXBpPLyMhQq1atNG3aNKNLgRNZsWKFRowYoT///FOxsbGyWq3q2bOnMjIyjC4NJlejRg29+uqr2rBhg9avX6+bbrpJt912m7Zv3250aXBxFpvNZjO6CMBI33//vfr376+srCx5enoaXQ6cyBtvvKEPPvhA+/btM7oUOImZM2fq6aef1pkzZ4wuBSbUsWNHtW/fXlOnTpUk5eXlKSwsTE8++aReeOEFg6uDs7BYLFqwYIH69+9vdClwMidOnFDVqlW1YsUKdenSxehy4GQqVqyoN954Q0OHDjW6FLgwemChTPvrr7/0xRdfqFOnToRXuGqpqamqWLGi0WUAcAHZ2dnasGGDevToUbDNzc1NPXr00OrVqw2sDEBZkZqaKkm8t8FVyc3N1VdffaWMjAyFh4cbXQ5cHAEWyqQxY8bI399flSpVUlJSkr777jujS4KTSUhI0HvvvafHHnvM6FIAuICTJ08qNzdX1apVK7S9WrVqSklJMagqAGVFXl6enn76aXXu3FnNmzc3uhw4ga1bt6pcuXLy9vbW448/rgULFqhp06ZGlwUXR4AFl/DCCy/IYrFc8mfXrl0Fxz/33HPatGmTFi9eLHd3dw0aNEiMpi2brrbtSNKRI0fUu3dv3X333Ro2bJhBlcNojrQdAADMaMSIEdq2bZu++uoro0uBk2jUqJHi4+O1Zs0aDR8+XIMHD9aOHTuMLgsujjmw4BJOnDihU6dOXfKYunXrysvL64Lthw8fVlhYmFatWkW31zLoattOcnKyunbtquuvv14zZ86UmxvfA5RVjrzuMAcWLiY7O1t+fn6aN29eobmLBg8erDNnztBTGFeMObBwtUaOHKnvvvtOK1euVJ06dYwuB06qR48eqlevnj788EOjS4EL8zC6AKA4VKlSRVWqVHHovnl5eZKkrKys4iwJTuJq2s6RI0fUrVs3tW3bVjNmzCC8KuOu5XUH+DcvLy+1bdtWS5cuLQge8vLytHTpUo0cOdLY4gC4JJvNpieffFILFizQ8uXLCa9wTfLy8vg8hRJHgIUyZc2aNVq3bp0iIiJUoUIFJSYm6qWXXlK9evXofYVLOnLkiLp27apatWrpzTff1IkTJwr2BQcHG1gZnEFSUpL++usvJSUlKTc3V/Hx8ZKk+vXrq1y5csYWB9MYPXq0Bg8erHbt2qlDhw6aMmWKMjIyNGTIEKNLg8mlp6crISGhYH3//v2Kj49XxYoVVbNmTQMrg5mNGDFCc+bM0XfffaeAgICC+fbKly8vX19fg6uDmY0dO1Z9+vRRzZo1dfbsWc2ZM0fLly/Xr7/+anRpcHEMIUSZsnXrVo0aNUqbN29WRkaGQkJC1Lt3b7344ouqXr260eXBxGbOnHnRD5G8jOJyHnroIc2aNeuC7cuWLVPXrl1LvyCY1tSpU/XGG28oJSVFrVu31rvvvquOHTsaXRZMbvny5erWrdsF2wcPHqyZM2eWfkFwChaLpcjtM2bM0EMPPVS6xcCpDB06VEuXLtXRo0dVvnx5tWzZUmPGjFFkZKTRpcHFEWABAAAAAADA1JjABQAAAAAAAKZGgAUAAAAAAABTI8ACAAAAAACAqRFgAQAAAAAAwNQIsAAAAAAAAGBqBFgAAAAAAAAwNQIsAAAAAAAAmBoBFgAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAUyPAAgAAMNCJEycUHBysmJiYgm2rVq2Sl5eXli5damBlAAAA5mGx2Ww2o4sAAAAoy37++Wf1799fq1atUqNGjdS6dWvddtttevvtt40uDQAAwBQIsAAAAExgxIgRWrJkidq1a6etW7dq3bp18vb2NrosAAAAUyDAAgAAMIFz586pefPmOnTokDZs2KAWLVoYXRIAAIBpMAcWAACACSQmJio5OVl5eXk6cOCA0eUAAACYCj2wAAAADJadna0OHTqodevWatSokaZMmaKtW7eqatWqRpcGAABgCgRYAAAABnvuuec0b948bd68WeXKldONN96o8uXL68cffzS6NAAAAFNgCCEAAICBli9frilTpujzzz9XYGCg3Nzc9Pnnn+v333/XBx98YHR5AAAApkAPLAAAAAAAAJgaPbAAAAAAAABgagRYAAAAAAAAMDUCLAAAAAAAAJgaARYAAAAAAABMjQALAAAAAAAApkaABQAAAAAAAFMjwAIAAAAAAICpEWABAAAAAADA1AiwAAAAAAAAYGoEWAAAAAAAADA1AiwAAAAAAACYGgEWAAAAAAAATI0ACwAAAAAAAKZGgAUAAAAAAABT+/8KYu/nUAr4BgAAAABJRU5ErkJggg==",
      "text/html": [
       "\n",
       "            <div style=\"display: inline-block;\">\n",
       "                <div class=\"jupyter-widgets widget-label\" style=\"text-align: center;\">\n",
       "                    Figure\n",
       "                </div>\n",
       "                <img src='data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABLAAAAH0CAYAAAAt0NQSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB4+klEQVR4nO3deVxUZf//8fewL4K4A4r7vptbKJmmuJSW7WmlmVmZlmV3mfzKULuxXSstW9VvZlmmtlqiuUSaO+4buKAibikIKAwwvz/mhiJxG4FzZng9Hw8eZ84yZz6jF8PMe67rOhabzWYTAAAAAAAAYFJuRhcAAAAAAAAAXAoBFgAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAUyPAAgAAAAAAgKkRYAEAAAAAAMDUCLAAAAAAAABgagRYAAAAAAAAMDUCLAAAAAAAAJgaARYAAAAAAABMjQALAAAAAAAApkaABQAAAAAAAFMjwAIAAAAAAICpEWABAAAAAADA1AiwAAAAAAAAYGoEWAAAAAAAADA1AiwAAAAAAACYGgEWAAAAAAAATI0ACwAAAAAAAKZGgAUAAAAAAABTI8ACAAAAAACAqRFgAQAAAAAAwNQIsAAAAAAAAGBqBFgAAAAAAAAwNQIsAAAAAAAAmBoBFgAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAUyPAAgAAAAAAgKkRYAEAAAAAAMDUCLAAAAAAAABgagRYAAAAAAAAMDUCLAAAAAAAAJgaARYAAAAAAABMjQALAAAAAAAApkaABQAAAAAAAFMjwAIAAAAAAICpEWABAAAAAADA1AiwAAAAAAAAYGoEWAAAAAAAADA1AiwAAAAAAACYGgEWAAAAAAAATI0ACwAAAAAAAKZGgAUAAAAAAABTI8ACAAAAAACAqRFgAQAAAAAAwNQIsAAAAAAAAGBqBFgAAAAAAAAwNQIsAAAAAAAAmBoBFgAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAUyPAAgAAAAAAgKkRYAEAAAAAAMDUCLAAAAAAAABgagRYAAAAAAAAMDUCLAAAAAAAAJgaARYAAAAAAABMzcPoAuB68vLylJycrICAAFksFqPLAQAAAAC4OJvNprNnzyo0NFRubvTVcUUEWCh2ycnJCgsLM7oMAAAAAEAZc+jQIdWoUcPoMlACCLBQ7AICAiTZXzgCAwMNrsbOarVq8eLF6tmzpzw9PY0uB06EtgNH0XbgKNoOHEXbgaNoO3CUmdpOWlqawsLCCj6PwvUQYKHY5Q8bDAwMNFWA5efnp8DAQMNfWOFcaDtwFG0HjqLtwFG0HTiKtgNHmbHtMI2N62JgKAAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAU2MOLBgmNzdXVqu1VB7LarXKw8ND58+fV25ubqk8Zknx9PSUu7u70WUAAAAAAFBqCLBQ6mw2m1JSUnTmzJlSfczg4GAdOnTIJSb1CwoKUnBwsEs8FwAAAAAALocAC6UuP7yqWrWq/Pz8SiWEycvLU3p6usqVKyc3N+cdOWuz2ZSZmanjx49LkkJCQgyuCAAAAACAkkeAhVKVm5tbEF5VqlSp1B43Ly9P2dnZ8vHxceoAS5J8fX0lScePH1fVqlUZTggAAAAAcHnO/UkeTid/zis/Pz+DK3Fu+f9+pTWHGAAAAAAARiLAgiGYu+na8O8HAAAAAChLCLAAAAAAAABgagRYQDGpXbu2pkyZYnQZAAAAAAC4HAIsAAAAAAAAmBoBFnCFzp49q/vvv1/+/v4KCQnR5MmT1bVrVz399NPq2rWrDh48qGeeeUYWi4U5qgAAAACYW06ONGGC1LOnfZmTY3RFwCURYAFXaPTo0frjjz/0/fffKzY2Vr///rs2btwoSZo/f75q1KihCRMm6OjRozp69KjB1QIAAADAJcTESNHRUmysfRkTY3RFwCURYDmxlStXql+/fgoNDZXFYtHChQsL7c/vCfTvnzfeeOOi54yOjr7g+MaNG5fwMzG/s2fPatasWXrzzTfVvXt3NW/eXDNmzFBubq4kqWLFinJ3d1dAQICCg4MVHBxscMUAAAAAcAlxcZLNZr9ts9nXARMjwHJiGRkZatWqlaZNm1bk/vyeQPk/n332mSwWi+68885LnrdZs2aF7hdnxheyUu7uum/fPlmtVnXo0KFgW/ny5dWoUaMSfVwAAAAAKBEREVL+1CcWi30dMDEPowuA4/r06aM+ffpcdP+/ewF999136tatm+rWrXvJ83p4eJi/B1F+d1ebTVqyxL5t3DhDSwIAAAAApxEVZV/GxdnDq/x1wKTogVVGHDt2TD/99JOGDh162WP37t2r0NBQ1a1bV/fff7+SkpJKocKrVMrdXevWrStPT0+tW7euYFtqaqr27NlTsO7l5VUwpBAAAAAATM3Dw94JYPFi+9KD/i0wN1poGTFr1iwFBATojjvuuORxHTt21MyZM9WoUSMdPXpU48eP1w033KBt27YpICCgyPtkZWUpKyurYD0tLU2SZLVaZbVaCx1rtVpls9mUl5envLw8x59Q586yLFkii80mm8UiW+fO0iXOZ/tf2JX/2FfL399fgwYN0nPPPaegoCBVrVpV0dHRcnOzZ8B5eXmqVauWVqxYoXvuuUfe3t6qXLmyY8/tCuTl5clms8lqtcrd3b3EHgcqaMP/bsvA5dB24CjaDhxF24GjaDtwlJnajhlqQMmy2PI/2cOpWSwWLViwQP379y9yf+PGjRUZGan33nvvqs575swZ1apVS2+//fZFe29FR0dr/PjxF2yfM2eO/Pz8Cm3LH54YFhYmLy+vq6qlkJwceb/9tjxWr1ZOeLiyRo8u8W8Mzp49q9GjR+vnn39WQECAnnrqKc2fP1833HCDXn75Za1bt07PPPOMEhISlJWVpdOnT5dYLdnZ2Tp06JBSUlKUw+VuAQAAAJRxmZmZGjhwoFJTUxUYGGh0OSgBBFgu4lIB1u+//64uXbooPj5erVq1uupzt2/fXj169NCkSZOK3F9UD6ywsDCdPHnygheO8+fP69ChQ6pdu7Z8fHyuuhZH2Ww2nT17VgEBAbLkT1R4jTIyMhQWFqY33njjioZmFqfz58/rwIEDCgsLK9V/x7LIarUqNjZWkZGR8vT0NLocOBHaDhxF24GjaDtwFG0HjjJT20lLS1PlypUJsFwYQwjLgE8//VRt27Z1KLxKT09XYmKiHnzwwYse4+3tLW9v7wu2e3p6XvAilpubK4vFIjc3t4Lhd6Uhf9hg/mM7YtOmTdq1a5c6dOig1NRUTZgwQZJ0++23l+pzkSQ3NzdZLJYi/41RMvi3hqNoO3AUbQeOou3AUbQdOMoMbcfox0fJYxJ3J5aenq74+HjFx8dLkvbv36/4+PhCk66npaXpm2++0SOPPFLkObp3766pU6cWrP/nP//RihUrdODAAa1atUq333673N3dNWDAgBJ9Ls7izTffVKtWrdSjRw9lZGTo999/L9G5rgAAAAAAAD2wnNr69evVrVu3gvXRo0dLkgYPHqyZM2dKkr766ivZbLaLBlCJiYk6efJkwfrhw4c1YMAAnTp1SlWqVFFERIT+/PNPValSpeSeiJNo06aNNmzYYHQZAAAAAACUOQRYTqxr16663BRmjz76qB599NGL7j9w4ECh9a+++qo4SgMAAAAAACg2DCEEAAAAAACAqRFgAQAAAAAAwNQIsAAAAAAAAGBqBFgAAAAAAAAwNQIsAAAAAAAAmBoBFmBCtWvX1pQpU4wuAwAAAAAAUyDAAgAAAAAAgKkRYAElJDs72+gSAAAAAABwCQRYwBXq2rWrRo4cqZEjR6p8+fKqXLmyXnrpJdlsNkn2YX8TJ07UoEGDFBgYqEcffVSSFBcXpxtuuEG+vr4KCwvTU089pYyMjILzHj9+XP369ZOvr6/q1KmjL774wpDnBwAAAACAWRFgAVdh1qxZ8vDw0Nq1a/XOO+/o7bff1ieffFKw/80331SrVq20adMmvfTSS0pMTFTv3r115513asuWLZo7d67i4uI0cuTIgvs89NBDOnTokJYtW6Z58+bp/fff1/Hjx414egAAAAAAmJKH0QUANptNmdbMEn2MvLw8ZVgz5J7tLjc3e27r5+kni8VyVecJCwvT5MmTZbFY1KhRI23dulWTJ0/WsGHDJEk33XSTnn322YLjH3nkEd1///16+umnJUkNGjTQu+++qxtvvFEffPCBkpKStGjRIq1du1bt27eXJH366adq0qRJMTxrAAAAAABcAwEWDJdpzVS5SeVK/XHTx6bL38v/qu5z/fXXFwq9wsPD9dZbbyk3N1eS1K5du0LHb968WVu2bCk0LNBmsykvL0/79+/Xnj175OHhobZt2xbsb9y4sYKCghx4RgAAAAAAuCYCLKAY+fsXDsTS09P12GOP6amnnrrg2Jo1a2rPnj2lVRoAAAAAAE6LAAuG8/P0U/rY9BJ9jLy8PKWdTVNgQGChIYRXa82aNYXW//zzTzVo0EDu7u5FHn/ddddpx44dql+/fpH7GzdurJycHG3YsKFgCOHu3bt15syZq64NAAAAAABXRYAFw1kslqseyne18vLylOuZK38v/4IAyxFJSUkaPXq0HnvsMW3cuFHvvfee3nrrrYseP2bMGF1//fUaOXKkHnnkEfn7+2vHjh2KjY3V1KlT1ahRI/Xu3VuPPfaYPvjgA3l4eOjpp5+Wr6+vwzUCAAAAAOBquAohcBUGDRqkc+fOqUOHDhoxYoRGjRqlRx999KLHt2zZUitWrNCePXt0ww03qE2bNho3bpxCQ0MLjpkxY4ZCQ0N144036o477tCjjz6qqlWrlsbTAQAAAADAKdADC7gKnp6emjJlij744IML9h04cKDI+7Rv316LFy++6DmDg4P1448/Ftr24IMPXlOdAAAAAAC4EnpgAQAAAAAAwNQIsAAAAAAAAGBqDCEErtDy5cuNLgEAAAAAgDKJHlgAAAAAAAAwNQIsAAAAAAAAmBoBFgxhs9mMLsGp8e8HAAAAAChLCLBQqjw9PSVJmZmZBlfi3PL//fL/PQEAAACg2OXkSBMmSD172pc5OUZXhDKMSdxRqtzd3RUUFKTjx49Lkvz8/GSxWEr8cfPy8pSdna3z58/Lzc15c1ubzabMzEwdP35cQUFBcnd3N7okAAAAAK4qJkaKjpZsNmnJEvu2ceMMLQllFwEWSl1wcLAkFYRYpcFms+ncuXPy9fUtlcCspAUFBRX8OwIAAABAiYiLs4dXkn0ZF2dsPSjTCLBQ6iwWi0JCQlS1alVZrdZSeUyr1aqVK1eqS5cuTj/sztPTk55XAAAAAEpeRIS955XNJlks9nXAIARYMIy7u3upBTHu7u7KycmRj4+P0wdYAAAAAFAqoqLsy7g4e3iVvw4YgAALAAAAAABcyMODOa9gGs47mzUAAAAAAADKBAIsAAAAAADMLidHmjBB6tnTvszJMboioFQxhBAAAAAAALOLiZGio+0Tqi9ZYt/G8D6UIfTAAgAAAADA7OLi7OGVZF/GxRlbD1DKCLAAAAAAADC7iAjJYrHftljs60AZQoDlxFauXKl+/fopNDRUFotFCxcuLLT/oYceksViKfTTu3fvy5532rRpql27tnx8fNSxY0etXbu2hJ4BAAAAAOCKREXZhxBGRtqXUVFGVwSUKubAcmIZGRlq1aqVHn74Yd1xxx1FHtO7d2/NmDGjYN3b2/uS55w7d65Gjx6t6dOnq2PHjpoyZYp69eql3bt3q2rVqsVaPwAAAADgCnl4MOcVyjQCLCfWp08f9enT55LHeHt7Kzg4+IrP+fbbb2vYsGEaMmSIJGn69On66aef9Nlnn+mFF164pnoBAAAAAAAcQYDl4pYvX66qVauqQoUKuummm/TKK6+oUqVKRR6bnZ2tDRs2aOzYsQXb3Nzc1KNHD61evfqij5GVlaWsrKyC9bS0NEmS1WqV1WotpmdybfLrMEs9cB60HTiKtgNH0XbgKNoOHEXbgaPM1HbMUANKFgGWC+vdu7fuuOMO1alTR4mJiYqKilKfPn20evVqubu7X3D8yZMnlZubq2rVqhXaXq1aNe3ateuijzNp0iSNHz/+gu2LFy+Wn5/ftT+RYhQbG2t0CXBStB04irYDR9F24CjaDhxF24GjzNB2MjMzjS4BJYwAy4Xdd999BbdbtGihli1bql69elq+fLm6d+9ebI8zduxYjR49umA9LS1NYWFh6tmzpwIDA4vtca6F1WpVbGysIiMj5enpaXQ5cCK0HTiKtgNH0XbgKNoOHEXbgaPM1HbyRwLBdRFglSF169ZV5cqVlZCQUGSAVblyZbm7u+vYsWOFth87duyS82h5e3sXOTm8p6en4S9i/2bGmuAcaDtwFG0HjqLtwFG0HTiKtgNHmaHtGP34KHluRheA0nP48GGdOnVKISEhRe738vJS27ZttXTp0oJteXl5Wrp0qcLDw0urTAAAAAAAgEIIsJxYenq64uPjFR8fL0nav3+/4uPjlZSUpPT0dD333HP6888/deDAAS1dulS33Xab6tevr169ehWco3v37po6dWrB+ujRo/Xxxx9r1qxZ2rlzp4YPH66MjIyCqxICAAAAAACUNoYQOrH169erW7duBev581ANHjxYH3zwgbZs2aJZs2bpzJkzCg0NVc+ePTVx4sRCw/0SExN18uTJgvV7771XJ06c0Lhx45SSkqLWrVvrl19+uWBidwAAAABACcjJkWJipLg4KSJCioqSPPjoDvBb4MS6du0qm8120f2//vrrZc9x4MCBC7aNHDlSI0eOvJbSAAAAAACOiImRoqMlm01assS+bdw4Q0sCzIAhhAAAAAAAmEVcnD28kuzLuDhj6wFMggALAAAAAACziIiQLBb7bYvFvg6AIYQAAAAAAJhGVJR9+c85sAAQYAEAAAAAYBoeHsx5BRSBIYQAAAAAAAAwNQIsAAAAAAAAmBoBFgAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAUyPAAgAAAAAAgKkRYKHsysmRJkyQeva0L3NyjK4IAAAAAIrG5xeUcR5GFwAYJiZGio6WbDZpyRL7tnHjDC0JAAAAAIrE5xeUcfTAQtkVF2d/8Zfsy7g4Y+sBAAAAgIvh8wvKOAIslF0REZLFYr9tsdjXAQAAAMCM+PyCMo4hhCi7oqLsy7g4+4t//joAAAAAmA2fX1DGEWCh7PLwYMw4AAAAAOfA5xeUcQwhBAAAAAAAgKkRYAEAAAAAAMDUCLAAAAAAAABgagRYAAAAAAAAMDUCLAAAAAAAAJgaARYAAAAAAABMjQALAAAAAAAApkaABQAAAAAAAFMjwAIAAAAA4Grk5EgTJkg9e9qXOTlGVwS4PA+jCwAAAAAAwKnExEjR0ZLNJi1ZYt82bpx9mZNj3x8XJ0VESFFRkgcfvYFrxW8RAAAAAABXIy7OHl5J9mVc3N/7LhVuAXAYQwgBAAAAALgaERGSxWK/bbHY1/NdKtwC4DB6YAEAAAAAcDWiouzLfw4TzBcRYe95ZbNdGG4BcBgBFsomxqUDAAAAcJSHx8WHBV4q3ALgMD6xo2xiXDoAAABQNpT2l9eXCrcAOIwAC2UT49IBAACAsoEvrwGXwCTuKJsuNekiAAAAANfBl9eASyDAcmIrV65Uv379FBoaKovFooULFxbss1qtGjNmjFq0aCF/f3+FhoZq0KBBSk5OvuQ5o6OjZbFYCv00bty4hJ+JAaKi7N/CREbal4xLBwAAAFwTX14DLoEhhE4sIyNDrVq10sMPP6w77rij0L7MzExt3LhRL730klq1aqXTp09r1KhRuvXWW7V+/fpLnrdZs2Zakt+1VpKHK05uzrh0AAAAoGxgUnXAJbhgMlF29OnTR3369ClyX/ny5RUbG1to29SpU9WhQwclJSWpZs2aFz2vh4eHgoODi7VWAAAAADAEX14DLoEAqwxJTU2VxWJRUFDQJY/bu3evQkND5ePjo/DwcE2aNOmSgVdWVpaysrIK1tPS0iTZhzFardZiqf1a5ddhlnrgPGg7cBRtB46i7cBRtB04irYDR5mp7ZihBpQsi82WP5sdnJnFYtGCBQvUv3//IvefP39enTt3VuPGjfXFF19c9DyLFi1Senq6GjVqpKNHj2r8+PE6cuSItm3bpoCAgCLvEx0drfHjx1+wfc6cOfLz83Po+QAAAAAAcKUyMzM1cOBApaamKjAw0OhyUAIIsFzEpQIsq9WqO++8U4cPH9by5cuv6pf5zJkzqlWrlt5++20NHTq0yGOK6oEVFhamkydPmuaFw2q1KjY2VpGRkfL09DS6HDgR2g4cRduBo2g7cBRtB46i7fxPTo701lvS6tVSeLj07LP24Ye4KDO1nbS0NFWuXJkAy4Xx2+jirFar7rnnHh08eFC//fbbVf8iBwUFqWHDhkpISLjoMd7e3vL29r5gu6enp+EvYv9mxprgHGg7cBRtB46i7cBRtB04qsy3nUmT7Fcot9mkRYukvDzmzrpCZmg7Rj8+Sp6b0QWg5OSHV3v37tWSJUtUqVKlqz5Henq6EhMTFRISUgIVAgAAAIBJxMXZwyvJvoyLM7YeAIUQYDmx9PR0xcfHKz4+XpK0f/9+xcfHKykpSVarVXfddZfWr1+vL774Qrm5uUpJSVFKSoqys7MLztG9e3dNnTq1YP0///mPVqxYoQMHDmjVqlW6/fbb5e7urgEDBpT20wMAAACA0hMRIVks9tsWi30dgGkwhNCJrV+/Xt26dStYHz16tCRp8ODBio6O1vfffy9Jat26daH7LVu2TF27dpUkJSYm6uTJkwX7Dh8+rAEDBujUqVOqUqWKIiIi9Oeff6pKlSol+2QAAAAAwEhRUfZlXJw9vMpfB2AKBFhOrGvXrrrUHPxXMj//gQMHCq1/9dVX11oWAAAAADgfDw/mvAJMjCGEAAAAAAAAMDUCLAAAAAAAAJgaARYAAAAAAABMjQALAAAAAAAApkaABQAAAAAAAFMjwAIAAAAAAICpEWABAAAAAADA1AiwAAAAAAAAYGoEWIARcnKkCROknj3ty5wcoysCAAAAAMC0PIwuACiTYmKk6GjJZpOWLLFvGzfO0JIAAAAAADAremABRoiLs4dXkn0ZF2dsPQAAAAAAmBgBFmCEiAjJYrHftljs6wAAAAAAoEgMIQSMEBVlX8bF2cOr/HUAAAAAAHABAizACB4ezHkFAAAAAMAVYgghYFZcqRAAAAAAAEn0wEIZZrPZlJOXI093T6NLKRpXKgQAAAAAQBI9sFCGvb/ufXX+rLP2n95vdClF40qFAAAAAABIIsBCGZVpzdR/f/+v1iWvU5sP22jejnlGl3QhrlQIAAAAAIAkAiyUUX6efvrzkT8VXiNcqVmpuvubuzX8x+E6Zz1ndGl/i4qyDyGMjLQvuVIhAAAAULqYlxYwDebAQplVs3xNrXhohV5e/rJejXtV0zdM16rDqzS3/xdq/NF8+5C9iAh7cORhwK8KVyoEAAAASl5Ojn3+2aLe/zMvLWAaBFgo0zzdPRXTPUZda3fVgwse1JZjW9T2w+v0/kKrBsfLfH+kLvXHFQAAAMDVu1RIxby0gGkwhBCQ1LNeT21+fLO61+muTFn1UH/pwdulNC+T/ZHK/+MaG2tfxsQYXREAAADg3C4VUjEvLWAaBFjA/wSXC9avD/yq/6q73POk2a2k1o9LqzvXMrq0v/ENEAAAAFC8LhVSMS8tYBqMPQL+wd3NXVH/7xd1/e/juv/8F9pf4bxusMzQuBVhirohSh5uBv/KRETYuzXbbHwDBAAAABSH/FDqn9N05GNeWsA0CLCAf/PwUKeXP1H8+bc04ucR+mLrF3p5+ctanLhYs++YrdpBtY2r7VJ/XAEAAABcPUIqwCkwhBC4iPI+5TX7jtmafftsBXgF6I9Df6jV9Fb6cuuXxhWV/8d18WL7kgncAQAAAABlAAEWcBn3t7xfmx/frPAa4UrLStPA+QM1aMEgpWWlGV0aAAAAAABlAgEWcAXqVKijlUNW6uUbX5abxU2fb/lcbT5soz8P/2l0aQAAAAAAuDwCLOAKebh5KLprtFY+tFK1ytfSvtP7FPFZhCaumKjcvFyjywMAAAAAwGURYAFXqXPNztr8+GYNbDFQubZcjVs+Tl1nddXBMwcdP2lOjjRhgtSzp32Zk1Ns9QIAAAAA4OwIsAAHlPcpry/u+EKf3/65ArwCFJcUp1bTW+mrbV85dsKYGCk6WoqNtS9jYoqzXAAAAAAAnBoBFnANHmj5gOIfj9f1Na5XalaqBnw7QA8tfEhns85e3Yni4iSbzX7bZrOvAwAAAAAASQRYwDWrW6Gufh/yu8Z1GSc3i5tmbZ6lNh+20ZrDa678JBERksViv22x2NcBAAAA2DHlBlDmEWA5sZUrV6pfv34KDQ2VxWLRwoULC+232WwaN26cQkJC5Ovrqx49emjv3r2XPe+0adNUu3Zt+fj4qGPHjlq7dm0JPQPX4eHmofHdxmvFQytUs3xNJZ5OVOfPOuu/K/97ZRO8R0XZhw5GRtqXUVElXTIAAADgPJhyAyjzCLCcWEZGhlq1aqVp06YVuf/111/Xu+++q+nTp2vNmjXy9/dXr169dP78+Yuec+7cuRo9erRefvllbdy4Ua1atVKvXr10/PjxknoaLiWiZoQ2P75Z9zW/T7m2XL247EXd9H83KSk16dJ39PCQxo2TFi+2Lz08SqdgAAAAwEwu1tOKKTeAMo8Ay4n16dNHr7zyim6//fYL9tlsNk2ZMkUvvviibrvtNrVs2VL/93//p+Tk5At6av3T22+/rWHDhmnIkCFq2rSppk+fLj8/P3322Wcl+ExcS5BPkObcMUez+s9SOa9yWnlwpVpNb6Wvt39tdGkAAACAuV2spxVTbgBlHt08XNT+/fuVkpKiHj16FGwrX768OnbsqNWrV+u+++674D7Z2dnasGGDxo4dW7DNzc1NPXr00OrVqy/6WFlZWcrKyipYT0tLkyRZrVZZrdbieDrXLL+O0qxnQNMB6hDSQYO/G6y1yWt177x79dOenzQ5crICvANKrQ5cGyPaDlwDbQeOou3AUbQdOMpUbWfNGsnHp/C61So995zk5iatXi2Fh0vPPmvfDkOZqe2YoQaULIvNlt8PE87MYrFowYIF6t+/vyRp1apV6ty5s5KTkxUSElJw3D333COLxaK5c+decI7k5GRVr15dq1atUnh4eMH2559/XitWrNCaNUVPSh4dHa3x48dfsH3OnDny8/O7xmfm/HJsOZqbMlfzjs2TTTYFewVrdK3Raujf0OjSAAAAAMAlZGZmauDAgUpNTVVgYKDR5aAE0AML12zs2LEaPXp0wXpaWprCwsLUs2dP07xwWK1WxcbGKjIyUp6enqX++LfqVj2e9Lge+v4hHUo7pKjEKI27YZyeC39O7m7upV4PrpzRbQfOi7YDR9F24CjaDhxlqraTkyO99VbhnlbMD2taZmo7+SOB4Lp4JXBRwcHBkqRjx44V6oF17NgxtW7dusj7VK5cWe7u7jp27Fih7ceOHSs4X1G8vb3l7e19wXZPT0/DX8T+zciabqp3kzY/vlmP//S4vt7+tcatGKclB5bo//r/n2oF1TKkJlw5M7ZnOAfaDhxF24GjaDtwlCnajqen9OKLxtaAq2aGtmP046PkMYm7i6pTp46Cg4O1dOnSgm1paWlas2ZNoeGB/+Tl5aW2bdsWuk9eXp6WLl160fvg6lTwraCv7vxKM26bIX9Pf608uFItp7fU/23+PzGaFwAAAACAohFgObH09HTFx8crPj5ekn3i9vj4eCUlJclisejpp5/WK6+8ou+//15bt27VoEGDFBoaWjBPliR1795dU6dOLVgfPXq0Pv74Y82aNUs7d+7U8OHDlZGRoSFDhpTys3NdFotFD7V+SJsf36xOYZ2UlpWmwQsH665v7tLJzJNGlwcAAAAAgOkwhNCJrV+/Xt26dStYz5+HavDgwZo5c6aef/55ZWRk6NFHH9WZM2cUERGhX375RT7/uKpHYmKiTp78OzS59957deLECY0bN04pKSlq3bq1fvnlF1WrVq30nlgZUa9iPa18aKVe/+N1jVs+TvN3ztcfSX/o434fq1+jfkaXBwAAAACAaRBgObGuXbtectiZxWLRhAkTNGHChIsec+DAgQu2jRw5UiNHjiyOEnEZ7m7uGnvDWPWu31sPLnhQ209s161f3aqhbYbq7V5vK9DbHJPgAwAAAABgJIYQAibQJqSN1j+6Xv8J/48ssujTTZ+q1fRWWnFghdGlAQAAAABgOAIswCR8PHz0Rs83tPyh5aodVFsHzhxQt1nd9Oyvz+p8zvlrO3lOjjRhgtSzp32Zk1M8RQMAAAAAUAoIsACT6VKri7Y8vkWPtHlENtn09p9v67oPr9P65PWOnzQmRoqOlmJj7cuYmOIqFwAAAACAEkeABZhQgHeAPr71Y/0w4AcFlwvWzpM7df0n1+vlZS8rOzf76k8YFyflz5dms9nXAQAAAABwEgRYgIn1bdhX24Zv073N7lWuLVcTVk5Qx086auuxrVd3oogIyWKx37ZY7OsAAAAAADgJAizA5Cr5VdJXd32luXfNVSXfSopPiVfbj9rqlZWvyJprvbKTREXZhw5GRtqXUVElWTIAAAAAAMWKAAtwEvc0u0fbntim2xrdJmueVS8te0nXf3r9lfXG8vCQxo2TFi+2Lz08Sr5gAAAAAACKCQEW4ESCywVrwb0LNPv22argU0Ebj268+t5YAAAAAAA4GQIswMlYLBbd3/J+bX9iu25tdGuh3lhbjm0xujwAAAAAAIodARbgpEICQrTw3oWFemO1+6idJq6YSG8sAAAAAIBLIcACnFh+b6wdI3YUzI01bvk4dfykozanbDa6PAAAAKBoOTnShAlSz572ZU6O0RUBMDkCLMAF5M+NNeeOOaroW1GbUjap3cftNGHFBHpjAQAAwHxiYuxXx46NtS9jYoyuCIDJEWABLsJisWhAiwHa/sR29W/cXzl5OXp5+cvq8EkHemMBAADAXOLiJJvNfttms68DwCUQYAEuJrhcsObfM7+gN1Z8SrzafdxO45ePV3ZuttHlAQAAAFJEhGSx2G9bLPZ1ALgEAizABeX3xtrxxA7d3vh25eTlKHpFtNp+1FZrDq8xujwAAACUdVFR9qGDkZH2ZVSU0RUBMDkCLMCFVStXTd/e862+vPNLVfarrG3Htyn803A9tegpnc06a3R5AAAAKKs8PKRx46TFi+1LDw+jKwJgcgRYgIuzWCy6r/l92jlipwa1GiSbbHpv7Xtq9n4z/bjnR6PLAwAAAADgsgiwgDKisl9lzeo/S4sfWKw6QXV0KO2Q+n3ZT/d8c49S0lOMLg8AAAAAgIsiwALKmMh6kdr2xDY91+k5uVvc9c2Ob9RkWhN9vOFj5dnyjC4PAAAAAIALEGABZZCfp59ej3xd64atU9uQtjpz/owe/fFRdZvVTbtP7ja6PAAAAAAACiHAAsqwNiFt9Ocjf+qtnm/Jz9NPKw+uVMvpLTVxxURl52YbXR4AAAAAAJIIsIAyz8PNQ6PDR2v7E9vVu35vZedma9zycbruw+u0+tBqo8sDAAAAAIAAC4Bd7aDa+nngz5pzxxxV8aui7Se2q/NnnTXy55FKy0ozujwAAAAAQBlGgAWggMVi0YAWA7RzxE491Poh2WTTtHXT1HRaU3236zujywMAAEBxycmRJkyQevaUXnvN6GoA4LIIsABcoJJfJc24bYaWPLhE9SrU05GzR9R/bn/d9fVdSj6bbHR5AAAAuFYxMVJ0tBQbK02aZHQ1AHBZBFiAM/jnN2QTJtjXS0H3ut21dfhWvdD5Bblb3PXtzm/VdFpTfbj+Q+XZ8kqlBgAAAJSAuDjJZrPfzl8CgIkRYAHO4J/fkEVH29dLia+nryb1mKQNj25Q+9D2Ss1K1eM/Pa4bZ96onSd2llodAAAAKEYREZLFYr+dvwQAEyPAApzBv78hi4sr9RJaBbfS6qGrNaXXFPl7+isuKU6tP2yt8cvHKysnq9TrAQAAwDWIirJ/MRoZKY0da3Q1AHBZBFiAM/j3N2QREYaU4e7mrlHXj9L2J7br5gY3Kzs3W9ErotXmwzb6I+kPQ2oCAACAAzw8pHHjpMWLpTFjjK4GAC6LAKuUDB48WCtXrjS6DDirf35DFh1tXzdQraBa+nHAj/rqzq9U1b+qdp7cqYgZERr+43Clnk81tDYAAAAAgOshwColqamp6tGjhxo0aKCYmBgdOXLE6JLgTP75Ddm4cfZ1g1ksFt3b/F7tHLFTD7d+WJI0fcN0NX2/qRbsXGBwdQAAAAAAV0KAVUoWLlyoI0eOaPjw4Zo7d65q166tPn36aN68ebJarUaXBzisom9FfXrbp/pt0G9qULGBks8m646v71D/r/orKTXJ6PIAAAAAAC6AAKsUValSRaNHj9bmzZu1Zs0a1a9fXw8++KBCQ0P1zDPPaO/evUaXCDisW51u2vz4ZkVFRMnDzUPf7f5OTaY10et/vC5rLiEtAAAAAMBxBFgGOHr0qGJjYxUbGyt3d3fdfPPN2rp1q5o2barJkycX62PVrl1bFovlgp8RI0YUefzMmTMvONbHx6dYa4Lr8vX01X+7/1fxj8Xrhpo3KNOaqTFLxqj1h6218iBzwAEAAAAAHEOAVUqsVqu+/fZb9e3bV7Vq1dI333yjp59+WsnJyZo1a5aWLFmir7/+WhMmTCjWx123bp2OHj1a8BMbGytJuvvuuy96n8DAwEL3OXjwYLHWBNfXrGozrXhohWbeNlOV/Sprx4kdunHmjXpwfCulnDlsdHkAAAAAACdj/EzQZURISIjy8vI0YMAArV27Vq1bt77gmG7duikoKKhYH7dKlSqF1l999VXVq1dPN95440XvY7FYFBwcXKx1oAzJyZFiYmSJi9PgiAj1yx2iqI1v6KO20mzLFn3/Tn290vsNDW8/XB5uvAQBAAAAAC6PT4+lZPLkybr77rsvORwvKChI+/fvL7EasrOzNXv2bI0ePVoWi+Wix6Wnp6tWrVrKy8vTddddp5iYGDVr1uyix2dlZSkrK6tgPS0tTZK915lZJqjPr8Ms9bi0116z/9hs0h9/KKBWLb13wFeDduTpychsbQzJ0lO/PKVPN32q93q9p+trXG90xZdE24GjaDtwFG0HjqLtwFG0HTjKTG3HDDWgZFlsNpvN6CJQOr7++msNHDhQSUlJCg0NLfKY1atXa+/evWrZsqVSU1P15ptvauXKldq+fbtq1KhR5H2io6M1fvz4C7bPmTNHfn5+xfoc4NxybbmKPRWr2UdnKz03XZLUvWJ3DQodpPIe5Q2uDgAAAICzyszM1MCBA5WamqrAwECjy0EJIMAqQ3r16iUvLy/98MMPV3wfq9WqJk2aaMCAAZo4cWKRxxTVAyssLEwnT540zQuH1WpVbGysIiMj5enpaXQ5ru2116RJk+w9sCwWacwYyc1NWr1aCg+Xnn1WJ7JOK2pZlGZtmSVJCvIJ0oQbJ2hYm2Fyd3M3+AkURtuBo2g7cBRtB46i7cBRtB04ykxtJy0tTZUrVybAcmEMISwjDh48qCVLlmj+/PlXdT9PT0+1adNGCQkJFz3G29tb3t7eRd7X6BexfzNjTS7nhRekvDwpLk6KiJCioiSPwi81ob6+mnn7TD3a7lGN+HmE4lPi9dSvT2nG5hmaevNUdQrrZFDxF0fbgaNoO3AUbQeOou3AUbQdOMoMbcfox0fJ4yqEZcSMGTNUtWpV3XLLLVd1v9zcXG3dulUhISElVBlcjoeHNG6ctHixfelx8Zy8U1gnrR+2XlP7TFWQT5A2pWxS5886a9CCQUo+m1yKRQMAAAAAzIwAqwzIy8vTjBkzNHjwYHn8K0wYNGiQxo4dW7A+YcIELV68WPv27dPGjRv1wAMP6ODBg3rkkUdKu2yUEe5u7hrRYYT2jNyjoW2GyiKLPt/yuRq+11CTfp+k8znnjS4RAAAAAGAwAqwyYMmSJUpKStLDDz98wb6kpCQdPXq0YP306dMaNmyYmjRpoptvvllpaWlatWqVmjZtWpolowyq4l9Fn9z6idYOW6vwGuHKsGYo6rcoNXu/mb7b9Z2Yrg8AAAAAyi7mwCoDevbsedEP/8uXLy+0PnnyZE2ePLkUqgKK1i60nf54+A/N2TpHzy95XvtO71P/uf0VWTdSU3pPUdMqhKkAAAAAUNbQAwuA6VgsFt3f8n7tHrlbURFR8nL3Uuy+WLX8oKVGLRql0+dOG10iAAAAAKAUEWABMIecHGnCBKlnT/syJ0flvMrpv93/q50jdqp/4/7KteXq3bXvqsF7DfTh+g+Vm5drdNUAAAAAgFJAgAXAHGJipOhoKTbWvoyJKdhVt0JdLbh3gWIfjFXTKk116twpPf7T42r7UVutPLjSsJIBAAAAAKWDAAuAOcTFSflztdls9vV/6VG3hzY/vlnv9n5XQT5B2nxss26ceaPunXevklKTSrlgAAAAAEBpIcACYA4REZLFYr9tsdjXi+Dh5qEnOz6pvU/u1eNtH5ebxU1fb/9ajac21vjl45VpzSzFogEAAEpAEVMrAEBZx1UIARgjJ8c+TDAuzh5WPf+8fXv+elTUJe9e2a+yPuj7gR5r95hG/TJKKw+uVPSKaH0W/5neiHxDdze9W5b8QAwAAMCZ5E+tYLNJS5bYt40bZ2hJAGA0AiwAxiimN2atg1tr+eDl+mbHN/rP4v8oKTVJ9867V++GvavJvSarffX2xVs3AACAo/79BV5UlORRxEeyK5ha4ZrODwBOiCGEAIzh6BuzIlgsFt3T7B7tGrlLL9/4snw9fPXHoT/U4ZMOenDBgzqcdriYigYAALgGRV20pqjhgp06Fb7fv9ev5vwA4CIIsAAY4wrnvLoafp5+iu4arT1P7tGgVoMkSbO3zFbD9xpq3LJxSs9Ov+bHAAAAuCJFBVNFfYFXVOiUf0y+f69fTDF+QQgAZkN/UgDGyJ/j6grnvLoaNQJraFb/WXqyw5Ma/eto/Z70uyaunKiPN36sV7q9oodaPyR3N/diezwAAIALFDVdQkSE/bbN9vcXeFcSOq1efWWPWdT5AcBFEGABMIaHR4lPRtoutJ1WPLRC83fO1/NLnte+0/v0yA+PaMqaKXq1+6u6ucHNTPQOAABKRlHB1M8//70v/wu8mJiiQydHgqgS/IIQAIxGgAXApVksFt3Z9E71bdhX09ZN08SVE7Xt+Db1/bKvbqh5g17r8ZrCw8KNLhMAALiaonpDFfUF3qVCp6sNokrhC0IAMAoBFoAywdvDW6PDR+uh1g/p1bhX9e6ad/V70u/q9Fkn9W/cXzE3xahJlSZGlwkAAMzuSq/0d6W9oS4WOhFEAUAhTOIOoEyp6FtRr0e+rr1P7tXDrR+Wm8VNC3ctVPMPmuuR7x/hioUAAODSrvRKf/nB1OLF9mVRIRcA4IoRYAFwbUVdAUhSWPkwfXrbp9o6fKtua3Sb8mx5+nTTp2rwXgO9sOQFnT532uDCAQCAKXGlPwAwBAEWANd2mW9Jm1ZpqoX3LdQfD/+hiJoROp9zXq/98ZrqvVtPb/zxhs5ZzxlSNgAAMKmICPucVhJX+gOAUkSABcC1XeG3pJ3COmnlQyv1/X3fq1mVZjp9/rSeX/K8mk1vpiWnlignL6cUiwYAAKYVFWX/Uiwy0r7kSn8AUCoIsAC4tqv4ltRisahfo37a/PhmzbxtpsICw3T47GFNPTRVbT9pq+92fSdbfhgGAACMdZFpAkrctc5tZVTdAODkmEkQgGu70isA/YO7m7sGtx6se5vfq3f/fFevLH9FO0/uVP+5/dUprJNe6/GaImoyXAAAAEPlTxNgs0lLlti3OcOV+5y1bgAwGAEWANd2sUtTXwEfDx890/EZVT9WXVsDt+rdte9q1aFVumHGDerXsJ9iuseoedXmxVwwAAC4ImabTD0nxx5O/fNLs6J6Z5mtbgBwEgwhBIDLKOdRThO7TlTCUwl6rO1jcre464c9P6jV9FYa8t0QJaUmGV0iAABlj9kmU7/MhWMKmK1uAHAS9MACgCsUGhCq6X2n65nrn9H/++3/6dud32pm/Ex9ufVLjewwUmMjxqqSXyWjywQAoGxwYJqAEnWlPavMVjcAOAkCLAC4So0qN9K8e+ZpzeE1emHpC1p+YLneWv2WPtn4icZ0HqNR14+Sn6ef0WUCAODarmGagBIREWGf08pmu3TPKrPVDQBOgiGEAOCgjjU66rdBv2nR/YvUqlorpWalKuq3KDV4r4E+3vCxcvK4qhAAAGVGVJR96GBkpH1JzyoAKFYEWABwDSwWi3rX762Nj23U7Ntnq3ZQbSWfTdajPz6q5u8317c7vlWeLc/oMgEAQEnL71m1eLF9WdQE7gAAhxFgAUAxcLO46f6W92vXiF16p/c7quxXWbtP7dZd39ylth+11Q+7f5Atf14MAAAAAMBVIcACgGLk7eGtpzo+pcSnEjWuyzgFeAUoPiVet351qzp80kGL9i4iyAIAAIDTsNlssuZajS4DIMACgJIQ6B2o8d3Ga/+o/Xqh8wvy8/TT+uT1unnOzer8WWct2beEIAsAgJKSkyNNmCD17Glf5jAvJXC1bDabfk34VeGfhuuVla8YXQ5AgAUAJamSXyVN6jFJ+0ft17Phz8rHw0erD69W5OeR6jqrq1YcWGF0iQAAuJ6YGPtE6rGx9mVMjNEVAU7DZrNpceJidf6ss3p/0VtrjqzRRxs/ohcWDEeABQCloKp/Vb3Z803te2qfnurwlLzcvbTy4Ep1ndVVPf6vh/5I+sPoEgEAcB1xcVJ+T2ebzb5uJHqEwQnYbDYt2bdEETMi1Gt2L60+vFo+Hj565vpntOmxTfJ09zS6RJRxBFgAXMfl3hya4M1jSECI3unzjhKfStTwdsPl6eappfuXKmJGhLrN6qbf9v/G0EIAAK5VRIRksdhvWyz2dSPRIwwmZrPZtHTfUnWZ2UWRn0dq1aFV8nb31qiOo7TvqX16u9fbCi4XbHSZgLi2KwDXkf/m0GaTliyxbxs37sr3l6IagTX0/i3v6/nOzyvm9xjNjJ+p5QeWa/mB5eoU1kkvdXlJver1kiX/zTcAALhyUVH2ZVycPbzKXzeK2XqEAbIHV8sOLFP08mj9nvS7JMnb3VuPtX1MYyLGKDQg1OAKgcLogQXAdVzuzaEJ3zzWDqqtj/p9pISnEjSi/Qh5u3tr1aFV6vNFH3X8pKO+3/09PbIAALhaHh72L6kWL7YvPQz+3t5sPcJQptlsNv205yd1/qyzuv9fd/2e9Lu83L00sv1IJT6VqHf6vEN4BVMiwHJh0dHRslgshX4aN258yft88803aty4sXx8fNSiRQv9/PPPpVQtUAwu9+bQxG8ea5avqak3T9W+Ufv0zPXPyNfDV+uS1+m2r25Tmw/baN6Oecqz5RldJgAAcERUlL0XeGSkfWl0jzCUSXm2PM3bMU/XfXSd+n7ZV6sPr5a3u7dGtB+hxKcS9d7N76l6YHWjywQuiiGELq5Zs2Zakj9USpLHJb59WrVqlQYMGKBJkyapb9++mjNnjvr376+NGzeqefPmpVEucG0uN1zAbMMJihAaEKq3e72tFyJe0Nur39a0ddO0+dhm3f3N3Wpapan+3w3/T/c2u1fubu5GlwoAAK5Ufo8wwAA5eTn6cuuXiomL0a6TuyRJ/p7+Gt5uuEaHj1ZIQIjBFQJXhgDLxXl4eCg4+Mom3HvnnXfUu3dvPffcc5KkiRMnKjY2VlOnTtX06dNLskygeFzuzaETvXms6l9Vr/Z4Vc91ek7vrHlH7655VztO7ND98+9X9PJoRd0Qpftb3M/VYAAAAFCkrJwszYyfqdf+eE37z+yXJAX5BOnJDk9qVMdRquRXyeAKgatDgOXi9u7dq9DQUPn4+Cg8PFyTJk1SzZo1izx29erVGj16dKFtvXr10sKFCy/5GFlZWcrKyipYT0tLkyRZrVZZrdZrewLFJL8Os9QD52F02wn0DNRLES/pqXZPadr6aXp33bva+9deDfluiMYvH6/nOz2vB1s8KG8Pb0Pqw8UZ3XbgvGg7cBRtB46i7biWjOwMfRr/qSavmawjZ49Ikqr4VdGoDqP0eNvHFegdKKl4/r/N1HbMUANKlsXG7MAua9GiRUpPT1ejRo109OhRjR8/XkeOHNG2bdsUEBBwwfFeXl6aNWuWBgwYULDt/fff1/jx43Xs2LGLPk50dLTGjx9/wfY5c+bIz8+veJ4MAEnSudxz+uXUL1p4fKFSc1IlSZU8K+mOqneoR6Ue8nYjyAIAACiLMnIztOjkIn1//Hul5do7FVTyrKT+VfurZ6WeLv8+MTMzUwMHDlRqaqoCAwONLgclgACrDDlz5oxq1aqlt99+W0OHDr1gv6MBVlE9sMLCwnTy5EnTvHBYrVbFxsYqMjJSnp4MucKVc7jt5ORIb70lrV4thYdLzz5brFdAyrRm6tNNn+qtP99ScnqyJCnYP1jPXP+MHm3zqPy9/A2rDXa87sBRtB04irYDR9F2nNupzFN6b917mrZ+mlKz7F9w1gmqo+fCnyvxnvpmajtpaWmqXLkyAZYL4xNLGRIUFKSGDRsqISGhyP3BwcEXBFXHjh277Bxa3t7e8va+8EXR09PT8BexfzNjTXAOV912Jk2yX2XIZpMWLZLy8oqefysnR4qJKTyx/BWESeU9y2t059F6ouMTmrFphl7941UlpSZpzNIxemP1G3q649N6ov0TquBbwfHaUCx43YGjaDtwFG0HjqLtOJejZ4/qrdVvafr66cqwZkiSmlRuoqgbonRf8/vk4VZ6H/fN0HaMfnyUPDejC0DpSU9PV2JiokJCir7KRHh4uJYuXVpoW2xsrMLDw0ujPMC1xMXZAyLJvoyLK/q4mBh7mBQba1/GxFzVw/h4+Gh4++Ha++RefdLvE9WtUFcnM0/qxWUvquaUmvrP4v/oSNoRx2oDAACA6Rw8c1AjfhqhOu/U0Vur31KGNUNtgtto3t3ztO2JbXqg5QOlGl4BpYUAy4X95z//0YoVK3TgwAGtWrVKt99+u9zd3QuGCA4aNEhjx44tOH7UqFH65Zdf9NZbb2nXrl2Kjo7W+vXrNXLkSKOeAuC8IiIki8V+22KxrxelmMIkL3cvDb1uqHaP3K3Zt89Wi6otlJ6drrdWv6U679TR0O+GFlw2+YprAwAAgGnsOLFDQ74bovrv1df7699XVm6WOoV10k8Df9KGRzfozqZ3ys3CR3y4LmJZF3b48GENGDBAp06dUpUqVRQREaE///xTVapUkSQlJSXJze3vF7hOnTppzpw5evHFFxUVFaUGDRpo4cKFat68uVFPAXBeUVH25T+HBhYlIkJassQeXhVDmOTh5qH7W96vgS0G6peEX/TqH69q5cGV+iz+M82In6H+jftrzOBn1fFKagMAAIDh4pLi9Pofr+uHPT8UbOtep7te7PKibqx1oyz5X0wCLo4Ay4V99dVXl9y/fPnyC7bdfffduvvuu0uoIqAM8fC4snmlrjToukoWi0V9GvRRnwZ9tPrQar32x2v6bvd3WrBrgRbsWqCutbvq2fue1c0NbuabOgAAAJPJs+Xph90/6PVVr2vVoVWSJIssur3J7Xq+0/PqWKOjwRUCpY8ACwCMdKVB1zUIDwvXwvsWaueJnXp91euavWW2lh9YruUHlqtehXp6ssOTGtJmiAK9uVoLAKAEOHjBEqAsOmc9p8+3fK7Jf04umP7By91Lg1sN1rPhz6pR5UYGVwgYh78cAFBGNKnSRDNum6GJ3Sbq3TXv6uONHyvxdKKe/vVpvbTsJQ1pPUQjO4xUg0oNjC4VAOBK8i9YYrPZh81LXP0W+JeU9BS9v+59fbD+A53MPClJKu9dXk+0f0JPdXxKweUufWV4oCxg3AgAlDE1Amvo9cjXdfiZw/rglg/UpHITnc0+q3fXvqtGUxup75y+ik2MlS1/cnkAAC4nJ0eaMEHq2dO+zMn5ex9XvwUuatvxbXr4u4dVa0otTVw5USczT6p2UG1N7jVZh545pJjuMYRXwP/QAwsAyih/L3893u5xPdb2MS3Zt0TvrHlHP+39qeCnSeUmeqrjU3qw5YPy9/I3ulwAgJldqpdVMV+wBHB2NptNixMX6+0/39bixMUF28NrhGt0+Gj1b9xfHm58VAf+jd8KACjjLBaLIutFKrJepPae2qupa6dqRvwM7Ty5U8N/Gq6xS8dq2HXDNKL9CNUKqmV0uQAAM7pUL6sSumAJ4GzO55zXF1u+0OQ/J2v7ie2SJDeLm+5scqeeuf4ZhYeFG1whYG4EWACAAg0qNdA7fd7RxJsmasamGXpv7XtKPJ2oN1a9obdWv6X+jftrVMdRuqHmDVyyGQDwt0v1siqFC5YAZnYi44Q+WP+Bpq2bpuMZxyVJ5bzK6ZE2j+ipjk+pToU6BlcIOAcCLADABQK9AzXq+lEa2WGkFiUs0jtr3tGSfUs0f+d8zd85X62DW2tUx1G6r/l98vHwMbpcAIDR6GUFXGDHiR2a8ucU/d/m/1NWbpYkKSwwTKM6jtIj1z2i8j7lDa4QcC4EWACAi3J3c1ffhn3Vt2FfbT++Xe+ueVefb/lc8SnxGvLdED0f+7wea/uYhrcfrtCAUKPLBQAYhV5WgCQpz5anXxN+1btr39UvCb8UbG8X2k7Phj+rO5vcKU93TwMrBJwXVyEEAFyRZlWb6cN+H+rw6MN6rcdrCgsM04nME3rl91dUa0otDfx2oNYcXmN0mQAAAKUuLStN7615T42nNtbNc27WLwm/yCKLbm98u34f8rvWPrJW9zW/j/AKuAb0wAIAXJWKvhX1fOfnNTp8tBbuWqh317yr35N+15fbvtSX275Ux+odNarjKN3Z9E55uXsZXS4AAECJ+ecFcM5mn5Vkn4phaJuhGtF+hOpVrGdwhYDrIMACADjEw81DdzW9S3c1vUsbj27Uu2ve1ZfbvtSaI2s0cP5AhSwO0RPtn9BjbR9TFf8qRpcLAABQLPJsefol4RdNXTtVixIWFWxvXLmxnuzwpAa1GqRyXuUMrBBwTQwhBABcs+tCrtPM/jOV9HSSJnSdoOBywTqaflQvLXtJYZPDNOS7IYpPiTe6TAAAAIf9de4vvbXqLTV4r4FumXOLFiUskkUW9W3YV78+8Ku2P7FdT7R/gvAKKCH0wAIAFJtq5arppRtf0piIMfpm+zd6Z807Wpe8TjPjZ2pm/Ex1qdVFozqO0q2NbpWHG3+CAACA+W06uknT1k3TnK1zdC7nnCQpyCdID7d+WMPbD1f9ivUNrhAoG/j0AAAodl7uXrq/5f0a2GKg/jz8p95d+67m7ZinlQdXauXBlapVvpZGtB+hR657RBV8KxhdLgAAQCHnrOf09fav9f7697X2yNqC7a2DW2tE+xEa2GKg/Dz9DKwQKHsIsAAAJcZisSg8LFzhYeF6M/JNvb/ufX244UMdTD2o55c8r+gV0RrUcpCe7PikmlZpanS5AACgjNt7aq+mr5+uGfEzdPr8aUmSp5un7mp6l0a0H6FOYZ1ksVgMrhIomwiwAAClonpgdf23+3/1YpcX9eW2L/XOmne05dgWTd8wXdM3TFdk3UiNaD9CtzS8heGFAACg1Fhzrfp+9/eavmG6luxbUrC9VvlaeqztYxp63VBV9a9qYIUAJAIsAEAp8/X01cNtHtaQ1kO04uAKvbPmHX2/+3vF7otV7L5YVQ+orqFthuqR6x5RWPkwo8sFAAAuKvGvRH2y8RPNiJ+hYxnHJEkWWdSnQR8Nbzdcfer3kbubu8FVAshHgAUAMITFYlHX2l3VtXZX7T+9Xx+s/0Az4mfoyNkjmrBygl75/RX1rt9bw64bplsa3CJPd0+jSwYAAE4uOzdb3+36Th9t/KhQb6tq/tX0cJuH9WjbR1U7qLZxBQK4KAIsAIDh6lSoo9cjX9fEbhM1f+d8fbjhQ604uEI/7/1ZP+/9WcHlgjWk9RA9ct0jqluhrtHlAgAAJ7P31F59vPFjzYyfqROZJyTZe1v1qt9Lw64bpn4N+/FlGWByBFgAANPw9vDWgBYDNKDFAO05tUefbPxEM+NnKiU9RZPiJmlS3CR1r9Ndw64bpv6N+8vbw9vokgEAgEmlZ6dr3o55+mzTZ/o96feC7SHlQjS0zVANvW4ova0AJ0KABQAwpYaVGur1yNf1yk2v6IfdP+jjjR9rceJiLd2/VEv3L1VF34oa2HyghrQZojbBbbgiEAAAkM1m06pDqzQjfobmbp+r9Ox0SZKbxU296vXSY20f44IxgJPitxYAYGpe7l66s+mdurPpnTpw5oA+2/SZPtv0mY6cPaKp66Zq6rqpalmtpYa0HqL7W9yvKv5VjC4ZAMqenBwpJkaKi5PCwyWLRVq1SoqIkKKiJA8+dqBkJZ9N1uebP9dn8Z9pz6k9BdvrV6yvIa2HaFCrQaoRWMPACgFcK/6SAACcRu2g2prQbYJevvFlLdm3RDPiZ2jhroXacmyLnvn1GT0f+7z6NuyrIa2HqE+DPny7CgClJSZGio6WbDYpNvbv7Uv+N0n2uHGGlAXXlp2brR/3/KjPNn2mRQmLlGfLkyT5e/rr7mZ36+HWDyuiZgS9tAEXwTt7AIDTcXdzV6/6vdSrfi+dPndaX277UjPiZ2h98not2LVAC3YtUDX/anqw5YMa0maImlZpanTJAODa4uLs4dW/2Wz2fUAx2npsqz7b9Jlmb52tk5knC7ZH1IzQkNZDdHfTuxXgHWBghQBKAgEWAMCpVfCtoCfaP6En2j+hrce2amb8TH2+5XMdyzimN1e/qTdXv6kO1TtoSOshuq/5fQryCTK6ZABwPRERhXte5bNY7PuAa3Qq85Tmbp9b8IVVvpByIRrcarCGtBmihpUaGlghgJJGgAUAcBktqrXQW73e0qs9XtVPe3/SjPgZ+mnPT1p7ZK3WHlmrUb+MUr+G/fRAywfUp34frmIIAMUlKkpavlxatuzvbXXrSoMH2/cBDsjKydLPe3/W51s+1497fpQ1zypJ8nTz1K2NbtXDbR5Wz3o9mTIAKCP4TQcAuBxPd0/1b9xf/Rv317H0Y5q9ZbZmbp6pbce36dud3+rbnd+qgk8F3dPsHt3f4n51rtlZbhY3o8sGgJLxzwnWS2pSdQ+PC89Zrx5zX+Gq2Ww2rTmyRp9v/lxfbf9Kf537q2Bfm+A2GtRqkB5o+YAq+1U2sEoARiDAAgC4tGrlqunZTs9qdPhobTm2RbO3zNacbXOUfDZZH274UB9u+FC1ytfS/S3u1wMtH1CTKk2MLhkAitc/J1gvyUnVIyLs57fZGDqIq7b75G59sfULzdk6R4mnEwu2hwaE6oEWD+jBVg+qedXmBlYIwGgEWACAMsFisahVcCu1Cm6lV3u8quUHlmv21tn6dse3Oph6UDFxMYqJi9F1IdfpgRYP6L7m9ykkIMTosgHg2v1zgvWSnFQ9f6jgP3t6AZdw9OxRfb39a83eOrvQvFb+nv66vcntGtRykG6qc5Pc3dwNrBKAWRBgAQDKHHc3d3Wv213d63bX+ze/rx/2/KDZW2ZrUcIibTy6URuPbtR/Yv+jHnV76IEWD6h/4/5czQhA6SnuIX+l1TPKw4Mhg7isw2mH9e2ObzVv5zz9kfSHbLKHq+4W+xWG729xv25rdJv8vfwNrhSA2RBgAQDKNF9PX93T7B7d0+wencw8af8meMtsrT68WosTF2tx4mL5ePjo5gY3695m96pvw77y8/QzumwArqy4h/zRMwoGO3jmoL7d+a3m7Zin1YdXF9p3fY3rNbD5QN3b/F5V9a9qUIUAnAEBFgAA/1PZr7KeaP+Enmj/hBL/StQXW7/QF1u/0J5TezR/53zN3zlf/p7+6teon+5tdq961+8tHw8fo8sG4GqKe8gfPaMcVxoT4Luofaf3FfS0WntkbcF2iyzqXLOz7mpyl+5ocofCyocZWCUAZ8KrLwAARahXsZ7G3ThOL3V5SfEp8Zq7fa7mbp+rA2cO6KttX+mrbV8p0DtQtzW6Tfc1v0896vaQl7uX0WUDcAVMhm4epTUBvotI+CtB83bM0zc7vtHGoxsLtltkUZdaXXRXU3toFRoQamCVAJwVAZYLmzRpkubPn69du3bJ19dXnTp10muvvaZGjRpd9D4zZ87UkCFDCm3z9vbW+fPnS7pcADAli8WiNiFt1CakjSZ1n6R1yes0d9tcfb3jax1OO6zPt3yuz7d8rgo+FXRHkzt0V9O71K12N7nJzejSATgrhvyZR2lNgO/Edp3cpXk75mnejnnafGxzwXY3i5u61u6qu5rcpdub3K7gcsEGVgnAFRBgubAVK1ZoxIgRat++vXJychQVFaWePXtqx44d8ve/+KSIgYGB2r17d8G6xWIpjXIBwPQsFos6VO+gDtU76I2eb2jVoVWau22uvtnxjY5lHNOnmz7Vp5s+VaB3oHrX662aGTUVkRWhSp6VjC4dgDNhyJ950BvuAjabTTtO7CjoabX9xPaCfe4Wd91U5ybd3fRu9W/cX1X8qxhYKQBXQ4Dlwn755ZdC6zNnzlTVqlW1YcMGdenS5aL3s1gsCg7mGxLAKTFXR6lxs7gpomaEImpGaErvKVp5cKW+3v61vtv9nY6mH9XXO76WJL075V11r9Ndtze+Xbc2ulXVylUzuHIAwBWjN5wke2i19fhWfbP9G83bOU+7Tu4q2Ofp5qkedXvorqZ36bZGt6mSH1/aACgZfKopQ1JTUyVJFStWvORx6enpqlWrlvLy8nTdddcpJiZGzZo1u+jxWVlZysrKKlhPS0uTJFmtVlmt1mKo/Nrl12GWeuA8nK7tvPaa/cdmk/74Q3Jzk8aMMbqqMiGiRoQiakRoSs8pWpe8TvN3ztfczXOVnJWsRQmLtChhkR778TGF1wjXrQ1v1W2NblO9CvWMLhsm5HSvOzAN2k4JGTv279s2m+SC/75FtR2bzab4Y/H6dte3mr9rvhL+SijY5+XupR51eujOJneqb/2+quBb4YJzoWww0+uOGWpAybLYbPmDuuHK8vLydOutt+rMmTOKu8TY/dWrV2vv3r1q2bKlUlNT9eabb2rlypXavn27atSoUeR9oqOjNX78+Au2z5kzR35+XGoeQNlls9l0OOuw/kz9U2vOrFHCuYRC+2v61NT15a9Xx/IdVde3LkO2AQCGstlsSjiXoFVnVmnVmVU6ln2sYJ+nxVPXBV6nTuU7qV35dvJ3v/iUJIARMjMzNXDgQKWmpiowMNDoclACCLDKiOHDh2vRokWKi4u7aBBVFKvVqiZNmmjAgAGaOHFikccU1QMrLCxMJ0+eNM0Lh9VqVWxsrCIjI+Xp6Wl0OXAiTtd2XntNmjTp77k6xo6lB5ZBimo7h9IO6Yc9P+j7Pd9rxcEVyrXlFhxfq3wt3drwVt3a8FZ1DussDzc6SZdVTve6A9Og7cARebY8rU5arSmLp2hT1iYlpSUV7PP18FXver11R+M7dHP9mxXgHWBgpTAjM73upKWlqXLlygRYLox3x2XAyJEj9eOPP2rlypVXFV5Jkqenp9q0aaOEhISLHuPt7S1vb+8i72v0i9i/mbEmOAenaTsvvCDl5f09V8cLLzAHlsH+2XbqVqqrUeGjNCp8lP4695d+2vOTFuxaoF8SftHB1IN6b917em/de6rkW0n9GvXT7Y1vV2TdSPl6+hr8LGAEp3ndgenQdnA52bnZikuK0/e7v9e3O7/V4bTDBfv8Pf3Vt2Ff3dX0LvWp30f+XvS0wuWZ4XXH6MdHyeNTjQuz2Wx68skntWDBAi1fvlx16tS56nPk5uZq69atuvnmm0ugQgDFjitXOY2KvhX1YKsH9WCrB5VpzVRsYqwW7FqgH/b8oFPnTmlm/EzNjJ8pP08/9a7fW/0b9VffhoXnGQFQzLgQBlzYiYwTWpSwSD/u+VG/Jv6qtKy0gn0BXgFq499GI28aqVsa3SI/T6YBAWA+/EV2YSNGjNCcOXP03XffKSAgQCkpKZKk8uXLy9fX/m3+oEGDVL16dU2aNEmSNGHCBF1//fWqX7++zpw5ozfeeEMHDx7UI488YtjzAABX5+fpp9sa36bbGt+mnLwc/X7wdy3ctVALdi3QobRDmr9zvubvnC93i7u61OqiWxrcopsb3KzGlRszbxZQnGJipOho+zDsJUvs2/hSAE4q/8qBP+75UT/u+VF/Hv5TNv09e0xV/6q6ucHNur3x7epWs5t+W/ybbm50M71YAJgWAZYL++CDDyRJXbt2LbR9xowZeuihhyRJSUlJcnNzK9h3+vRpDRs2TCkpKapQoYLatm2rVatWqWnTpqVVNgCUaR5uHupWp5u61emmKb2naFPKJi3YuUALdy/UtuPbtOzAMi07sEz/if2P6laoq5vr36xbGt6irrW7ysfDx+jyAecWF2cPryT78hIXvgHMKD07Xcv2LyvoaXUo7VCh/a2DW6tfw37q27Cv2oW2k5vF/jmAq7cBcAYEWC7sSubnX758eaH1yZMna/LkySVUEQBcAYbwFLBYLLou5DpdF3KdJt40UQl/JeinPT/p54SftfzAcu07vU9T103V1HVT5evhq251uqlP/T7qU7+P6lWsZ3T5gPOJiLD3vMq/EEZEhNEVAZdks9m05dgW/ZLwi35N/FVxSXGy5v0dRvl4+KhH3R7q26Cvbml4i2oEXt18uABgJmXzEwEAwLwuN4SnDAdc9SvW16jrR2nU9aOUnp2upfuW6qe9P+nnvT/ryNkj+nnvz/p578+SpAYVG6hnvZ7qWa+nutXuxpWjgCsRFWVf/vP1BTCZk5knFZsYq18Tf9Wvib8qJT2l0P46QXXUq14v9W3YV93qdGM+KwAuo2y84wcAOI/LDeFhjhpJUjmvcgXzZuXPc7Jo7yItSlikPw79ob1/7dXev/Zq2rpp8nDzUHiN8IJAq21IW7m7uRv9FADz4UIYMKGcvBytObymoJfV+uT1heay8vP0U7fa3dS7fm/1qtdL9SvWZ35EAC6JAAsAYC6XG8LDHDUXsFgsalmtpVpWa6kxEWOUlpWmZfuXaXHiYi3et1gJfyXo96Tf9XvS73pp2Uuq4FNB3et2V8+69kCrVlAto58CAOAf9p3ep6X7lurXxF+1ZN8SpWalFtrfslpL9arXS73r91bnsM7y9vA2qFIAKD0EWAAAc7ncEB7mqLmsQO/Agt5Zkv2DUGxirGL3xWrJviU6ff605u2Yp3k75kmyD028qfZN9snja3dTtXLVjCwfAMqcExkntOzAMi3Zt0RL9i3R/jP7C+2v6FtRPev1VK96vdSzXk+FBoQaVCkAGIcACwBgLpcbwsMcNVetboW6eqzdY3qs3WPKycvR+uT19t5ZiYv15+E/lfBXghL+StBHGz+SJDWt0lQ31b5JN9W5STfWvlEVfSsa/AwAwLWknk/VyoMr9dv+3/Tbgd+05diWQvs93DzUsXpH9azXU73r92boNwCIAAsA4GyYo+aaeLh56Poa1+v6Gtdr3I3jlHo+Vb8n/a7f9v+mZQeWKT4lXjtO7NCOEzs0dd1UWWRR6+DW6la7m26qc5NuqHWDAr0DjX4aAOBU0rLS9EfSH1p2YJmWHVimjUc3Ks+WV+iYFlVbqEfdHupep7u61OrCxTcA4F8IsAAAKMPK+5RX34Z91bdhX0n2q1utOLBCyw4s02/7f9POkzu1KWWTNqVs0tt/vi13i7vahbZTl1pdFFEzQp3COqmyX2WDnwVgMmX4aqmwO33utOKS4rTy4EqtOLhCG45uuCCwql+xvrrX6a6b6tykrrW7qqp/VYOqBQDnwF9SAABQoLJfZd3Z9E7d2fROSdLRs0e1/MDygkAr8XSi1hxZozVH1uiNVW9IkppUbqKImhEFP3WC6nAFLJRtRl8t9bXXpJUrCc9KUfLZZMUlxen3g/YLZmw5tqXQlQIlqV6Ferqx1o3qVqebbqx1o8LKhxlULQA4J/6aAQCAiwoJCNGAFgM0oMUASVJSapKWH1iuuKQ4xSXFaefJnQU/H2/82H6fciGFAq2W1VrKw423HChDjL5a6qRJUmamMeFZGZBny9Puk7vtr4OH7KHVvyddl6TGlRurS80u6lKri26sfaNqBNYwoFoAcB28mwQAAFesZvmaGtRqkAa1GiTJPuRw1aFVBYHW+uT1Opp+VN/s+Ebf7PhGklTOq5zCa4Src1hnRdSMUMcaHVXOq5yRTwNGKStD64y+WqqR4ZkLysjO0LrkdVp1aJVWHVql1YdX669zfxU6xs3iptbBrRURZg/uu9TqwhVdAaCYueA7BgAAUFoq+1XWrY1u1a2NbpUknbOe0/rk9QU9E/5I+kOpWamK3Rer2H2xkiR3i7taB7dWx+od1aF6B7Wv3l6NKzeWm8XNyKeC0mD00LrSYvTVUvOH8BoRnjm5nLwcbT++XWuPrLX/JK/V9uPblWvLLXScr4evOtboqBtq3qCImhG6vsb1XOACAEoYARYAACg2vp6+uqHWDbqh1g2S7ENtth/fXhBoxSXFKSk1SRuObtCGoxv0/vr3JUkBXgFqF9pO7UPbq0P1DupQvYNqBNZgLi1XUxpD68zQy8voq6WOHVt4DiwUyWazaf+Z/X+HVUfWauPRjTqXc+6CY6sHVFfnmp3VqUYnda7ZWa2qtZKnu6cBVQNA2UWABQAASoybxU0tqrVQi2otNLz9cEnSodRDWnVoldYlr9O65HVan7xeZ7PPFlxePl81/2oFYVb70PZqX729KvpWNOqpuL7SCH5KY2hdWenldSljxkgvvnhlx5oh8CslxzOOa92RdQU9q9YdWadT505dcFygd2BBmJ6/rB5Y3YCKAQD/5Jp/nQAAgGmFlQ/TveXv1b3N75VkH7Kz88ROrUteV9ALYuvxrTqWcUw/7PlBP+z5oeC+9SrUKwi12oW2U6tqrRTgHWDUU3EtpRH8lMbQutKcQN0Vwh8XDfxSz6cqPiW+0OvKwdSDFxzn5e6l1sGt1SG0Q8GQ5oaVGjKkGQBMyMn+wgIAAFfj4eZR0Evr4TYPS7LPpRWfEq+1R9YWfADd+9deJZ5OVOLpRH257UtJkkUWNajUQG2C29h/QtqoVbVWxkye7OxhRmkEP6UxtK40J1B3hfDH6CsmFoOU9BRtOrpJm1LsPxuPbtS+0/suOM4iixpXblyoZ2fLai3l7eFtQNUAgKvlRO+qAABAWeHr6avwsHCFh4UXbDt97rTWJ68vGP6zIXmDjpw9oj2n9mjPqT2au31uwbHV/KupVXArtar2v5/gVmpUqVHJzlnj7GGG0VfOKy6lOYG6C4Q/zvT/fub8GW0/vl3bjm/T9hP25bbj23Qi80SRx9csX1PtQtupQ6i9Z1XbkLYq71O+lKsGABQXAiwAAFyJs/cCuoQKvhUUWS9SkfUiC7YdSz+m+JT4gp4X8Snx2ntqr45lHNPixMVanLi44Fgvdy81rdJUzao0U9MqTQt+6lWoJ3c392sv0NnDDKOvnFdcSnMCdScKfy7KhP/vGdkZ2nFiR6GQavuJ7TqcdrjI4y2yqFHlRoV6YrYJbqNKfpVKuXIAQElyjXe0AAAY7d/B0XPPGVOHs/cCukrVylVTr/q91Kt+r4JtGdkZ2nZ8mzYf26zNKZu1+dhmbTm2RWezzyo+JV7xKfGFzuHt7q1GlRvZA63K9lCrWdVmqleh3tX12HL2MMPoK+cZ4VoDXxOGP1fNwP/3rJws7T61u1BIte34Nu0/vV822Yq8T43AGmpetbmaV2muZlWbqXnV5mpSuYn8vfxLuXoAQGkjwAIAGM8Veg39Ozhyc5NatCj9Opy9F1Ax8PfyV8caHdWxRseCbXnjo3XgnfHaUk3aWUXa0b2ldoR4aOeJnTqXc05bjm3RlmNbCp3H081TDSs1LNRbq1mVZmpQqYG83L0ufGBXCDPKmmsNfMti6OeArJwsJfyVoJ0nd9qHAJ6wB1Z7T+1Vri23yPtU9a96QVDVtEpTBfkElW7xAADTcLJPBwAAl+QKvYb+HRytXm1MgOXsvYBKiNsfq1T3tFT3tNR/lySfatLixcrNy9XB1IPacWLHBT8Z1gxtP7Fd209sL3Qud4u76lSoo4aVGqphxYZqWKGeGv6yXg3WJapG++5y+/ln5wtgy6rLBb6uEK6Xkpy8HB1KPaSEvxK059Qe7f1rr/ac2qPdp3brwJkDyrPlFXm/IJ+gC4KqZlWaqYp/lVJ+BgAAs+MvMADAeK7Qa+jfwVF4+OXvUxLoBVS0iwR77m7uqluhrupWqKu+DfsWHJ5ny9Oh1EOFQ62T9mVaVpoS/kpQwl8J+lk///0Y7SXvnDjV/u801anfTnWC6qhOUB3VDqqtOhXstyv6VpTFYintZ4+LuVzg6wrhejGx2Ww6kXlC+0/v177T+7T/zN/L/af3Kyk16aK9qSQp0DtQjSs3LhRUNa/aXCHlQvidAABcEQIsAIDxXKHX0L+Do2eflRYvvvR9SgJDmop2lcGem8VNtYJqqVZQLfVp0Kdgu81mU/LZ5ILeJXtO7dGen2ZpT95J7asgZXlIu3VSuxN+KfK8AV4BBYFW7fJ/B1t1KtiDrkDvwL8Pfu01aeVKev6UpMu1C1cI169CRnbG38HUP4Kq/JAqw5pxyft7uXupboW6Bb0TG1RqoEaVGqlR5Uaq5l+NoAoAcE14JwQAMJ4r9Br6d3BktRpXCy5UTMGexWJR9cDqqh5YXV1rd7Vv/DNQio5WjsWmpPLS/lGDtb9fhPaftn/wP3DmgPaf2a+U9BSdzT6rrce3auvxrUWev6JvRdUuX1ve57y17M91qnU2RyHfxCrEdkDBI8YoJCBEAV4BBAHF5XLtwhXC9f+x5lqVfDZZh9MO61DaIfsy9VDB7YOpB3U84/glz2GRRaEBoapboa7qVKijukF1C0LYuhXqKiQgRG4Wt1J6RgCAsoYACwBgvJLuNcQ8NihJ/wtcPeLiVDciQnUv0r7OWc/pYOrBC4Kt/aftt0+dO6W/zv2lv879JUla3f6f954hTZshSfLz9FNwuWCFlAspWIYEhBTeFhCiKn5V5O7mXtLP3rWZPFy32WxKzUrV8YzjOpZ+TMlpyfrt5G9av3K9jmceV3J6so6kHVHy2WQdzzh+0Sv7/VOQT5A9oAr6O5iqU8G+rFW+lrw9vEvhmQEAcCHevQMAXB/z2BSNYK94XGEA6+vpq8aVG6tx5cZF7k/LStOBMweUcDJBP6/+WYELZinZL0cp5aSjtSspxduqtKw0ZVozte/0Pu07ve+Sj+dmcVNV/6pFBl3V/Kupkl8lVfarrEq+lVTJr5J8PHwcevourZSH5NpsNqVnp+v0+dM6fe60jmcct4dTGccKlsfSjxXalp2bfeGJDhd9fk83T1UPrK6wwDCFlQ9TjYAa9mVgDdUsX1N1guqogm+Fkn2SAAA4iHepAADXV8bmsbliBHumEugdqJbVWqpJxSZyT3DXzeG15blypdTi73AxIztDKekpSklP0dH0o/bl2aN/3/7f8njGceXZ8gqOvRJ+nn6q6FtRFX0rKtA7sOAnwCvgousB3hfuK8u9vvJseUrPTtfZrLP2ZfZZnT53WqfPn9aZ82cK3/5fSPXv25eaCP1iArwCVK1cNVXxqyLbWZta1G2h0MBQVQ+ortCAUIUGhCokIERV/asyxA8A4LQIsAAArs+F5rEpVgR75jZmjPTii4U2+Xv5q17FeqpXsd4l75qTl6MTGSeKDLqOph/VycyTBT+nMk8p15arTGumMq2ZOpx2ke47V8jP0++CoKucVzn5efrJ19NXvh72H28Pb3m7e8vT3VMebh7ydPvf8n/rl9uWv92iC+cDs8mmPFuecvNylWvLVW5ernLycgpu/3uZnZut7NxsWfOsys7NVlZOls7nnNf5nPPKys3SOes5ncs5V/Bv9M/bGdkZBWFVpjXzmv7t8nm6eaqCbwVV9a+qqv5VVc2/WuFluWoFt6v6V5Wvp68kyWq16ueff9bNfW6Wp6dnsdQCAIBZEGABAFyfyeexMQzBnsvycPNQSIB9yGAbtbnksXm2PKVlpemvc3/pVOYpnT5/WmezziotK01pWWk6m/337aLW07LSdDbrrLJysySpINhJ0ZX1/HJFbhY3BXgFKMA7QEE+QargU0EVfCv8fdvnf7d9K1y4z7eCfD18magfAIB/IcACALg+R+excfU5ogj2IHvYEuQTVDB5t6OycrIKwq1/B2Bns87qXM65gp5M56znlJVr7+WUk5dT8GPNs9qXudYLtv1zuzXPqpzcHFn/OiGdPy/5+EhBQRc8L3c3d7lb3OXu5i4PN4+C2/9eerl7Ffx4unnK28NbPm5e8t64Rb6HU+Rdo7b8buolP+9y8vXwlZ+nn/zcvOU7b6H8t+xSuZbtFPDE0yrnF6QArwD5ePgQQAEAUMxc6F04AADFzNXniCrlCarh2rw9vOXt4a3KfpVL5wEnTJDGR/+vB+F5Kfrp4m3PEyZI4z/+3/kPSpZu0riR/9o/275/wXrJEszvEwAAJYhZHAEAuBjmiAKuTU6OPejp2dO+zMkpvnOX9O/n5c7P6wMAAKWKAAsAgIuJiLDPDSUxRxTgiPxejLGx9mVMTPGdu6R/Py93fl4fAAAoVQwhBADgYpgjCrg2JdlLqaR/Py93fl4fAAAoVQRYZcC0adP0xhtvKCUlRa1atdJ7772nDh06XPT4b775Ri+99JIOHDigBg0a6LXXXtPNN99cihUDgEkwR9SFXH1iexSvkrzSZUn/fl7u/Lw+AABQqhhC6OLmzp2r0aNH6+WXX9bGjRvVqlUr9erVS8ePHy/y+FWrVmnAgAEaOnSoNm3apP79+6t///7atm1bKVcOADClkhwShtJXknNUSfaAMzpaioy0L+mlBAAAHESA5eLefvttDRs2TEOGDFHTpk01ffp0+fn56bPPPivy+HfeeUe9e/fWc889pyZNmmjixIm67rrrNHXq1FKuHABw1Uo6jJCYuNrVlHQgmd9LafFi+5LeegAAwEG8i3Bh2dnZ2rBhg8aOHVuwzc3NTT169NDq1auLvM/q1as1evToQtt69eqlhQsXXvRxsrKylJWVVbCelpYmSbJarbJardfwDIpPfh1mqQfOg7YDRxnSdl57zf5js0l//CG5uUljxhTvY3TpYj93/pCwLl0kI34/cnKkt96SVq+WwsOlZ591mXCkVNvOmjWSj0/hdV7vnBZ/s+Ao2g4cZaa2Y4YaULIsNlv+16hwNcnJyapevbpWrVql8PDwgu3PP/+8VqxYoTVr1lxwHy8vL82aNUsDBgwo2Pb+++9r/PjxOnbsWJGPEx0drfHjx1+wfc6cOfLz8yuGZwIAAAAAwMVlZmZq4MCBSk1NVWBgoNHloAS4xleVMNTYsWML9dpKS0tTWFiYevbsaZoXDqvVqtjYWEVGRsrT09PocuBEaDtwlCFt57XXpEmT/u4dNXZs8ffAMov+/aVly/5e79ZNukRvYWdS0HYeflie587ZN44ZUzLzR7lwT7ayiL9ZcBRtB44yU9vJHwkE18U7FBdWuXJlubu7X9Bz6tixYwoODi7yPsHBwVd1vCR5e3vL29v7gu2enp6Gv4j9mxlrgnOg7cBRpdp2XnhBysv7+wqBL7zgumFEx47SokV/h3UdO0ou9jvqee7c3wHW//2f9PLLJfAgntKLL/59dcnbbuPqki6Av1lwFG0HjjJD2zH68VHyeGfiwry8vNS2bVstXbpU/fv3lyTl5eVp6dKlGjlyZJH3CQ8P19KlS/X0008XbIuNjS00BBEAYFL5E2aXBfm9kfLDOq5ud23yJ3O32aQlS+zbykpbAgAAToEAy8WNHj1agwcPVrt27dShQwdNmTJFGRkZGjJkiCRp0KBBql69uiZNmiRJGjVqlG688Ua99dZbuuWWW/TVV19p/fr1+uijj4x8GgAAFFaWwjpJevDBkj1/aV9dMr/H1z8DSHp8AQCAS+Cdgou79957deLECY0bN04pKSlq3bq1fvnlF1WrVk2SlJSUJDc3t4LjO3XqpDlz5ujFF19UVFSUGjRooIULF6p58+ZGPQUAAMqmqChp5crS6WEWEWHveZU/JDMiomQfb+JEacIE++3YWCk3VyrigjAAAAD5CLDKgJEjR150yODy5csv2Hb33Xfr7rvvLuGqAADAJY0ZY5+fqjhdrOdTaQ/JnD37wnUCLAAAcAkEWAAAAGXFxea6KmtDMgEAgNNxu/whAAAAJpKTYx9+1rOnfZmTY3RFzqO057q6mH/P6VXSc3wBAACnRw8sAADgXLhinuNKe66ri3nxRcnNjatIAgCAK0aABQAAnItZehE5o9Ke6+piGLIIAACuEgEWAADO7GKTcrsyo3oRucK/NcERAABwUk72rgsAABRSFofTGdWLqCz+WwMAAJgEARYAAM6sLA6nM6oXUVn8twYAADAJrkIIAIAzi4iwD6OTjJ2Uuyzg3xoAAMAw9MACAMCZmWVS7rKAf2sAAADDEGABAODMmJS79PBvDQAAYBiGEAIA4AxycqQJE6SePe3LnByjKwIAAABKDT2wAABwBlwBr+zIDyf795c6drQPVfTgLRsAACjbeDcEAIAz4Ap4Zcdbb0ktWkjLlkmLFtm3EVYCAIAyjiGEAAA4A66AV3asXv33bWcPKxn6CgAAigk9sAAAcAZcAa/sCA//+7azh5UMfQUAAMWEAAsAAGfAFfDKjmeflRYvlrp1+3sOLGfF0FcAAFBMCLAAAADMJH/C9oULJU9PQ0u5ZhER9p5XNpvz9yYDAACGIsACAABAyWDoKwAAKCYEWAAAwPnl5NjnW/pnUOLB2xzDMfQVAAAUE97ZAQAA58dk4QAAAC7NzegCAAAArhmThQMAALg0AiwAAFB6cnKkCROknj3ty5yc4jlvRIR9knCJycIBAABcEEMIAQBA6SmpoX5MFg4AAODSCLAAAEDpKamhfkwWDgAA4NIYQggAAEoPQ/0AAADgAHpgAQCA0sNQPwAAADiAAAsAAJQehvoBAADAAQwhBAAAAAAAgKkRYAEAAAAAAMDUCLAAAIBzysmRJkyQeva0L3NyjK4IAAAAJYQ5sAAAgHOKiZGioyWbTVqyxL6N+bUAAABcEj2wAACAc4qLs4dXkn0ZF2dsPQAAACgxBFgAAMA5RURIFov9tsViXwcAAIBLIsByUQcOHNDQoUNVp04d+fr6ql69enr55ZeVnZ19yft17dpVFoul0M/jjz9eSlUDAHAVoqLsQwgjI+3LqCijKwIAAEAJYQ4sF7Vr1y7l5eXpww8/VP369bVt2zYNGzZMGRkZevPNNy9532HDhmnChAkF635+fiVdLgAAV8/DgzmvAAAAyggCLBfVu3dv9e7du2C9bt262r17tz744IPLBlh+fn4KDg4u6RIBAAAAAACuCAFWGZKamqqKFSte9rgvvvhCs2fPVnBwsPr166eXXnrpkr2wsrKylJWVVbCelpYmSbJarbJarddeeDHIr8Ms9cB50HbgKNoOHEXbgaNoO3AUbQeOMlPbMUMNKFkWmy3/8j1wZQkJCWrbtq3efPNNDRs27KLHffTRR6pVq5ZCQ0O1ZcsWjRkzRh06dND8+fMvep/o6GiNHz/+gu1z5sxh+CEAAAAAoMRlZmZq4MCBSk1NVWBgoNHloAQQYDmZF154Qa+99tolj9m5c6caN25csH7kyBHdeOON6tq1qz755JOrerzffvtN3bt3V0JCgurVq1fkMUX1wAoLC9PJkydN88JhtVoVGxuryMhIeXp6Gl0OnAhtB46i7cBRtB04irYDR9F24CgztZ20tDRVrlyZAMuFMYTQyTz77LN66KGHLnlM3bp1C24nJyerW7du6tSpkz766KOrfryOHTtK0iUDLG9vb3l7e1+w3dPT0/AXsX8zY01wDrQdOIq2A0fRduAo2g4cRduBo8zQdox+fJQ8AiwnU6VKFVWpUuWKjj1y5Ii6deumtm3basaMGXJzc7vqx4uPj5ckhYSEXPV9AQAAAAAAisPVJxpwCkeOHFHXrl1Vs2ZNvfnmmzpx4oRSUlKUkpJS6JjGjRtr7dq1kqTExERNnDhRGzZs0IEDB/T9999r0KBB6tKli1q2bGnUUwEAAAAAAGUcPbBcVGxsrBISEpSQkKAaNWoU2pc/7ZnVatXu3buVmZkpSfLy8tKSJUs0ZcoUZWRkKCwsTHfeeadefPHFUq8fAAAAAAAgHwGWi3rooYcuO1dW7dq19c85/MPCwrRixYoSrgwAAAAAAODqMIQQAAAAAAAApkaABQAAAAAAAFMjwAIAAAAAAICpMQcWil3+vFppaWkGV/I3q9WqzMxMpaWlydPT0+hy4ERoO3AUbQeOou3AUbQdOIq2A0eZqe3kf/785zzPcC0EWCh2Z8+elWSfFB4AAAAAgNJy9uxZlS9f3ugyUAIsNuJJFLO8vDwlJycrICBAFovF6HIk2dP4sLAwHTp0SIGBgUaXAydC24GjaDtwFG0HjqLtwFG0HTjKTG3HZrPp7NmzCg0NlZsbsyW5Inpgodi5ubmpRo0aRpdRpMDAQMNfWOGcaDtwFG0HjqLtwFG0HTiKtgNHmaXt0PPKtRFLAgAAAAAAwNQIsAAAAAAAAGBqBFgoE7y9vfXyyy/L29vb6FLgZGg7cBRtB46i7cBRtB04irYDR9F2UJqYxB0AAAAAAACmRg8sAAAAAAAAmBoBFgAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAUyPAQplz6623qmbNmvLx8VFISIgefPBBJScnG10WTO7AgQMaOnSo6tSpI19fX9WrV08vv/yysrOzjS4NTuC///2vOnXqJD8/PwUFBRldDkxs2rRpql27tnx8fNSxY0etXbvW6JLgBFauXKl+/fopNDRUFotFCxcuNLokOIFJkyapffv2CggIUNWqVdW/f3/t3r3b6LLgBD744AO1bNlSgYGBCgwMVHh4uBYtWmR0WSgDCLBQ5nTr1k1ff/21du/erW+//VaJiYm66667jC4LJrdr1y7l5eXpww8/1Pbt2zV58mRNnz5dUVFRRpcGJ5Cdna27775bw4cPN7oUmNjcuXM1evRovfzyy9q4caNatWqlXr166fjx40aXBpPLyMhQq1atNG3aNKNLgRNZsWKFRowYoT///FOxsbGyWq3q2bOnMjIyjC4NJlejRg29+uqr2rBhg9avX6+bbrpJt912m7Zv3250aXBxFpvNZjO6CMBI33//vfr376+srCx5enoaXQ6cyBtvvKEPPvhA+/btM7oUOImZM2fq6aef1pkzZ4wuBSbUsWNHtW/fXlOnTpUk5eXlKSwsTE8++aReeOEFg6uDs7BYLFqwYIH69+9vdClwMidOnFDVqlW1YsUKdenSxehy4GQqVqyoN954Q0OHDjW6FLgwemChTPvrr7/0xRdfqFOnToRXuGqpqamqWLGi0WUAcAHZ2dnasGGDevToUbDNzc1NPXr00OrVqw2sDEBZkZqaKkm8t8FVyc3N1VdffaWMjAyFh4cbXQ5cHAEWyqQxY8bI399flSpVUlJSkr777jujS4KTSUhI0HvvvafHHnvM6FIAuICTJ08qNzdX1apVK7S9WrVqSklJMagqAGVFXl6enn76aXXu3FnNmzc3uhw4ga1bt6pcuXLy9vbW448/rgULFqhp06ZGlwUXR4AFl/DCCy/IYrFc8mfXrl0Fxz/33HPatGmTFi9eLHd3dw0aNEiMpi2brrbtSNKRI0fUu3dv3X333Ro2bJhBlcNojrQdAADMaMSIEdq2bZu++uoro0uBk2jUqJHi4+O1Zs0aDR8+XIMHD9aOHTuMLgsujjmw4BJOnDihU6dOXfKYunXrysvL64Lthw8fVlhYmFatWkW31zLoattOcnKyunbtquuvv14zZ86UmxvfA5RVjrzuMAcWLiY7O1t+fn6aN29eobmLBg8erDNnztBTGFeMObBwtUaOHKnvvvtOK1euVJ06dYwuB06qR48eqlevnj788EOjS4EL8zC6AKA4VKlSRVWqVHHovnl5eZKkrKys4iwJTuJq2s6RI0fUrVs3tW3bVjNmzCC8KuOu5XUH+DcvLy+1bdtWS5cuLQge8vLytHTpUo0cOdLY4gC4JJvNpieffFILFizQ8uXLCa9wTfLy8vg8hRJHgIUyZc2aNVq3bp0iIiJUoUIFJSYm6qWXXlK9evXofYVLOnLkiLp27apatWrpzTff1IkTJwr2BQcHG1gZnEFSUpL++usvJSUlKTc3V/Hx8ZKk+vXrq1y5csYWB9MYPXq0Bg8erHbt2qlDhw6aMmWKMjIyNGTIEKNLg8mlp6crISGhYH3//v2Kj49XxYoVVbNmTQMrg5mNGDFCc+bM0XfffaeAgICC+fbKly8vX19fg6uDmY0dO1Z9+vRRzZo1dfbsWc2ZM0fLly/Xr7/+anRpcHEMIUSZsnXrVo0aNUqbN29WRkaGQkJC1Lt3b7344ouqXr260eXBxGbOnHnRD5G8jOJyHnroIc2aNeuC7cuWLVPXrl1LvyCY1tSpU/XGG28oJSVFrVu31rvvvquOHTsaXRZMbvny5erWrdsF2wcPHqyZM2eWfkFwChaLpcjtM2bM0EMPPVS6xcCpDB06VEuXLtXRo0dVvnx5tWzZUmPGjFFkZKTRpcHFEWABAAAAAADA1JjABQAAAAAAAKZGgAUAAAAAAABTI8ACAAAAAACAqRFgAQAAAAAAwNQIsAAAAAAAAGBqBFgAAAAAAAAwNQIsAAAAAAAAmBoBFgAAAAAAAEyNAAsAAAAAAACmRoAFAAAAAAAAUyPAAgAAMNCJEycUHBysmJiYgm2rVq2Sl5eXli5damBlAAAA5mGx2Ww2o4sAAAAoy37++Wf1799fq1atUqNGjdS6dWvddtttevvtt40uDQAAwBQIsAAAAExgxIgRWrJkidq1a6etW7dq3bp18vb2NrosAAAAUyDAAgAAMIFz586pefPmOnTokDZs2KAWLVoYXRIAAIBpMAcWAACACSQmJio5OVl5eXk6cOCA0eUAAACYCj2wAAAADJadna0OHTqodevWatSokaZMmaKtW7eqatWqRpcGAABgCgRYAAAABnvuuec0b948bd68WeXKldONN96o8uXL68cffzS6NAAAAFNgCCEAAICBli9frilTpujzzz9XYGCg3Nzc9Pnnn+v333/XBx98YHR5AAAApkAPLAAAAAAAAJgaPbAAAAAAAABgagRYAAAAAAAAMDUCLAAAAAAAAJgaARYAAAAAAABMjQALAAAAAAAApkaABQAAAAAAAFMjwAIAAAAAAICpEWABAAAAAADA1AiwAAAAAAAAYGoEWAAAAAAAADA1AiwAAAAAAACYGgEWAAAAAAAATI0ACwAAAAAAAKZGgAUAAAAAAABT+/8KYu/nUAr4BgAAAABJRU5ErkJggg==' width=1200.0/>\n",
       "            </div>\n",
       "        "
      ],
      "text/plain": [
       "Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib ipympl\n",
    "from ipywidgets import *\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "x = xs_sorted\n",
    "\n",
    "def f(x, i):\n",
    "    return model(x, a_list[i], b_list[i], c_list[i])\n",
    "\n",
    "fig = plt.figure(figsize=(12, 5))\n",
    "ax = fig.add_subplot(1, 1, 1)\n",
    "ax.set_xlabel(\"x\")\n",
    "ax.set_ylabel(\"y\")\n",
    "\n",
    "ax.scatter(xs, ys, label=\"gt\", s=5, c=\"r\")\n",
    "line, = ax.plot(x, f(x, i), label=\"pred\", c=\"g\")\n",
    "\n",
    "ax.legend()\n",
    "ax.grid(True)\n",
    "\n",
    "\n",
    "def update(i=0):\n",
    "    line.set_ydata(f(x, i))\n",
    "    fig.canvas.draw_idle()\n",
    "    \n",
    "interact(update, i=(0,num_epochs-1,1));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(12, 5))\n",
    "\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.plot(loss_list)\n",
    "plt.xlabel(\"epoch\")\n",
    "plt.ylabel(\"loss\")\n",
    "plt.grid(True)\n",
    "plt.yscale(\"log\")\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "for p_name, p_gt, p_list, color in zip((\"a\", \"b\", \"c\"), (_a, _b, _c), (a_list, b_list, c_list), (\"r\", \"g\", \"b\")):\n",
    "    plt.hlines(y=p_gt, xmin=0, xmax=num_epochs, linestyles=\"dashed\", colors=color)\n",
    "    plt.scatter(range(len(p_list)), p_list, label=p_name, s=1, c=color)\n",
    "plt.xlabel(\"epoch\")\n",
    "plt.ylabel(\"param value\")\n",
    "plt.legend()\n",
    "plt.grid(True)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Вопрос:** если бы это была \"настоящая\" задача, что следовало бы изменить в данном пайплайне?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Переопределение `backward`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "g616ocbHv78t"
   },
   "source": [
    "Что, если нам хочется релизовать кстомный градиент для произвольной функции. \n",
    "\n",
    "Зачем?\n",
    "\n",
    " - Мы можем знать лучший способ посчитать градиент, чем делать бэкпроп для суперпозиции элементарных функций\n",
    " - Можем реализовать численно более устойчивый метод\n",
    " - Можем использовать функции из внешних библиотек\n",
    " - Использовать недифференцируемые функции?.."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nkYTgy6ZO_Ru"
   },
   "source": [
    "Рассмотрим сигмоиду:\n",
    "\n",
    "$$ \n",
    "  \\sigma(x) = \\frac{1}{1+e^{-x}}\n",
    "$$\n",
    "\n",
    "Если честно распишем суперпозицию функций, то получим:\n",
    "\n",
    "$$\n",
    "  \\sigma(x) = f_1 \\odot f_2  \\odot f_3 \\odot f_4(x), where \n",
    "$$\n",
    "\n",
    "$$\n",
    "f_1 = \\frac{1}{u}, f_2 = 1 + u, f_3 = \\exp(u), f_4 = -u\n",
    "$$\n",
    "\n",
    "Тогда:\n",
    "\n",
    "$$\n",
    "\\frac{\\partial \\sigma}{\\partial x} = \\frac{\\partial \\sigma}{\\partial f_2}\\frac{\\partial f_2}{\\partial f_3}\n",
    "\\frac{\\partial f_3}{\\partial f_4}\n",
    "\\frac{\\partial f_4}{\\partial x}\n",
    "$$\n",
    "\n",
    "Но зная как устроена производная можно упростить:\n",
    "\n",
    "$$\n",
    "\\frac{\\partial \\sigma}{\\partial x} = \\sigma(x)(1 - \\sigma(x))\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DRpFQBmuR-V_"
   },
   "source": [
    "Вручную задать градиени функции в библиотеке PyTorch можно создав дочерний класс от [`torch.autograd.Function`](https://pytorch.org/docs/stable/notes/extending.html#extending-torch-autograd).\n",
    "\n",
    "**NB**: Для того, чтобы сделать `backward()`, требуется знать результат `forward()`. Для этого у переменной контекста есть метод [`save_for_backward()`](https://pytorch.org/docs/stable/generated/torch.autograd.function.FunctionCtx.save_for_backward.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 244,
     "status": "ok",
     "timestamp": 1631815233191,
     "user": {
      "displayName": "Filipp Nikitin",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gib7R6zFSdJ1Mn4MWbQTRjNvdPBqipg3uQtaNBFNg=s64",
      "userId": "00003629133353694743"
     },
     "user_tz": -180
    },
    "id": "okxelJTDO9_L"
   },
   "outputs": [],
   "source": [
    "class MySigmoid(torch.autograd.Function):\n",
    "    @staticmethod\n",
    "    def forward(ctx, x):\n",
    "        # YOUR CODE HERE\n",
    "        \n",
    "        # val = ...\n",
    "        \n",
    "        # END OF YOUR CODE\n",
    "        return val\n",
    "    \n",
    "    @staticmethod\n",
    "    def backward(ctx, grad_output):\n",
    "        # YOUR CODE HERE\n",
    "        \n",
    "        # val, = ...\n",
    "        # grad = ...\n",
    "        \n",
    "        # END OF YOUR CODE\n",
    "        return grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 211,
     "status": "ok",
     "timestamp": 1631815238628,
     "user": {
      "displayName": "Filipp Nikitin",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gib7R6zFSdJ1Mn4MWbQTRjNvdPBqipg3uQtaNBFNg=s64",
      "userId": "00003629133353694743"
     },
     "user_tz": -180
    },
    "id": "n89TIozkVt27"
   },
   "outputs": [],
   "source": [
    "from torch.autograd import gradcheck"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 227,
     "status": "ok",
     "timestamp": 1631815280552,
     "user": {
      "displayName": "Filipp Nikitin",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gib7R6zFSdJ1Mn4MWbQTRjNvdPBqipg3uQtaNBFNg=s64",
      "userId": "00003629133353694743"
     },
     "user_tz": -180
    },
    "id": "42MVi8npWlAR",
    "outputId": "15653e25-a437-4e4b-b093-5c610ee3e56f"
   },
   "outputs": [],
   "source": [
    "sigmoid = MySigmoid.apply\n",
    "x = torch.rand(2, requires_grad=True)\n",
    "print(gradcheck(sigmoid, x, eps=1e-4, atol=1e-3))\n",
    "\n",
    "# be sure to use double for better approximation\n",
    "x = torch.rand(2, requires_grad=True).double()\n",
    "print(gradcheck(sigmoid, x, eps=1e-6, atol=1e-4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aXYgzHcz7E_1"
   },
   "source": [
    "PyTorch умеет считать матрицу Якоби или матрицу Гессе для заданной функции."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 229,
     "status": "ok",
     "timestamp": 1631815491399,
     "user": {
      "displayName": "Filipp Nikitin",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gib7R6zFSdJ1Mn4MWbQTRjNvdPBqipg3uQtaNBFNg=s64",
      "userId": "00003629133353694743"
     },
     "user_tz": -180
    },
    "id": "a65TCapsZBlf"
   },
   "outputs": [],
   "source": [
    "from torch.autograd.functional import hessian, jacobian"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 246,
     "status": "ok",
     "timestamp": 1631815519585,
     "user": {
      "displayName": "Filipp Nikitin",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gib7R6zFSdJ1Mn4MWbQTRjNvdPBqipg3uQtaNBFNg=s64",
      "userId": "00003629133353694743"
     },
     "user_tz": -180
    },
    "id": "nS-DSoplZ6E_",
    "outputId": "db8a907c-8365-451f-d6f0-d1d186a54d97"
   },
   "outputs": [],
   "source": [
    "jacobian(sigmoid, x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 215,
     "status": "ok",
     "timestamp": 1631815472226,
     "user": {
      "displayName": "Filipp Nikitin",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gib7R6zFSdJ1Mn4MWbQTRjNvdPBqipg3uQtaNBFNg=s64",
      "userId": "00003629133353694743"
     },
     "user_tz": -180
    },
    "id": "z_nXiJsFXoDR"
   },
   "outputs": [],
   "source": [
    "def sum_sigmoid(x):\n",
    "    return torch.sum(sigmoid(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hessian(sum_sigmoid, x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vLPPlL6Sj39B"
   },
   "source": [
    "## Итоги\n",
    "\n",
    "* Узнали, что такое `pytorch` и как в нем работать с тензорами\n",
    "* Немного погрели GPU, запустив вычисления на нем\n",
    "* Решили пример задачи, используя только библиотеку `pytorch`\n",
    "* Написали собственную реализацию сигмоидальной нелинейности"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Что еще почитать:\n",
    "* [A gentle introduction to `torch.autograd`](https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html)\n",
    "* [Extending pytorch](https://pytorch.org/docs/stable/notes/extending.html#extending-torch-autograd)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "В следующий раз: \n",
    "* (наконец-то) нейросети на pytorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "authorship_tag": "ABX9TyM3x5e0hXQ/KRZ2lfc2ew+0",
   "name": "02. PyTorch autograd (classwork).ipynb",
   "provenance": [
    {
     "file_id": "1ZysOsvyIEv_yWYw9vvt8yWt5Q0MgRA57",
     "timestamp": 1631790599254
    }
   ]
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}