 Building a perceptron.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Building a perceptron\n",
    "\n",
    "\n",
    "\n",
    "This post is mostly for my own reference so I can find my notes and explanation to myself on this issue easily. It might have some value to others.\n",
    "\n",
    "Sigmoid Function\n",
    "The sigmoid function is one of many possible functions that are used as a nonlinear activation function between layers of a neural network. \n",
    "I think the most appealing property that makes sigmoid fuction so usefull is that, it gives output value between 0 and 1, with is usefull for binary classification. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Input values $x_1, x_2,x_3$ synapse weigths $w_1.w_2.w_3$ neuron $  f(\\displaystyle\\sum_{i=1}^n x_i w_i)$ output $\\hat{y}$ <br>\n",
    " So in this case $(\\displaystyle\\sum_{i=1}^n x_i w_i)=\\displaystyle\\sum_{i=1}^3 x_1 w_1 + x_2 w_2 +x_3 w_3 $ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Sigmoid  and sigmoid derivative plot\n",
    "The reason why calculating the derivative of this function is important, is because the learning process for neural networks involves making small changes to parameters, proportional to the partial derivatives of those parameter values, and relative to the loss function. This requires stepping backwards through the layers, applying the chain rule along each step. So, one of those components that require calculation, is the derivative of the activation function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 227,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/povilassabaliauskas/anaconda3/envs/tensorflow/lib/python3.6/site-packages/ipykernel_launcher.py:30: UserWarning: Matplotlib is currently using module://ipykernel.pylab.backend_inline, which is a non-GUI backend, so cannot show the figure.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1MAAAEvCAYAAACgx5sPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdd3hUVf4/8PedPpMeUkgmjSQktBDBJIJIEwSlRKIirJUv67KyrmVVxLUgrj8VFXfXFWVl7QUQpQrIUqQIC2IoSk1CSC+k1+kz9/fHQGRIJSSZTPJ+PU+ezJxz5+aTKDPznnPuOYIoiiAiIiIiIqKrI3F2AURERERERK6IYYqIiIiIiKgdGKaIiIiIiIjagWGKiIiIiIioHRimiIiIiIiI2oFhioiIiIiIqB1krfRz3XQiInKqW2+9Fdu2bXN2GURE1HsJzXVwZIqIiLq1srIyZ5dARETUJIYpIiIiIiKidmCYIiIiIiIiageGKSIiIiIionZgmCIiIiIiImoHhikiIiIiIqJ2YJgiIiIiIiJqB4YpIiLqEHPnzkVAQACGDBnSZL8oinjssccQHR2NoUOH4ujRo11cIRERUcfqMWHqoYcewunTpzv1Z0yZMgVVVVWN2hcvXoylS5d26s8mIuru5syZ0+Lmut9//z0yMjKQkZGBFStWYP78+V1YHRERUceTObuAjvLhhx92+s/YunVrp/8MIiJXNWbMGGRnZzfbv3HjRjzwwAMQBAEjRoxAVVUVioqKEBQU1HVFEhH1ElabCLNVhMVqg9Vmv28VRft3mwiLTfyt3aGt5WN+62vc7vh4+4wEqwjYLv5smw2wiSJsV7RZRRGiaD+fTRTxzG0R8PNQOPtP2CYuGabq6+tx9913Iz8/H1arFS+++CKWL1+OpUuXIiEhAR999BHeeOMNeHt7Iz4+HkqlEsuWLcOcOXOgVqtx7NgxlJSU4OOPP8bnn3+OgwcP4oYbbsCnn34KAFi1ahVee+01iKKIqVOn4o033gAAREREIDU1FX5+fnj11Vfx2WefISAgAKGhobj++uud+BchIur+CgoKEBoa2nA/JCQEBQUFDFNE5PIsVhFGiw1Giw0miw1Gs63hvtEiNm4z/9ZusthguRh87F82WC67bb7itqXRMY37zFZ7YHFVj0wIhZ+zi2ijdoepYS8d6sg6Gjn28ohm+7Zt24bg4GBs2bIFAFBdXY3ly5cDAAoLC/HKK6/g6NGj8PDwwM0334z4+PiGx1ZWVuLgwYPYtGkTkpOTceDAAXz44YdITEzE8ePHERAQgIULF+LIkSPw8fHBpEmTsGHDBsyYMaPhHEeOHMHq1atx/PhxWCwWDB8+nGGKiKgDrVixAitWrAAAlJaWOrkaInJ1JosNOpMVepMNerMNepPV/mW2XWyzQme0f790X2+yORyjM1lhMF8ehi4GJ4t95Ic6js2F/p4uOTIVFxeHp556CgsXLsS0adMwevTohr7Dhw9j7Nix8PX1BQDMnDkT6enpDf3Tp0+HIAiIi4tDYGAg4uLiAACDBw9GdnY2cnJyMG7cOPj7+wMA7r33Xuzbt88hTP34449ISUmBRqMBACQnJ3f670xE5Oq0Wi3y8vIa7ufn50Or1TZ57Lx58zBv3jwAQEJCQpfUR0TdjyiKMJhtqNZbUGuwot5oRa3BgnqjFXUX79dd3ma0NvTVGa2oM9rbTRYXHqbphWyi6/z3cskwFRMTg6NHj2Lr1q144YUXMGHChDY/VqlUAgAkEknD7Uv3LRYL5HJ5h9dLRET2D56WLVuG2bNn46effoKXlxen+BH1IiaLDZX1ZlTpLKjWX/y6eLtGf1nbZfdr9BYGoXYSBEAuFSCVCJBJHL9LG+7D4f7lx0gkaHT8b/1odPzl9yUCIJEIkAoCBMF+rKTh++X99u8NtwX7zw3wdI3rpQAXDVOFhYXw9fXFfffdB29vb4fFJxITE/HEE0+gsrISHh4eWLt2bcPoU1skJSXhscceQ1lZGXx8fLBq1So8+uijDseMGTMGc+bMwV//+ldYLBZ89913+OMf/9hhvx8RkSv63e9+hz179qCsrAwhISF4+eWXYTabAQAPP/wwpkyZgq1btyI6OhoajQaffPKJkysmomshiiJqDVZU1JtRUW9GeZ0ZlZfdrqi3oLLejPKLbXUGq7NL7jQSAVDKJVDKJFDI7N+VMgFK+eX37W0KmeSKYwXIpRLIpQJkUvttmUSAXCZALv2tz97/2235Fbdll46R2O9LJYAgCM7+0/R47Q5TLV3T1NlOnDiBBQsWQCKRQC6XY/ny5Xj66acB2KeRPPfcc0hKSoKvry8GDBgALy+vNp87KCgIS5Yswfjx4xsWoLj99tsdjhk+fDhmzZqF+Ph4BAQEIDExsUN/PyIiV7Rq1aoW+wVBwHvvvddF1RDRtTBZbCitNaGkxoSSWhNKaswO90trTCitNcHoIqNGUgmgUUihVkihlkugUUihUkiglkugVkihUUiglkuhVkgajlEr7Pc1l9+X/xaElBdDkUJmDzAMLr2TILY8J9E1/oVcoa6uDu7u7rBYLEhJScHcuXORkpLi7LKIiKgdEhISkJqa6uwyiHoMURRRpbOgsMqIomojCiuN9ttVRhTX2ANTlc7i7DIbyKUCvDQyeKhkcFdK4aGSwk0phbtSZv+uksL94vdL7Y73pVDJJQw7dC2a/Z/HJaf5tWbx4sXYuXMnDAYDJk2a5LB4BBEREVFPZzDbkFdhQG65AfkVht+CU5X9S2/q+uXSJALg4yaHt0YGL/XFr4u3PZu7r5YxCFG31iNHpoiIqOfgyBRR04xmG/IrDcitMCCv3B6cci8GqAs1pi6pQa2QoI+bHD5ucvRxl8PX7bIvd5nDfS+1DBIJQxG5pN41MkVERETUU+hNVmSXGZBZqkNmiR7nS3Q4X6pHQZURnbWCtEQA+rjLEeCpQICHAv4eCgR4KuB/8b69XQ53Fd9KUu/GfwFERERE3YDFKiK7TI+04npkluiRWWoPTp0RmtQKCYK9lQj2ViLIy/492EeJvl4KBHoq4esmh0zKUSSi1jBMEREREXUxvcmK9As6pBXVI61Yh7NF9ThXouuwPZUkAhDkrUSorwphvipofZQIuhiegr2V8NbIeB0SUQdgmCIiIiLqREazDWeL63Eyvw4n8uuQVlyPnHLDNY82CQIQ6KlAWB97YArro264rfVRQiGTdMwvQETN6jFh6qGHHsKTTz6JQYMGddrPmDJlClauXAlvb2+H9sWLF8Pd3b1hr6vmuLu7o66urkN+ZmuqqqqwcuVK/OlPfwJg3+j4sccew7fffntV5yEiIqK2E0URuRUGnMivawhP6Rd0sFjbn5wEAdB6KxEZoEGUvxpRAWpE+msQ4aeCWiHtwOqJ6GpxNb8O0BlhShRFiKIIiaR9nyplZ2dj2rRpOHnyZLseT0TUXXA1P+rOzBYbThXW42hODY7m1OJkfh2q9e3fo8nPXY7YIDf0D9QgKkCNKIYmou6g2TmxLjf+W19fj6lTpyI+Ph5DhgzB119/DQAYN25cw4vtRx99hJiYGCQlJeEPf/gD/vznPwMA5syZg/nz52PEiBGIjIzEnj17MHfuXAwcOBBz5sxp+BmrVq1CXFwchgwZgoULFza0R0REoKysDADw6quvIiYmBjfddBPS0tKarDUrKwsjR45EXFwcXnjhBYe+t956C4mJiRg6dCheeuklAPYAFBsbiwceeABDhgxBXl5ew8989tln8d577zU8fvHixVi6dCnq6uowYcIEDB8+HHFxcdi4cSMA4Nlnn0VmZiauu+46LFiwANnZ2RgyZAgAYMSIETh16lTDuS797err6zF37lwkJSVh2LBhDeciIiIiO73JisPnq/Hv3XmY9+lpjH79Z/zfR6fw7s48HMiouqogFdZHhUmDffHnCaFYdt8A7Hh6OHYsuB7L7huAx28Jw7R4fwwMdmOQIurG2j3Nb3lkXkfW0cj886FNtm/btg3BwcHYsmULAKC6utqhv7CwEK+88gqOHj0KDw8P3HzzzYiPj2/or6ysxMGDB7Fp0yYkJyfjwIED+PDDD5GYmIjjx48jICAACxcuxJEjR+Dj44NJkyZhw4YNDhv/HjlyBKtXr8bx48dhsVgwfPhwXH/99Y1qffzxxzF//nw88MADDkFo+/btyMjIwOHDhyGKIpKTk7Fv3z6EhYUhIyMDn332GUaMGOFwrlmzZuGJJ57AI488AgBYs2YN/vvf/0KlUmH9+vXw9PREWVkZRowYgeTkZCxZsgQnT57E8ePHAdiD2uXnWrNmDV5++WUUFRWhqKgICQkJeO6553DzzTfj448/RlVVFZKSkjBx4kS4ubm15T8ZERFRj2O22nAivw4Hz1XjcFY1ThfWt2vKXrC3EkNC3BGndcdgrRti+rrBTcmQROTqXO6aqbi4ODz11FNYuHAhpk2bhtGjRzv0Hz58GGPHjoWvry8AYObMmUhPT2/onz59OgRBQFxcHAIDAxEXFwcAGDx4MLKzs5GTk4Nx48bB398fAHDvvfdi3759DmHqxx9/REpKCjQaDQAgOTm5yVoPHDiAtWvXAgDuv//+hlGu7du3Y/v27Rg2bBgAoK6uDhkZGQgLC0N4eHijIAUAw4YNQ0lJCQoLC1FaWgofHx+EhobCbDbjueeew759+yCRSFBQUIALFy60+De8++67MWnSJLz88stYs2YN7rrrroa6Nm3ahKVLlwIADAYDcnNzMXDgwBbPR0RE1FOIoojsMgMOZVbjUGYVUrNroDPZruoc7kopBmndEKd1R1yoB4Zo3dDHXdFJFRORM7lcmIqJicHRo0exdetWvPDCC5gwYQIWLVrU5scrlUoAgEQiabh96b7FYoFcLu/QeptadlQURfz1r3/FH//4R4f27OzsFkeBZs6ciW+//RbFxcWYNWsWAOCrr75CaWkpjhw5ArlcjoiICBgMhhZr0mq16NOnD3799Vd8/fXX+Pe//91Q19q1axEbG3u1vyYREZHLqjNYcDCzGgcyqvDT+WoUV5uu6vEBngpcH+6B4eGeGBbugX5+akgkXHacqDdwuWumCgsLodFocN9992HBggU4evSoQ39iYiL27t2LyspKWCyWhpGhtkpKSsLevXtRVlYGq9WKVatWYezYsQ7HjBkzBhs2bIBer0dtbS2+++67Js81atQorF69GoA99FwyefJkfPzxxw2LURQUFKCkpKTV2mbNmoXVq1fj22+/xcyZMwHYpzkGBARALpdj9+7dyMnJAQB4eHigtra2xXO9+eabqK6uxtChQxvqevfdd3FpUZJjx461WhMREZErKqw0YNWhYsz//AzGv3kEz6zJwMZjpW0KUqG+Stw+zB8vp0Rh8xPXYduTw/DaXf1xV2IgogI0DFJEvUi7R6aau6aps504cQILFiyARCKBXC7H8uXLHfq1Wi2ee+45JCUlwdfXFwMGDICXl1ebzx8UFIQlS5Zg/PjxEEURU6dOxe233+5wzPDhwzFr1izEx8cjICAAiYmJTZ7rnXfewT333IM33njD4RyTJk3CmTNnMHLkSAD2Vf6+/PJLSKUtz50ePHgwamtrodVqERQUBMA+DXH69OmIi4tDQkICBgwYAADo06cPRo0ahSFDhuC2225ruNbqkrvuuguPP/44XnzxxYa2F198EU888QSGDh0Km82Gfv36YfPmzW38yxEREXVfNpuIkwV12JtWiX1plThXom/zY/u4yzEi0gs3RHnhhkgvBHhyyh4R2fXIpdHr6urg7u4Oi8WClJQUzJ07FykpKc4ui4iI2oFLo1N72WwifsmrxY5TFdh1pgIlNW2bvqeSSzA83AM3RHphZLQ3ogPUTU7bJ6Jeo9knAJe7ZqotFi9ejJ07d8JgMGDSpEkOi0cQERFRz2WziTh+KUCdLkdprblNjwvvo8KYWB+M6u+N60I9oJS73JUQROQEPXJkioiIeg6OTFFrRFHE6cJ6bPmlDDtOl6OsDQFKKgHiQz0wNtYHY2J9EOGn7oJKichF9a6RKSIiIur5CquM+P7XMmz5pRRZZS2vZAsAaoUEo/t7Y+wAX4yK9oaXhm+DiOja8FmEiIiIXEadwYKdpyuw+ZdSHMluftXaS9QKCcbE+OCWwX1wY7QX1ApulEtEHYdhioiIiLo1URTxa34d1qWWYPupchjMLW+iq1FIMCb2UoDyhorXPxFRJ2GYIiIiom6pRm/B5l9Ksf5ISatLmcskAkb198bUeD+MjvFhgCKiLsEwRURERN2GKIr4Ja8O36ZewM5T5TBaWl4La4jWDdPi/TFpSB/4uMm7qEoiIjuGKSIiInI6o9mG/54qx6pDRThbpGvx2L5eCkyL98fUeD+uwkdETsUwRURERE5TUmPCNz9fwNojF1BZb2n2OKkEGB3jgzuuD8CN0d6QSriJLhE5H8MUERERdbmT+XX46lARdp6qgMXW/FS+YG8lUoYHIHmYPwI8FV1YIRFR6ximiIiIqEuIooj/navGJ/sLWlzWXBCAsbE+mJkYiBGRXpBwFIqIuimGKSIiIupUFquInafL8cn+QqQXN389lLtKihnDAjArKRAhvqourJCIqH0YpoiIiKhTGMw2fHe8FJ8fKER+pbHZ4yL8VPjdDX0xLd4fGiU31SUi18EwRURERB3KYLZhXeoFfLK/EGV15maPS4r0xAM3BmNkFKfyEZFrYpgiIiKiDmE027DuSAk+3l+AstqmQ5QgABMG+mLOTcEYrHXv4gqJiDoWwxQRERFdE5PFhg1HS/DRj4UoqTE1eYxMKmB6vD8eHBWEcO4NRUQ9BMMUERERtYvFKmLTsRL8Z18BiqubDlEquQQzEwNx38ggLm1ORD0OwxQRERFdFVEUsedsJf61MxfZZYYmj1HJJbg7MRAPjgqGr7u8iyskIuoaDFNERETUZsdza/HP7Tn4Ja+uyX6lTMBdiYGYMyoYfh4ciSKino1hioiIiFqVXabHuzvz8MOZiib7FTIBd14fiP8bHQx/higi6iUYpoiIiKhZlfVm/Ht3PtYeuQCrrXG/RACSh/nj4XEhCPRSdn2BREROxDBFREREjVisItamXsD7u/NQo7c2ecyYGG88dksYogI0XVwdEVH3wDBFREREDlKzqvHm9znIuKBrsn+w1g1PTApHQoRnF1dGRNS9MEwRERERAKCoyoh/bM/BjlNNXxcV6qvEoxPDMHGQLwRB6OLqiIi6H4YpIiKiXs5kseHT/YX4ZH8hDObGF0ZpFBLMGxuCe0b0hVwmcUKFRETdE58RiYiow2zbtg2xsbGIjo7GkiVLGvXn5uZi/PjxGDZsGIYOHYqtW7c6oUq6XGpWNWYt/xXLd+c3GaSmxfthw2PX4cGbghmkiIiuIIii2FJ/i51ERESXWK1WxMTEYMeOHQgJCUFiYiJWrVqFQYMGNRwzb948DBs2DPPnz8fp06cxZcoUZGdnt3jehIQEpKamdnL1vU+Vzox/bs/FxmOlTfYPDHbDwikRiA/16OLKiIi6nWbnNXOaHxERdYjDhw8jOjoakZGRAIDZs2dj48aNDmFKEATU1NQAAKqrqxEcHOyUWnszURSx5dcyvL0tB1U6S6N+HzcZHp0QhtuH+UMi4XVRREQtYZgiIqIOUVBQgNDQ0Ib7ISEh+OmnnxyOWbx4MSZNmoR3330X9fX12LlzZ5PnWrFiBVasWAEAKC1teuSErl5OuR6vbc7C4fM1TfbfeX0AHrslDJ5qvj0gImoLTn4mIqIus2rVKsyZMwf5+fnYunUr7r//fthsja/TmTdvHlJTU5Gamgp/f38nVNqzWG0iPj9QiFnv/9pkkIr0V+OT3w/GC8mRDFJERFeBz5hERNQhtFot8vLyGu7n5+dDq9U6HPPRRx9h27ZtAICRI0fCYDCgrKwMAQEBXVprb5JdpsfiDZn4Ja+uUZ9SJuAPY0PwwI1BXFyCiKgd+MxJREQdIjExERkZGcjKyoLJZMLq1auRnJzscExYWBh27doFADhz5gwMBgNHnjrJpdGo2ct/bTJIjYjywpo/xeP3Y7QMUkRE7cSRKSIi6hAymQzLli3D5MmTYbVaMXfuXAwePBiLFi1CQkICkpOT8fbbb+MPf/gD/vGPf0AQBHz66afc/LUT5JTp8VIzo1GeaikW3BaBqUP9+LcnIrpGXBqdiIi6NS6N3nY2m4iVh4qxbFcujJbGL+FjY33w/PR+8PdQOKE6IiKXxaXRiYiIerKSGhNeXH+uyQUmPNVSPHNbBKZwNIqIqEMxTBEREbm4nafL8cqm86jRWxv1jYn1wQscjSIi6hQMU0RERC5KZ7Tize+zsfFY4724PFT2a6OmxXM0ioioszBMERERuaAT+bV4fu055FUYG/Ul9fPE31KiEOildEJlRES9B8MUERGRC7HaRHz8YwE+2JMP6xX7HcukAh6dEIr7RgZBIuFoFBFRZ2OYIiIichFltSY8v/YcDmc1XmSin78ar90ZjQFBbk6ojIiod2KYIiIicgGHz1fjubXnUF5nbtQ3KykQj98SBrVC6oTKiIh6L4YpIiKibsxqE/Hh3gJ8sDcfV24N6eMmw8szojA6xsc5xRER9XIMU0RERN1US9P6Evt54tU7o7nkORGREzFMERERdUM/na/G801M6xME4A9jtJg3LgRSLjJBRORUDFNERETdiCiK+GR/Id7blQfbFdP6fN3kePXOaIyI8nJOcURE5IBhioiIqJuoM1jw0oZM/HCmslEfp/UREXU/DFNERETdQFapHk+tTkNWmcGhndP6iIi6L4YpIiIiJ9t1ugKL1p+DzuS4C6+XWobX74rGyGhvJ1VGREQtYZgiIiJyEqtNxHu78vDJ/sJGfQOCNHh7VgyCfVROqIyIiNqCYYqIiMgJqnUWPPttBg5lVjfqm36dH56bFgmVXOKEyoiIqK0YpoiIiLpYVqkej69MQ16F4/VRMqmABbeGY2ZiIASB10cREXV3DFNERERd6EBGFZ79JgN1RqtDu5+HHG/dHYPrwjycVBkREV0thikiIqIuIIoivjpYjH9sz2m0f9R1YR546+7+8OOy50RELoVhioiIqJOZLDa8tjkLG4+VNuq7fZg/npvWDwoZr48iInI1DFNERESdqKLOjKe/Tsex3FqHdokA/GVSOO4d2ZfXRxERuSiGKSIiok6SXlyPx1emobja5NDurpTi9ZnRuKm/j5MqIyKijsAwRURE1AkOZFThmTXpjTbiDfVV4Z/3xCLSX+2kyoiIqKMwTBEREXWwtakX8PqWLFgdcxSS+nnizbtj4KXhyy8RUU/AZ3MiIqIOYrOJeHdXHj7dX9iob2ZiIBbcFg65lAtNEBH1FAxTREREHcBotmHR+nPYfqrCoV0QgKcnh+OekUFOqoyIiDoLwxQREdE1qqw34y+r0vBLXp1Du0ouwWt3RmP8QF8nVUZERJ2JYYqIiOga5JTr8eiXacirMDi0+7rJ8a97YzFY6+6kyoiIqLMxTBEREbXT8dxa/GVVGqp0Fof2fv5qLLs3FsE+KidVRkREXYFhioiIqB1+OFOBv36bAZNFdGhP7OeJt2fFwEPNl1giop6Oz/RERERX6ZufL2DJlizYHHMUpsX7YVFyJOQyrthHRNQbMEwRERG1kSiK+GBPPj7YU9Co7+HxIZg3VgtBEJxQGREROQPDFBERURtYbSJe35yFtUdKHNqlEuDF5EjcPizASZUREZGzMEwRERG1wmC24blvM7D7bKVDu0ouwRsz+2NMrI+TKiMiImdimCIiImpBjd6CJ1al4VhOrUO7l1qGd+6NRXyoh5MqIyIiZ2OYIiIiakZJjQmPfHEG50r0Du19vRR47/6BiPRXO6kyIiLqDhimiIiImnC+VI9HvjiD4mqTQ3tUgBrv3TcAgV5KJ1VGRETdBcMUERHRFU4V1OHPX55ttBnvsDAP/POeWHhyDykiIgLDFBERkYPUrGo8vjINOpPNoX38AB+8dld/qOTcQ4qIiOwYpoiIiC7am1aJZ9akw2Rx3I33zusD8OzUfpBJuYcUERH9hh+vERFRh9i2bRtiY2MRHR2NJUuWNHnMmjVrMGjQIAwePBj33HNPF1fYsq2/luGp1WmNgtTc0cF4fjqDFBERNcaRKSIiumZWqxWPPPIIduzYgZCQECQmJiI5ORmDBg1qOCYjIwOvv/46Dhw4AB8fH5SUlLRwxq615nAxlmzNhuiYo/D4LWGYc1Owc4oiIqJujyNTRER0zQ4fPozo6GhERkZCoVBg9uzZ2Lhxo8Mx//nPf/DII4/Ax8e+wW1AQIAzSnUgiiI+2leA17c4BilBAF6Y3o9BioiIWsQwRURE16ygoAChoaEN90NCQlBQUOBwTHp6OtLT0zFq1CiMGDEC27Zt6+oyHYiiiHd25GLZrjyHdplEwOt3RePOhEAnVUZERK6C0/yIiKhLWCwWZGRkYM+ePcjPz8eYMWNw4sQJeHt7Nzp2xYoVWLFiBQCgtLS0w2ux2kS8tjkL6444TjVUygS8NSsGo2N8OvxnEhFRz8ORKSIiumZarRZ5eb+N8OTn50Or1TocExISguTkZMjlcvTr1w8xMTHIyMho8nzz5s1DamoqUlNT4e/v36G1mi02PLf2XKMg5aaU4r37BzJIERFRmzFMERHRNUtMTERGRgaysrJgMpmwevVqJCcnOxwzY8YM7NmzBwBQVlaG9PR0REZGdmmdBrMNf1mdju0nyx3avTUyrJgzENdHeHZpPURE5NoYpoiI6JrJZDIsW7YMkydPxsCBA3H33Xdj8ODBWLRoETZt2gQAmDx5Mvr06YNBgwZh/PjxeOutt9CnT58uq1FntOKxr87iQEaVQ3uApwIfzx2MQcHuXVYLERH1DIJ45TqwjlrsJCIi6mwJCQlITU29pnPUGix47Ks0HM+tdWgP9VXh3w8MQLCP6prOT0REPVqzGw1yAQoiIurRqnUWPPLlGZwqqHdojw5QY/kDA+HnoXBSZURE5OoYpoiIqMeqqDdj/udnkF6sc2gfGOSG9x8YAG+N3EmVERFRT8AwRUREPVJprQl//OwMskr1Du1xIe54774B8FDzJZCIiK4NX0mIiKjHKa42Ygols5gAACAASURBVN6nZ5BXYXBoHx7ugX/dOwBuSqmTKiMiop6EYYqIiHqUgkoD5n16BoVVRof2EVFe+PvsGKgVDFJERNQxGKaIiKjHyCnTY95nZ1BSY3JoHxPjjTfvjoFSzh1BiIio4zBMERFRj3Dugg4Pf34G5XVmh/YJg3zx+p3RkMsYpIiIqGMxTBERkcs7W1SP+Z+fQZXO4tA+ZagfXp4RBZm02S1CiIiI2o1hioiIXNqJ/Fo88sVZ1BqsDu0zhvvjhemRkEoYpIiIqHMwTBERkcs6llODR79KQ73RMUjNSgrEM7dFQMIgRUREnYhhioiIXNJP56vxxMo0GMw2h/YHbgzCE5PCIAgMUkRE1LkYpoiIyOXsz6jEU6vTYbKIDu3zxmrx8PgQBikiIuoSDFNERORSdp+twDNrMmCxOgapP08Ixe/HaJ1UFRER9UYMU0RE5DJ2nCrHc9+eg8XmGKSevjUc944MclJVRETUWzFMERGRS/j+RBleXHcOVsdLpPDctH6YmRjonKKIiKhXY5giIqJub9PxUry8IROXD0gJArAoORIzhgc4rzAiIurVGKaIiKhbq9JZsHhDJsTLgpREAP6WEoWp8f7OK4yIiHo9ibMLICIias6aw8UorDI6BCmpBHjtrmgGKSIicjqOTBERUbf01cEiLN2W49AmkwhYMrM/JgzydVJVREREv2GYIiKibufT/YV4Z0euQ5tcKuCtWTEYG+vjpKqIiIgcMUwREVG38p+9+Xj/h3yHNoVMwN9nx2JUf28nVUVERNQYwxQREXULoihi+e58/GdvgUO7RAD+de8A3BDp5aTKiIiImsYwRURETieKIv61Mw+f7i90aFcrJAjro2KQIiKibomr+RERkVOJooi3/5vTKEi5KaV4//6B0CikTqqMiIioZRyZIiIip7HZRLz5fTa+PnzBod1dJcX79w9AXIiHkyojIiJqHcMUERE5hc0m4tXNWVh3pMSh3Ustw/IHBmJgsJuTKiMiImobhikiIupyVpuIlzdm4rvjZQ7t3hoZ/v3gQMT2ZZAiIqLuj2GKiIi6lMUqYtH6c/j+RLlDex93OT54cCCiAjROqoyIiOjqMEwREVGXMVtteH7tOew4VeHQ7u8hx4o5gxDhp3ZSZURERFePYYqIiLqEyWLDwm8ysOdspUN7Xy8FPnhwEML6qJxUGRERUfswTBERUaczmm14+ut07M+ocmgP9lZixZyB0PowSBERkethmCIiok6lN1nx1Op0HMysdmgP9VXigwcHIchb6aTKiIiIrg3DFBERdZo6gwWPrUzDsZxah/YIPxU+eHAQAjwVTqqMiIjo2jFMERFRp6jWWfDnL8/gZEG9Q3ukvxofPDgQfh4MUkRE5NoYpoiIqMNV1Jnx8OdnkHFB59Ae01eD5fcPhK+73EmVERERdRyGKSIi6lAlNSb88bPTyC4zOLTHhbhj2X0D4KnmSw8REfUMfEUjIqIOU1BpwB8/O4OCSqND+/BwD/zr3gFwU0qdVBkREVHHY5giIqIOkV2mx8OfncGFGpND+43RXlg6KwZqBYMUERH1LAxTRER0zdKL6zH/87OoqDc7tI8f4IMlM/tDIZM4qTIiIqLOwzBFRETX5FRBHf70xRnU6K0O7bfG9cHfUqIglzJIERFRz8QwRURE7XYspwaPfpWGeqNjkJox3B8vTI+EVCI4qTIiIqLOx48LiYioXQ5lVuNPX5xtFKTMadvxxVO34q0332j2sWvXroUgCEhNTe3sMomIiDoNwxQREV213Wcr8PjKszCYbQ7txpMbsfHVe3H69CmsWrUKp0+fbvTY2tpavPPOO7jhhhu6qlwiIqJOwTBFRERXZdPxUiz4Oh0mi+jQntxfRGjdUURFRUGhUGD27NnYuHFjo8e/+OKLWLhwIVQqVVeVTERE1CkYpoiIqM2+OliEl9Znwuo4IIWnbw1HnKoAoaGhDW0hISEoKChwOO7o0aPIy8vD1KlTu6JcIiKiTsUwRURErRJFEe/tysPSbTkO7RIBeDE5EveODGr1HDabDU8++STefvvtVo9dsWIFEhISkJCQgNLS0nbXTURE1JkYpoiIqEU2m4glW7Lx4T7HUSaZVMCSmf1xx/UBAACtVou8vLyG/vz8fGi12ob7tbW1OHnyJMaNG4eIiAgcOnQIycnJTS5CMW/ePKSmpiI1NRX+/v6d9JsRERFdGy6NTkREzTJbbVi0PhPbTpQ7tKsVEvx9dgxGRHk3tCUmJiIjIwNZWVnQarVYvXo1Vq5c2dDv5eWFsrKyhvvjxo3D0qVLkZCQ0Pm/CBERUSfgyBQRETVJb7LiyVXpjYKUp1qKDx4c6BCkAEAmk2HZsmWYPHkyBg4ciLvvvhuDBw/GokWLsGnTpq4snYiIqEsIoii21N9iJxER9Uy1egseX5mGY7m1Du3+HnIsf2AgogI0XVZLQkIC96MiIiJnanYHek7zIyIiB2W1Jjzy5VmkF+sc2kN9lVj+wEBofbikOREREcAwRUREl8mrMOCRL84gr8Lo0N4/UIP37x8APw+FkyojIiLqfhimiIgIAHCqoA6PfnUWlfUWh/brwjzwr3ti4aHmSwYREdHl+MpIREQ4kFGFBWvSoTc57sY7qr833rq7P9QKqZMqIyIi6r4YpoiIerlNx0vxysbzsNgc1xyaOtQPL90eCbmMC78SERE1hWGKiKiXEkURH/9YiGW78hr1zbkpGI9NDIUgNLuAERERUa/HMEVE1AtZbSLe3JqNNT9fcGgXBGDBrRH43Yi+TqqMiIjIdTBMERH1MgazDc+vPYcfzlQ4tMulAl69Mxq3DO7jpMqIiIhcC8MUEVEvUqUz48lV6Y0243VXSfGP38UiIcLTSZURERG5HoYpIqJeIqdcj8e+SkNuucGhPdBTgWX3DUB0oMZJlREREbkmhikiol7gaE4NnlyVjmq94x5SUQFqLLtvAPp6KZ1UGRERketimCIi6uG+/7UML23IhNnquPT58HAP/ON3sfDkZrxERETtwldQIqIeShRF/GdvAZbvzm/UNzXeD4uSI6HgHlJERETtxjBFRNQDmS02/G3TeWz+paxR3/zxIfjDWC33kCIiIrpGDFNERD1Mtc6Cp79OR2p2jUO7XCpg8YwoTBnq56TKiIiIehaGKSKiHiSrVI8nVjVesc9LLcPffxeD4eFc+pyIiKijMEwREfUQ+zMq8ddvzqHOaHVoD/VV4d37YhHeR+2kyoiIiHomhikiIhcniiK++F8R/rkjF6Ljgn0YFu6Bv8+OgbdG7pziiIiIejCGKSIiF2Y02/DKpvPY8mvjhSaSr/PH89P7ccU+IiKiTsIwRUTkokpqTHhqdRpOFtQ7tEsE4MnJ4bhnRF+u2EdERNSJGKaIiFzQyfw6/GV1GspqzQ7tHiop3pjZHyOjvZ1UGRERUe/BMEVE5GI2HSvBq5uzYLI4XiAV4afCP38Xi3A/LjRBRETUFRimiIhchMliw5vfZ2NtakmjvlH9vfH6XdHwUPFpnYiIqKvwVZeIyAUUVxux4Ov0RtdHAcCDo4Lw6MQwSCW8PoqIiKgrMUwREXVzP52vxrPfZKBKZ3FoV8kleGF6P0yN93dSZURERL0bwxQRUTcliiI+3V+IZbvyYLti/6gQHyWWzo5BbF835xRHREREDFNERN1RrcGCxRsy8cOZykZ9o2O88f/uiIanmk/hREREzsRXYiKibuZUQR0WfpOBgkqjQ7sgAA+PC8FDY7SQ8PooIiIip2OYIiLqJkRRxFcHi/HOzlxYrI7z+jzVUrx2Z3+M6s/9o4iIiLoLhikiom6gSmfGSxvOY19a42l9A4I0WDorBloflRMqcw5RFAERAAfgiIioG2OYIiJysuO5tXj2mwxcqDE16puVFIi/TAqHUi5xQmXXThRF6MtsqC2woK7QCn2ZFfpKGwwVNhgqrdCX22CsscGiF2Ex/PZlNYgQbfZzlBaYsDwyDxAAiRyQqyWQawTI3QTINRLI3AQoPSVQ95FA4yeF2k8KjZ8Eaj8pPIKl0ARKIZEylRERUcdjmCIichKrzb5a3/LdebDaHPvcVVIsvj0KEwb5Oqe4qyCKIuqLrKjMtKAiw4yqTDNqci2oLbCirtACa+OM2M4fBNhMgNFkg7G67Q+TKACPYBk8w2TwCJHCK1wGn2g5fGPkcA+WQhAYtIiIqH0YpoiInKCoyohF6zORml3TqG+I1g1LZvbvltP6rEYR5WdNKD1pRskJEyrSzKjMNMNcJ7b+YCexmYDqbAuqsy2N+uTuAnz724OVb6wcAXEK+A2WQ6ZyzZFAIiLqWgxTRERd7Ptfy/DalizUGayN+u6/MQiPTgiFXOb8N/OiKKIq04KiVCNKfjWh9IQJFelm2MzOrqzjmOtEXDhmwoVjvw2fCVLAN0aOgKEKBMQrEHidAr4xcghcQZGIiK4giGKLnyZ2348aiYhcTI3egte3ZGHbifJGfV5qGf6WEoUxsT5OqMzOZhFRfsaMwsNGFP1sRFGqEYYKW+sPbIXCQ4CHVgZ3rRRuAVKofCRQ+Uqh9pVA5SuBylsKuUaAVCVAdtmXRGYPLwkJCfj58M8QRcBmFmHWXfyqt19rZa4Toa+0QV9uvyZLV2azfy+xorbACkPltf8OSi8JghIVCE5SIShJCb9B8ob6iIiox2v2CZ8jU0REXeDnrGosWp+J4urGFxDdEOmFv6VEIcBT0aU1iaKI6iwL8n40IO9HAwoPG9s9XU/hLsCnvxw+UXL49JfBq58cnlop3LUyKD2vfZRNkAgQAEikAmQqQH0Vl5KZam2oybegJteCmjwLqs5bUJlhRkWaGaY2/r7GahuydxqQvdMAAJC7CeiboEToTSqEjlHBJ1rGa6+IiHohjkwREXUio9mG93/IwxcHi3Dl061CJuDxiWGYfUPfLtuE11htQ/7/7OEp/0cDagsaTzVsjYdWCv84BfyHKOAfZ58CpwmQdFqYSEhIQGpqaoefVxRF1BdbUZFuRnmaGWUnzSj51Yia3Kv/m7gFSRE6WoWwMSqEjFJB6eX8aZpERNRhmn2BY5giIuokv+TV4uUNmcgqMzTqi+mrwat3RCM6UNPpdVRnm5G1U4/sHQYUHzE2LDneFnJ3AX2vVyLoegX8h9oDlNpX2nnFNqGzwlRzDJVWlJwwofRX+7VURalGmGrb/nIoSIDAYQpETFAj4hY1fKLknVgtERF1AYYpIqKuojdZ8f4P+fjqUOPRKEEAHrgxCH+6ORSKTlpkQrSJuPCLCdk79MjeqUflucar2DVH1UeCoAQlgpOUCEpUos8A518b1NVh6ko2q4iKdDOKDhtR+JMRhT8bYShveyL1jpQhYqIa/W5RI3CYggtZEBG5HoYpIqKucDSnBos3nEdeRePRqL5eCrySEoWEfl4d/nNtFhEFB43I/F6H7J166Mva9mZfqgCCkpQIHaNC6E0q+MbKu921P84OU1cSRRGV5yzI329A3j4DCn8ywmJo28ul2k+CiAlq9JukRsgoFaSK7vW3JiKiJjFMERF1Jr3Jin/tzMPXh4sbjUYBQMrwAPxlchg8VB237o/NIqLwJyMyt+pw/r/6Nq+85xMjQ9hoFULHqBGUqOj2eyp1tzB1JYtRRNFho30hj30GVKS3be14pZcE/SapET1NA+1IpdNHAImIqFkMU0REneV/56rw2uYsFFQaG/X19VLgpdsjMSLKu0N+ls0qouhnIzK36HB+mx76Nkw3E6RAcJISEbeoETFRDc+QzlnIddu2bXj88cdhtVrx0EMP4dlnn3Xo//vf/44PP/wQMpkM/v7++PjjjxEeHt7qebt7mLpSXZEF2bsMyN6hR8EhQ5v25VL5ShA52R6sgpKUkEgZrIiIuhGGKSKijlZaa8LS77Ox/VRFk/0zEwPx+C1hcFNe24INoiii9FcT0jfokLlVB11p6wFK7i4gbKwKERPUCB+v7vTV5axWK2JiYrBjxw6EhIQgMTERq1atwqBBgxqO2b17N2644QZoNBosX74ce/bswddff93quV0tTF3OWGND7l57sMrdo2/TUuxqPwmibtMgeroGfa9XdLtpl0REvRD3mSIi6ihWm4hvf76AZbvyUGdsvIy21keJl26PROI1XhtVV2RB+gYd0tfXt2kRCYWngMhJGkROUSNkpApSZde9CT98+DCio6MRGRkJAJg9ezY2btzoEKbGjx/fcHvEiBH48ssvu6w+Z1F6StB/ugb9p2tgNdmnZZ7fZh9VbG4zYX2ZDSe/qMPJL+rgESpFzAw3xNyugXckVwUkIupuGKaIiK5CWlE9/t9353GyoL5RnyAAs5P64tGJoVAr2jcaZdbZkLVdj7S19cj/n7HV+QEKdwERk9SInqpx6oIGBQUFCA0NbbgfEhKCn376qdnjP/roI9x2223N9q9YsQIrVqwAAJSWlnZcoU4kVQgIHa1C6GgVblosovCQEec263D+vzqYapr+D12bZ8WRd2tw5N0aBMQrEDNDg+hpGqj7dO3y9ERE1DSGKSKiNqjRW/Dv3flY83MxrE0MKAwMcsPz0/thsNb9qs8t2kQUHjYifZ0Omd/rYK5vOUHJ3QRETLRfXxN6U9eOQHWEL7/8Eqmpqdi7d2+zx8ybNw/z5s0DYJ/m19NI5b8FqzGv+CB/vwHntuiQtUMPczNTAUt+MaHkFxP+92oVQseoEDPDDRETVd1+AREiop6MYYqIqAVWm4j1R0rw3g95qNI1nmqnUUjwyM2huDupL2RXuWhAdbYZaevt0/hq8xtPF7ycRAaEjVMhJsUN4eO73xtorVaLvLy8hvv5+fnQarWNjtu5cydeffVV7N27F0qlsitL7LakCgHhN6sRfrMaFqOIvH0GZGyqR/YOPaymxsfbLEDODwbk/GCA3F1A1K0axNyhQXCSkntYERF1MS5AQUTUjGM5NXhjazbSinVN9t880BfP3BaOQK+2hwJjjQ2ZW3RIW1eP4iNNvFO+gt8QOWLvcEP0NA00ft13apfFYkFMTAx27doFrVaLxMRErFy5EoMHD2445tixY7jrrruwbds29O/fv83nduUFKK6FscaG89t0SN+gQ+GhxitFXsk9WIqYFDfEpvD6KiKiDsbV/IiI2upCtRH/3JGLbSfKm+zX+ijxzG0RGBPr06bz2Swi8n40IG1d86MNl9P4S9B/hhti79CgT6ziast3mq1bt+KJJ56A1WrF3Llz8fzzz2PRokVISEhAcnIyJk6ciBMnTiAoKAgAEBYWhk2bNrV63t4api5XW2BBxiYd0jfUozKj9cVIAocpEJPihuhpaqi8u28IJyJyEQxTREStqTVY8MmPhVh5qAhGS+OnP5Vcgt+P1uL+G4OglLc+za78jAlp6+qRvlEHfVnLy5lLlQL6TVIj9g77QhLcwPU3DFO/EUURZafNSF9fj4xNrf9/JVEA4ePViL3DDWFjnbdACRGRi2OYIiJqjtliwzepF/CfvQVNXhcFALfG9cETt4S1OqVPV2pFxib7NL7yM63v1hqUqETMHRpE3aaB0rN7XQfVXTBMNc1mEZF/wID09TpkbdfDYmj5JVvla1+mPSbFDf5xcu5fRUTUdgxTRERXEkUR20+VY9nOPORXNn1NSmxfDZ6ZEoHh4Z7NnsdiFJGzy76cee4+A8SW15KAR6gUsSluiElxg1c41wFqDcNU60y19uur0tbpUPhT69dX+fSXITbFDf1v18A9iP8PEhG1gmGKiOgSURRxMLMa7/+Qh1NN7BcFAH4ecswfF4LbhwdA2sQKaaIo4sIx+zS+c5ub3yfoErm7gKgpGsTe4YagBAVXXbsKDFNXpybfgowN9Uhbp0N1divXVwlAyI1KxNzhhshJasjdODpKRNQEhikiIlEUcTirBst/yMMveXVNHqNRSDDnpmDcNzKoyY13awssSN9gn8ZXndXyG1VBAoTcpELsHRpE3KKGXM03qu3BMNU+oijiwnET0ta2LfDLNAIib7VfX6UdwWXWiYguwzBFRL3bkewaLN+dhyPZtU32yyQC7kwIwLyxIfB1d1xWumEK1QYdCg+2YQpVjAyxd7gh5nY3uAVyJbVrxTB17axGEdk/6JG+vh65ewywtTJg5R4kRf8Z9pFUnygus05EvR7DFBH1Tsdza/Hv3fn46Xx1s8fcPNAXj00MRbifuqHNZhGR/z8D0tddxcX9yRrEprjBbwgv7u9IDFMdS19uRcZ3OqSvq0fpydYXSQmIVyA2RYPo6RqofPjhABH1SgxTRNR7XLom6uMfC5odiQKA0THeeHh8CAYFuze0lZ8xIW29Dhkb66ErbWXZaTkQfjOXne5sDFOdpyLdvsx6+kYd6otbXjlFIrcvsx6TokH4eDX/fyei3oRhioh6PqtNxK7TFfhkfwHOFumaPW5klBfm3xyCuBAPAEB9iRUZm+qRvl7XpuXMA4YqEHsHP6nvKgxTnc9mFVF40Ii0dfU4/189LPqWX/6V3heXWb9Dg4ChCo7EElFPxzBFRD2XyWLD5l/K8NmBQuSWG5o9LqmfJx4eH4Jh4Z4w623I3qFH2nod8n80QGx5EAruwVLEzLDv0cNrSLoWw1TXMtXZcP6/eqSvq0fBIWOr7wS8I2WISXFDzAwNPLRcZp2IeiSGKSLqeSrqzPg29QK+Sb2AstrmR5RuiPTCQ2OCMUzribwfDTj3nQ5ZO/Ww6FpZztxNQNRt9k/fg5O4upmzMEw5z6XVK9PX16PqfOvLrGtHXFxmfbIaCneuXklEPQbDFBH1HGnF9Vh1qBjfnyiDydL005QgAOMH+OL/bgyCT4EcGd/pcH6bHsaqloegBAkQMlqF2BQuZ95dMEw5nyiKKPnVhPR1OmR8p2v135FMLSBysv16wuCRSkik/CCCiFwawxQRuTarTcS+tEqsPFSM1OyaZo+TSQRMHeqHGb7+qP/RiswtetRfaPnCegDoM1COmBQN+ie7wS2A10F1JwxT3YvVJCJntx7p63XI2a2HrZXLDN36ShE9VY2oqRoExPP6KiJySQxTROSaSmpM2HSsFOuPlqCwqvk9ntRyCe7sG4BhJR4o3mFCdU4rU5Jw8U3eNA1iUjTwG6joyLKpAzFMdV/6CisyN+uQtk6Hkl9NrR7vHixF1BQNoqaoGayIyJUwTBGR67DaRBzKrMa6IxewN60S1uZmFInAQJMbJuh9oPxFgtrc1kegVD4SRN6qRv9kDYISeR2UK2CYcg2V58xIW1+P9A061Be1/m+RwYqIXAjDFBF1f5dGodYdvYCiqmY+5RaBgDIFRlZ5QXteCUtJ609TcjcB/W5RI3q6BiE3qSCV802bK2GYci2iTUTBISPS19cj8/vWF3oBAA+tFJG3XQxWQxX8kIOIuhuGKSLqnvQmK/acrcTmX0pxKLMatiaedQQb0LdUgeg8NWIK3SCtbv2NlkQBhI+zj0CFjVdxIQkXxjDlusw6G3J2G5C5VYfc3QZYDK2/rXALlCJ8ggr9JqmhHcHNsImoW2CYIqLuw2YTcTSnBpt/KcOOU+XQmRrP45ObBYQUKhGer0ZEoQoKQ+thSKIAQkepEHmbBv0mqaH0ZIDqCRimegZzvQ05ewzI3KJD7p62BSu5u4CwsSr0u0WNsHH8N01ETsMwRUTOJYoi0op12H6yHN+fKENxdeNpfO51UoTnqxCWr0LwBSWkttY/kZYq7W+2Im9VI/xmvtnqiRimeh5z/WUjVm0MVhIZEHyDEv1uUSNiohruwdwgmIi6DMMUEXU9URSRfkGHHSfLsf1UBfIqDA79gg3wK5cjvMAeoPwq27ainkwjIHy8ClG3aRA2VgW5GwNUT8Yw1bM5BKt9hjZdYwUAfQbIETZWhbBxKgQOV/JaSCLqTAxTRNQ1RFFExgUddpyqwI5T5cgpdwxQGp0EIUUqhBYooS1SQWVqWxBSekkQPt4+AhU6RgWZigGqt2CY6j0sRhEFBwzI2qlH9k499GUtbw58icJdgHaUPViFjVXBvS9HrYioQzFMEVHnMVttOJZTi71pldiXVon8yt/2g5JYgb4lSoQWKhFSpEKfSnmbz+sVLkPERBUiJqrR93olJDJ+8twbMUz1TqJNxIXjJmTv0CNrhx5V51vfO+4S39iLo1Zj7aNWMiWfO4jomjBMEVHHqtVbsP9cFfaercSBc1WoM1zcV0YE+lTKEVysQHCxCsHFCsitbRtFEiRA4HAFIiaqETFBDe9IGfeeIYYpAgBUnjcje4d9xKr4qKnN71CkSgFBCQpob1Qh5EYl/IYoIJHyeYWIrgrDFBFdG5tNxNniehzKrMbBc9U4nlsLi00ERMCnWobgYiWCi5UIuqBs89Q9wD59L2SUEuE3qxE2TgW1r7QTfwtyRQxTdCVDpRV5PxqQu9eA3H0GGMrbNh0QABQeAoJH2IOV9kYVfKL5oQ0RtYphioiuXkmNCQczq3DwXDV+Ol+NKp0FEAGvGhmCLygawpPG0PYAJEiAgHgFQkerEDpWhYCh/JSYWsYwRS0RbSJKT5qRu1eP3L0GlBw3QWx7toLGX4KgJCWCEpUISlDCN1bO5yQiuhLDFBG1rkpnxrHcWhzJqsGh89XILNFDagX8yhXoW6JA31IFAksVUBmvbvTIra/UHp7GqBAySgmVN0efqO0YpuhqGCqtyNtvRN4+PfL/Z0T9/2/v7mPkuO86jr+/M7O79/zgu/PTnanj2m7sNHFJ7LahAtEW0QKV+0+EgkQKpFVElVRBqkTTVlSV+AcBAiqVf6KmUIlWDgqFIAShQSD+iZrECc1zUjtxHN/l7POd7/lud3Zmvvwxe+dz4se17/YePi9ptDOzc+uvPbfe/czvYYbTa/r5Yrux9fY8XG09WGLzgaLGXImIwpSIvN/YTMzzJ6d57u0pnj85zbEzczTNB2w5ez449Y0Vr+p+T0sVO4ztH2ui/+Ml+n+pxKa9BXWjkbopTEm9U+0wWgAAEdZJREFU3J3JtxOGnqow+FSZd39aoTx+Dc1W5DcD33xbkW13lNj8kSKbDxQ1W6DIxqMwJbLRZZlzcqzMS4PTvHhqhudOTjE0XKF3rEDfWJG+sQJ9o0U6Zq/9S0Khzdh+qMT2O0v0f7yJnn3qJiM3jsKU3CieOaOvVRl6qszQUxXefbZy1fe1Wqp1a8jm24psqYWrvluLFNt0uwaRdUxhSmSjmZir8tLgDC8PzvDS4AyvnJyhNByweay4GJ66JiPs0v8/XFKxzdhye4ntHyvRf2eJvg8XNW25LBuFKVkuaeycfSXm9NEKw89WGD4aU5m4tpYrAAy690RsOVCi75YCvbcU6bm5oBuKi6wfClMi69nUfMIbp2d5fXiO14dneeONWconMnrGC/ScK9AzXqBrKiLw+gJP+0DI1jtKbL2jxLY7inTvVcuTrByFKVkpnjnjbyZ5sHq2wumjFaaHrm3M1SKDzp1RHq72F+nZnz+29GrMqMgapDAlsh64OyNTMW+croWmwVkGX6+QvuN5cKqFp2uZXe+9ggh69hXy4HQwD1CtW/ThL42jMCWNNPNuwvDRCmd+FjPyYszoyzFpXP/rtW4J6dlXoOdDBbr3Fti0p0D37oioSa1YIquYwpTIWjM1n/DmyBxvjszz1uA8775RZuJ4QnEkoGsyonsyomO6/tYmAAy6dkVsvq2YLweK9OzTzFWyuihMyWqSxs65N6qc+VmFMy/EjLwQM/Fmcn0vatDxCxGb9kR078kD1qa9Bbo+WND/xyKrg8KUyGrk7ozPJpwcm+fNM3OcOFbm9PEKUycTwhGjeyIf11TPpBAX0z4Q0ndrcTE89X24SLFdV0NldVOYktWuMpUx8kLM2Zdjxl6LOftKlcm3k+v+FmUBtO+I6NwZ0XVTRNfOiM6bCnTeFNG2LVR3a5GVozAl0kgTc1XeGStz8uw8p45XGDkeM/VOQjLsNI8HdExHdMxEROmN+WC0AmzaW6Bvf5He/YW8S8nNRUodCk6y9ihMyVpUnc0Ye73K6CsxZ1+tMvZqzNjPq2TX0UVwqaAInR+I6NxZoOumPHB17MiX1m0hYUFBS+QGUpgSWU7zccq7ExWGRssMvVXh7ImYicGE+eGU5KxTnAxon41omw2v+Z5NV1LcbPTtLdK3v0jPvgK9+4p07Yo0u56sGwpTsl6kVWfizSpjr1c59/Mq545VGT9WZepUekO/cVmQT9/ePhDRsSOkvT+ifUdEx0C+r3WrWrVErpHClEi90swZm6lyZrzC8KkKIydjRgerTA0nlEcyslGnNBnQNhvROhfUNdX4FRkUthpduwts31daHLDcvaugbnqy7ilMyXpXncuYeDPh3LE8ZI0fy4PW9GCdMwleQRDlYat1a0jb1pDWrdGS9fyxZXOoi3Ii513yzaBbeMuG5e5Ml1POnKswPFhhZKjK+GCVqdMJ8yMp1XOOj0Nh2miZC2kunw9KJQL6KN74olqh1B+w6aYC2/YU2bSnyKbdEZ27NAhZRGS9KrQE9N2a3/x3qepcxtTJhIkT+TL5dj4Wa+JEQvlcHffDqskSmB5MLxvWLIDm3oC2WtBq2RzS0hfQ0hvS3BvS3LuwHlBo1kU92bgUpmRdmY9TxiZjRoarjA7HjJ9OmBpJmB1NqZzLqE442ZQTTEM0F9BUDijFF34ItBDSwvJNBe7tTnFbQMcHIrbsLrJtb4nOnRGdH4ho6tIU5CIikiu0BPTsy2dZfa/KZMbE21UmF4NWwvRQwvSphLmz9QetBZ7B3EjG3EgML16hzjajpScPVs29IS2LjyFN3UG+dAWUugOaukNdHJR1RWFKVh13Z66Scm484dyZmPHRhMnRhNlzKbPnUsoTKfGUU51yshnHZyGYg3DeKFYCStX3XyErEVBiZa6cpe1O2Gc0bw3oGIjYuqvItl0lOgYi2vsjdcsTEZHrVuoM2HKgxJYDpfc9l5QzpodSpgcTpk7lj/mSMjWUUB67/rC1VHXGmZxJmDx5dcdHzUZTVx6ySl3nw1ZTd7i4r9QRUGw3iu0L6wHFNsMCBTFZXRSm5IbJMme+kjI1kTA1njA1njIzkTA7mTE7mVKezqjUlmTWSWaddNbxeYcyWNkIYwgrRrEaXHKihoigYb+4SWuGd0KhJ6B1S0jX9oi+nUW27yrRuSOibXukK24iItJQUVNA9wcDuj9YuOjz1dmMmTMps8MpM6dTZk+nzJ5O8vXa/vkbHLiWSuadmfmUmeFrHxNWbMsDVrE9oNhhlBbX8/C1sF1oNQotRqElIFpYbw0WH8MimOnzWq6fwtQGkGVOpZIxP5syP5dRnk8pzznl+ZR4PqM875RnUypzGeVZJ57JiOcyknmnOu8k805adrKy4xXwClABiyGIwRIjqhphYlc1tXeEES3HJA11cnOSZsfbIew2Sr0BbbWg1DNQZNvOEl3bI1r6QsLi6qlbRESkHoXWgO5dAd27Lh62ANKKMzuSB62Z0ylzZ1Pmz6bMjabMj2bMjaXMn82YH0vJrvOexdcinnHimRTqCGJLWciFAeuCsGVELQGFZiMsGVFTvoRNS9Yvtr/0/uPUkrb+KUxdB3enWnXi2IkrKXHZqcYZccWpVpw4zkhq20nVSSpONXaSakYSO0nspFUnqebTpaZxRlqFrOqkSf6YVSFLnDRxvApZ7GQxeDXf9ipQBRKwKliSh5sggSA1ghTC1Ai8vjfzags+V6valJG1Ae0QdRnF7oCWnoC2vpCuLQU2bSuwub9I5+aIUlegKWJFbpAnnniCBx98kDRN+dKXvsRDDz10wfOVSoUvfOELPPfcc/T09PDoo4+yc+fOxhQrIpcUlmzxvlWX45lTmcyYG82YH02ZG83y4DWat26VJzLK4ymViYzyeL7tyzNJ4TXxFOJpJ55e3mKCIudDVtEIikZYMMIi+XptOyhS228EBfL9i8dT279wPASl8z8bFiGI8uctzGdrDCIjCFmyz2pLvm5h/uctPd5CtdbVY02Fqb//0yGm3knxzPEUPHU8A7L8TXF+3fPH2jZp7bG2zxb2e/6c+fltSy1/9Pw4ywC389sOlhmBU3dAuRYGhBjhGgw01ysN8hajrNmhFYI2I2o3Ch1GU1dIS3dA66aIjp6Qrt6I7s0FNm2OKHVoOleRRkjTlPvvv58nn3ySgYEBDh06xOHDh9m/f//iMY888gjd3d0cP36cI0eO8LWvfY1HH320gVWLyPWwwGpjnULYc+mWrgXuTjzttXCV1sJWdv5xPKMymRFP50tlOstDz1RGdXbt3bEni8kvuk+vjdqDCCxcEroiCKP3hK4IgnAhfNWOD/PfBQvzmSCD0LAgbwG02voFx7zvZy887iP3ddDStzYm5VpTYWrkJzHtQxGXmepdVoEkcpKik5UcbwKaIWiGsMUIW41Cm1FqyweUNrcHtHSGtHaFtHdFdHSHdHQXaO8O8+ZxXSERWTOeeeYZdu/eza5duwC4++67efzxxy8IU48//jjf/va3Abjrrrt44IEHcHe910U2CDOj1GGUOgI6fuHavoZmSW0owrTnQWvqvesZ8VRGPOtUZ53qXEZ1Lh+jXZ3Pavvy/Vm8TH/BNS5LgMRJK9DI283e/NttClPLQpOg1S0zJw2dLIQscjwCD8ELDgUgAisZ1gRhkxE0GVGzUWgyohaj2BpQajFKrSFNrUZze0hzW0hLW0BrR0hbe0hrR0ixOVD/YJENamhoiB07dixuDwwM8PTTT1/ymCiK6OzsZGxsjN7e3hWtVUTWniDKe6Y0dV3/a6VxPia8OpsHrsXHhfVZJylfuKRL1ysX2V95/zFSH1sbOQpQmLpujpMFkAV5UPEAPPA8qIT5OrV1aotFtfVaU6pF1JZa02nhfN9Wi2r9ZKO8b2uh1u+20BRQaDJKTfnN8kpNRqkloNQc0NQc0NQS0NwS0tSSb0fFVfiPJyJyCQ8//DAPP/wwAGfPnm1wNSKy3iyMSSp1Lt/3I888D221cJVV8xCXxfn+dGG76qTxkvXKe587/7MLS3aR57PUyZK8BS9LwBMnS89vZ4njyaWPWw1j2Raspc4KaypM7bqrmYmhhCCo9eUMF/p11gbZ1fpdLvTtDEPL16N8PQhrA/VqA/HC0Ihqg/HC0IgK+WtFUf5zUe3YMDIKhYAwovZcQKFgRAUIQoUUERGA/v5+Tp06tbg9ODhIf3//RY8ZGBggSRImJyfp6el532vdd9993HfffQAcPHhweQsXEVkGFizM6tfoSq6OZ+fD12LoWhLSFsNZ9Xz48iyfv2Bh/YL9KWS1uQwW5jbI0iXHp477+fVsyes096ydpqk1Fabuun9ro0sQEZFLOHToEMeOHePEiRP09/dz5MgRfvSjH11wzOHDh/nBD37AnXfeyWOPPcanPvUpjZcSEVkFLDDCIO8JJVdvTYUpERFZvaIo4rvf/S6f+cxnSNOUe++9l1tuuYVvfetbHDx4kMOHD/PFL36Re+65h927d7Np0yaOHDnS6LJFRETqZu6XHRynkXMiItJQBw8e5OjRo40uQ0RENq5LNtdpwI+IiIiIiEgdFKZERERERETqoDAlIiIiIiJSB4UpERERERGROihMiYiIiIiI1EFhSkREREREpA4KUyIiIiIiInW47H2mzOwJoHflyrlqvcBoo4uQhtC537h07jeuW4BXGl2ENITe9xuXzv3GtRrP/ai7f/ZiT1zppr2rkpkddfeDja5DVp7O/calc79x6dxvXDr3G5fO/ca11s69uvmJiIiIiIjUQWFKRERERESkDms1TD3c6AKkYXTuNy6d+41L537j0rnfuHTuN641de7X5JgpERERERGRRlurLVMiIiIiIiINtabDlJl9xcxeN7NXzOzPG12PrCwz+6qZuZmtxun7ZRmY2V/U3vMvmtk/m1lXo2uS5WVmnzWzN8zsuJk91Oh6ZGWY2Q4z+x8ze7X2Gf9go2uSlWNmoZn9n5n9W6NrkZVjZl1m9ljtc/41M7uz0TVdjTUbpszsk8DngQPufgvwlw0uSVaQme0Afh14p9G1yIp6Eviwu98G/Bz4eoPrkWVkZiHwt8BvAPuB3zGz/Y2tSlZIAnzV3fcDHwfu17nfUB4EXmt0EbLivgM84e43AwdYI78DazZMAV8G/szdKwDuPtLgemRl/TXwx4AG/W0g7v4Td09qmz8FBhpZjyy7jwLH3f0td4+BI+QX0WSdc/dhd3++tj5N/qWqv7FVyUowswHgt4DvNboWWTlm1gn8CvAIgLvH7j7R2KquzloOU3uBXzazp83sf83sUKMLkpVhZp8Hhtz9hUbXIg11L/AfjS5CllU/cGrJ9iD6Qr3hmNlO4BeBpxtbiayQvyG/WJo1uhBZUTcBZ4G/q3Xx/J6ZtTa6qKsRNbqAyzGz/wK2XuSpb5LXvom8+f8Q8I9mtss1PeG6cIVz/w3yLn6yDl3u3Lv747VjvkneDeiHK1mbiKwsM2sD/gn4I3efanQ9srzM7HPAiLs/Z2a/2uh6ZEVFwO3AV9z9aTP7DvAQ8CeNLevKVnWYcvdfu9RzZvZl4Me18PSMmWVAL3mqlTXuUufezG4lv3rxgplB3s3reTP7qLufXsESZZlc7n0PYGa/D3wO+LQunqx7Q8COJdsDtX2yAZhZgTxI/dDdf9zoemRFfAI4bGa/CTQBHWb2D+7+uw2uS5bfIDDo7gst0I+Rh6lVby138/sX4JMAZrYXKAKjDa1Ilp27v+Tum919p7vvJH/z3a4gtTGY2WfJu38cdve5Rtcjy+5ZYI+Z3WRmReBu4F8bXJOsAMuvlj0CvObuf9XoemRluPvX3X2g9vl+N/DfClIbQ+173Ckz+1Bt16eBVxtY0lVb1S1TV/B94Ptm9jIQA7+nq9Qi6953gRLwZK1l8qfu/oeNLUmWi7snZvYA8J9ACHzf3V9pcFmyMj4B3AO8ZGY/q+37hrv/ewNrEpHl9RXgh7WLZ28Bf9Dgeq6KKX+IiIiIiIhcu7XczU9ERERERKRhFKZERERERETqoDAlIiIiIiJSB4UpERERERGROihMiYiIiIiI1EFhSkREREREpA4KUyIiIiIiInVQmBIREREREanD/wPTyN0FZOw/+gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1080x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "%matplotlib inline\n",
    "\n",
    "def sigmoid(x):\n",
    "    return 1./(1+np.exp(-x))\n",
    "\n",
    "def sigmoid_derivative(x):\n",
    "    return x * (1-x)\n",
    "\n",
    "# Calculate plot points\n",
    "x = np.arange(-6., 6., 0.01)\n",
    "y = sigmoid(x)\n",
    "dx = sigmoid_derivative(y)\n",
    "#dz = dx*(1-sigmoid(dx))\n",
    "\n",
    "# Setup centered axes\n",
    "fig, ax = plt.subplots(figsize=(15, 5))\n",
    "ax.spines['left'].set_position('center')\n",
    "ax.spines['right'].set_color('none')\n",
    "ax.spines['top'].set_color('none')\n",
    "ax.xaxis.set_ticks_position('bottom')\n",
    "ax.yaxis.set_ticks_position('left')\n",
    "\n",
    "# Create and show plot\n",
    "ax.plot(x,y, color=\"#307EC7\", linewidth=4, label=\"sigmoid\")\n",
    "ax.plot(x,dx, color=\"#9621E2\", linewidth=4, label=\"sigmoid derivative\")\n",
    "#ax.plot(x,dz,color='#abc222', linewidth=4,label='just')\n",
    "ax.legend(loc=\"upper left\", frameon=False)\n",
    "fig.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sigmoid normalizing function f(x) in this case so that all values would be from 0 to 1 we use this fuction $f(x)=\\frac{1}{1+e^{-x}} \\to f(x)=\\frac{1}{1+e^{\\sum_{i=1}^n x_i w_i}}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_input = np.array([[0,0,1],\n",
    "                            [1,1,1],\n",
    "                            [1,0,0],\n",
    "                            [0,1,0],\n",
    "                            [0,1,1],\n",
    "                            [1,0,1]])\n",
    "'''This is a row but we transpose it becouse I'm lazy, without transpose I should write, \n",
    "training_outputs = np.array([[0],\n",
    "                             [1],\n",
    "                             [1],\n",
    "                             [0],\n",
    "                             [0])\n",
    "\n",
    "'''\n",
    "\n",
    "training_outputs = np.array([[0,1,1,0,0,1]]).T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(199)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Briefly explaining this line of code, we create matrix 3 by 1, and generate number between 1 and 0 \n",
    "#then we multipy by 2 and subtract -1 so that the range would be between 1 and -1\n",
    "s_weights = 2 * np.random.random((3, 1)) -1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "printing initial value of weights: \n",
      "[[ 0.6883052 ]\n",
      " [-0.0723153 ]\n",
      " [ 0.08968613]]\n",
      "training our outputs \n",
      " [[0.15759797]\n",
      " [0.86627626]\n",
      " [0.99485543]\n",
      " [0.15186876]\n",
      " [0.03241353]\n",
      " [0.97310225]]\n",
      " and our error_rates after each itteration \n",
      "[[-0.15759797]\n",
      " [ 0.13372374]\n",
      " [ 0.00514457]\n",
      " [-0.15186876]\n",
      " [-0.03241353]\n",
      " [ 0.02689775]]\n",
      "training our outputs \n",
      " [[0.1114534 ]\n",
      " [0.90460001]\n",
      " [0.99836847]\n",
      " [0.10995447]\n",
      " [0.01525934]\n",
      " [0.98713912]]\n",
      " and our error_rates after each itteration \n",
      "[[-0.1114534 ]\n",
      " [ 0.09539999]\n",
      " [ 0.00163153]\n",
      " [-0.10995447]\n",
      " [-0.01525934]\n",
      " [ 0.01286088]]\n",
      "training our outputs \n",
      " [[0.09040252]\n",
      " [0.92238928]\n",
      " [0.99917649]\n",
      " [0.08971502]\n",
      " [0.00970031]\n",
      " [0.99177552]]\n",
      " and our error_rates after each itteration \n",
      "[[-0.09040252]\n",
      " [ 0.07761072]\n",
      " [ 0.00082351]\n",
      " [-0.08971502]\n",
      " [-0.00970031]\n",
      " [ 0.00822448]]\n",
      "synaptic weigts after training: \n",
      " [[ 7.10673503]\n",
      " [-2.31898343]\n",
      " [-2.31063398]]\n",
      "outputs after training\n",
      "[[0.09040252]\n",
      " [0.92238928]\n",
      " [0.99917649]\n",
      " [0.08971502]\n",
      " [0.00970031]\n",
      " [0.99177552]]\n"
     ]
    }
   ],
   "source": [
    "print(f\"printing initial value of weights: \\n{s_weights}\")\n",
    "for i in range(1):\n",
    "    for iteration in range(1000):\n",
    "        input_layer = training_input\n",
    "        #so basically this line is the whole formula from above, to normalize the product of the input layer with the synaptic weights\n",
    "        outputs = sigmoid(np.dot(input_layer,s_weights))\n",
    "        #calculating errror rate for adjusting s_weights for better rezults\n",
    "        # comparing given rates, to calculated rates \n",
    "        error_rate = training_outputs - outputs\n",
    "        # making the adjustmend with sigmoid_derivative function\n",
    "        adjustment = error_rate * sigmoid_derivative(outputs)\n",
    "        #Updating weights\n",
    "        s_weights += np.dot(input_layer.T, adjustment)\n",
    "    print(f\"training our outputs \\n {outputs}\\n and our error_rates after each itteration \\n{error_rate}\")\n",
    "print(f\"synaptic weigts after training: \\n {s_weights}\")\n",
    "print(f'outputs after training\\n{outputs}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6 (tensorflow)",
   "language": "python",
   "name": "tensorflow"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}