1. Basic Concepts.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets.samples_generator import make_regression \n",
    "from scipy import stats \n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Basic Concepts\n",
    "\n",
    "## What is \"learning from data\"?\n",
    "\n",
    "> In general **Learning from Data** is a scientific discipline that is concerned with the design and development of algorithms that allow computers to infer (from data) a  model that allows *compact representation* (unsupervised learning) and/or *good generalization* (supervised learning).\n",
    "\n",
    "This is an important technology because it enables computational systems to adaptively improve their performance with experience accumulated from the observed data. \n",
    "\n",
    "Most of these algorithms are based on the *iterative solution* of a mathematical problem that involves data and model. If there was an analytical solution to the problem, this should be the adopted one, but this is not the case for most of the cases.\n",
    "\n",
    "So, the most common strategy for **learning from data** is based on solving a system of equations as a way to find a series of parameters of the model that minimizes a mathematical problem. This is called **optimization**.\n",
    "\n",
    "The most important technique for solving optimization problems is **gradient descend**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preliminary: Nelder-Mead method for function minimization.\n",
    "\n",
    "The most simple thing we can try to minimize a function $f(x)$ would be to sample two points relatively near each other, and just repeatedly take a step down away from the largest value. This simple algorithm has a severe limitation: it can't get closer to the true minima than the step size.  \n",
    "\n",
    "The Nelder-Mead method dynamically adjusts the step size based off the loss of the new point. If the new point is better than any previously seen value, it **expands** the step size to accelerate towards the bottom. Likewise if the new point is worse it **contracts** the step size to converge around the minima. The usual settings are to half the step size when contracting and double the step size when expanding. \n",
    "\n",
    "This method can be easily extended into higher dimensional examples, all that's required is taking one more point than there are dimensions. Then, the simplest approach is to replace the worst point with a point reflected through the centroid of the remaining n points. If this point is better than the best current point, then we can try stretching exponentially out along this line. On the other hand, if this new point isn't much better than the previous value, then we are stepping across a valley, so we shrink the step towards a better point.\n",
    "\n",
    "> See \"An Interactive Tutorial on Numerical Optimization\": http://www.benfrederickson.com/numerical-optimization/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradient descend (for *hackers*) for function minimization: 1-D\n",
    "\n",
    "Let's suppose that we have a function $f: \\Re \\rightarrow \\Re$. For example: \n",
    "\n",
    "$$f(x) = x^2$$\n",
    "\n",
    "Our objective is to find the argument  $x$ that minimizes this function (for maximization, consider $-f(x)$). To this end, the critical concept is the **derivative**.\n",
    "\n",
    "The derivative of $f$ of a variable $x$, $f'(x)$ or $\\frac{\\mathrm{d}f}{\\mathrm{d}x}$,  is a measure of the rate at which the value of the function changes with respect to the change of the variable. It is defined as the following limit:\n",
    "\n",
    "\n",
    "$$ f'(x) = \\lim_{h \\rightarrow 0} \\frac{f(x + h) - f(x)}{h} $$\n",
    "\n",
    "The derivative specifies how to scale a small change in the input in order to obtain the corresponding change in the output: \n",
    "\n",
    "$$ f(x + h) \\approx f(x) + h f'(x)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4.0000\n"
     ]
    }
   ],
   "source": [
    "# numerical derivative at a point x\n",
    "\n",
    "def f(x):\n",
    "    return x**2\n",
    "\n",
    "def fin_dif(x, \n",
    "            f, \n",
    "            h = 0.00001):\n",
    "    '''\n",
    "    This method returns the derivative of f at x\n",
    "    by using the finite difference method\n",
    "    '''\n",
    "    return (f(x+h) - f(x))/h\n",
    "\n",
    "x = 2.0\n",
    "print(\"{:2.4f}\".format(fin_dif(x,f)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The limit as $h$ approaches zero, if it exists, should represent the **slope of the tangent line** to $(x, f(x))$. \n",
    "\n",
    "For values that are not zero it is only an approximation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25.000000 25.000000\n",
      "27.562500 27.500002\n",
      "30.250000 30.000005\n",
      "33.062500 32.500007\n",
      "36.000000 35.000010\n"
     ]
    }
   ],
   "source": [
    "for h in np.linspace(0.0, 1.0 , 5):\n",
    "    print(\"{:3.6f}\".format(f(5+h)), \"{:3.6f}\".format(f(5)+h*fin_dif(5,f)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAADFCAYAAACWyLzbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XlcVPX+x/HXDLuICIIguOCCgKIsKoIgy0DuCy6pZZlm2y+zLMsWzTKta+XSvlu5rym4pKmsgrggCCiKAS4hoLlcVxCE+f1xdLikhSYwoJ/n4/F9DMOcOfM9871c333P93yO6vTpi1qEEEIIIcQ9U+u7A0IIIYQQ9wsJVkIIIYQQ1USClRBCCCFENZFgJYQQQghRTSRYCSGEEEJUEwlWQgghhBDVRIKVEEIIIUQ1kWAlhBBCCFFNJFgJIYQQQlQTQ319sJtba5ycnPT18fXOlStXMDc313c3xF/IuNQ9MiZ1k4xL3SNjcneOHj3K4cPHqtxOb8HKycmJ5ORkfX18vRMbG0twcLC+uyH+Qsal7pExqZtkXOoeGZO74+npfUfbyalAIYQQQohqIsFKCCGEEKKaSLCqZ4qKiggKCqKsrKxa9xsVFcXjjz9e6XclJSUEBgZy/fr1Kt8/efJkPDw8ePrpp2ukf9XRRyGEEKKmSbCqZ3788UeGDh2KgYFBte43LS0NLy+vSr8zNjYmNDSUlStX/uN7c3JySExMJC0tDU9Pzxrp3732UQghhKgN92+wunQJhg2DiAgoLdV3b6rN0qVLGTx4MAAhISFs27YNgGnTpjFx4sR/vd+0tDQKCwsJDAykZcuWbN++HYDw8HCWLl36t+/LysoiODiY48eP4+XlxQ8//KDrX13poxBCCFFb7t9gdeQIJCXBkCHQogW8/rryu3qspKSE3NxcXZmKGTNm8P7777N06VJSU1P55JNP/vW+09LSsLW1JT4+nk8//VQXVNzd3dm7d+/fvs/FxYUnnniCmTNnsnv3bgoKCiqV0agLfRRCCCFqy/0brLp0gRMnYP168PWFuXPBxQUCA2HhQrhyRd89vGtnzpyhcePGuueBgYFotVrmzZvHihUrbjn9FhYWhru7+y0tMjKy0nalpaWcPXuWyZMn657f/BwDAwOMjY25dOnS3/YrIyMDDw+PW/pXl/oohBBC1Aa91bGqFYaGMHCg0goLlUC1YAGMHQsTJ8KoUTB+PPj4gEql795WyczMjOLiYt3zjIwMCgoKaNKkCRYWFrdsf/NUWVUOHTqEh4cHarWSs9PT03F3d9e9fu3aNUxNTf/2/QcPHsTd3Z3i4uJK/atLfRRCCCFqw/07Y/VX9vbK6cCsLIiPh6FDYelSZTarUyeYNw/+/FPfvfxHVlZWlJWVUVxcTEFBAaNHjyYyMpKGDRuyZcuWf73ftLQ0PDw8dM/T09Pp3LkzAGfPnsXGxgYjIyMAQkNDOXnypG7bS5cuYWRkhJmZWaX+AXrroxBCCKEvD06wukmlgp494eefoaAAvv0WzM1h8mRwcFAWvG/aBHX08v1evXqxdetWhg4dyty5c3Fzc+Ptt99mxowZ/3qfaWlpupACcODAAd1sUExMDP379wegvLyc7OxsrK2tb7vtzf4lJCRw9epVvfRRCCGE0Kf7+1RgVRo1gmeeUdrBg/DTT7BoEaxdq4SsMWNg3Dho317fPdWZMGEC8+fPJykpSfe7wMDASs/v1pw5cyo9z83N1f28bNkyZs+eDUBmZibDhg3DzMxM97qfnx+rV6++pX9hYWF66aMQQgihTw/ejNXf6dgR5syBvDxYtw68veHjj5UF7wEB8OOPSgkHPfP29iYkJKRGCnD+VUlJCeHh4bS/ESzd3d2ZN29enekf3NpHIYQQQp8kWP2VsTGEh8OGDfDHH/Dhh3D2rLLIvVkzZeF7XByUl+uti08++WSNFOD8K2NjY8aMGXPX76ut/sG/76MQQghREyRY/ZNmzWDKFMjMhJ074ZFHlNOEwcHg7AzvvQfHj+u7l0IIIYSoIyRY3QmVCvz84PvvlbINixdD69bwzjvg5AShocrv6mFtLCGEEEJUHwlWd6tBA3jsMdi+HY4dgxkzlMcxY5QZrqeegoQE0Gr13VMhhBBC1DIJVveiVSuYPh1+/11ZdzV8OKxYoZRzcHaGmTPlVKEQQgjxAJFgVR3UauVWOT/+WFHhvUULJXQ5OYFGo5RxuHxZ3z0VQgghRA2SYFXdGjZUTgvGxMDRo8oC9xMn4IknlOrvY8dCdLReryoUQgghRM2oMlidPJnHkCH9CQjoRs+ePnz33Ve3bKPVannrrdfw8fEgKMiP9PT9NdLZesfJCd5+WzlVuGOHclXhunXKYvfWrWHaNDhyRN+9FEIIIUQ1qTJYGRoaMmPG+yQk7GXz5ih+/PF7srIOV9omKmorubk57N69n7lzP2XKlJdrrMN3Kjs7m3nz5rF//37K9T07pFIpRUZvXlW4fDl06AD/+Y9SgNTPD77+Gs6d028/hRBCCHFPqgxWdnb2dO7sCUDDhha0b+9CQUF+pW02b/6VESMeQaVS0bWrDxcuXODUqcKa6fEdWr9+PZMnT8bLy4umTZvy8MMP880333DkyBG0+rxiz8wMRo2CzZuVKu8ff6ysvXr+eeWqwmHDIDISSkqU7dNXwXx3gmLDYb678lwIIYQQddJd3SvwxInjZGSk06VL10q/LyzMx8Ghue65g4MjBQX52NnZV9pu0aKfWLz4Z917YmNj/12v74BKpaJPnz6kpKRw+vRp1qxZw5o1awCwtbXFy8sLb29vvL29sbW1rbF+VKlrV+jShYY5Odj99ht2UVEYr11LaaNGXBraActWv2NAKSqAC39QFvECWYcOcdouSH99FjqXL1+u0f8di7snY1I3ybjUPTImNUN1+vTFO5q+uXz5MuHh/Zg06VUGDBhU6bXRox9m4sRX8PX1A2DYsIG8/fYMPD29/3Z/ffuGkJycfA9dvzNarZacnByio6OJiooiOjqaM2fOVNqmffv2aDQaQkNDCQ4OxsbGpsb79beuX4ffflMKjjb7FSxVt25j2QJePlD7fRO3iI2NJTg4WN/dEP9DxqRuknGpe2RM7o6npzfbtsVVud0dzViVlpby5JOPMWzYiFtCFYC9vQP5+Xm65/n5J2nWzOEuultzVCoV7dq1o127djzzzDOUl5dz4MABXdCKi4vjyJEjHDlyhG+++QYAT09PXdDq2bMnFhYWtddhQ0Po319p7zYGbs295f/NY+YbRYQ/YkbnzsoSLiGEEELoX5VrrLRaLZMmTaB9exf+7/9euO02ffr0ZdWq5Wi1WpKT99CoUaNbTgPWFWq1ms6dOzNp0iQ2bNjAuXPnSEpK4v3330ej0WBiYsL+/fuZN28e/fv3x9raGn9/f95++21iY2MpLi6uvc5aNr/tr09caM67H5rh6Qlt7K8waWIZsbHKZJcQQggh9KfKGavdu3exevUK3Nw6EhLiD8DUqdPJy1NmqMaOHU9YWG+2b9+Kj48HDRo04NNPby3JUFcZGhri6+uLr68vb731FsXFxezcuVM3o7V371527tzJzp07mTVrFqampgQEBOhmtLy9vTE0vKulancudDpseBFKi3S/0hqacd7hGZ5238n6zHYcO92UT7+AT78A60alDBxiSHi4il69lLvvCCGEEKL23PEaq+pWW2us7tXFixeJj4/XBa309PRKr1taWhIUFKQLWh07dkRVnefm0ldB1HtoL+ShsmyuhK3OIwAov1bK7i+TifjhDOsOu/K71ln3NlOTcnr1VhMeDgMGgD7X59/PZI1C3SNjUjfJuNQ9MiZ3p1rXWD3IGjVqxIABAxgwYAAAf/75JzExMbqglZ2dzfr161m/fj0ATZs2RaPR6IJWmzZt7q0DnUdA5xHE3eYPQG1ihN8rfvi9ArMvXuLwV5FELjhDRHZHdl/zZf16WL8e1GotAQEqwsNh8GC41y4JIYQQ4vYkWN0lW1tbRowYwYgRyqzRiRMniI6O1gWt/Px8VqxYwYoVKwBwcnLSBS2NRkOzZs1qpF+qRha4vTEYtzfgjYIC8r/9jvU/nSXihBfR5Rri442Jj4dXXoFOnSA8XGleXrL4XQghhKgucq/Ae9SyZUvGjh3LokWLyMvL4/Dhw3z55ZcMGzYMa2trjh07xo8//shjjz2Gg4MDHTp0YOLEiaxbt47z58/XTKeaNcPh3Wd47vibbDnkxJ+vfcyKpi8yiuVYcJGMDJg5E7p0gVatYOJEiIqC0tKa6Y4QQgjxoJAZq2qkUqlwcXHBxcWF559/nvLyctLS0nT1s+Lj4zl06BCHDh3iiy++QKVS4e3tTWhoKBqNhoCAAMzNzau3U66uWH40lZEfahm5ezfXFr1D7LJ8Ii4EE6kK548/mvHFF/DFF9C4sbIea/Bg6NNHuZ+0EEIIIe6cBKsapFar8fLywsvLi1dffZWSkhL27t2rC1pJSUns27ePffv28dFHH2FkZISvr68uaHXv3h1jY+Pq6YxKBb6+mPj60vuz6/Tevp0vl7xO8toTRBT1IsJwOIf+254lS2DJEjAxgbAw5XThwIFgZ1c93RBCCCHuZxKsapGxsTH+/v74+/szffp0rl69SmJioi5o7du3jx07drBjxw7effddzM3N6dmzJxqNBktLS3r27ImBgcG9d8TQEPr0Qd2nDz5Xr+KzcSMfLJvCkU2/E3m9HxGmo0gq9mLTJjWbNimZrEcPZSYrPBycnav+CCGEEOJBJOUW6pDz588TFxenWwifmZlZ6XUrKytCQkJ0Vxy6uLhUb2mH8+fhl19g+XIKozPZwAAiGo1h+xU/SsoqMniHDhWL37t0AfUDvFJPLleue2RM6iYZl7pHxuTuSLmFesjKyorw8HDCw8MBKCws1F1xuGnTJgoLC1m7di1r164FwMHBoVJph5YtW95rB+Cpp+Cpp7DPz+fpVat4evmrXNqTyW/0JsL2GTZeDiIz04TMTPjgA3BwqJjJCg6G6jpzKYQQQtRHD/BcQ91nb2/Po48+yg8//MDy5cvJzc3l+++/55FHHsHOzo78/HyWLFnCk08+SatWrWjXrh3PPvssK1eu5PTp0/f24Q4OMGkS7N6Nxe+pDJ/lxZKmr/BnUUO2qXvzQstImltdIT8fvv4aevdWipA++iisXAkXL1bPdyCEEELUJxKs6pHWrVvz1FNPsWzZMgoKCjhw4ACffvopgwcPxtLSkpycHL777jtGjRqFnZ0dnTt35uWXX2bDhg1cvJek064dTJ0KBw5glJ5C2Otd+NzgZU6cb0iyoS/T2q/CvcV/uXgRli+HUaPAxgb69oVvv4WCgur7DoQQQoi6TIJVPaVSqejYsSMvvvgiERERnDlzhj179vCf//yHhx56CDMzMzIyMvjkk08YNGgQ1tbW+Pr6MnXqVKKioigqKqr6Q26nUyflHGBODqrdu+kysQczL79Mxh9WZJt0ZK7HInq6neH6dS1btsBzzymTX76+MHs2HD5cvd+DEEIIUZdIsLpPGBoa0q1bN9544w22bt3K+fPniY2N5e2338bf3x+VSsXu3bv54IMPCAsLw8rKCo1Gw/vvv09SUhLXr1+/uw9UqcDHB+bNgz/+gLg42o4L5JWTk4k/ZEthg7Ys8P2egT6nMDXVsns3vPkmuLmBqyu88Qbs2gXl5TXzfQghhBD6IMHqPmViYkJQUBDvvfceCQkJnDt3jl9//ZXJkyfj5eVFSUkJMTExTJs2jR49emBtbc2AAQOYP38+aWlplN9N4lGrITBQWWxVUABbt9J0lIYnD09h/R57zpg0Z63mC8aEncTKSktWFnz4Ifj5gaMjPPssbN4M167V3PchhBBC1Aa5KvABYWFhQd++fenbty8AZ8+eJTY2VldDKysri02bNrFp0yYAbGxsCAkJ0RUrbdeu3Z2VdjA0hIceUtpXX8G2bZivXMmQiLcYcmki162bktD/NSJMRhCxrwXHj6v47jv47jul0nu/fspVhv36KZXghRBCiPpEgtUDqkmTJgwbNoxhw4YBkJeXR0xMDFFRUURFRZGXl8fq1atZvXo1AC1atNCVddBoNDg6Olb9IcbG0L+/0oqL4bffMFy5kuD17xJ85TXm29iSNvwlIho8SkSaE2lpKlatglWrlHwWEqKUcRg0CJo3r8lvQwghhKgeEqwEAM2bN+fxxx/n8ccfR6vVkp2drStUGhMTwx9//MHChQtZuHAhAC4uLrqgFRwcTJMmTf75A0xNlamowYPh6lXYsgXVqlV4bvgAz6vTeNfOjmOjnyKy8RNEHGhH/A4V27bBtm0wYQJ07VpRlLRDB2WJlxBCCFHXSLASt1CpVDg7O+Ps7Myzzz5LeXk5GRkZuqAVFxdHVlYWWVlZfP3116hUKjw9PXVBq2fPnjT8pzs4N2gAQ4cq7epV+PVXWLkSp7XzeKnofV6ys+PMmMfZZPckEYdd+W2riuRkSE6GadOU6g/h4UpG8/OD6rjLjxBCCFEdJFiJKqnVajw8PPDw8ODll1+mtLSU5ORkXdDauXMnqamppKamMnfuXAwNDenevbsuaPn6+mJiYnL7nTdoAMOHK+3KFSVkrVqFzcoveaJoDk/Y2XF19Ei2txpPRLY76zeqyc6GOXOUZmurnCoMD4fQUDAzq93vRgghhPhfEqzEXTMyMsLPzw8/Pz+mTp1KUVERO3fu1AWtvXv3kpiYSGJiIjNnzsTMzIyAgABd0PL29r79zaTNzeHhh5V25Qps2gSrV9Ng6fcMKvqMQba2XB86nJ1u44k84UXEejW5ubBggdLMzZUK8OHhyrIua+va/26EEEI82CRYiXtmZmZGaGgooaGhvP/++1y4cIH4+Hhd0MrIyGDbtm1s27YNAEtLS4KDg3VBq0OHDrdecWhuDiNGKO3KFaUew5o1GC5bROCVrwls0oQ54UM48NpYIgp9idxowL59sHat0gwMICio4pThvd5GUQghhLgTEqxEtbO0tGTgwIEMHDgQgNOnTxMTE6MLWjk5OURGRhIZGQmAnZ1dpZtJt27duvIOzc0rThcWFcGWLbBmDapVK+m04Ac6WVnx9qBBnHj+MdZfDCJioxGxsRAdrbQXXwQvr4rF7506yeJ3IYQQNUOClahxTZs2ZeTIkYwcORKA48ePEx0drQtaBQUFLF++nOXLlwPg5OSkK+ug0Wiwt7ev2JmZGQwZorTiYuWywTVrICKClgsX8oKFBS8MHMj5H0exqbQXkVtM2LwZUlOV9s470Lp1xUyWv79S2kEIIYSoDvJPiqh1rVq1Yty4cYwbNw6tVktWVpauUGlMTAzHjh1jwYIFLFiwAIAOHTroglZwcDCNb1YONTWFgQOVVlICUVHwyy8QEYHVsmU81qABj/XtS/GXDxPVYCARWxuwfj0cPQrz5yutSRPl7eHhSk3TBg30+MUIIYSo9yRYCb1SqVS4urri6urKhAkTKCsrIy0tTRe04uPjyczMJDMzk88//xy1Wo23t7cuaAUEBNCgQQOlGGnfvkr75huIi1NC1rp1mP7yC/2Njen/0EN888FwdjUbQmSsJevWQXY2/Pyz0szMoFcvJWQNGAA2Nvr+doQQQtQ3Vd4r8KWXnqdDhzYEBna/7euJiTto27Y5ISH+hIT4M2fO7GrvpHhwGBgY4O3tzWuvvcbmzZs5f/48O3bs4N133yUwMBADAwOSk5P58MMP6d27N40bN650T8TS0lLl3F5oqHJLnZMnISEBXngBDhzA4Klx+A9qwkf7Qjny0pccjD7F++8r95MuKoLISBg3DuzsIDhYmdU6elTf34oQQoj6ospgNWrUaFasWPuP2/j6+hETk0hMTCKvvvpGtXVOCGNjYwICAnjnnXeIi4vj/Pnz/Pbbb0yZMoWuXbty/fp14uPjeeedd+jZsydWVlb069ePOXPmkJKSQjkoC6nmzlUSUnIyvPEGFBSgmvgCHTT2vLXBj93DPyZvx1G++kqZtVKrlUmvV16BNm3Aw0NZn5WaClqtvr8VIYQQdVWVpwL9/Pw5ceJ4bfRFiCqZm5vTq1cvevXqBcD58+eJi4vTnTrMzMxk8+bNbN68GQBra2uCg4N15SDae3uj6tIFZs2CQ4dg3TqlPsOUKTgyhf9zd+f/hgzhwrThbM7rRESkil9/hfR0pb33nlK6YfBg5ZRheblcXiiEEKKC6vTpi1X+9/eJE8d57LERxMfvvuW1xMQdPPnkYzRr5oi9vT3vvvs+rq5ut93PokU/sXjxzwAUFuazYsWKe+v9A+Ty5cv/fJsYAcDZs2dJTU0lJSWFlJQUTp06Vel1GxsbvLy88Pb2xsvLCzs7OwBMCguxSUjANiEBy4wMVOXlFNnbcyYggHzfIOJK/ElIsiMxsQnnzlVUkW/YsAQ/v3MEBJyhW7dzmJmV1+rxilvJ30rdJONS98iY3J1Jk15h27a4Kre752B16dJFVCo1DRs2ZPv235g69XV2795f5Qf37RtCcnJyldsJRWxsLMHBwfruRr2Tm5urK+sQHR3N6dOnK73erl07Xf2skJAQbG1t4c8/Yf16ZTZr+3a4dk1ZyT5oEOWDh7DXMoyILaZERMDhwxX7MjWFsDBlJmvgQGjatJYPVgDyt1JXybjUPTImd8fT0/uOgtU9XxVoYdFI93NYWG9ef30yZ8+epUmTJve6ayHuWZs2bWjTpg1PPfUUWq2WgwcP6oJWbGws2dnZZGdn89133wHQuXNn3RWHgcuW0UilUgqSRkTAmjWof/yR7ubmdO/Th/+8NZillxzJu6QhMhKSkmDjRqWpVMrSrpunDNu10/MXIYQQolbcc7A6deoUTZs2RaVSkZKSTHl5OdZykzZRB6lUKtzd3XF3d+fFF1/k+vXrpKSk6IJWQkIC6enppKenM3/+fAwMDOjWrZsyozV+PD2+/hrTXbuUkBUZCb/8wqNqNargYF4fNZiCT4eyIbU5ERFKSa2EBKW99hp07FhR+b1LF6n8LoQQ96sqg9Wzz44jMTGBc+fO4uHhypQpbymXtANjx45n48YIfv55AQYGhpiZmfLttz/det83IeogQ0NDfHx88PHx4Y033uDatWskJSXpgtbu3bvZtWsXu3bt4oMPPsDExAR/f39lRmv1aroCJ7/8klapqfDSSzTjJZ7x9OSZwYO59NYQtuR3JiJSxaZNcPCg0t5/HxwdK2aygoKUElxCCCHuD3e0xqomyBqruyPnwmvfpUuX2LFjhy5o7d9fee2ghYUF7u7ujBgxAk3btrgfOoR6/XrYuVOpydCyJQwaREnfwcQRRMQmIyIjldJaN1laQv/+Ssjq0wcsLGr5IO9D8rdSN8m41D0yJnen1tZYCXG/srCwoF+/fvTr1w+AM2fOEBsbq1sIf+TIEZKSkkhKSgLA1taWkJAQQj/6CE1pKW137UK1YAHGX3zBQ40a8VC/fnzx0SD2NRtARJQFERHKLNayZUozNlbqmoaHw6BB8L+3SBRCCFE/SLAS4g7Z2NgwfPhwhg8fDkBeXh5ffvklBQUFREVFkZeXx6pVq1i1ahUALVu2RDNsGKG2tmgKCnCIjka1YgVdDQ3pGhjIrKcH8bv7ECJTWxIZCYmJsHmz0p57Dnx9K04Zurjo88iFEELcKQlWQvxLzZs3p3fv3gQHB6PVasnOziYqKoqoqChiYmI4ceIEPy9Zws83tnd1dUUTEEAoEHzwINaTJuHMJF7t2JFXBw7k1OtD2VjQhYj1arZtU64yTEpSCsW7ulYsfu/WTakML4QQou6RYCVENVCpVDg7O+Ps7Mxzzz1HeXk56enpuvVZ8fHxHD58mMOHD/PVje29OnZEY2tL6LlzBHz0EXblsxlva8v4/v25vCCcrfQi4jczNm5U6mXNnq20Zs2UU4Xh4RASAiYmVXZPCCFELZFgJUQNUKvVeHp64unpySuvvEJpaSl79+7VBa2dO3eScvAgKcAclCsUfZ2d0RgZEbpmDd1//pmhxsYMDQ6m9J3B7LAdSsQueyIj4cQJ+PZbpVlYQL9+yinDfv2UxfBCCCH0R4KVELXAyMiIHj160KNHD6ZNm0ZRURGJiYm6oJWcnExCVhYJwHuAmYkJPe3t0aSkELp1K0FMQOPuzqePDGC/y0gicjsTsV5NejqsXKk0IyNlBuvm4ndHR30ftRBCPHgkWAmhB2ZmZoSFhREWFgbAhQsXiIuL0wWtAwcOsPX4cbbe2L6xmRnBhYVoPv6I0PLZvGtlxYx+/Tg6fiSRl0OJ2NqAHTtg61alPf88+PhULH53c5OipEIIURskWAlRB1haWjJo0CAGDRoEKHc0iImJ0QWt3NxcIoqKiLixvX1xMZo1a9AsXUq4Ws0kf3/OTB3ORpNhROx1YOtWFXv2wJ49MHUqODsrAWvwYOVqQwMD/R1rXTV58mS2b9+Oj48PR44cITo6GoO/+aJKSkoICwsjOjoaQ0P5v1EhRAW5tkiIOsjOzo5Ro0bx3XffkZOTw9GjR1mwYAGjR4/G3t6ewqIill27xlNA6/Jy2uzezZszX8JkWnO+3deSM6NfImJaMmMfu06TJvD77/DxxxAQAA4O8PTTsGkTFBfr+0jrhpycHBITE0lLS8PT05OhQ4f+bagCMDY2JjQ0lJUrV9ZiL4UQ9YEEKyHqAScnJ5588kmWLFlCfn4+mZmZfP755wwZMoTGjRtztKSEH4BHAfu8PHwWfE70rG6Er2zAYY/exL64lpefvICTE5w+DT/8AAMGgI0NPPwwLFkC58/r+SCrWUhICNu2bQNg2rRpTJw48bbbZWVlERwczPHjx/Hy8uKHH35g8ODBVe4nPDycpUuX1vBRCCHqG5nDFqKeUalUuLm54ebmxgsvvEBZWRn79+/XVYTfsWMHB69e5SDwWWkp6uitdIneSijwbfMWWD7yPL8ZjyAizYnU/WrWrIE1a8DQULl34c1Thi1a6PtI782MGTOYPn06p0+fJjU1lfXr1992OxcXF5544gmcnJwYM2YMLVu2xMnJqcr9uLu7s3fv3to4FCFEPSLBSoh6zsDAgC5dutClSxemTJlCSUkJu3fv1gWtXbt2sbe0lL3A7Lw/MF7+Jn68yRAjI6akHPn4AAAgAElEQVR2HcQfrV9iw8luxO02JSoKoqJg4kTw9q4oSuruXv8WvwcGBqLVapk3bx6xsbH/eGovIyODwYMHc+bMGRo3bnxH+zEwMMDY2JhLly5hITd5FELcIMFKiPuMsbExPXv2pGfPnrz77rtcuXKFhIQEXdBKSUkhTqslrrQUkn/BPPkXAoF3bF3AeRKp6iH8ltKUlBQVKSkwfTq0aVMxk+XvXz8Wv2dkZFBQUECTJk2qDD4HDx7E3d2d4uJiiv+y8Oyf9nPt2jVMTU2rve9CiPpL1lgJcZ8zNzend+/efPTRRyQnJ3PmzBnWrl3LhAkTcHNz4wqwGZj+ZxbTd/4fcQn29Cq24P9avcBwrxRsrcvIzYV585RThfb28OSTsH49FBXp++hur6CggNGjRxMZGUnDhg3ZsmWL7rXQ0FBOnjype37p0iWMjIwwMzPDysqKsrIyXbj6p/2cPXsWGxsbjIyMau/AhBB1ngQrIR4w1tbWDBkyhC+++ILMzExOnjzJkiVLGDduHC1btOAcEFF+ha+Pf8ma1C4YnTOmt1kv+rRaTaumFzlzBn76SZm9srGBIUNg4UI4e1bfR6YoLi5m6NChzJ07Fzc3N95++21mzJgBQHl5OdnZ2VhbW+u2P3DgAO7u7rrnvXr1IiEhgatXr/7tfgBiYmLo379/7R2YEKJekFOBQjzgHBwcGD16NKNHj0ar1ZKbm6urnxW9bRv5586RX7QNjitXxrWkI3aNxnDOaDg5Z9sQEQEREcrpwZ49K04Z/s/671plampKUlKS7nlgYKDueWZmJsOGDcPMzEz3up+fH6tXr9Y9nzBhAvPnzycsLOxv9wOwbNkyZs+eXZOHIoSohyRYCSF0VCoVbdu2pW3btjz99NNotVoOHDigBK2tW4mLjeXE1YOcuPg68DrgiKN6CAZmj3CyqDuxsQbExsKkSeDhUbH43cOjbix+d3d3Z968ef+4jbe3NyEhIZSVlf1jgdDw8HDat29fE90UQtRjEqyEEH9LpVLRqVMnOnXqxEsvvcT169fZt2+fErQ2bSJxzx5Oln4BV74ALFHRD2vD4VzS9iEtrQFpaTBjBrRqVXF7nZ49ldIOddmTTz75j68bGxszZsyYWuqNEKI+kTVWQog7ZmhoSPfu3XnzzTfZnpDA+YsXiY6OZtpbb+Hn3gK1agVnrw+jpMwK6IsB32KsOs3x4/DZZ6DRgJ0dPPEErFsHV67o+4iEEKJ6SbASQvxrpqamhISEMPP999mZkcH5CxfYuHEjrzz/FB6tDlHGc5Ro7QE/YDZqsjh3DhYtgqFDwcZGy6BB8OOP8Oef+j4aIYS4dxKshBDVxsLCgv79+zP3yy/Zf+wYf/75J6tWruC5UU44286jHFfAFXgD2EVxsYoNG2D8eLC31xIYqGXePMjJ0fOBCCHEvyTBSghRY2xsbHh4xAi+Xr6cI6dPc+LECX7+4TXG9N6Po0UvwAF4FthMeXkpO3aomDwZ2rUDN9dS3n4b9u0DrVbPByKEEHdIgpUQota0aNGCJ8aPZ+GWLfxx4QJZWbF8Nac9w33fw8rYERgJLAMucDjLiFmzoGtXsG1ymaefLmb7digt1fNBCCHEP6gyWL300vN06NCGwMDut31dq9Xy1luv4ePjQVCQH+np+6u9k0KI+49KpaJ9+/b83+TJrE5K4kzRKVJT32TOmwfo0yEYE3Vf4CvgJGfPN+SHH0x56CGwML+MJugES5YUc+lSxf4iUk/iPzuasVuu4D87mojUk3/30UIIUWOqvOh51KjRjB//DC+88OxtX4+K2kpubg67d+9n3769TJnyMlu2xFR7R4UQ9ze1Wo2npyeenp5M/uADSktL2bNrF9uXvk/khgLS8rtRzmCulXYkJr4hMfGg4hrtHA/jNeoa+03Pcu16OQAn/1vEm2szAAj3ctTnYQkhHjBVzlj5+fnTuLHV376+efOvjBjxCCqViq5dfbhw4QKnThVWayeFEA8eIyMj/Hv25J1vviLl5DouXZnE1o3HeKb/K7RoNBNIRIsRv5/0IPHKZV2ouqmotIyPf8vST+eFEA+sey7TV1iYj4NDc91zBwdHCgrysbOzv2XbRYt+YvHin3Xvi42NvdePf2BcvnxZvq86SMaldhmZm/PIq4N45FW4fPk8O7d/SNQGKGzU+bbbnzx/ldceewz3sDBatmqFqi6Uf39Ayd9K3SNjUjNqtf7xmDHjGDNmHAB9+4YQHBxcmx9fr8XGxsr3VQfJuOjXgAED4BPwnx3Nyf8W3fL69Yt/MmfpUli6lGampmg6d0YTHk7oI4/QSl83M3xAyd9K3SNjUjPu+apAe3sH8vPzdM/z80/SrJnDve5WCCHu2Gu9XTAzqnxfPxNDFcObXeNRX1/sTU0pKC5m6Z49jH/rLZxat6athQXPBAWx4vPPOX36tJ56LoS439xzsOrTpy+rVi1Hq9WSnLyHRo0a3fY0oBBC1JRwL0f+M7QTjo3NAHBsbMaHwzz4ZPpzLE1KIv/qVQ4eOMBnU6cS7uFBYyMjci9f5vv4eB558UXs7OzoZGXFpN69Wb9gARcuXNDzEQkh6qsqTwU+++w4EhMTOHfuLB4erkyZ8halNwrJjB07nrCw3mzfvhUfHw8aNGjAp59+VeOdFkKIvwr3ciTcy/G2pzdUKhUdOnakw6xZTJw1i7KyMlJTUohatozoLVvY8fvvHPjvfzmwdSufbt2K+qmn6GpjQ6ifH5pRo/AfMgQzMzP9HJgQol6pMlh9++1P//i6SqXiww/nVVuHhBCiphkYGNC1Wze6duvG6/Pnc+3aNXYnJRG1fDnR27ez6+hR9pw5w54NG/jPhg0YAz3s7Qnt0QPN6NF0GzgQIyMjfR+GEKIOksrrQogHnomJCYHBwcz49lt25ORw/uJFNm/axKtjxuDdvDmlQGxhIW+vXYv/sGFYm5jQv3lz5o0axf6NGykvK9P3IQgh6ohavSpQCCHqg4YNG9KnXz/69OsHwNmzZ4mLiSFq9Wqi4+I4fOoUv548ya8rV8LKlTRRqwlp3pzQgAA0jz+Oc69eqNTy361CPIjkL18IIarQpEkThg4fzpcrV3KosJC8vDwWL1zI2MGDaWFlxdnyctacOMH/LVuGS9++tDQy4gknJxY9/jh5v/0GMqMlxANDZqyEEOIuOTo68tiYMTw2ZgxarZacnByio6KIiowkescO8i5fZtHx4yw6fhyWLKG9Wo2meXNCe/Yk+NFHsQkLA2NjfR+GEKIGSLASQoh7oFKpaNeuHe3ateOZZ5+lvLycAwcOEB0dTdTGjcQlJnKkuJgjJ07wzY1ipZ4qFRpHR0IDAug5ciQWoaFgYaHvQxFCVAMJVkIIUY3UajWdO3emc+fOTJo0ievXr5OcnKwErc2bSdy9m/2lpezPy2PeihUYrliBD6C5cdWh77BhmGo0YC/1AIWojyRYCSFEDTI0NMTX1xdfX1/eeustiouL2blzpxK0tm5l77597CwvZ2dhIbPWrsV07VoCAI21NaG+vngPHIhhUBC4uoLc61CIOk+ClRBC1CJTU1M0Gg0ajYZZs2Zx8eJF4uPjlaC1fTvpGRlsB7afOwe//orlr78SBGgaNCC0Wzc69umDKiAAunYFU1N9H44Q4i8kWAkhhB41atSIAQMGKDeUBv78809iYmKUoBUVRXZ2NuuB9VevQlwcTePi0AAaAwNCO3emTWgo+PtDjx7QtKlej0UIIcFKCCHqFFtbW0aMGMGIESMAOHHiBNHR0bqglZ+fzwpgRVkZpKbilJqKZs4cJWw5OdEsMFAJWT16QIcOYGDwj58nhKheEqyEEKIOa9myJWPHjmXs2LFotVqOHDlCVFQU0dHRxMTEcOzcOX4EfgQ4dgy3P/4gdNEiNEBww4ZY+fnBzebrC40b6/eAhLjPSbASQoh6QqVS4eLigouLC88//zzl5eWkpaXpglZ8fDyHrlzhEPAFoL5yBe+EBDTbthEK+APmHTpUBC0/P2VRvFSJF6LaSLASQoh6Sq1W4+XlhZeXF6+++iolJSXs3btXF7SSkpJILioiGfgIMDIwwK+wEM2yZYQuWIAPYGxpCd27K7NZvr7Kz9bWej4yIeovCVZCCHGfMDY2xt/fH39/f6ZPn87Vq1dJTEzUBa19+/YRf+4c8cC7gLmJCT0bNUKTkUHotm14aLUYALi4VA5bnTqBofxzIcSdkL8UIYS4TzVo0ICHHnqIhx56CID//ve/xMXF6YLWwYMH2fLHH2y5sb2VhQUhLVuiUasJ3bgRl0WLUAGYmUGXLkrYutlatJC6WkLchgQrIYR4QDRu3JjBgwczePBgAAoLC4mJiSEqKoqoqCiOHTvG2oMHWXtjewc7OzTOzmjMzAg9e5aWX3wBc+cqL9rbg4+PErJ8fJS6WrIwXggJVkII8aCyt7fnkUce4ZFHHgHg6NGjurIO0dHR5J86xZJTp1hyY/u2bdoQ6uGBpnFjQi5domlGBqxfX7FDFxclZN1sHh5gYlL7ByaEHkmwEkIIAUDr1q0ZP34848ePR6vVkpmZqQtasbGx5OTmkpOby3c3tu/UqROhzz2HplkzgkpKaJSRAVu3wuLFygZGRkq46tYNewsLsLVVrkKU2lriPibBSgghxC1UKhUdO3akY8eOTJw4kbKyMlJSUnRBKyEhgYyMDDIyMvgEMDAwoGvXroQ++SSaTp3oodVilpYGe/fCkiW4XroEH30E5ubKeq2uXaFbN+WxbVtZryXuGxKshBBCVMnAwIBu3brRrVs3Xn/9da5du8auXbt0QWv37t269gFgYmJCjx49CA0NRTNzJtqsLHoYGipBa+9e+PJLuHZN2XnjxkrAuhm4unaFVq0kbIl6SYKVEEKIu2ZiYkJQUBBBQUHMmDGDy5cvs2PHDl3Q2r9/PzExMcTExADKFYohISFK0HrqKTq5uqI+dAiSk5W2dy/MmwelpcoHNGmiBK3/bRK2RD0gwUoIIcQ9a9iwIX379qVv374AnD17ltjYWF3QysrKYtOmTWzatAkAGxubiqA1eTLt2rVDVVICGRlK0Nq3T3n8+GO4fl35kCZNwNtbCVne3kpr00bClqhTJFgJIYSodk2aNGHYsGEMGzYMgNWrV1NcXKwr7ZCXl8fq1atZvXo1AC1atECj0ShBa+BAHJ97TtlRcbEStm4GrX37lJIPN2e2LC0rQtbN5uwsC+SF3kiwEkIIUeNsbW0JDg7m8ccfR6vVkp2drZvNiomJ4Y8//mDhwoUsXLgQABcXF13QCg4Opkm3bhU7u3YNDhyAlBQlaO3bB198UbFmy9xcuRrRy6uidewopR9ErbijYBUdvY2pU1+nrKyMxx57ghdffKXS6ytWLGXGjGnY2zsAMH78Mzz22BPV31shhBD1nkqlwtnZGWdnZ5599lnKy8vJyMjQBa24uDiysrLIysri66+/RqVS4enpqQtaPXv2pOHNdVdPP63stLQUDh9WwtbNtnChskgelNIPHTpUBC1PTyV8WVrq74sQ96Uqg1VZWRmvvz6Z1asjcXBwpFevYHr37oeLi2ul7QYPHsrs2XNrrKNCCCHuT2q1Gg8PDzw8PHj55ZcpLS0lOTlZF7R27txJamoqqampzJ07F0NDQ7p3764LWr6+vpiYmCj3NOzUCZ648R/25eWQkwOpqRXt11/h558rPrxNGyVk3WweHnK7HnFPqgxWKSnJtG7dBien1gAMGTKMLVs23RKshBBCiOpgZGSEn58ffn5+TJ06laKiInbu3KkLWnv37iUxMZHExERmzpyJmZkZAQEBuqDl7e2NgYEBqNXKeitnZxgxouIDCgpg/36lpaYqj+vWgVarvG5lpQSs/21yKlHcoSqDVWFhAY6OzXXPmzVzICUl+ZbtNm5cT1LSTtq2bcfMmf+p9J6bFi36icWLf76x33xiY2P/fc8fMJcvX5bvqw6Scal7ZEzqpnsdFwMDA90NpS9fvkx6ejqpqamkpKSQm5vLtm3b2LZtGwDm5uZ4enri5eWFt7c3Tk5OqP46A2VmBn5+SgMMioowz82lYXa2rpknJWFwY92WVq3masuWXG7blstt23LlxmOJtXW9nd2Sv5WaUS2L13v16sOQIcMxMTFh4cIfmTjxOdau3XjLdmPGjGPMmHEA9O0bQnBwcHV8/AMhNjZWvq86SMal7pExqZuqe1wGDBig+/n06dPExMToZrRycnJ0M1oAdnZ2aDQa3YxW69at7+xDysogOxvS0lClpWGeloZ5ejp2UVEV29jYQOfOSuvUSXns0AEaNKi2Y60p8rdSM6oMVvb2zTh5Mk/3vKAgn2bNHCptY23dRPfzY489wXvvTa/GLgohhBB/r2nTpowcOZKRI0cCcPz4caKjo3VBq6CggOXLl7N8+XIAnJyclLION8KWvb397XdsYKDcWNrFpfKpxPPnlRIQ6emQlqb8/N13cPWq8rpardym5+aar5utbVspA/EAqDJYeXl1ITc3l+PHj9GsmQPr1v3CN98sqLTNqVOF2Nkp/8PcsuVX2rdvXzO9FUIIIarQqlUrxo0bx7hx49BqtWRlZREVFUV0dDQxMTEcO3aMBQsWsGCB8m9Zhw4ddEErODiYxo0b//MHWFlBYKDSbiovh9xcJWylpythKyOj8totU1NlNsvdXQla7u5Kc3Sst6cTxa2qDFaGhobMnv0xI0cOoaysjEcffRxXVzdmz56Fp6c3ffr04/vvv+G3337FwMAQKysrPvvsm9rouxBCCPGPVCoVrq6uuLq6MmHCBMrKykhLS9MFrfj4eDIzM8nMzOTzzz9HrVbj7e2tC1oBAQE0uJPTemo1tGuntKFDK35/9SpkZioh68ABpW3bBosWVWxjaakErI4dKx47doSmTSVw1UOq06cvavXxwX37hpCcfOsieHF7ci68bpJxqXtkTOqmujouJSUl7NmzRxe0kpKSKL1Z1Z2KKxRvBq3u3btjZGR07x989qwSsg4erPx47lzFNk2aVISsjh2V2a6OHcHWtloCV10dk7rK09ObbdviqtxOKq8LIYR4YBkbGxMQEEBAQADvvPMOV65cITExURe09u3bR3x8PPHx8bzzzjuYm5sTGBioW5/l6emJWq2++w9u0gSCgpR2k1YLhYVKyPrftmwZXLhQ+b03Q5abm/Kzmxs4OMgMVx0gwUoIIYS4wdzcnF69etGrVy8Azp8/T1xcnC5oZWZmsnnzZjZv3gyAtbU1wcHBhIaGEhoaSvv27W8t7XCnVCpo1kxpYWEVv9dqIT9fOaV48KDymJkJK1bAf/9bsV2jRkrA+mtr3VoWzdciCVZCCCHE37CysiI8PJzw8HAACgoKiImJ0d1M+vjx46xdu5a1a9cC4ODgUOmKw5YtW957J1QqZYG7oyM89FDF77VaOHUKDh1SgtbNxy1bKleXNzGB9u3B1VVpbm7g6oq6qOje+yZuIcFKCCGEuEPNmjXj0Ucf5dFHHwUgNzdXV9YhOjqa/Px8Fi9ezOLFiwFo166drn5WSEgItra21dcZlQrs7ZUWElL5tfPnlXsnHjqktMOHlSrzv/yiXMEIBIJy+x5XV6WkxM1HFxdo3lxOK/5LEqyEEEKIf6lNmza0adOGp556Cq1Wy8GDB3VBKzY2luzsbLKzs/nuu+8A6Ny5s25GKzAwkEaNGtVMx6ysKlWW1ykuVoqeHjrE0S1baH3tGmRlKTNcly9XbNeggTLLdTNo3fy5fXvllKP4WxKshBBCiGqgUqlwd3fH3d2dF198kevXr5OSkqILWgkJCaSnp5Oens78+fMxMDCgW7duuhmtHj16YGpqWrOdNDXV1c86bmtL65tXBd5cx5WVVbnt2QOrVlXU4gKws1MC1s3m7Kw8tm2r7P8BJ8FKCCGEqAGGhob4+Pjg4+PDG2+8wbVr10hKStIFrT179rBr1y527drFBx98gImJCf7+/roZra5du2JoWEv/TP/vOi6NpvJrxcVK8dMjRyoC1++/w4YNcPp05X20aFFx4+ubrV07ZQH9AxK6JFgJIYQQtcDExITg4GCCg4N57733uHTpEjt27NAFrf379+tuxQNgYWFBUFCQLmi5u7v/u9IO9+pmxfgOHW597cIFJWQdOaI83mwrVyrrvG5SqaBly4oiqu3aKTNc7dpBmzZgbl57x1PDJFgJIYQQemBhYUG/fv3o168fAGfOnCE2NlYXtI4cOcLGjRvZuHEjALa2toSEhOiCVtu2bf99aYfqYmkJXbsq7a/OnVNCVna20m7+vHp15UKooJSYaNu2cmvTRnm0salXC+klWAkhhBB1gI2NDcOHD2f48OEA5OXlVbqZdF5eHqtWrWLVqlUAtGzZUrc+S6PR4ODgoM/u38raGrp3V9pfnT8POTlKuxm8cnKU2/0sXFh5WwsLJWTdDFqtW1c8b9VKKSdRh0iwEkIIIeqg5s2bM2bMGMaMGYNWqyU7O1tX1iE6OpoTJ07w888/8/ONmlWurq66oBUcHIy1tbV+D+CfWFn9/UxXUREcPaoErdzcinb4MGzerKz5uunm2rCnn4bp02uv//9AgpUQQghRx6lUKpydnXF2dua5556jvLycjIwMXdCKi4vj8OHDHD58mK+++gqVSoWXl5cuaAUEBNCwYUN9H8adMTP7+zVd5eXKbX+OHlXC1s1HR8fa7+ffkGAlhBBC1DNqtRoPDw88PDx45ZVXKC0tJTk5WVcRfufOnaSkpJCSksKcOXMwNDTE19dXF7S63+70XH2gViv3RHRwAH9/fffmtiRYCSGEEPWckZERfn5++Pn5MW3aNIqKiti5c6duRmvv3r0kJCSQkJDAe++9h5mZGR07dmT48OGEhobi5eWFgdxPsFpIsBJCCCHuM2ZmZrobQwNcuHCB+Ph43YzWgQMHSE5OJjk5GYDGjRsTHBysm9Fyc3PT/xWH9ZQEKyGEEOI+Z2lpycCBAxk4cCAAp06d4ssvv6SwsJDo6GhycnKIiIggIiICAHt7e92NpENDQ3FyctJj7+sXCVZCCCHEA8bOzg6NRkPwjVvaHD9+XFfWISoqisLCQpYtW8ayZcsAaN26ta6sg0ajwc7OTo+9r9skWAkhhBAPuFatWjFu3DjGjRuHVqvl8OHDuqAVExPD0aNH+eGHH/jhhx8A6Nixoy5oBQUF0bhxYz0fQd0hwUoIIYQQOiqVCjc3N9zc3JgwYQJlZWXs379ftxB+x44dHDx4kIMHD/LZZ5+hVqvp0qWLLmj5+/vToEEDfR+G3kiwEkIIIcTfMjAwoEuXLnTp0oUpU6ZQUlLC7t27dUFr165d7N27l7179zJ79myMjY3x8/PTBS0fHx+MjIz0fRi1RoKVEEIIIe6YsbExPXv2pGfPnrz77rtcuXKFhIQEXdBKSUkhLi6OuLg4pk+fjrm5OYGBgbqg5eHhoZ+bSdcSCVZCCCGE+NfMzc3p3bs3vXv3BuDcuXPExcXpgtahQ4fYvHkzmzdvBsDa2rrSzaTbt29/X5V2kGAlhBBCiGpjbW3NkCFDGDJkCAD5+fnExMTorjg8ceIEv/zyC7/88gsAjo6OlW4m3aJFC312/55JsBJCCCFEjXFwcGD06NGMHj0arVZLbm6u7orD6OhoTp48yeLFi1m8eDEAzs7OlW4mbWtrq+cjuDt3dJIzOnobfn7e+Ph48Nln8255/dq1azz99Fh8fDzo0yeEEyeOV3tHhRBCCFG/qVQq2rZty9NPP82KFSs4deoU6enpfPLJJwwcOJBGjRrx+++/8+233zJixAiaNm2Kp6cnr7zyCps2beLSpUv6PoQqVTljVVZWxuuvT2b16kgcHBzp1SuY3r374eLiqttm6dJFWFo2Zs+eNNatW8PMme/w/fc/12S/hRBCCFHPqVQqOnXqRKdOnXjppZe4fv06+/bt081oJSYmkpaWRlpaGvPnz8fAwAAfHx/daUM/Pz9MTU31fRiVVDljlZKSTOvWbXByao2xsTFDhgxjy5ZNlbbZsmUTI0c+AsDAgeHs2BGLVqutmR4LIYQQ4r5kaGhI9+7defPNN9m+fTvnz58nOjqaadOm4efnB0BSUhKzZs1Co9FgZWVFWFgYH3zwAeXl5XruvaLKGavCwgIcHZvrnjdr5kBKSvLfbmNoaIiFRSPOnTtHkyZNKm23aNFPLF7884335BMbG3uP3X9wXL58Wb6vOkjGpe6RMambZFzqnvoyJiqVSndD6atXr5KWlkZqaiopKSnk5OQQFRXF77//To8ePfTdVaCWF6+PGTOOMWPGAdC3b4juHkWiarGxsfJ91UEyLnWPjEndJONS99TXMenXr5/u5zNnzhATE0NZWVmdOZYqg5W9fTNOnszTPS8oyKdZM4fbbuPg4Mj169e5dOki1tbW1d9bIYQQQogbbGxsePjhh/XdjUqqXGPl5dWF3Nxcjh8/RklJCevW/ULv3v0qbdO7dz9WrlwOwIYNEQQEBN1Xxb6EEEIIIe5ElTNWhoaGzJ79MSNHDqGsrIxHH30cV1c3Zs+ehaenN3369GP06DFMmPAMPj4eWFlZ8e23P9VG34UQQggh6pQ7WmMVFtabsLDelX73xhvTdD+bmpqyYMGi6u2ZEEIIIUQ9c//eBVEIIYQQopZJsBJCCCGEqCZ6u1fg0aNH8fT01tfH1ztnz56hSRMbfXdD/IWMS90jY1I3ybjUPTImd+ePP/6/vfsNifoO4Dj+viU9iGs+0U7LgY52usjMmZqdf9IIs38mapKuLWIxomjJiFa40LVBMFjggwVu1KP+SLOgZVppZfnfMluDJS1ZLTUzVm0+SL3DPYjFmFG/4c+70/u8np18Dz7w4eDj/X7cz9jj+iwPH/6pn0ifAJYuTeH8+TpPx5D/UC/eR514J/XifdTJ+NClQBERERGTaFiJiIiImGTKjh27ij0dQoyJior2dAR5CfXifdSJd1Iv3kedmNrD1HMAAASoSURBVE/3WImIiIiYRJcCRUREREyiYSUiIiJiEg0rL3Xq1EmSkuKw2fzp6Gh/5VmXy0VaWiIFBd71hO/JyEgv3d33ycpaQWJiLElJcZSVfevmlL7F6GflwoXzJCS8R1xcFKWl37gxoW96/PgPcnIyiY+fT05OJk+ePH7puZKSz0lKisPhWMDu3TsYGdHdKePFaCf37/9Obm4mDscCEhNjuXfP2O83yXMaVl4qImIOhw4dJiHB8dqzZWUHsNvtbkglRnrx8/OjpOQr6uvbqKqq5eDB7+jsvOXGlL7FSCcul4udOz/l6NEK6uvbOHHiB3UyzkpL95OcnEJLSwfJySmUlu4fdaa1tYXW1mYuXWri8uUWrl9vp7Gx3gNpfYORTgC2bv2YLVs+oaHhKtXVFwkICHRz0olNw8pL2e3hzJ79zmvP9fR0U1NzloKCD92QSoz0YrMFMW/efACs1unY7eH09va4I55PMtJJe/tVwsLeJjQ0jKlTp5KVlU11daWbEvqm6upK8vLyAcjLy6eq6vSoMxYLDA4OMjQ0xODgIE6nk8DAGe6O6jOMdNLZeQun08nixWkAWK1Wpk2b5tacE52G1QRXVPQZe/Z8wRtvqEpvdO/eXW7e/ImYmAWejuLTHjzoZdaskBevg4NnauyOs/7+fmy2IABmzLDR398/6kxsbDwORxKRkXYiI+2kpi7Bbg93d1SfYaSTO3d+xd/fnw0bCkhLS6S4uAiXy+XuqBOax54VKJCdvZr+/r5Rf9+1aw8ZGSte+/5z56oICAggKiqahoYr4xHRJ421l38MDAywceN69u7dx/Tpb5oZ0eeY1YmY61W9/JvFYsFisYw619V1h9u3O+no+AWA3NxMmpsbWbhw0fgE9gFj7cTlctLc3ERt7RVCQt5i06YNHDt2mIKCD8Yt82SjYeVBFRWnxvT+1tYWzp6torb2PM+ePWNg4C82b/6IAwe+NymhbxprLwDDw8Ns3Pg+2dlrWblytQmpfNtYOwkKCqa7+/6L1729PQQHzxxrLJ/3ql4CAwPp63uAzRZEX98DAgJGP+z3zJnTxMTEYrVaAViyZCltba0aVmMw1k6Cg2cyd24koaFhAGRkrODatTYNq/9B148msKKiYm7cuMW1az9TVnaIxMRkjSovMDIywvbtW7Dbw9m8eaun4wgQHR1DV1cXd+/+xtDQECdPVpCevtzTsSa19PTllJcfAaC8/AjLlo3+ZjEkJITGxgacTifDw8M0NTXoUuA4MtJJdHQMT58+5dGjRwDU11/Gbo9wa86JTsPKS1VW/khUVARXr7aSn5/L2rVrgOf3iqxbl+3hdL7LSC8tLc0cP36MK1cuk5rqIDXVQU3NWU/GntSMdOLn58e+fV+Tl5eFw7GAzMwsIiLe9WTsSW/btkLq6i4SHz+furpLbNtWCEBHRzuFhc//4Vi1ag2hoWGkpCwkNXURc+bMJT09w5OxJzUjnUyZMoXi4i/Jzl5FSspCRkZGWL9+gwdTTzx6pI2IiIiISfSNlYiIiIhJNKxERERETKJhJSIiImISDSsRERERk2hYiYiIiJhEw0pERETEJBpWIiIiIib5G00BPWrCqswDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95e18cb550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-1.5,-0.5, 100)\n",
    "f = [i**2 for i in x]\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,f, 'r-')\n",
    "plt.plot([-1.5, -0.5], [2, 0.0], 'k-', lw=2)\n",
    "plt.plot([-1.4, -1.0], [1.96, 1.0], 'b-', lw=2)\n",
    "plt.plot([-1],[1],'o')\n",
    "plt.plot([-1.4],[1.96],'o')\n",
    "plt.text(-1.0, 1.2, r'$x,f(x)$')\n",
    "plt.text(-1.4, 2.2, r'$(x-h),f(x-h)$')\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It can be shown that the “centered difference formula\" is better when computing numerical derivatives:\n",
    "\n",
    "$$ \\lim_{h \\rightarrow 0} \\frac{f(x + h) - f(x - h)}{2h} $$\n",
    "\n",
    "The error in the \"finite difference\" approximation can be derived from Taylor's theorem and, assuming that $f$ is differentiable, is $O(h)$. In the case of “centered difference\" the error is $O(h^2)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The derivative tells how to chage $x$ in order to make a small improvement in $f$. \n",
    "\n",
    "Then, we can follow these steps to decrease the value of the function:\n",
    "\n",
    "+ Start from a random $x$ value.\n",
    "+ Compute the derivative $f'(x) = \\lim_{h \\rightarrow 0} \\frac{f(x + h) - f(x - h)}{2h}$.\n",
    "+ Walk a small step (possibly weighted by the derivative module) in the **opposite** direction of the derivative, because we know that $f(x - h \\mbox{ sign}(f'(x))$ is less than $f(x)$ for  small enough $h$. \n",
    "\n",
    "The search for the minima ends when the derivative is zero because we have no more information about which direction to move. $x$ is a critical o stationary point if $f'(x)=0$. \n",
    "\n",
    " + A **minimum (maximum)** is a critical point where $f(x)$ is lower (higher) than at all neighboring points. \n",
    " + There is a third class of critical points: **saddle points**.\n",
    "\n",
    "If $f$ is a **convex function**, this should be the minimum (maximum) of our functions. In other cases it could be a local minimum (maximum) or a saddle point."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAADGCAYAAADhV5X4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd81FXWx/HPJIFIrwESkKZgoiudQARSXCAUFVkQWMQIPCKuuCAooLDqoqvSLLDqKo8CgjQVUJGitCBFUIyIhaJ00whNDCqBkOePswR5BALJTH4zme/79ZpXkmEmc3KZTM7ce+65roMHj+cgIiIiIm4V4HQAIiIiIkWRkiwRERERD1CSJSIiIuIBSrJEREREPEBJloiIiIgHKMkSERER8YA8k6zk5B/p2rUzrVs3p02bSKZMeQWA8eOfoUGD64iLa0VcXCtWrPgo9z6TJj1HZGRDoqKasGrVCs9FLyIiIuKlXHn1yUpPTyM9PY0GDRqRmfkzbdtG8+abc3j//QWUKlWaQYMGn3f7HTu2M3Bgfz76aDVpaal0734bGzd+SWBgoEd/EBERERFvkudMVtWq1WjQoBEApUuXoX7960hNTbno7ZctW0zXrt0IDg6mVq3a1KlTl6Skze6LWERERMQHXFFN1v79+/j66600bdoMgKlTpxATE8WQIfdz7NhRAFJTUwgLq557n7Cw6qSlpboxZBERERHvF3S5N8zMzKR//7t46qmxlClTlr597+Ghh0bicrkYO/ZfPPHEaCZNeuWyH3jGjGnMnDkdgJ07t3P11VdfcfBX6syZMwQEqNbfXTSe7qcxdS+Np/tpTN1L4+l+hTGmhw4dYvv2vXne7rKSrFOnTtG/fx+6devBLbfcBkCVKlVy/71Pn7vp06cHAKGhYaSkJOf+W0pKMtWqhf7heyYk9CMhoR8AHTvGsXmz55cUExMTiY2N9fjj+AuNp/tpTN1L4+l+GlP30ni6X2GMaaNGTS7rdnmmejk5OTz44CDq17+Ov/3tgdzr09PTcj9fsmQR4eERAMTHd2LhwvmcPHmSffv2snv3bpo0aXal8YuIiIj4tDxnsjZt2sg778wlIuIG4uJaATB69OMsWPAu3377NeCiZs2aTJw4CYDw8Ai6dOlK69bNCQoKYty4idpZKCIiIn4nzySrZcsoDh48/ofr27aNv+h9hg4dztChwwsWmYiIiIgPU7WdiIiIiAcoyRIRERHxACVZIiIiIh6gJEtERETEA5RkiYiIiHiAkiwRERERD1CSJSIiIuIBSrJEREREPEBJloiIiIgH+EeSdeYM/Oc/uLKynI5EREREPGXBAoLT0vK+XSHxjyRr7Vq4/34inn3WEi4REREpWhYvhh49qDN1qtOR5PKPJCsmBiZMoEpiIgwZAjk5TkckIiIi7rJxI9xxBzRqxPcPPuh0NLn8I8kCePhh9vfsCS+9BM8843Q0IiIi4g7btkHnzlC9OixZQnbJkk5HlCvI6QAK0+5776Vm8eLwj39AlSowYIDTIYmIiEh+HTgA7dtD8eLw8cf2t/2775yOKpdfJVkEBMAbb8ChQ3DffRASArff7nRUIiIicqWOHIEOHeD4cVizBurUcTqiP/Cf5cKzihWDd96B5s2hVy/45BOnIxIREZErceIE3Hor7NoF778PjRo5HdEF+V+SBVCqlO1CqFvX/pO+/NLpiERERORyZGVB9+5W7D5rFsTGOh3RRflnkgVQqZKt35YvD/HxsHOn0xGJiIjIpWRnQ0ICLFsGr70G3bo5HdEl+W+SBVCjBixfbp+3awc//uhsPCIiInJhOTnwwAMwbx6MGwf33ON0RHny7yQLoH59y4iPHbMdCocOOR2RiIiI/H+PPQavvgojR8KIEU5Hc1mUZAE0aQKLFsGePdCxI/z8s9MRiYiIyFnPPw9PP22tl5591uloLpuSrLOio+Htt60IvksX+PVXpyMSERGRqVPhoYes2P0//wGXy+mILpuSrN+79VZ4801ITLT2/KdOOR2RiIiI/5o3z2qv4uPhrbcgMNDpiK5InklWcvKPdO3amdatm9OmTSRTprwCwNGjR+jevQstWjSie/cuHDt2FICcnBxGjRpOZGRDYmKi2Lp1i2d/Ane7807LlBcvhrvusp0MIiIiUrgWL4Y+faB1a1iwAIKDnY7oiuWZZAUFBTFmzNOsW/c5S5euZOrU/2XHju1MnvwC0dExbNq0hejoGCZPfgGAlSs/ZvfuXWzatIXnnpvEiBFDPf5DuN3AgTBhgmXQAwfCmTNORyQiIuI/Vq+29gwNG1rNtBedR3gl8kyyqlatRoMG1km1dOky1K9/HampKSxbtpiePXsD0LNnb5Yu/RCApUuX0KPHX3G5XDRrFslPP/1EenqaB38ED3n4YdvJ8MYbMGyYbR0VERERz9q40cp3rrnGdv+XK+d0RPl2RWcX7t+/j6+/3krTps3IyMigatVqAFSpUpWMjAwA0tJSCAurkXufsLDqpKam5N72rBkzpjFz5vTc+yQmJhbgx7g8mZmZV/Y4cXFc+9131Jg0ib1HjrC3f3+PxeaLrng8JU8aU/fSeLqfxtS9NJ7nK/XDDzQaOpTT5crx5ZgxZH3zzRV/D28a08tOsjIzM+nf/y6eemosZcqUPe/fXC4Xrius9k9I6EdCQj8AOnaMI7YQ2uInJiZe+ePExsKAAdR+4w1qh4fDqFGeCM0n5Ws85ZI0pu6l8XQ/jal7aTx/57vvbNNZ+fIUW7uWm2rXzte38aYxvawk69SpU/Tv34du3Xpwyy23ARASEkJ6ehpVq1YjPT2NypUrA1CtWhgpKec6p6ekJBMaGuaB0AuJy2Wt+3/7DUaPhhIlYKgP1pmJiIh4q++/hz//GYKCYOVKyGeC5W3yrMnKycnhwQcHUb/+dfztbw/kXh8f34l582YDMG/ebDp06AxAhw4defvtOeTk5LB582eULVv2D0uFPicwEKZPtx4dw4bBK684HZGIiEjRsGcP3HwznD5tCVb9+k5H5DZ5zmRt2rSRd96ZS0TEDcTFtQJg9OjHGTx4KAMG9GXWrBnUqFGT11+fDkDbtvGsWPExkZENKVmyJJMmFZGEJCjITvs+eRIGDYKrrgLVaImIiOTfgQM2g3XihO0ovP56pyNyqzyTrJYtozh48PgF/23+/EV/uM7lcjFu3PMFj8wbFS9uXeG7dLHmaMHB1ldLRERErkxqqiVYhw/bDFbDhk5H5Hbq+H6lrroKFi6EmBhISIC5c52OSERExLekpdkSYUoKLF0KzZo5HZFHKMnKj5Il4cMPoVUr60b7zjtORyQiIuIbDh60Gaz9+2HJErjpJqcj8hglWflVqpQ9OVq2hL/+FebPdzoiERER75aRYQnWnj12bE50tNMReZSSrIIoXdqmOSMjoVcveO89pyMSERHxTocOQdu28MMPthrkJb2sPElJVkGVKWNt/5s2hR494IMPnI5IRETEuxw5Au3awc6ddhbhzTc7HVGhUJLlDmXLwkcfQePG1ktLiZaIiIg5fNiWCLdtg/fft9ksP6Eky13KlTs/0dLSoYiI+LuMDJu12r7dEqz27Z2OqFApyXKn8uXh449t6fCOO1QMLyIi/uvgQUuwzi4Rxsc7HVGhU5LlbmdntCIjoWdPtXcQERH/k54OcXGwa5ftIvSjJcLfU5LlCWXLWjF8VJS1d5g3z+mIRERECkdamiVYe/daqyM/KXK/ECVZnlKmjLV3uOkm6N0b3nrL6YhEREQ8KznZWjPs329/A/2gTcOlKMnypLN9tGJj7Qie1193OiIRERHP2LfPmoumpNhqThFvNHo5lGR5WqlS1nStQwcYMABeftnpiERERNzrhx+gTRvrh7ViBbRu7XREXkFJVmEoUcIOle7SBR54ACZOdDoiERER99i2zWatfvkFVq2yjV8CKMkqPMHBttOwRw8YPhyeesrpiERERApm61aIiYEzZ2DNGusVKbmCnA7ArxQrBrNmWcL1+ONw4gQ8+yy4XE5HJiIicmU++8xKYUqWtBms+vWdjsjrKMkqbEFBMH261WqNGwfHj8NLL0GAJhVFRMRHJCbCrbdCSIjVYNWt63REXklJlhMCAuCVV6yf1vjx8PPPMG2aJWAiIiLebPFiOz6ubl075aR6dacj8lr6q+4UlwvGjrUO8aNHQ2YmzJ1rS4kiIiLeaN486NMHGja0Ng2VKzsdkVfTGpWTXC4YNQomT7YDpW+5xeq0REREvM0bb9gpJi1bwsqVSrAug5Isb/D3v9ty4apVdr7TkSNORyQiInLOhAlwzz3Qvr2dz1uunNMR+QQlWd6ib194911ISrJ+I8nJTkckIiL+LicHRo6EESOgZ0/44APbTSiXJc8ka8iQ+7n++rpER7fIvW78+Gdo0OA64uJaERfXihUrPsr9t0mTniMysiFRUU1YtWqFZ6Iuqrp2tTXu/futW+733zsdkYiI+KvTp+2kkvHj4b77rAVR8eJOR+VT8kyyevW6k7lzF/zh+oEDB7F69XpWr15P27bxAOzYsZ2FC+ezdu1nzJ27gJEjh5Gdne3+qIuyuDhYvdoK4Vu3hi+/dDoiERHxN7/9Zs2z33gDHnvMdsQHBjodlc/JM8mKimpF+fIVLuubLVu2mK5duxEcHEytWrWpU6cuSUmbCxyk32naFNats52GsbHWj0RERKQwHD8OnTvbcXAvvghPPqmm2fmU7xYOU6dO4e2359CoUWPGjHma8uUrkJqaQtOmzXNvExZWnbS01Avef8aMacycOR2AtLQUEgshkcjMzCyUx3GX4Oeeo8Hw4ZRo355to0eTERPjdEjn8bXx9AUaU/fSeLqfxtS9vG08ix8+zI2PPEKpPXvY8eijpDds6HNv9L1pTPOVZPXtew8PPTQSl8vF2LH/4oknRjNp0itX9D0SEvqRkNAPgI4d44iNjc1PKFckMTGxUB7Hrf78Z7j1Vm4YMwb+/W8YNMjpiHL55Hh6OY2pe2k83U9j6l5eNZ47d0K/fpCRAR9+SESHDkQ4HVM+eNOY5mt3YZUqVQgMDCQgIIA+fe7myy+/ACA0NIyUlHO74lJSkqlWLdQ9kfqrihXtyIJbb4UHHrDGpTk5TkclIiJFyWefQatWVg+8erWdSSgFlq8kKz09LffzJUsWER5uuW58fCcWLpzPyZMn2bdvL7t376ZJk2buidSflSgB8+fDvffCM8/A//wPnDrldFQiIlIULF1qm67KlIENG6B587zvI5clz+XCgQP7sX79Oo4cOUzDhuGMGDGK9evX8u23XwMuatasycSJkwAID4+gS5eutG7dnKCgIMaNm0igdiO4R1AQvPoqhIbCmDGQlgZvvw2lSzsdmYiI+KqpU+0NfIMGsGQJVKvmdERFSp5J1muvTfvDdXfemXDR2w8dOpyhQ4cXLCq5MJcL/vlPO4zzb3+DmBj48ENLvERERC5XTo79PXnySWjXzpphly3rdFRFjjq++6IBA6zr7o4dEBUF333ndEQiIuIrsrKswP3JJ+3j4sVKsDxESZav6tQJ1qyxhnGtWtnnIiIil3K2B9abb9pM1htvQLFiTkdVZCnJ8mVNm8LGjbaG3r49zJ7tdEQiIuKtfvwR2rSxvlfTpsETT6jJqIcpyfJ1tWvbbpCoKLjzTpv+VYsHERH5vS++gMhI2LPHlgf79nU6Ir+gJKsoqFABPvoI7rrL3pncfTecPOl0VCIi4g3efx+io21ZcP16W/mQQqEkq6gIDrY19qeegpkzbbfI4cNORyUiIk7JyYEXXoCuXeGGG2DTJrjxRqej8itKsooSlwv+8Q+rzfrsM2jZ0o5JEBER/3L6NNx/PwwbBn/5i9VhqQdWoVOSVRT99a+wciUcO2aJ1qpVTkckIiKF5ehR24H+6qswcqQ1ri5Z0umo/JKSrKKqVSubGg4NtfX3//zH6YhERMTTdu60N9eJidaeYexYCNCfeqdo5IuyunXh008hPt6mjR94wKaQRUSk6FmxAlq0gCNHbDWjf3+nI/J7SrKKurJlrTv8Qw/Byy9Dx442lSwiIkXHK69Ahw5Qo4bV5LZp43REgpIs/xAYCBMn2tTxmjX2TmfbNqejEhGRgsrKspWKQYPsTfT69VCnjtNRyX8pyfIn/ftbEfyxY5ZoLVrkdEQiIpJfBw9C27ZWcztiBLz3ns4g9DJKsvxN69bW+bdePejSBZ5+Wh3iRUR8zRdfQLNm8Pnn1rZn3DhbtRCvoiTLH119NaxbB717W1+tHj0gM9PpqERE5HLMnm1vmF0uWx7861+djkguQkmWvypRwjrDT5wICxbATTfBrl1ORyUiIhdz+jQMH27n1EZG2ixWkyZORyWXoCTLn7lctutw6VI7nb1ZM1iyxOmoRETk/zt40HoeTpxohe4rVkCVKk5HJXlQkiX2i/vFF1C7NtxyC4wZA2fOOB2ViIiANZZu2tT6Hr75prXjKVbM6ajkMijJElOnDmzYAHfdBf/8J9x2m/ppiYg4KScHpkyB6GgICrLX6IQEp6OSK6AkS84pUQKmT7d3SR99BM2bw5YtTkclIuJ/fv0V7rkHBg6EuDjYvBkaN3Y6KrlCSrLkfC6XrfevWWO/5C1bWhNTtXkQESkc338PUVEwdSqMHg2LF0OlSk5HJfmgJEsu7Kab4Msv7WiGe+6Bfv3gl1+cjkpEpGhbsMA2IR04YMnVv/6l/lc+LM8ka8iQ+7n++rpER7fIve7o0SN0796FFi0a0b17F44ds9qdnJwcRo0aTmRkQ2Jioti6VUtNPq1KFVi2DB5/HGbMsFmtnTudjkpEpOg5dQqGDYNu3SA8HJKSoFMnp6OSAsozyerV607mzl1w3nWTJ79AdHQMmzZtITo6hsmTXwBg5cqP2b17F5s2beG55yYxYsRQz0QthScw0HYbLl0KKSn2DmvuXKejEhEpOvbvh5gYeOEF+PvfYe1aqFXL6ajEDfJMsqKiWlG+fIXzrlu2bDE9e/YGoGfP3ixd+iEAS5cuoUePv+JyuWjWLJKffvqJ9PQ0D4QthS4+3pYP//Qn6y48cCABJ086HZWIiE+rtG4dNGoE33xjb2AnT4bixZ0OS9wkXzVZGRkZVK1aDYAqVaqSkZEBQFpaCmFhNXJvFxZWndTUFDeEKV7h6qutIP6RR2DKFJrcfz9s2+Z0VCIivufkSXjwQW587DFroZOUBD17Oh2VuFlQQb+By+XC5XJd8f1mzJjGzJnTAUvOEhMTCxpKnjIzMwvlcYq8+HgqVqjAdc88Q3aTJuwcMoT0Dh2cjqpI0HPUvTSe7qcxLbirkpO54cknKbNzJ3tuvZX9DzxAzo8/2skbUmDe9BzNV5IVEhJCenoaVatWIz09jcqVKwNQrVoYKSnnniQpKcmEhoZd8HskJPQjIaEfAB07xhEbG5ufUK5IYmJioTyOX4iNZUPdutz08stEjBtHRHKy9dcqW9bpyHyanqPupfF0P41pAc2eDX/7m9W7LlzIvvLlNZ5u5k3P0XwtF8bHd2LevNkAzJs3mw4dOgPQoUNH3n57Djk5OWze/Blly5bNXVaUoiercmU7P2vMGJgzx+oKNm50OiwREe9z/Lh1a7/zTmjQwBo9336701GJh+WZZA0c2I9Ondryww/f07BhOLNmzWDw4KGsWbOaFi0asWZNIoMH2y7Ctm3jqVWrNpGRDXnoocGMG/e8x38AcVhgoLV4+OQTa1jaujU88wxkZzsdmYiId/jsM+vWPmuWHVuWmKjdg34iz+XC116bdsHr589f9IfrXC6XEit/ddNN9s7svvusQ/HHH8PMmVYsLyLij7KzYfx4eyNavbq9GW3VyumopBCp47u4T7lyVm/w5pvwxRc2JT5njtNRiYgUvj17IDYWRo2yBqNbtijB8kNKssS9XC6rO9iyBa6/Hnr3tr5aR486HZmIiOfl5MD06dCwIWzdaqdlzJkD5cs7HZk4QEmWeMY111hPrX/9C959F2680YrkRUSKqowMm7Xq1w+aNLEk66677M2n+CUlWeI5QUFWn7VxI5QpA+3aweDBOmhaRIqeRYvszeTixTBhAqxcqeJ2UZIlhaBpU+tmPHgw/PvfNo2+fr3TUYmIFNyxY9C3L9x2G1SpAp9/Dg8/bDuvxe8pyZLCUaIETJoEq1fD6dPQpg0MHw6//eZ0ZCIi+bNsmZ3n+tZb8I9/wObNtuFH5L+UZEnhio21OoV774WJE613zGefOR2ViMjlO37cXsM6drRTLj79FJ56Sgc7yx8oyZLCV6YMvPoqfPQRZGZCVJTNaqlWS0S83ZIlcMMN8MYbMGKElUI0b+50VOKllGSJc9q3h2++gXvusVmthg1tR6KIiLc5fNja03TubLNX69fDuHFw1VVORyZeTEmWOKtcOXjtNVi1Cs6cseXE+++36XgREW/w7rvW92/OHHjsMZu9atnS6ajEByjJEu8QF2e1WkOH2lLiDTfABx84HZWI+LMff4SuXeGOO+yIsM2b4cknITjY6cjERyjJEu9RqhQ8/zxs2GDdkbt0scZ+yclORyYi/iQ729rNXH+91Y6OG2f9/ho2dDoy8TFKssT7tGxp0/HPPmtFphER8MortpwoIuJJX31lB94PHmwfv/nGCtyDgpyOTHyQkizxTsWKwSOP2AtcixYwaJAdrrpli9ORiUhRlJlpu5ybNrXDnWfPhqVLoW5dpyMTH6YkS7zbNdfAxx/DzJmwa5e9AA4ZAj/95HRkIlIU5ORYYXtEhO1y7tsXtm+3g+115qAUkJIs8X4uF/TpAzt2wH33Wa1EeLi908zJcTo6EfFV339vDUXvuAMqVbK2DK+/DhUrOh2ZFBFKssR3VKgAL79sHeKvvhruvBP+/GdbUhQRuVy//AKPP25H4nz6qR35tXmz1WCJuJGSLPE9zZrZC+Orr1qRaqNGVqR69KjTkYmIN8vJgXnzbCb8qadsBmv7dnv9UGG7eICSLPFNgYEwcCDs3GkfX34Z6tWDKVNs+7WIyO999ZU1O+7VCypXhrVr7WDn0FCnI5MiTEmW+LZKlSzBSkqyBqYDB9o5YjqeR0QAMjLsFIkmTeDbb+2Eic8/h9atnY5M/ICSLCkaGjaExERbCjh0yN6xdu1qha0i4n9++w3Gj4drr7UZ7kGD7PXg3nttJlykECjJkqLD5YIePWwX4tNPw4oV1rF56FA4csTp6ESkMJytu4qIgJEjITraNsdMnmybZ0QKkZIsKXpKlIBRo+xda79+9uJ67bXw3HP27lZEiqZ166xpca9eULasvdFatMgK3UUcUKAkq2nTPxET05K4uFa0axcDwNGjR+jevQstWjSie/cuHDumHV/ikGrVbJlgyxaIjISHH4brroMZM1QcL1KUfPst3HYbtGkDe/dar6ukJGvxIuKgAs9kLViwmNWr17N8uRUaT578AtHRMWzatIXo6BgmT36hwEGKFMiNN8KyZfauNiQE7r4bGje2cxHVzFTEdx04AP37Q4MGttnlmWfghx/gf/5HdVfiFdy+XLhs2WJ69uwNQM+evVm69EN3P4RI/vz5z9bIdN48a0bYuTPExMAnnzgdmYhciYMHYdgwa9syaxY8+CDs3g2PPgolSzodnUgu18GDx/P9Vr5ZsxspV648LpeLhIR+JCT049prr+aHHw4AkJOTQ716NXO//r0ZM6Yxc+Z0ANLSUpg7d25+w7hsmZmZlC5d2uOP4y98eTxdp04Rungxtd56i+DDhznSrBl7+vfn54gIR+Py5TH1RhpP93NyTIN+/pmr582jxvz5BGRlkd6uHXv79uW3atUciccd9Bx1v8IY0wcfHJa7gncpBUqyUlNTCA0NIyMjgzvu6MKzz07grrt6nZdU1atXk++/33/J79OxYxybN2/ObxiXLTExkdjYWI8/jr8oEuP5yy/wn//As8/C4cNW1/Hkk9YSwgFFYky9iMbT/RwZ0+PHbQPLxIl2OHzPnjBmjNVY+jg9R92vMMa0UaMml5VkFWi5MDQ0DICQkBA6dbqFpKQvCAkJIT09DYD09DQqV65ckIcQ8aySJeGhh2DPHjtmY80aO6ana1crnBUR5/z0k/1e1q4Njz1my/tffQVz5xaJBEuKvnwnWSdOnCAz8+fczxMTVxEREUF8fCfmzZsNwLx5s+nQobN7IhXxpDJl4B//sGTriSdg9Wpo2tRmtgphllVEfufoUfjnP6FWLTvIuXVr69L+/vtW5C7iI/KdZGVkHOSWW+KJjb2JDh3iaNcunptvbsfgwUNZs2Y1LVo0Ys2aRAYPHurOeEU8q0IFe3Hft8+WDdets2N6OnWys85ExHMOHrQed7Vr23JgXJzNKH/wgR0ML+Jj8n3seO3adUhM3PCH6ytWrMT8+YsKFJSI48qVs+WJIUPgpZfghResc3SrVraDqVMn6zAvIgW3d6/VW73xBpw8Cd262cyyQ7WRIu6iju8il1K2rL2z3rfPCm8PHIBbbrEX/9mz4dQppyMU8V3ffAMJCefOF+zdG7Ztg3feUYIlRYKSLJHLUbIk/P3v1uhw+nQ4fRruvBOuucaO6zl+3OkIRXxDTg4sXw7x8dYoeP58+93avdtmslTQLkWIkiyRK1GsmHWM/+YbOxOtbl07rqdGDfu4/9LtSkT8VlYWvPmm7d5t3x62brWD3Pfvt+X4GjWcjlDE7ZRkieRHQIAtGyYm2u7DW26BF1+0pKt7d+siryN7RCA11TaT1KwJffvCmTMwbZrVYY0aBZUqORygiOcoyRIpqKZNrT5r927rubV6tfXzadTIDqr95RenIxQpXDk58OmnVmNVs6bt1G3WDD76yGaw+vaF4GCnoxTxOCVZIu5SsyaMG2fF8f/7v3bdgAG2DDJsGOzY4Wx8XmznHbeROvm53K9zsrPZ1b8Pyc8+CUDm55s4svDdS36P00cOkzJxrEfjlDxkZtpzv3lzuOkmWLwYHngAdu6EDz+0ZULtyhU/oiRLxN1KloR77oEtW2w5sW1b+Pe/ITzc+v7Mm2f1KZLLFXwVWfv3c+bkSQB+2bqFoIrnlpFKN29Bxa7dL/k9gipWIuzhRzwap1zE1q0waBCEhcG991obhpdfhuRkq7e69lqnIxRxRL77ZIkWRMzNAAAOpklEQVRIHlwuWzaMiYG0NKtDmTIFevWCKlVs63q/fnD99U5H6hVKNWnKiaTNlIlqxfF1n1CmdTS/bvsWgJ9Wr+S3Xd9T9Z77SHvpRQJKluS3XT+Qfewolfv0pUxUK04dTCd57FPUfv4lflq9kszPN1IpNZXdb79FhVtvh9OnOf7JalzFilH90ScILFOGA0+MIiShH1ddU4/s48fZ98gw6r7yeu79c377jay01Ive368dP25vGKZOhY0bbfmvRw+47z6IitKMlQiayRIpHNWqWRPTXbtg6VJravrii3DDDdCypSVfP/3kdJSOKtOqDT+vX8uZrCyy9u3lqnr1L3rb00ePcPVTYwl75DEOzXrzgrfJ2r+PIx1vo+azz3F4zlu4goOpNWESV9UP5/gnq/KMJ2v/PsKGP5rv+xdJOTmU27LFdthWq2azVsePw/PPQ0oKzJhhy4RKsEQAzWSJFK6AAOjQwS4HD8KsWdYbaOBAePBBIqKirFC+fXsI8q9fz+BadTiVcZCf139CqSZNL3nb0pEtcQUEEHx1TU5fJDkt8acG5BQvTlC5cgSULEmpppH2ODVrcXLf3jzjKfGnBgSUKElACfJ1/yJl5057rr71Fo1377YmvQkJ0L+/1V8pqRK5IM1kiTilShUYOhS+/ho++wz69qXi5s3QubPVtgwebNf7USuI0s0iyZgxjTKtoi95O1dQsXNfXGR8XL9PUgMCCCj23/u4AsjJzrZPAwPJOWP3P3Mq64rvX6QdPGi1hC1aWIPQp56CunXZ9uij1pbh1VchMlIJlsglKMkScZrLZbMBr7zChnffhffeszquKVPsD1y9etZPaMuWIp9wlb25LZXu6EVwrdqF8nhBIVU4ufsHADI3ri+Ux/Rqhw9b25H27c8l+llZdq7ggQOwfDnp7dvb5g4RyZOSLBEvklOsGHTpYme3paXZH7xrroHx46FxY9uh+NhjRTbhKlapMhU63Vpoj1fxtq4c+3gp+4YPIfv4z4X2uF7l8GHblNGxo9VZDRhgPd9GjLBZ1i+/tP5v1as7HamIz3EdPHjc8Vfqjh3j2Lx5s8cfJzExkdjYWI8/jr/QeLrfRcc0IwMWLrTdXImJ1jW7Th24/Xa7tGoFgYGFHa7X03P0IvbvtxnThQth7VrIzobatW13YM+eltBfZBlQY+peGk/3K4wxbdSoCcuXr8nzdv5VWSviq0JCbCfXvfdCejp88IH9kXz5ZetDFBJiR/t07gzt2llhsshZZ87AF19Yc9BFiyApya6//noYORK6drWTC1RfJeJWSrJEfE3VqrakM2CAbZ9ftsxmJBYssGWfYsWgTRvo1Mku4eH64+mPjh2D5cstsVq61ArZXS5rGTJunM2A1r94mwwRKTglWSK+rGxZW+Lp0QNOnbLz4hYvtsvDD9ulRg2b3WrXzrrPh4Q4HbV4QlaWNQVdvtwun39uM1jly1vLkM6d7WPlyk5HKuI3lGSJFBXFikF0tF3GjYN9++xA3uXLbWlx2jS7XcOGEBtrlzZtoFKlS31X8VanTtmyX2IirFkDn3wCJ05YL7bISBg92nYJtmzpdz3XRLyFfvNEiqpatc7VcWVn2x/k5cth5Up47TWYNMlud+ON1jKiVSs7DqVmTS0veqPMTJud2rDBitXXrbOkCiAiwpqDtmtn52OWL+9srCICKMkS8Q+BgdaLq3lz67l18qT9wV6zxi7TpsFLL9ltw8Is2YqKshmRxo2hdGln4/c32dnWZf2LL2wJcMMG+OorW/4DO46pb19LjqOjrU5PRLyOkiwRfxQcDK1b22X0aDh92noibdhgdV0bNsD8+XZbl8uK55s1sx1ojRpBgwZQoYKzP0NRceoU7NhhSVRSEmzebB8zM+3fS5e2Jb/Ro+1cwBYtNPYiPkJJlohYzU7jxnYZNMiuS0+3mZTNm+2yYgXMnHnuPjVqWLLVoIHNrISH2/ErZco48zN4u+xs2LMHtm+Hbdssqd26Fb77zhItgKuusv+Dvn0tqW3WzMZVPdBEfJLHkqxVq5YzevRIsrOz6dPnbgYPHuaphxIRT6ha9VwbiLNSUy0x+Oor+7h1K3z8sc2EnVWjhiUG9epZt/q6dc99LOrLjqdPw48/wq5d1jV91y67bN9uy39ZvzsfsXp1S1A7dLCPN95otVUqUhcpMjzy25ydnc3IkQ/xzjvvExZWnfbtY4mP78R114V74uFEpLCEhtolPv7cdVlZ5xKJbdvOfZwzx3o1/V6lSnD11ecuNWrYpWrVc5eQkALP3Lz3ZTITPtpB8rFfqb5xFcPjr+P2xgU4FiYnx3qSpaefu6Sm2nl+Bw5YYnXgACQnn59wFitmndTDw+3YmoiIczN+FSsW6GcUEe/nkSQrKWkzderUpXbtOgB07dqNZcsWK8kSKYqKF7fkISLCOof/3tGj58/q7NtnycjevbZD7v8nYWA1YJUqWd1RxYr2sUIF2zFXpgyUKmUzYqVK2SU42JKZ4sWheHHeOwiPfnuSX7Pt2yUf+5VH39kC337L7ZXP2NJcVpZdfv3Vap9OnDj38aefLO4jR+zj0aN2vt9vv134Zz+bKLZubYnj72fvatTQUp+IH/NIkpWWlkr16jVyvw4NDSMpyfNnE4qIl6lQ4Vxt0YVkZkJKyvkzROnpdlbj2UTn0CH4/nv7+sQJ2xl5CRPum8qv5aqcd92vZ2DC+mRuf7X/xe8YEGDJW9my5xK8a6+1zytVOn+2rWpVO0w5JMTuJyJyAY4t/s+YMY2ZM6cDkJaWQmJioscfMzMzs1Aex19oPN3Pr8e0cmW73HDDJW/mys4m4NdfCfztNwJ//RXX6dMEnD6N69QpArKzSUm+cEf7lHIhJL30EmeCgsj57yU7OJjsEiU4U6IEZ4oVu/z+YGdnuLZtu9Kf0uf59XPUAzSe7udNY+qRJKtatVCSk3/M/To1NYXQ0LDzbpOQ0I+EhH4AdOwYVyinkOu0c/fSeLqfxrTgwsauIvnYr3+8vnxJmpzdOSn5pueoe2k83c+bxtQj89yNGzdl9+7d7Nu3l6ysLBYunE98fKe87ygiUkDD46+jRLHz66BKFAtkePx1DkUkIv7KIzNZQUFBjB07gZ49u5KdnU3v3ncRHh7hiYcSETnP2V2EubsLy5co+O5CEZF88FhNVtu28bRtG5/3DUVE3Oz2xtW5vXF1r1o2EBH/o20xIiIiIh7gFa2F9+zZQ6NGTTz+OIcPH6JSpcoefxx/ofF0P42pe2k83U9j6l4aT/crjDE9cGDfZd3OdfDg8RyPRuJF2rWLYfnyNU6HUWRoPN1PY+peGk/305i6l8bT/bxpTLVcKCIiIuIBSrJEREREPCBw+PBH/+l0EIWpYcPGTodQpGg83U9j6l4aT/fTmLqXxtP9vGVM/aomS0RERKSwaLlQRERExAOKfJL1wQcLadMmkqpVy7FlS1Lu9fv376NmzSrExbUiLq4VDz/8oINR+paLjSnApEnPERnZkKioJqxatcKhCH3X+PHP0KDBdbnPyxUrPnI6JJ+1atVyoqKaEBnZkMmTn3c6HJ/XtOmfiIlpSVxcK9q1i3E6HJ80ZMj9XH99XaKjW+Red/ToEbp370KLFo3o3r0Lx44ddTBC33OhMfWm19Ein2SFh1/PtGmziIpq9Yd/q127DqtXr2f16vVMnPiiA9H5pouN6Y4d21m4cD5r137G3LkLGDlyGNnZ2Q5F6bsGDhyU+7zUqQn5k52dzciRDzFnznzWrfucBQveZceO7U6H5fMWLFjM6tXrvWZ7vK/p1etO5s5dcN51kye/QHR0DJs2bSE6OobJk19wKDrfdKExBe95HS3ySVb9+tdx7bX1nA6jSLnYmC5btpiuXbsRHBxMrVq1qVOnLklJmx2IUPxdUtJm6tSpS+3adShevDhdu3Zj2bLFToclfi4qqhXly1c477plyxbTs2dvAHr27M3SpR86EZrPutCYepMin2Rdyv79+7j55tZ06dKRjRs3OB2Oz0tNTSEs7NwhvGFh1UlLS3UwIt80deoUYmKiGDLkfi0d5FNaWirVq9fI/To0NIzU1BQHI/J9LpeLHj1up23baGbMmOZ0OEVGRkYGVatWA6BKlapkZGQ4HFHR4C2vo15xrE5Bdet2GxkZ6X+4/tFHH6djx84XvE/VqtVISvqWihUr8dVXX3L33b1Zu3YTZcqU9XS4PiE/YyqX51Jj27fvPTz00EhcLhdjx/6LJ54YzaRJrzgQpcj5Fi36iNDQMDIyMrjjji7Uq1f/gmUYkn8ulwuXy+V0GD7Pm15Hi0SSNX/+B1d8n+DgYIKDgwHrp1G7dh127fqhUM5Q9AX5GdPQ0DBSUpJzv05JSaZatVB3hlUkXO7Y9ulzN3369PBwNEVTtWqhJCf/mPt1amoKoaFhDkbk+86OX0hICJ063UJS0hdKstwgJCSE9PQ0qlatRnp6GpUr6xzDgqpSpUru506/jvrtcuGhQ4dyi7L37t3D7t27qFWrtrNB+bj4+E4sXDifkydPsm/fXnbv3k2TJs2cDsunpKen5X6+ZMkiwsMjHIzGdzVu3JTdu3ezb99esrKyWLhwPvHxnZwOy2edOHGCzMyfcz9PTFxFRISem+4QH9+JefNmAzBv3mw6dNBKQUF50+tokZjJupTFixcxatRwDh8+RO/ed/CnP93I22+/x6efrmf8+KcJCipGQEAAEya8SIUKFZ0O1ydcbEzDwyPo0qUrrVs3JygoiHHjJhIYGOh0uD5lzJjH+PbbrwEXNWvWZOLESU6H5JOCgoIYO3YCPXt2JTs7m96971LCWgAZGQfp2/dOALKzT/OXv9zBzTe3czgq3zNwYD/Wr1/HkSOHadgwnBEjRjF48FAGDOjLrFkzqFGjJq+/Pt3pMH3KhcZ0/fq1XvM6qo7vIiIiIh7gt8uFIiIiIp6kJEtERETEA5RkiYiIiHiAkiwRERERD1CSJSIiIuIBSrJEREREPEBJloiIiIgHKMkSERER8YD/A3zHQieIihwEAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95df75a400>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-15,15,100)\n",
    "y = x**2\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-')\n",
    "plt.plot([0],[0],'o')\n",
    "plt.ylim([-10,250])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "ax.text(0,\n",
    "        20,\n",
    "        'Minimum',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['pale red'],\n",
    "       )\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmIAAADFCAYAAAD30J6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XlclWX6x/HPAQSzzF0ENzAXtNxAUdyQxHBJyd3M1JapadOs1CwzbdU0FadfM9Oipbm0IKVpboO4ZRqh6ViaiWKxiUtuKSDw++MecRx3OfCc5ft+vc4LeM52cXs45/K+7+e6bAcPHi9AREREREqch9UBiIiIiLgrJWIiIiIiFlEiJiIiImIRJWIiIiIiFlEiJiIiImIRJWIiIiIiFlEiJiIiImIRJWIiIiIiFlEiJiIiImIRL6sDuFYNGwYSEBBQrM9x6tQpbr755mJ9Dnei8bQ/jan9aUztS+NpfxpT+yqp8dy3bx+7du2/6u2cJhELCAggMTGxWJ8jISGBjh07FutzuBONp/1pTO1PY2pfGk/705jaV0mNZ7Nmwdd0Oy1NioiIiFjEskQsPn4VYWHBhIY2ZebMaVaFISJu6sutqbSdFM+w5adoOymeL7emWh2SiLghSxKxvLw8xox5lgULYtmw4XsWLfqC3bt3WRGKiLihL7emMnbRDlL/OA1A6h+nGbtoh5IxESlxluwRS0pKJDCwDgEBgQD06tWH5cuX0qBBkBXhiIizOXUKDh+Go0fN5cgR8/XECXPdyZPnv2ZnQ06OueTmQk4OUxo/yGmfchc85OncPKbMXcc9ExZCqVLg7Q0+PnDzzXDLLee/3nILVKhw/lKxormULQs2m0UDIiLOypJELCMjnerVaxT+7OfnT1LSxRvx58yZzdy5H/3nPmkkJCQUa1wnT54s9udwJxpP+3P1MfXIzsYnKwufgwfxOXiQ0llZeB86hPeRI+Zy9CjeR4/ieebMFR8n39OTvJtuIr90afJ8fCjw8qLAy4v8UqUo8PQkLeTWS94vrVRZTvz0Ex65udjy8vDIycHzzBk8T5/GIzf3ys9ZqhQ5FSuSU6GCuVSsSE7lypypUoXsqlXJ/s/XvDJlbnh8nIGrv0atoDG1L0cbT4c+a3LIkAcYMuQBALp2jSj2sxx0Zop9aTztzyXG9Ngx2LULfvkF9u6F5OTzXzMyLr59pUrg52cuzZqBr6+5VKp04axUhQpmVuqWW/Dw9r7ivgv/SfGFy5IXHK9QhrJ79lz6Trm5ZpbtxInzM3HnZuOOHMHj4EFKZ2ZSOjPT/B7JyXDwIBQUXPz71KkDt91mvtapA/XrQ1AQVKly7ePooFziNepgNKb25WjjaUkiVq2aH6mpvxf+nJ6ehp+fvxWhiEhxOXYMtm83l507TfL1888XJls2G9SoYZKSbt0gMBBq1YKaNc2lRg0oXdruoY2KasDYRTs4nZtXeOymUp6Mimpw+TuVKgXly5tLzZrX9kQ5OZCWBr/9Br//DgcOwL59Jkn7/nv4/HPIOx8DlSqZhCwoCG6/HZo0MRcXSNBE5NIsScSaNw8hOTmZlJT9+Pn5ExcXyz/+8aEVoYiIPaSlQWKiuWzbZpKvlJTz15crBw0bQpcu5mtQkJkFCgw0+7BK2D3NqwMwZcVuUv84TfXyNzEqqkHhcbvx9oaAAHO5lLNnTXL2yy/nE9Vdu2DxYvjwv94Tq1UzCVnTptCihbkEBmpPmogLsCQR8/LyYtKkKQwY0Iu8vDwGDbqfoKCGVoQiItfr+HHYsgW+/dbM6iQmnp/l8vAwSVabNvDXv56f0ale3eGShnuaV+ee5tWtXabw8jq/PNmly4XXHTwIO3acn1X88UeIiTGzbGCWYkNCoGVLCAszl8qVS/53EJEisWyPWGRkFJGRUVY9vYhcqwMHYO1ak3h9+61JDgoKTGIVFAR33WUSghYtzB4uF9+MXmKqVoVOnczlnJwc+Pe/z88+JibClClmZg2gXj2TBLdpAx06QIMGDpcAi8iFHHqzvohYYN8+SEgwydfatbB/vzl+663QujX06mU+6Fu1MkuOUnK8vSE42FweecQc+/NPk5Bt2mQS5WXL4OOPzXW+vhAebi4dO5plYSVmIg5FiZiIu/vjD4iPh5UrYdUqs5EczMbx8HAYOdJ8veMO8PS0Nla5WJkyZvarQwfzc0EB7NlzPpFOSIDPPjPX+flBZCR07my++vlZFraIGErERNxNfj5s3QpLl8I335j9Xvn5plBpRAQ8/bT52qiR2fMlzsVmMydC1K8Pf/mLScySk2HNGli92syYzZ1rbtu4sdmb1r27meUsVcra2EXckBIxEXdw8iSsWHE++crIMB/YLVvCiy+afV6tWumD2BXZbKY8yG23wcMPm6R72zYz+7liBUyfbvaZlSsHUVEmKevWTRv/RUqIEjERV5WVZcogfPml+dDNzjY1sM592HbpovpU7sjD4/w+szFjzFmwq1ebJH3ZMrOM6eFhljrvucdcate2OmoRl6VETMSVpKVBbCx88QVs2GBmP2rXhsceg+hoaNfOlEwQOefWW6F3b3PJz4ekJPjqK4iLM8vUTz9tkrbevaF/f3NmpojYjTaAiDi7jAz4v/8zG+pr1IDhw03bnXHjzF6wffvM8lPHjk6ZhP3SryfpM98u/LkgL4+9Dw4m9c1XbujxTn6/mSNxX9grPNfi4WHKkLz6qimT8csv8NZbpujuuHFm31lwMEyadP6kDhEpEiViIs7o+HH46CNz5pu/Pzz5JBw+DBMmwE8/mVpfEyeaul5OXq7A5lOanAMHyM/OBuDP7dvwqljphh/vlpatqNirr73Cc2316sGoUaYsxoEDMG2aKaExdqzZcxYaCjNnmuKzInJDnO+/xyLuKjeXSps2wT/+YZaOzpwxH4bjxsGAAaY3oYu6OTiEU0mJlA1ry/EN6yjbrgOnf94JwOk9v5A1+30KcnOweftQ7fHheFevwdGvvyL7wH6qPT6C7JT9pMdMpdabb3Pi2w2c2bsH34f/SsY7Myh36BAHVn5N3rFj+D4+nONr4znzy25K161PtSefBmDP4P7U+8SUgDixaSOnfvieak8+TcY7M7B5e5O9P/mK93cJNWuaUiYjR5racp9/DvPnw4gR8Mwz5oSPwYPxqFDB6khFnIpmxEQc3c6d8NxzUKMGjV94wWysfughU8Bzzx545RWXTsIAyrZtz4mN68nPySEnZT+l69UvvM67eg1qvjqJ2lNiqDRgEIcWmNIM5bv1IDcjnRObN5Hx7kx8H3kcj0v0tfTIzqbm61OoMuwh0ia/RoXu0dSe9g7ZB1I4s+/qy2/5p04V6f5OKSDAzJRt3WpmX0eNMkuZ991Hm759TdmMTZtM6QwRuSLNiIk4omPHYOFCmDXL1Pny8oKePdkREkLjUaPcrsyET+1AcrMOcmLjOm4ODrnguvw/T5HxzgxyM9IAGwV5pt2PzcMD3yeeJuXZ4ZTvHMVNQY0u+dhnAupgs9nwrhWAZ7ny+NQOAMC7Zi3OZh2EwDpXjO3mFi2LdH+nd8cd8Oab8PrrsH49WZMm4bdgAXzwgWmB9eCDcP/9pnG5iFxEM2IijqKgwDTRfughs+/rr3817WumTy88G/KwGxfdvKVFKFlzZlO2bYcLjh9eOI8ydzQmYNo7VH9+HAW5uYXX5aan4VG6NGePHrns4xb8p1uAzWbD9l9ja7PZKMjLO/fD+dvn5lxwf5tXqavf3x14eEB4OLvHjIH0dPjwQ9OdYfRos6zZt6+Zzc3PtzpSEYeiREzEaidOwHvvmcbZoaHw6adw331mJmz7dlM+QPW+uPXOSCr1G1g443RO3p9/Fm7eP5bwr/PHT53i4Kz3qfnKm+SdOMGJTRtv+Lm9ypUj+/ffKMjP5+SW7274cdxG2bJmJmzDBti1y7yGExJMa6X69U0B2awsq6MUcQhKxESssmeP2ehcvTo8+iicPQvvvmtmv957z1S9d/IzHu2pVKXKVOjW46LjFaN7c2j+HFJGjYC887MtWR9/QPku3fD2r47vY09xaN7HnD32xw09d+X7hpI26VV+Gzcaz/LajH5dGjQwidfvv8O8eWa2d/RoU2pl6FDTsFzEjdkOHjzuFLspu3aNILGY/2ATEhLo2LFjsT6HO9F4XkJ+vmkr87e/mVZDpUqZIpmPPw5hYVdNvDSm9qcxta9rGs+ffjL/6fj4Y9N+KywMnnoK+vQx5THkAnqN2ldJjWezZsGsWrX2qrfTjJhISfjzT/PBExRk+vht3Wpqfh04AJ98Yhoua/ZL3EWjRvDOO2aWLCbGLFMOGmTOxnzjDVOQWMRNKBETKU4ZGabOV82a8MQTptfj/PmQkgIvv6wzycS9lStnOkHs3m36XDZubJrQ16xpihT/+qvVEYoUOyViIsVh1y5z9mPt2uZ/+B06wPr1sHkz3Huvll9E/puHB3Ttapbtt283y/XvvWc29vfuDd/pBAlxXUrEROwpMdHsc2nUCBYsMMnY7t2mgXK7dlp+FLmaxo1h9mwzazx2rDnbMiwMIiJg5UoViRWXo0RMpKgKCiA+3vR9bNkS/vUveOEF80Hy7rumX5+IXB8/P1Mk9sABePtt04A8Ksr8jX3xBbhTjTZxaUrERG5UQQEsXw5t20KnTqYV0VtvmQ+O115T7S8Re7jlFtPLMjkZ3n/fNLzv18/MnM2fr4RMnJ4SMZHrVVAAS5dC69ZmX8vvv5uZr337TM+9W2+1OkIR1+PjAw8/DD//bNp/eXqawseNGsHcuaYOn4gTKrZE7K233qBJkwZERLQlIqItq1evKLwuJuZtQkObEhYWTHz86uIKQcS+ziVgoaFw992QmQn//Kc5s+uxx6B0aasjFHF9np4wYAD8+KNZoixdGoYMgYYNYc4czZCJ0ynWGbFHH32CNWs2smbNRiIjowDYvXsXcXGxrF+/hYULFzFmzDPk6Q9HHF18vFmCvPtuOHzYNDTeswceeURnQIpYwcPDnBizdas5GeaWW0yl/jvugM8+U09LcRolvjS5fPlSevXqg4+PD7VrBxAYWIekJLW4EAe1aZPZ/9Wpk9n79Y9/mLMgH3rIbZtvizgUDw+45x744QczQ+bhYWbMgoNhyRKdZSkOz6s4H3zWrPf47LMFNGvWnIkTX6d8+Qqkp6cREtKy8Db+/tXJyEi/5P3nzJnN3LkfAZCRkUZCQkJxhsvJkyeL/TnciTOP58379hH4wQdU/vZbcipUIOWJJ0jv2ZN8b2/YeOPNo4vKmcfUUWlM7cvS8axUCWbOxDc+noCPPuKmnj051qgRyY8+yrEmTayJyQ70GrUvRxvPIvWa7NOnJ1lZmRcdHzt2PCEhLalUqRI2m41Jk14jMzODmJh3ef75ZwkJaUm/fgMBePrpJ+jUqTM9etxzxedSr0nn45Tj+dtvpuL9xx9D2bKmOfHw4WbZwwE45Zg6OI2pfTnMeObmmnpkEydCWprZVvDmm2bp0sk4zJi6CEfrNVmkGbHY2MXXdLvBg4cyeHB/APz8/ElLSy28Li0tlWrV/IoShkjRHTkCkybBzJlmKWPkSFNMslIlqyMTkRtRqpTZwzl4sPm7njQJmjQx+8heecW0URJxAMW2RywzM6Pw+2XLlhAU1BCAqKhuxMXFkp2dTUrKfpKTkwkOblFcYYhcWU4OzJgBdevC1KkwcKApHDl1qpIwEVdQpgw8/zzs3Wvqkc2fb1onvfginDhhdXQixbdHbOLEl9i5cwdgo1atWkydGgNAUFBDoqN70a5dS7y8vJg8eSqenp7FFYbIpRUUwFdfmbpfv/4KnTub5MuJ95GIyBVUqmT+xp96yiRhb7wBH34Ir74KDz5oymKIWKDYErF3333/steNHDmKkSNHFddTi1xZUpL5n/Hatab20LJl0KWL+kCKuIPateGTT8zez2eeMcuXf/ubaaPUubPV0YkbUmV9cR9ZWeZNt0UL+OknUw1/+3ZTHV9JmIh7CQ2F9evh88/h5Em46y5TBiM52erIxM0oERPXl5sLMTGm+fbs2WYj/p49phq+V7FWcBERR2azQd++pm3SpEmwerVpmTRuHJw6ZXV04iaUiIlr+9e/oFkzePppaNXKzIC9/TaUK2d1ZCLiKHx8YMwYc6JOv37w+uvQoAEsWKCCsFLslIiJa0pNNWdARkbCmTPw5ZewfLnZEyYicin+/qaB+IYN4OsLgwaZrho//WR1ZOLClIiJa8nNhWnTICjIJF8TJ8LOnRAdrX1gInJt2raFLVtMS7Nt26BpUzNjdvKk1ZGJC1IiJq5jwwYICYFnn4UOHUwCNn48lC5tdWQi4mw8PeHRR01v2fvvh7feMvvHYmO1XCl2pURMnN+RI/Dww9C+PRw7BnFx8PXXcNttVkcmIs6uShWYNcv8R69CBbO5v2dPOHDA6sjERSgRE+dVUGA20zZsCB99ZIqz/vSTOQVdy5AiYk9t28IPP5iTfeLjzezY9Olw9qzVkYmTUyImzmnfPlP/a9AgU6AxMdEsHdx8s9WRiYir8vIyRWB37oTwcPN9q1amSLTIDVIiJs4lL89sxr/9dti40TTz3bTJlKgQESkJAQFm+8Onn5oztFu2hNGj4fRpqyMTJ6RETJzHzp1meeDZZ8+fUv7UU+oRJyIlz2aD/v1h1y546CGYMsX0ql23zurIxMkoERPHl5trGvM2bw5798L8+bB4MdSsaXVkIuLuypeH994zxaPz8syS5RNPwIkTVkcmTkKJmDi2rVtNb8jx46FPHzMLdu+92owvIo7lzjthxw7TxePvfzfbJ1autDoqcQJKxMQx5eaaYqyhoaZZ91dfmTMkq1SxOjIRkUu7+WZzJuXGjeb7qChTi0yzY3IFSsTE8ezYYc5EmjDBtCn6979N3R4REWcQFmZm8597Dt5/Hxo3NiUvRC5BiZg4jrNn4Y03THX81FRYtMj0fatY0erIRESuT+nSZgP/hg3g7W1OMHrySTh1yurIxMEoERPHsGcPtGsHL75oCrL++9/Qq5fVUYmIFE2bNqZf5YgR8H//Z0rtbN5sdVTiQJSIibUKCswZR82amZ5uCxbAZ59pL5iIuI4yZWDGDFizBnJyTBmeCRPMXlhxe0rExDqZmWbv16OPmv817thh9oSJiLiijh1h+3bTEWTiRJOQ/fKL1VGJxZSIiTWWLDEbWFevhpgYWLECatSwOioRkeJVrhzMmWNm/vfuNasB//iHWR0Qt6RETErW6dOm2GHPnibx+uEHGD4cPPRSFBE30q+fWQVo3x4ee8zsiT10yOqoxAL69JOSs2OH6cn27rvmtO5Nm6BRI6ujEhGxhr8/fPONqT32zTfQtKnKXLghJWJS/AoK4J13TBJ2+LCpNj1lCvj4WB2ZiIi1PDxMNf7vvoNbb4XISHj+ebOpX9xCkRKxxYvjaN8+FF/fcmzblnTBdTExbxMa2pSwsGDi41cXHo+PX0VYWDChoU2ZOXNaUZ5enMHhw2YZ8qmnzBvM9u3QubPVUYmIOJbmzSExEf7yF5g82ZTzSU62OiopAUVKxIKCGjF79jzCwtpecHz37l3ExcWyfv0WFi5cxJgxz5CXl0deXh5jxjzLggWxbNjwPYsWfcHu3buK9AuI4yq3Y4fZiLpypdmQv2SJylKIiFzOzTfDP/8JX3xhais2bw6ff251VFLMvIpy5/r1G1zy+PLlS+nVqw8+Pj7Urh1AYGAdkpISAQgMrENAQCAAvXr1YfnypTRoEFSUMMTR5OfDpEk0e+klqFPH7AULDrY6KhER59Cnj+kwMnAg9O9PvZ49oXVrU61fXE6RErHLSU9PIySkZeHP/v7VychIB6B69fMlCvz8/AsTtEuZM2c2c+d+BEBGRhoJCQnFEW6hkydPFvtzuLpSR47Q8M03qZiYSFqHDiSPHk3e8eOgcbULvUbtT2NqXxpP+7G99hqBH3xArU8/5eTtt7Pz5Zc5XauW1WE5PUd7jV41EevTpydZWZkXHR87djxdu3YvlqDOGTLkAYYMeQCArl0j6NixY7E+X0JCQrE/h0tbu9aUpvjjD3j/ffbcdhsdIyKsjsql6DVqfxpT+9J42llkJNubNaPJ1Km0evxx+OADFb4uIkd7jV41EYuNXXzdD+rn509aWmrhz2lpqVSr5gdAaurvhcfT09Pw8/O/7scXB5Ofb86CfOEFqFvX7Alr3FizYCIidnCkdWvTr3LgQLj3XtNI/O23dea5iyiW8hVRUd2Ii4slOzublJT9JCcnExzcgubNQ0hOTiYlZT85OTnExcUSFdWtOEKQknL0KERHm9Ot+/aF7783SZiIiNhPjRqmV+Wzz5rm4e3bw/79VkcldlCkRGzp0iU0bRpEYuIWBg3qR//+9wAQFNSQ6OhetGvXkoEDezN58lQ8PT3x8vJi0qQpDBjQi7ZtWxAd3YugoIZ2+UXEAomJZhP+ihUwcyYsXGjq4IiIiP2VKgVTp0JsLOzebd5/ly61OiopoiJt1u/evQfdu/e45HUjR45i5MhRFx2PjIwiMjKqKE8rjuCDD8x+MF9fWL8eWrWyOiIREffQuzc0aWJWIe6+G156CV5+GTw9rY5MboAq68v1yc6GRx4xRQfDwyEpSUmYiEhJq1vXlAYaNgxefRV69IAjR6yOSm6AEjG5dr/9ZvYlvP++2Zj/zTdQubLVUYmIuKebboJZs+Dvf4fVq6FFC7OpX5yKEjG5NvHxZj/Crl0QFwevv65pcBERq9ls8Ne/wrp1ZsUiLAzmzrU6KrkOSsTkygoKYPp00x+yalVzVuQ991gdlYiI/LfWrc9vFRkyBEaOhLNnrY5KroESMbm806fN/oNnnjElKr77Dhpcuq2ViIhYzNcXVq2CESNgxgzo0gUOH7Y6KrkKJWJyab//Dh06wJw58Morpglt2bJWRyUiIldSqpRJwmbPNme0t2wJ27dbHZVcgRIxudjGjWbT565d8OWX5tRoD71UREScxrBhF+4b++ILqyOSy9Cnq1xo1iyIiDCzX999Z5YkRUTE+bRqZQpvN2kC/fqZWmP5+VZHJf9DiZgYeXmmdcZDD5n6YFu2wO23Wx2ViIgUhZ+f6fs7bJjZZjJgAPz5p9VRyX9RIiZw/Dj07AnTpsGTT5r6YBUqWB2ViIjYg4+PWe041x6pfXuzD1gcghIxd5ecbPYPrFhhigL+7W/gVaTOVyIi4mhsNrPqsWQJ7NkDoaFm5UMsp0TMna1fb/4Y09Nh5UpTFFBERFxX9+6mNVLp0mYbyqefWh2R21Mi5q7mzoVOnaBSJdi8Ge680+qIRESkJNx+u3nfb9kSBg6E114zxbvFEkrE3E1BgTlzZsgQaNvWnBlZr57VUYmISEmqUsUUfx082JQoGjbMlLqQEqfNQO7kzBl48EFYsMD80f3zn+DtbXVUIiJiBR8fU7S7bl2YMAH274dFi8xKiZQYzYi5i0OHIDLSJGFvvGHOoFESJiLi3mw2s0ryySdmhSQszGzmlxKjRMwd/Pqr+eNKTDQbM8eONX98IiIiAPfdB//6Fxw5Yj4vNm2yOiK3oUTM1Z37H87RoxAfD/37Wx2RiIg4onbtTAJWvrw5gSsuzuqI3IISMVcWF2faFZUrZ/642rSxOiIREXFk9eqZz4umTaFPH5g50+qIXJ4SMVf1t7+ZP6KmTeHbb3VmpIiIXJsqVcwKSnQ0jBgBzzyjHpXFSImYq8nPh+eeg+HDTdui+HioWtXqqERExJmUKQNffAFPPQXTp5selWfOWB2VS1L5CleSk2PKU8ybB088ATEx4OlpdVQiIuKMPD3N50hAgGmPlJUFX35p9pCJ3RRpRmzx4jjatw/F17cc27YlFR4/cCCFWrWqEhHRloiItjz33NOF1/3441bCw1sTGtqUF14YRYGq+drHiRNw990mCXvjDbM0qSRMRESKwmYzS5Pz5pltLh06QFqa1VG5lCIlYkFBjZg9ex5hYW0vui4gIJA1azayZs1Gpk6dUXh89OiRvP32TDZv3kZy8l7i41cVJQQByMw0m/Lj42H2bJWnEBER+xo0CJYtg337zJn4u3ZZHZHLKFIiVr9+A+rWvfZN4JmZGZw4cYIWLUKx2Wz0738vy5YtLUoIsnevaVX088+weLGpmC8iImJvkZGwdq3ZK9a2rWqN2Umx7RE7cCCFO+9sR9myZRk79iVat25Denoafn7VC2/j71+djIzLT3HOmTObuXM/AiAjI42EhITiCheAkydPFvtz2NMtv/5Kk9GjseXlsX3KFE6UKQMOFL+zjacz0Jjan8bUvjSe9udoY1p62jSajBmDT0QEOydO5EirVlaHdF0cbTyvmoj16dOTrKzMi46PHTuerl27X/I+vr7VSEraScWKlfjxx60MHTqI9es3X3dwQ4Y8wJAhDwDQtWsEHTt2vO7HuB4JCQnF/hx2s26d2TxZrhysXElIUJDVEV3EqcbTSWhM7U9jal8aT/tzyDHt3Bm6dKHJuHGmX+W991od0TVztPG8aiIWG7v4uh/Ux8cHHx8fAJo2bU5AQCB79/6Kn58/6emphbdLS0ulWjX/6358t7dkiamQHxAAK1dCzZpWRyQiIu6kalVYs8aUSbrvPtMa6YknrI7KKRVLHbFDhw6Rl5cHwP79+0hO3kvt2gH4+lajbNmyJCZuoaCggM8+W0DXrt2KIwTXNWcO9OoFjRvD+vVKwkRExBrlysHy5dCjBzz5JEycCKqEcN2KlIgtXbqEpk2DSEzcwqBB/ejf/x4ANm3aSMeOYUREtOWhh4YwZcoMKlSoCMDkydN45pmnCA1tSkBAIJ063VX038JdzJgBQ4dCx46mOWvlylZHJCIi7uymmyA21nw2TZhgiomrCv91KdJm/e7de9C9e4+LjvfoEU2PHtGXvE+zZsGsW3f9+8XcWkEBvPKKeZH37g3z58N/ln5FREQs5eUFs2ZBpUowbRocPw4ffmiOy1VplBxdQYFpWTRtmilN8f77enGLiIhj8fCAqVNN1f3x4+HkSU0aXCP1mnRkeXnwyCMmCRs+XP/DEBERx2WzwUsvmW00ixaZjfynTlkdlcNTIuaocnPNmSgffADjxpkXtof+uURExMGNGGEmDlavhqgoOHbkPKSeAAAPFUlEQVTM6ogcmj7ZHdGZM2Yv2KefwuTJ8OqralkkIiLO48EHYcEC2LwZ7rwTDh2yOiKHpUTM0fz5p5nO/fprePddGD3a6ohERESuX//+8NVXsHOn6YeceXFxeFEi5lhOnICuXU1pitmz4bHHrI5IRETkxnXrBkuXQnIyhIdDaurV7+NmlIg5ij/+gLvugo0bYd48Ne8WERHX0KmTKfyalgYdOkBKitURORQlYo7g8GHzQv3hB/j8cxg40OqIRERE7Kd9e1i1yrRC6tABfv3V6ogchhIxq2VlmbXznTvhyy9N+yIRERFX06oVxMebkhbh4bB7t9UROQQlYlbKzDRJ2K+/ms353dR3U0REXFjz5pCQAGfPmnZ9P/9sdUSWUyJmlYwMk4Tt22c2MkZGWh2RiIhI8bvjDlizxnSOiYiAn36yOiJLKRGzQnq6efGlpMCyZeZ7ERERd9GokZkZ8/AwM2P//rfVEVlGiVhJS0szL7rffjNnkYSHWx2RiIhIyQsKMslYqVJmQmLHDqsjsoQSsZKUmmqSsLQ0WLHCnEUiIiLirurXN8mYj49JxrZvtzqiEqdErKSkp5s2D+npJglr29bqiERERKxXrx6sXQs33WRKObnZMqUSsZJwbmN+aqpZjmzTxuqIREREHMdtt5kN/D4+ZtJi506rIyoxSsSKW2ameVH9/jt8841mwkRERC6lbl2TjHl5mc9NNyltoUSsOB08aKZZU1JMiQrtCRMREbm8evVMMubhYVaSdu2yOqJip0SsuBw6ZJKw5GRTrFVnR4qIiFxdgwamAj+YmbFffrE2nmKmRKw4HD0KnTubivlLlqhOmIiIyPVo2NAkY2fPmmQsOdnqiIqNEjF7O34cunQxlYLj4sysmIiIiFyfRo1g9Wo4fdp8lv72m9URFQslYvZ08qTpF5mUBJ99ZhIyERERuTFNmsDKlXDkyPkSUC6mSInYhAnjaNMmhPDwMIYOHcSxY38UXhcT8zahoU0JCwsmPn514fH4+FWEhQUTGtqUmTOnFeXpHcvp09CzJ2zaBPPnQ3S01RGJiIg4v5AQU/opPd3MjB08aHVEdlWkRCw8PIJ16zazdu0mbrutLjExJrHavXsXcXGxrF+/hYULFzFmzDPk5eWRl5fHmDHPsmBBLBs2fM+iRV+we7cLnBGRnQ29e5vqwB9/DP36WR2RiIiI6wgLM9UH9u+HyEg4fNjqiOymSIlYREQnvLy8AAgJaUlaWioAy5cvpVevPvj4+FC7dgCBgXVISkokKSmRwMA6BAQE4u3tTa9efVi+fGnRfwsrnT0LAweabP2992DwYKsjEhERcT3h4fDVV+Ysyi5dzJ5sF+BlrwdasGAu0dG9AUhPTyMkpGXhdf7+1cnIMOu61avXKDzu5+dPUlLiZR9zzpzZzJ37EQAZGWkkJCTYK9xLOnny5PU9R34+QZMmUW3VKvY89RSpdeuaWTEBbmA85ao0pvanMbUvjaf9aUz/S6lSVBo/ntvHj+d4+/ZsnzyZ/NKlr+shHG08r5qI9enTk6yszIuOjx07nq5duwMwffoUPD296Nt3gF2DGzLkAYYMeQCArl0j6Nixo10f/38lJCRc+3MUFMATT8CqVfD669R74QXqFWt0zue6xlOuicbU/jSm9qXxtD+N6f/o2BHq1KH8oEF0iIkxs2Te3td8d0cbz6smYrGxi694/cKF81i5cjmxsUuw2WyAmek6t0wJkJaWSrVqfgCkpv5eeDw9PQ0/P/8bCtxSBQXw/PPw97/DmDEwdqzVEYmIiLiPgQNNpYK//AUGDYKFC01rJCdUpD1i8fGreOedGcyd+yllypQpPB4V1Y24uFiys7NJSdlPcnIywcEtaN48hOTkZFJS9pOTk0NcXCxRUd2K/EuUuDffhLfegsceM9//JwEVERGREvLwwzBtGsTGmoQsP9/qiG5IkdLH559/jpycHPr1M6UaQkJaMnXqDIKCGhId3Yt27Vri5eXF5MlT8fT0BGDSpCkMGNCLvLw8Bg26n6CghkX/LUrSO+/Aiy/C/feb75WEiYiIWGPkSLNpf8IEKFsWYmKc7nO5SInYli0/Xva6kSNHMXLkqIuOR0ZGERkZVZSntc4nn8BTT8E998CsWaYpqYiIiFhn/HiTjE2bBhUrmqTMiTjngqoVvv4ahg0zlX0XLHDatWgRERGXYrPB1Kmmz/PEiSYZGz7c6qiumbKJa7FunSnSGhwMX34J13mqrIiIiBQjm83U8jx6FEaMMMmYk9T11Nra1WzdCj16QGAgLFtm1qBFRETEsXh5mRWrO+80K1hLllgd0TVRInYlv/wCUVFQvrxpOlq5stURiYiIyOWULm1WroKDoX9/s6Ll4JSIXU5aGtx1l/l+1SqoUePKtxcRERHrlS1rVrACA82K1o+XP7HQESgRu5Q//jB9rA4fhm++gfr1rY5IRERErlXlyrBiBdx6q/k837fP6oguS4nY/zp9Gnr2hF27IC4OQkKsjkhERESuV82aJhnLyTErXAcPWh3RJSkR+y+2vDy4917YsMHUDIuMtDokERERuVGNGpnyU6mp0K0bnDhhdUQXUSJ2TkEB9aZPN81DZ840m/xERETEuYWFweefw7Zt0Ls3ttxcqyO6gOqInfPSS/gvXQrjxsGTT1odjYiIiNhL9+7w4YcwbBhB+fnQqZPDdMdxjCisdvYsJCaS1r07vPKK1dGIiIiIvQ0dCpMnc3NKChw7ZnU0hTQjBqYI3OLF7Fm/Hn8naxYqIiIi12jUKJKaNqVDhQpWR1JIM2LneHtT4OlpdRQiIiJSXGw28n18rI7iAkrERERERCyiRExERETEIkrERERERCyiRExERETEIkrERERERCyiRExERETEIkrERERERCyiRExERETEIkrERERERCyiRExERETEIkVKxCZMGEebNiGEh4cxdOggjh37A4ADB1KoVasqERFtiYhoy3PPPV14nx9/3Ep4eGtCQ5vywgujKCgoKNpvICIiIuKkipSIhYdHsG7dZtau3cRtt9UlJmZa4XUBAYGsWbORNWs2MnXqjMLjo0eP5O23Z7J58zaSk/cSH7+qKCGIiIiIOK0iJWIREZ3w8vICICSkJWlpqVe8fWZmBidOnKBFi1BsNhv9+9/LsmVLixKCiIiIiNPystcDLVgwl+jo3oU/HziQwp13tqNs2bKMHfsSrVu3IT09DT+/6oW38fevTkZG2mUfc86c2cyd+xEAe/b8QrNmwfYK95IOHz5EpUqVi/U53InG0/40pvanMbUvjaf9aUztq6TG87ffUq7pdldNxPr06UlWVuZFx8eOHU/Xrt0BmD59Cp6eXvTtOwAAX99qJCXtpGLFSvz441aGDh3E+vWbryd+AIYMeYAhQx647vvdqM6dw1m1am2JPZ+r03jan8bU/jSm9qXxtD+NqX052nheNRGLjV18xesXLpzHypXLiY1dgs1mA8DHxwcfHx8AmjZtTkBAIHv3/oqfnz/p6eeXL9PSUqlWzb8o8YuIiIg4rSLtEYuPX8U778xg7txPKVOmTOHxQ4cOkZeXB8D+/ftITt5L7doB+PpWo2zZsiQmbqGgoIDPPltA167divYbiIiIiDipIu0Re/7558jJyaFfv2jAbNifOnUGmzZt5K23XsfLqxQeHh5MmTKDChUqAjB58jSGD3+M06dP06lTZzp1uqvov4Wd3H//MKtDcCkaT/vTmNqfxtS+NJ72pzG1L0cbT9vBg8dVyEtERETEAqqsLyIiImIRJWIiIiIiFnH7RGzx4jjatw/F17cc27YlFR6/UpsmubLLjSlATMzbhIY2JSwsmPj41RZF6NzeeusNmjRpUPjaXL16hdUhOaX4+FWEhQUTGtqUmTOnXf0OclUhIXcQHt6aiIi2dO4cbnU4TmfEiMdp1KgOHTq0Kjx29OgR+vaNplWrZvTtG80ffxy1MELnc6kxdbT3ULdPxIKCGjF79jzCwtpedN3l2jTJlV1uTHfv3kVcXCzr129h4cJFjBnzTOHZtXJ9Hn30icLXZmRklNXhOJ28vDzGjHmWBQti2bDhexYt+oLdu3dZHZZLWLRoKWvWbHSoOk3OYuDA+1i4cNEFx2bOnE6HDuFs3ryNDh3CmTlzukXROadLjSk41nuo2ydi9es3oG7delaH4VIuN6bLly+lV68++Pj4ULt2AIGBdUhKSrQgQnF3SUmJBAbWISAgEG9vb3r16sPy5Wq3JtYKC2tL+fIVLji2fPlSBgwYBMCAAYP45puvrQjNaV1qTB2N2ydiV3KuTVN0dFe+++5bq8Nxeunpafj7/2+Lq3QLI3Jes2a9R3h4GCNGPK6lihuQkZFO9eo1Cn82xaYv325Nro3pIXwPkZEdmDNnttXhuISsrCx8fasBULWqL1lZWRZH5Boc6T3Ubr0mHdm1tGn6X5dr01S27K3FHa5TuJExlWt3pfEdNuxhnn12DDabjUmTXuPll18kJuZdC6IUudCSJSvw8/MnKyuLfv2iqVev/iW3fciNsdlshR1s5MY52nuoWyRiV2vTdCmXa9NU3I3HncWNjKmfnz9paf/b4srPnmG5jGsd38GDhzJ4cP9ijsb1VKvmR2rq74U/p6en4eendmtFdW4Mq1SpQrdud5OU9IMSsSKqUqUKmZkZ+PpWIzMzg8qV1fy7qKpWrVr4vSO8h2pp8jIu16ZJblxUVDfi4mLJzs4mJWU/ycnJBAe3sDosp5OZmVH4/bJlSwgKamhhNM6pefMQkpOTSUnZT05ODnFxsURFqd1aUZw6dYqTJ08Ufp+QEE/DhnptFlVUVDc+/XQ+AJ9+Op8uXbTiUFSO9h7qFjNiV7J06RJeeGEUhw8fYtCgftxxR2M+++zLK7Zpkiu73JgGBTUkOroX7dq1xMvLi8mTp+Lp6Wl1uE5n4sSX2LlzB2CjVq1aTJ0aY3VITsfLy4tJk6YwYEAv8vLyGDTofsvfjJ1dVtZBhg27D4C8vLP07t2PO+/sbHFUzuXRRx9g48YNHDlymKZNgxg9+gWGDx/JX/4yjHnz5lCjRi0++OAjq8N0Kpca040b1zvUe6haHImIiIhYREuTIiIiIhZRIiYiIiJiESViIiIiIhZRIiYiIiJiESViIiIiIhZRIiYiIiJiESViIiIiIhb5fzqi56vTj9ljAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95df76aac8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-15,15,100)\n",
    "y = -x**2\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-')\n",
    "plt.plot([0],[0],'o')\n",
    "plt.ylim([-250,10])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "ax.text(0,\n",
    "        -30,\n",
    "        'Maximum',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['pale red'],\n",
    "       )\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmgAAADGCAYAAABmZnz+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XtYlHX6x/H3AIKHPGAqJ1N0zbAsPIWxqEiaoGZmmiQmqb/MrXZt1U3LdjusWyt5tt12aytdXRMttFITzRBPlUpk2kFLSTROYoqKIsgwvz++iZrnZmBm4PO6ruea4ZmHee65L4Tb79Fy8OAxGyIiIiLiMjycHYCIiIiInE8FmoiIiIiLUYEmIiIi4mJUoImIiIi4GBVoIiIiIi5GBZqIiIiIi7G7QDt16hTR0d3p3v23dO0aRkLCiwBkZu4jJiaKsLBQRo0aTklJCQDFxcWMGjWcsLBQYmKi2L8/s/y9Zs+eTlhYKOHhHUhJWWtvaCIiIiJuye4CzcfHh6SkFaSmfkJKymbWrVtLWtpWJk9+jtGjH2fr1i+pX78BCxfOB2DhwvnUr9+ArVu/ZPTox5k8+TkAdu/exbJlSWzcuJXExKVMnDgOq9Vqb3giIiIibsfuAs1isXDdddcBcPr0aU6fLsVisbBp03r69bsXgNjYIaxatQKA5OSVxMYOAaBfv3vZuDEVm81GcvJKBgwYiI+PD82bB9OiRUvS09PsDU9ERETE7ThkDJrVaiUqKoKbb/4NkZFRBAe3pF69+nh5eQEQGBhEbm4OALm5OQQFNQXAy8uLunXrcfjwYXJysgkMDCp/z3O/R0RERKQ68XLEm3h6erJu3WaOHi1g+PChfP/9d45420uaP38uCxbMA+C773Zxww03VOj9ysrK8PDQfApHUk4dS/l0POXUsZRPx1NOL8+rsJBa2dlYa9akqGlTbFfIVWXl89ChQ+zate+K1zmkQDujfv0GRER0JS1tK8eOHaW0tBQvLy+ys7Pw9w8AwN8/gKysHwkMDKK0tJTjx4/RsGFDAgICyc7OKn+vc7/nl+LjRxAfPwKA3r2jSEur2K7Q1NRUunfvXqH3qG6UU8dSPh1POXUs5dPxlNPLWLIE4uIgPBxWrYL69a/4LZWVz3btOlzVdXaXiocOHeLo0QIAioqKWL9+Ha1btyYiohvLl78HwOLFi4iJ6QtAdHQfFi9eBMDy5e/RpUskFouF6Og+LFuWRHFxMZmZ+8jIyKBDh072hiciIiLVydtvw5AhpjhbvfqqijNXZHcLWl5eLn/4w++wWq3YbGXcc88AevXqTevWIYwePYK//30yt94aytCh8QAMHRrP448/QlhYKL6+vrz22lwAQkLa0L//ALp0uR0vLy8SEqbh6elpb3giIiJSXfz3vzByJHTrBsuXw8+TGN2R3QXaLbe0JSVl0wXng4NbsHp16gXna9asyZtvzr/oe40d+yRjxz5pb0giIiJS3bzxBjzyCPToAe+/D7VrOzsiu2h0oYiIiLi3mTNh1CiIjoYPPnD74gxUoImIiIi7stng2Wdh3DgYNMi0nNWq5eyoHMKhszhFREREKkVZGfzxj/DKK2bc2euvQxUau64WNBEREXEvpaUwYoQpzsaONePPqlBxBirQRERExJ0UFcH998P8+fDXv8L06WCxODsqh1MXp4iIiLiHI0fgnntg82aYMwf+8AdnR1RhVKCJiIiI6ztwAGJiYM8eSEyEwYOdHVGFUoEmIiIiru2rr0xxdvw4JCdDVJSzI6pwGoMmIiIirmvDBuja1cza3LChWhRnoAJNREREXFViIvTqBX5+8OmnEBrq7IgqjQo0ERERcS02G7z4otn0PCzMTApo3tzZUVUqjUETERER11FSAqNHw7x5MHQovPkm+Pg4O6pKpxY0ERERcQ1HjpjJAPPmwfPPw4IF1bI4A7WgiYiIiCv4/nvo1w9++MEUZg8+6OyInEoFmoiIiDjXRx+Zdc08Pc3zbt2cHZHTqYtTREREnMNmg1mzTLfmDTfAtm0qzn6mAk1EREQqX3ExjBxpNjvv3x8++QRatHB2VC5DBZqIiIhUrpwcs+DsvHnw3HPw7rtw3XXOjsql2F2gZWX9yIABfenS5Xa6dg3j9ddfBeDIkcMMGtSfzp3bMWhQfwoKjgBgs9mYNOlJwsJCiYwMZ8eO7eXvlZi4kM6d29G5czsSExfaG5qIiIi4mo0boUMH+PJLeOcdM1vTQ+1Fv2R3Rry8vHjhhRfZtGkbq1Z9zFtv/Yfdu3cxZ85MunWLZMuW7XTrFsmcOTMB+PjjNWRk7GXLlu1Mnz6bCRPGAqagmzYtgeTkFFavXse0aQnlRZ2IiIi4uTPjzaKioG5d2LIFBg1ydlQuy+4Czc/Pn9tuawfAddfVpXXrm8jJySY5eSWxsXEAxMbGsWrVCgBWrfqQwYOHYLFY6NQpjKNHj5KXl8u6dR8TGRmFr29DGjTwJTIyipSUtfaGJyIiIs5WWGh2BRg7Fu6+20wGaNvW2VG5NIe2Ke7fn8nOnTvo2LET+fn5+Pn5A9CkiR/5+fkA5OZmExjYtPx7AgODyMnJJicnh6CgoHPOB5KTk+PI8ERERKSy7doFd9xhujP//ndYuhTq13d2VC7PYeugFRYWMnLkMCZPnkLduvXOe81isWCxWBx1K+bPn8uCBfMAU/ClpqY67L0vprCwsMLvUd0op46lfDqecupYyqfjuUNO/dasofXMmVh9fPj25Zc50rEjbNjg7LAuytXy6ZAC7fTp04wc+SADBw7m7rvvAaBx48bk5eXi5+dPXl4ujRo1AsDfP5Ds7B/Lvzc7O4uAgEACAgLYvHnTOeeziYjoctH7xcePID5+BAC9e0fRvXt3R3yMS0pNTa3we1Q3yqljKZ+Op5w6lvLpeC6d0xMn4Pe/N7M0u3bFc9EiQs/pJXNFrpZPu7s4bTYbf/zj47RufROPPvr78vPR0X1YvPhtABYvfpuYmL4AxMT0ZsmSRdhsNtLStlKvXj38/PyJiurB+vUpFBQcoaDgCOvXpxAV1cPe8ERERKQyff01hIXBf/8Lf/4zpKSAixdnrsjuFrQtWz7jnXcSadPmFqKiIgB45plnGTNmLKNGDWfhwvk0bdqMN96YB0DPntGsXbuGsLBQateuzezZZlkOX9+GjBs3gV69ugMwfvxEfH0b2hueiIiIVAabDd54A554AurVgzVroGdPZ0fltuwu0O64I5yDB49d9LWkpOUXnLNYLCQkzLjo9XFxw4iLG2ZvSCIiIlKZDh2Chx+G9983RdmCBeDv7+yo3JpWhhMREZFfb80auPVWWLUKpk+H1atVnDmACjQRERG5dqdOmXXNoqOhYUPYuhXGjdOuAA6iLIqIiMi1SUuDTp3MzgB/+IP5OjTU2VFVKSrQRERE5OqUlMBf/mIWni0oMN2ac+ZArVrOjqzKcdhCtSIiIlKFbd8ODz0EO3aYx1mzoEEDZ0dVZakFTURERC6tpAReeAFuvx0OHjQzNefNU3FWwdSCJiIiIhf36acwapRZfDYuznRnXn+9s6OqFtSCJiIiIuc7ftwM/o+IgGPHYMUKWLhQxVklUoEmIiIiZ61YATffDP/8pynSvv4a+vZ1dlTVjgo0ERERgcxMuPde6NcP6teHTz6B2bOhbl1nR1YtqUATERGpzoqL4aWXoE0b+OgjmDIF0tPNUhriNJokICIiUl2tXQuPPw7ffQf33QczZ0KzZs6OSlALmoiISPWzZw8MGAB33QVWq1lwNilJxZkLUYEmIiJSXRw7BhMnwi23mO7MF1+Er76CmBhnRya/oC5OERGRqs5qNYvLPvMM5OWZnQBeegkCA50dmVyCCjQREZGqymaDDz+Ep54yLWUREWYZjU6dnB2ZXIG6OEVERKqiLVsgKgruvhtOnYIlS2DjRhVnbkIFmoiISFWyaxfcf79ZJuPbb82Cs998Y85ZLM6OTq6SujhFRESqgj174K9/NVsy1aoFzz0H48droVk35ZAWtCeeeIybb25Jt26dy88dOXKYQYP607lzOwYN6k9BwREAbDYbkyY9SVhYKJGR4ezYsb38exITF9K5czs6d25HYuJCR4QmIiJStWVmwsMPQ0gIvPsujBsHP/wAzz+v4syNOaRAe+CBoSQmLj3v3Jw5M+nWLZItW7bTrVskc+bMBODjj9eQkbGXLVu2M336bCZMGAuYgm7atASSk1NYvXod06YllBd1IiIicr6a2dnwyCNw442wYIFZcHbvXpg6FRo3dnZ4YieHFGjh4RE0aOB73rnk5JXExsYBEBsbx6pVKwBYtepDBg8egsVioVOnMI4ePUpeXi7r1n1MZGQUvr4NadDAl8jIKFJS1joiPBERkarjm29g2DA6DxsG8+eb1rM9e8y+mQEBzo5OHKTCxqDl5+fj5+cPQJMmfuTn5wOQm5tNYGDT8usCA4PIyckmJyeHoKCgc84HkpOTc9H3nj9/LgsWzCt/v9TU1Ir5ED8rLCys8HtUN8qpYymfjqecOpbyab/rdu+m+dtv02jjRsp8fNh3zz3kDR1KSaNGpuVs715nh+jWXO1ntFImCVgsFiwOnDkSHz+C+PgRAPTuHUX37t0d9t4Xk5qaWuH3qG6UU8dSPh1POXUs5fNXKisz2zBNmwapqVCvHkyahOcf/8iBr75STh3I1X5GK2yZjcaNG5OXlwtAXl4ujRo1AsDfP5Ds7B/Lr8vOziIgIJCAgACysrLOOZ9NgJpqRUSkOiouhjffhLZtzTpme/aYIm3/fvjb3+Dnv6lSdVVYgRYd3YfFi98GYPHit4mJ6QtATExvlixZhM1mIy1tK/Xq1cPPz5+oqB6sX59CQcERCgqOsH59ClFRPSoqPBEREdfz44/w5z/DDTeYsWXe3vC//0FGhlkyo359Z0colcQhXZyjR49g8+ZNHD78E6GhIUyYMIkxY8YyatRwFi6cT9OmzXjjjXkA9OwZzdq1awgLC6V27drMnv0qAL6+DRk3bgK9enUHYPz4ifj6NnREeCIiIq7LZoNNm+CVV2DpUtOteffdMGYM9OihxWWrKYcUaK+9Nvei55OSll9wzmKxkJAw46LXx8UNIy5umCNCEhERcW0FBWZR2ddfhx07oEEDGDsWHn0UWrZ0dnTiZNpJQEREpLLYbPDZZ6YoW7wYioqgQwfz9dChULu2syMUF6ECTUREpKLl5JjWsnnz4Ouv4brrID4eRo2Cjh2dHZ24IBVoIiIiFeHUKfjgA1OUrV5txpaFh5vWsiFDTJEmcgkq0ETEpf2UtITjm9aDhwcWiwdNRj9GrRtvuqrvPX0wj6wpkwme8Y8LXjvw3CQax4+g5m9uJOOxh2k+ZQae9epd1fseeG4SpUeOYPGugUfNmvg/OgbvoKaXvD73X6/ge3d/fG5odslrCrd+Ro2AwMteI26gtBTWrYNFi8yA/6NHoWlTeOop02J209X97IqoQBMRl1W0excnPt9Gs5dn4VGjBtZjx7CVnnZ2WAAEPDGOmr+5kYKPkslfMI+gp/58yWv9H/3DFd+vcOtn1Ol4uwo0d1RWBp9+ComJsGQJHDxoFpQdMAAefBCiosDT09lRiptRgSYiLqu04DCe9erhUaMGwHktXD+9k0jh51uxlZRQq3UITUY/jsVi4dTePeT+aw4AdW5rX359WXExea/OoTjzB2oENsVWUnLRex7bsI4jH66gcUEBebu/psnDv8NymT+utW9uS8FKM2P95M4vyZ//FjZrGTVbtaLJqMfwqFHjvNa67x8cjG+ffhSmb8PD25vACc9wOi+XwrStnPzmKw4nLSHgT0/h7a+Ful1aaSls2GBayZYtg+xsqFnTLI8xZAj06WO+FvmVKmyhWhERe9W5rT2lhw7xw5jfkfeff3Hy66/KX2sQ05fmU2YQPOMflJWUcOLzbQDkvjqbJiMfIXjanPPe6+iaVVh8vAme9SqNYodwKmPPBfcr/vEAxz/ZRLO/JZD/wDDw8DDdq5dRmLYV72bNKSspIfefswkYO4HgGa9gs5ZxdM2HF1xvKz5FzdY3ETxtDrXa3MLRj9dQ66Y2XNcpjMbDRtB82mwVZ67qxAkzpuz//g/8/c0aZW+9BXfcYRaTzcuDd96B++5TcSZ2UwuaiLgsj1q1aJYwg6Jd33Dyq53kzHyZRkMfon5UD05+vZMj7y+lrLgYa+FxfG5ohrXNzZSdOEHtm9sCUC+yOye2fw7AyW+/xrf33QD4NG+BT/PgC+53cueXnMrYy/6nxtO4sJCT3t54XmLl9pzZM/Dw9sarSROajHyE09lZ1GjSBO/AIADqd7+TguSV+Pbtf973Wby8qNPxdgBqtmzFiR3bHZIrqSA//ggrVsDy5ZCSYgb+16tnWsruuw9iYqBOHWdHKVWQCjQRcWkWT09q33IrtW+5FZ9mzTm2PoW6EV05+Ma/aDZlBjUaNebQkrcpO33xLstrVS8yisZDH7rixslnxqCdUVxYeHU38PTCcmZleA8PsFrtiFYc7tQp2LjRzLpcvRq++rnVtmVLGD0a+vWDrl3NFkwiFUhdnCLiskqyfqQkJ7v86+J9P+DVqDG2n4sxz7r1KCsqovCzT8zXda7Do04dir79BoBjG892T9ZucwvHNm0w77M/k+LMfRfcr3bb2yj87BNKjxYAYD1+nNP5B68q1hqBQZw+eLA83mPr11Hr55a8q+FRqxZlp4qu+npxEKsVPv/cbETeuzc0bAi9epltl5o0gYQEs27Znj0wa5bp1lRxJpVALWgi4rLKTp3i4FuvU3biBHh64O0fgN/o3+NZ5zrq94gmc/wf8GzQ4LyWLP/HniD3X3OwYKF2aLvy8/V79Sbv1Tns++NjeAc1pWbLVhfcz+eGZjR64EGyJj9H48Lj/Pjh+zR5eDQ1Gje5Yqwe3t74P/4EOTMSyicJ1O/V+6o/a92IruT9+58UfLicgPGaJFBhrFbYuRPWrzfLYaxfb7ZcArMExqhREB0NkZHquhSnshw8eMzm7CDs0bt3FGlpaRV6jyt1dci1U04dS/l0POXUsZyWz5MnYds2sxn5xo1mOYxjx8xrLVuaJTDOHIGBlR+fHfQz6liVlc927Trw0UeXn3wEakETEZGqoqwMdu2CLVvMsXWr2YT8zDi/tm0hLg66dDFH8+bOjVfkMlSgiYiI+ykthd27IT3djCFLT4cvvoAzkzXq14fbbzcr+IeHw29/C76+zo1Z5BqoQBMREddls5n1xXbuPHvs2AHffGNmXALUrg3t2sHw4dCpE3TuDK1bm1myIm5KBZqIiDjf6dOwbx989x18++3ZY9cuOHLk7HX+/nDbbfD449C+PXToYIoxbaUkVYwKNBERqRxHj5oi7Icfzh7ff2+OffvOXxPOzw9CQiA2Ftq0gVtvNUejRs6KXqRSqUATERH7HT9u9qPMyYGsLDhwAPbvN48HDhCRkWGuOVfdunDjjaZbcsgQaNXKfB0SYtYjE6nGXK5AS0n5iGeemYjVauXBBx9izJhxzg5JRKqR977IYurq3WQVFBH0WQpPRt/Eve2DnB1W5Ssuhp9+gkOHzONPP0F+Phw8aMaEHTxojtxcU5idOHHhezRsCDfcADfcwMHmzQnq0gVatDBHcLB5/cyuCiJyHpcq0KxWKxMnjuedd94nMDCIXr26Ex3dh5tuCnF2aCJSDbz3RRZPL91J0WnT1ZZVUMTTS3cCuH6RVlZmBs0XFZnj5ElTNJ04YWY2nnl+7NiFx9GjZpxXQYE5jhwx338p119vVtn38zNjwO6+GwICzDpiAQEQFARNm5630Ov3qakEac0ukavmUgVaenoaLVq0JDi4BQADBgwkOXmlCjQRd3emeCguNse5z0tKzADxM4+nT5slFC52WK1nj7Ky8w+b7fzn5x5w4eNFTD15C0U2n/POFZ22MnXJFu5d/tXZkxd7r1/e89w4fhnjuZ/j3OPcz3omF2eOM/k5k7df5vLMjMarVaOG2fS7bl2z/ESDBmYl/QYNzNGwoSnEGjUyj9dfD40bm69r1Li2e4nINXOpAi03N4egoKblXwcEBJKefuEuAfPnz2XBgnk/f082qampFRpXYWFhhd+julFOHcvh+bRa8TpxAq/jx/EqLKRGYSFehYV4njiB18mTeP58eJ04gWdREZ6nTuFZVITHqVPm+alTeBQX41FcjGdxMR6nTzsutgqUPeEDuEiPW3ZZDXjuuat6D5vFAhZL+SMWCzYPj/PO2Tw8wMMDm6cnNg8Pc3h6Xvzw8qKsRg1sXl7YatSgrFYtyry9zfMzh7c3ZTVrYvXxOfvc2xtrrVqU1aqFtWbNs0edOpTWro3tWveTPHzYHLt3X9v3/Uz/5h1POXUsV8unSxVoVys+fgTx8SMAs9VTRW/NoO00HE85dawr5rO09Ox4oXPHD505zh1ndOjQ2b0JL6d2bdP6Ureu6cqqU8e0rtSpY16rVevsY61aULOmOXx8zj+8vU2LzJnHM4eX19nD0/Ps45nDw+Pso8Vy4eMvD7jw8RcCp6aSVXBhS1Sgb+3zZxhe5r0sv3gUQ//mHU85dSxXy6dLFWj+/gFkZf1Y/nVOTjYBAe61N5pIpSsuplZWltn4+ecZcxw4YGbSnZlVl5dnutZ+qWZNM5aocWPThdWy5dnuLF/fs11fDRqYldkbNDjbLeblUr8+HOLJ6JDzxqAB1KrhyZPRIVr0VEQqlUv9hm3fviMZGRlkZu4jICCQZcuS+Pe/33R2WCLOZbOZ2XN79pj1ovbuPX8tqexsOv9yXFWjRmaQdmCgGcQdEHD28PMzR5MmprVLs+jKnZkIUD6Ls0Gt6juLU0ScyqUKNC8vL6ZMmUps7ACsVitxccMICWnj7LBEKkdxsSnCzl1F/bvvTFF27NjZ6zw8TPHVogX07AktWrCrqIiQXr3MkgZNm5ouRflV7m0fxL3tg1yuu0NEqheXKtAAevaMpmfPaGeHIVJxyspMy9eOHefvLbh37/njnIKDzay68PCzC3jeeCM0b27Ga50jNzWVEBUTIiJVhssVaCJVSmmpaQlLTzfH55/D9u1nF/W0WOA3vzFb2Nx/P9x8s9nW5qabzAB7ERGpllSgiTiKzWbGhm3ZYo6tW01RdmZ9qjp1zObOI0dCaKgpym655bzFPEVEREAFmsivV1ICX3wBmzaZ45NPzJIVYGZHduwIjz5q9hns0MF0T3p6OjdmERFxCyrQRK5WcTF89plZzmL9etNKVlRkXmvVCnr3hjvugM6doW1brbYuIiK/mgo0kUuxWmHbNli71hRln3xiuis9PKBdOxg9Grp0gYgI8Pd3drQiIlKFqEATOdePP8Lq1eZYu9ZsGg1w222mILvzTujWzSzYKiIiUkFUoEn1ZrWawfzLl8OKFWbJCzALuvbvD716mbXGGjd2bpwiIlKtqECT6ufECdNC9v778OGHZu9JT0/o2hWmToXoaDOGTCvsi4iIk6hAk+rh6FHTQpaUBMnJZnC/ry/06QN33w0xMeq2FBERl6ECTaqu48fhvfcgMRE++ghOnzZ7U44cCQMHmhazKrjht4iIuD/9dZKq5dQp0225aJFpMTt1Cpo1gzFjTFHWubOZhSkiIuLCVKCJ+7PZzBIY//0vLFliujObNIH/+z8YMsTsZamiTERE3IgKNHFf+/fD/Pnm+P57s3floEEwdKhZDkPdlyIi4qb0F0zcy+nTZkmM11+HNWtM61n37jBpkunCrFvX2RGKiIjYTQWauIeMDHjjDXjrLcjLg6ZN4dlnYfhwCA52dnQiIiIOpQJNXFdZmVmv7JVXYNUqM46sb1945BGzLIa6MEVEpIrSXzhxPUePwrx58M9/mrFl/v6mtWzUKNNyJiIiUsWpQBPX8cMPMGsWvPmmWe3/jjvg+efNwH9vb2dHJyIiUmnsWnvggw+W0bVrGH5+9dm+Pf2812bPnk5YWCjh4R1ISVlbfj4l5SPCwzsQFhbKnDkzys9nZu4jJiaKsLBQRo0aTklJiT2hiTvZtg1iY6FVK3j1VbjvPnPu008hLk7FmYiIVDt2FWghITczd+5CwsMjzju/e/culi1LYuPGrSQmLmXixHFYrVasVisTJ45n0aIkNm3axtKl77J79y4AJk9+jtGjH2fr1i+pX78BCxfOtyc0cXU2G6xcCZGREBZmtl/6059MK9r8+dCpk7MjFBERcRq7CrTWrW+iVasbLzifnLySAQMG4uPjQ/PmwbRo0ZL09DTS09No0aIlwcEt8Pb2ZsCAgSQnr8Rms7Fp03r69bsXgNjYIaxatcKe0MRVlZVBUhIdR482e2D+8APMmAEHDkBCgsaYiYiIUEFj0HJysunY8fbyrwMDg8jNzQEgKOjsH+CAgEDS09M4fPgw9erVx+vnWXnnXn8x8+fPZcGCeQDk5maTmprq+A9xjsLCwgq/R1VnsVppkpJCs4ULqZOZiSUwkF0TJpB3113YvLwgPf3KbyKXpJ9Rx1NOHUv5dDzl1LFcLZ9XLNAGDryH/Py8C84//fSz9O7dt0KCupL4+BHEx48AoHfvKLp3716h90tNTa3we1RZVqvZF/OFF2DPHmjbFhYtIq1xY7r36EGIs+OrIvQz6njKqWMpn46nnDqWq+XzigVaUtIH1/ymAQGBZGdnlX+dnZ2Fv38AAFlZP5afz8nJJiAgkIYNG3Ls2FFKS0vx8vI673pxU2Vl8O67Zhbmt9/CbbfBsmVwzz1mPTMX+l+KiIiIq6mQHaSjo/uwbFkSxcXFZGbuIyMjgw4dOtG+fUcyMjLIzNxHSUkJy5YlER3dB4vFQkREN5Yvfw+AxYsXERPjnNY5sZPNBu+/D+3bm5mZFgu88w588QXce682LRcREbkKdv21XLlyOaGhIaSlbSUu7n4GDzaD/ENC2tC//wC6dLmdBx64j4SEaXh6euLl5cWUKVOJjR1AREQn+vcfQEhIGwD+8pcX+Pe//0FYWChHjhxm6NB4+z+dVK5PPoEuXUwhVlQECxfCjh1mHTMVZiIiIlfNrkkCffv2o2/ffhd9bezYJxk79skLzvfsGU3PntEXnA8ObsHq1an2hCPOsmsXPP00vPeeWfX/tddg5EhtxSQiIvIrqVlDfr2DB+E4uVSlAAAO/UlEQVR3vzMD/z/+GCZPNhMBHnlExZmIiIgd9FdUrl1JCcyZYwqykyfh0UfNXpmNGzs7MhERkSpBBZpcPZsNli+H8eNNS1mfPjB9OoRosQwRERFHUhenXJ1du6BXL+jf33RfrlpltmpScSYiIuJwKtDk8k6cgEmTzDpmaWkwe7aZmRkT4+zIREREqix1ccrFnVnP7IknYP9+eOghePllaNLE2ZGJiIhUeSrQ5EL79sHvf2+6MNu2hQ0boGtXZ0clIiJSbaiLU86yWmHWLLjlFrMV0/TpZhNzFWciIiKVSi1oYuzYAQ8/DNu2mdmZ//oXNGvm7KhERESqJbWgVXenTsEzz0DHjqZrc9EiWLFCxZmIiIgTqQWtOtu2zQz+//ZbGD4cpk2D6693dlQiIiLVnlrQqqPiYvjznyE8HI4dg+RkmDtXxZmIiIiLUAtadfPFF6bVbOdO02o2cyY0aODsqEREROQcakGrLqxW+NvfICwM8vPNlk1z56o4ExERcUFqQasO9u2DBx+EzZshNhZefRUaNnR2VCIiInIJKtCquoUL4bHHzPMFC2DoULBYnBuTiIiIXJa6OKuqggKIizMtZ7feCl9+aZ6rOBMREXF5KtCqoq1boX17WLIEJk82uwIEBzs7KhEREblKdhVozz//Z377245ERobz0ENxHD1aUP7a7NnTCQsLJTy8Aykpa8vPp6R8RHh4B8LCQpkzZ0b5+czMfcTERBEWFsqoUcMpKSmxJ7TqyWaDGTMgIgLKymDjRrOchpd6skVERNyJXQVaZGQUGzZsYf36T/nNb1oxe7YpuHbv3sWyZUls3LiVxMSlTJw4DqvVitVqZeLE8SxalMSmTdtYuvRddu/eBcDkyc8xevTjbN36JfXrN2Dhwvn2f7rq5PBh6N8fxo+Hvn1h+3azzpmIiIi4HbsKtKioHnj93DrTsePtZGdnAZCcvJIBAwbi4+ND8+bBtGjRkvT0NNLT02jRoiXBwS3w9vZmwICBJCevxGazsWnTevr1uxeA2NghrFq1ws6PVo18+im0a2cWnJ01C5YtA19fZ0clIiIiv5LDxqAtWrSAHj3uAiAnJ5vAwKDy1wIDg8jNzSE3N4egoKbl5wMCAsnJyebw4cPUq1e/vNg7c71cgc0Gr7wC3bqZbsxPPoEnntBEABERETd3xcFJAwfeQ35+3gXnn376WXr37gvAzJlT8fT0YtCgWMdHeBHz589lwYJ5AOTmZpOamlqh9yssLKzwe1wrj6IibpoxA7+1azn029+y6+mnKS0sNBMC3IAr5tSdKZ+Op5w6lvLpeMqpY7laPq9YoCUlfXDZ1xMTF7JmTTJJScux/NxyExAQWN7dCZCdnYW/fwAAWVk/lp/PyckmICCQhg0bcuzYUUpLS/Hy8jrv+ouJjx9BfPwIAHr3jqJ79+5X+hh2SU1NrfB7XJM9e+C+++Crr+Bvf6PR00/TxcO9JuS6XE7dnPLpeMqpYymfjqecOpar5dOuv+opKR/xj3/MYsGCxdSuXbv8fHR0H5YtS6K4uJjMzH1kZGTQoUMn2rfvSEZGBpmZ+ygpKWHZsiSio/tgsViIiOjG8uXvAbB48SJiYvra98mqquXLoVMnyMqCVavgmWfAzYozERERuTy71l946qk/UVJSwv339wfMRIFp02YREtKG/v0H0KXL7Xh5eZGQMA1PT08ApkyZSmzsAKxWK3FxwwgJaQPAX/7yAqNHj+Dvf5/MrbeGMnRovJ0frYqx2eCll8yyGR07wrvvam0zERGRKsquAm3r1i8v+drYsU8yduyTF5zv2TOanj2jLzgfHNyC1atT7Qmn6jp5EkaOhMWLzVZN//kP1Krl7KhERESkgmgFU1e3fz/ce69Z1ywhAZ58UrM0RUREqjgVaK5s82YzGeDUKVixAvr0cXZEIiIiUgk0utxV/e9/EBUF9evDli0qzkRERKoRFWiuxmaD55+HYcPMnpqffQYhIc6OSkRERCqRujhdSXExPPywaT0bPhxeew28vZ0dlYiIiFQyFWiu4qefYMAA2LgRXnwRnn5akwFERESqKRVormDvXoiJgQMHYNEieOABZ0ckIiIiTqQCzdk+/9xMACgthY8/NuPOREREpFrTJAFnWrMGuneHmjXNkhoqzkRERAQVaM7zv/9B377QsiV8+qlmaoqIiEg5FWiVzWaDqVPNMhpdu8KGDRAY6OyoRERExIWoQKtMNhs89RRMmACDB8OqVWYhWhEREZFzqECrLFYr/O538PLL8NhjZramj4+zoxIREREXpAKtMpw+DQ8+CK+/DpMmwT/+AR5KvYiIiFycltmoaCdPwv33w4cfQkKC6d4UERERuQwVaBXp2DHo18/sDvDaa/DII86OSERERNyACrSKUlBgdgf4/HN4+23tDiAiIiJXTQVaRTh8GHr1gh074J134N57nR2RiIiIuBG7RqpPmTKZyMhwoqIiuP/+/uTm5gBgs9mYNOlJwsJCiYwMZ8eO7eXfk5i4kM6d29G5czsSExeWn//yyy+IjLyDsLBQJk16EpvNZk9oznPoEPToATt3wrJlKs5ERETkmtlVoD3++BOsX/8p69ZtplevGKZNSwDg44/XkJGxly1btjN9+mwmTBgLwJEjh5k2LYHk5BRWr17HtGkJFBQcAWDChLFMnz6HLVu2k5Gxl5SUj+z8aE5w8CBERcGuXfDBB2anABEREZFrZFeBVrduvfLnJ0+exGKxALBq1YcMHjwEi8VCp05hHD16lLy8XNat+5jIyCh8fRvSoIEvkZFRpKSsJS8vl+PHj9OpUxgWi4XBg4fw4Ycr7ftklS031+yrmZEBK1dCdLSzIxIRERE3ZfcYtJde+itLliyiXr16LF1qiqrc3GwCA5uWXxMYGEROTjY5OTkEBQWdcz6QnJwccnKyCQgIOu/63Nxse0OrPHl5cOedsH+/2R2gWzdnRyQiIiJu7IoF2sCB95Cfn3fB+aeffpbevfsyadKzTJr0LLNnT+fNN19j4sRnKiTQc82fP5cFC+YB8P3339GuXYcKvd9PPx3i+usbXf4i75rQqjWM+WOFxlJVXFVO5aopn46nnDqW8ul4yqljVVY+DxzIvKrrrligJSV9cFVvNHDgYOLiBjFx4jP4+weSnf1j+WvZ2VkEBAQSEBDA5s2bzjmfTUREFwICAsnJyTrven//S28gHh8/gvj4EVcVlyPcdVckH320vtLuVx0op46lfDqecupYyqfjKaeO5Wr5tGsMWkbGnvLnyckradWqNQAxMb1ZsmQRNpuNtLSt1KtXDz8/f6KierB+fQoFBUcoKDjC+vUpREX1wM/Pn7p165KWthWbzcaSJYvo3buPfZ9MRERExE3ZNQZt8uTn2bv3eywWD2644QamTp0FQM+e0axdu4awsFBq167N7NmvAuDr25Bx4ybQq1d3AMaPn4ivb0MAEhJmMGbMoxQVFdGjx1306NHLntBERERE3JZdBdrcuf+76HmLxUJCwoyLvhYXN4y4uGEXnG/XrgMbNmyxJ5wKM2zYcGeHUOUop46lfDqecupYyqfjKaeO5Wr5tBw8eMxNV4QVERERqZrsGoMmIiIiIo6nAu0yPvhgGV27huHnV5/t29PLz+/fn0mzZk2IioogKiqCP/1JS2tcjUvlE2D27OmEhYUSHt6BlJS1TorQvb388kvcdttN5T+Xa9eudnZIbikl5SPCwzsQFhbKnDkXH6oh16Zjx7ZERt5BVFQEd90V6exw3NITTzzGzTe3pFu3zuXnjhw5zKBB/encuR2DBvUv35lHruxi+XS136Eq0C4jJORm5s5dSHh4xAWvBQe3YN26zaxbt5lp02Y5ITr3c6l87t69i2XLkti4cSuJiUuZOHEcVqvVSVG6t9GjHy//uezZU7tZXCur1crEieNZtCiJTZu2sXTpu+zevcvZYVUJS5euZN26zS61jIE7eeCBoSQmLj3v3Jw5M+nWLZItW7bTrVskc+bMdFJ07udi+QTX+h2qAu0yWre+iVatbnR2GFXGpfKZnLySAQMG4uPjQ/PmwbRo0ZL09DQnRCjVXXp6Gi1atCQ4uAXe3t4MGDCQ5GQ323ZOqqTw8AgaNPA971xy8kpiY+MAiI2NY9WqFc4IzS1dLJ+uRgXar7R/fyZ33tmF/v1789lnnzg7HLeWk5NNYOAvt/rKcWJE7uutt14nMjKcJ554TN0dv0Jubg5BQWe3qTOLaLvRtnMuyuyxfC89e3Zj/vy5zg6nysjPz8fPzx+AJk38yM/Pd3JE7s+VfofavRenu7vSVlYX4+fnT3r61zRseD1ffvkFDz0Ux8aNW87bPL66+jX5lKt3ufwOH/4w48dPxGKxMGXK33juuWfK1yAUcably1cTEBBIfn4+99/fnxtvbH3RoSPy61ksFiwWi7PDcGuu9ju02hdoV7uV1bl8fHzw8fEBIDS0PcHBLdi7d0+F7wnqDn5NPgMCAsnO/uVWXwGODKvKuNr8PvjgQzz44OAKjqbq8fcPICvr7DZ1OTnZBARcets5uTpncti4cWP69Lmb9PTPVaA5QOPGjcnLy8XPz5+8vFwaNdK+nPZo0qRJ+XNX+B2qLs5f4dChQ+WD2Pft+4GMjL00bx7s3KDcWHR0H5YtS6K4uJjMzH1kZGTQoUMnZ4fldvLycsuff/jhckJC2jgxGvfUvn1HMjIyyMzcR0lJCcuWJREdrW3n7HHixAkKC4+XP09NTaFNG/1sOkJ0dB8WL34bgMWL3yYmRr0U9nC136HVvgXtclauXM6kSU/y00+HiIu7n7Ztb2XJkvf49NPNvPzyi3h51cDDw4OpU2eVb1kll3apfIaEtKF//wF06XI7Xl5eJCRMw9PT09nhup0XXvgLX3+9E7DQrFkzpk2b7eyQ3I6XlxdTpkwlNnYAVquVuLhhTv8l7e7y8w8yfPhQAKzWUu67737uvPMuJ0flfkaPHsHmzZs4fPgnQkNDmDBhEmPGjGXUqOEsXDifpk2b8cYb85wdptu4WD43b97oUr9DtZOAiIiIiItRF6eIiIiIi1GBJiIiIuJiVKCJiIiIuBgVaCIiIiIuRgWaiIiIiItRgSYiIiLiYlSgiYiIiLgYFWgiIiIiLub/AZtOm6T32V1vAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95df74f400>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-15,15,100)\n",
    "y = x**3\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-')\n",
    "plt.plot([0],[0],'o')\n",
    "plt.ylim([-3000,3000])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "ax.text(0,\n",
    "        400,\n",
    "        'Saddle Point',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['pale red'],\n",
    "       )\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are two problems with numerical derivatives:\n",
    "+ It is approximate.\n",
    "+ It is very slow to evaluate (two function evaluations: $f(x + h) , f(x - h)$ ).\n",
    "\n",
    "Our knowledge from Calculus could help!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We know that we can get an **analytical expression** of the derivative for **some** functions. \n",
    "\n",
    "For example, let's suppose we have a simple quadratic function, $f(x)=x^2−6x+5$, and we want to find the minimum of this function. \n",
    "\n",
    "#### First approach\n",
    "\n",
    "We can solve this analytically using Calculus, by finding the derivate $f'(x)  =  2x-6$ and setting it to zero:\n",
    "\n",
    "\\begin{equation}\n",
    "\\begin{split}\n",
    "2x-6  & = & 0 \\\\\n",
    "2x  & = & 6 \\\\\n",
    "x & = & 3 \\\\\n",
    "\\end{split}\n",
    "\\end{equation} "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAADGCAYAAADhV5X4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xt8zvX/x/HHtY05E4bNWTmHMcaisaI5hJZTX4eFksq5ol/pm0r1rb6Vpn4qFaIcQ5KGHOZUlK+U+FLCsJM5m7CZ6/fHO5Ofw07Xtc91eN5vt+vWdrm26+XdtXle78/7/Xrbjhw5bUdEREREHMrH6gJEREREPJFCloiIiIgTKGSJiIiIOIFCloiIiIgTKGSJiIiIOIFCloiIiIgTZBuyEhIOExXVhTZtWnDnnaFMnToFgDfeeJXGjesSEdGaiIjWrFq1IutrYmLeIjS0CWFhzVizZpXzqhcRERFxUbbs+mSlpCSTkpJM48bBpKWdoX37cD79dA5LliyiePESDBs28qrH79mzm6FDB7NixVqSk5Po2bMbmzf/hK+vr1P/IiIiIiKuJNuZrIoVK9G4cTAAJUqUpE6duiQlJd7w8cuXLyMqqgf+/v5Ur16DmjVrsW3bVsdVLCIiIuIGcrUm6+DBeHbs+IWQkOYATJs2lbZtwxg16nFOnjwBQFJSIkFBlbO+JiioMsnJSQ4sWURERMT1+eX0gWlpaQwePICJE1+jZMlSDBz4ME8++TQ2m43XXnuZCRPGExMzJcdPPHPmdGbNmgHAb7/tpmrVqrkuPrcuXbqEj4/W+juKxtPxNKaOpfF0PI2pY3n9eNrtFDt8GN/z5/mzalUyixTJ97csiDE9evQou3cfyPZxOQpZGRkZDB7cnx49enPvvd0AqFChQtaf9+//IP379wYgMDCIxMSErD9LTEygUqXAa75ndPQgoqMHAdCpUwRbtzr/kmJcXBzt2rVz+vN4C42n42lMHUvj6XgaU8fy6vG02yE6Gn7/HebNg969HfJtC2JMg4Ob5ehx2UY9u93O6NHDqFOnLo89Njzr/pSU5KyPv/lmKfXq1QcgMrIzixcv5MKFC8THH2Dfvn00a9Y8t/WLiIiIJ3vlFfjsM3jpJYcFLFeT7UzWli2bWbBgLvXrNyQiojUA48c/z6JFX7Bz5w7ARrVq1XjzzRgA6tWrT/fuUbRp0wI/Pz9ef/1N7SwUERGRK2bPhn/+E/r3h+ees7oap8k2ZLVqFcaRI6evub99+8gbfs2YMWMZM2Zs/ioTERERz7NhAwwaBG3bwscfg81mdUVO48Wr7URERKRA/fYb3Hcf1KwJixaBv7/VFTmVQpaIiIg439Gj0KUL+PjAsmVQtqzVFTldjls4iIiIiOTJ+fPQvTscOgRr18Ktt1pdUYFQyBIRERHnuXQJBg6E776D+fMhLMzqigqMLheKiIiI8zz9tOmD9cYb0KuX1dUUKIUsERERcY733oM334Rhw+Cpp6yupsApZImIiIjjffkljBxp1mLFxHh0q4YbUcgSERERx9q8Gf7xDwgNNY1HvbQpuUKWiIiIOM7evdC1K1SuDEuXQrFiVldkGYUsERERcYyUFIj860SY2FgICLC2HouphYOIiIjkX1qaaTaanAxr1kDt2lZXZDmFLBEREcmf9HTo0QO2b4clS6BlS6srcgkKWSIiIpJ3djs8/DCsXAnTppnZLAG0JktERETy45lnYNYsePllGDTI6mpcikKWiIiI5M3kyfD66/D44/Dss1ZX43IUskRERCT3Zs+GUaPg/vtN2PLCZqPZUcgSERGR3Fm+HB58ENq1g88/99pmo9lRyBIREZGc27zZ7CRs1MjsJCxSxOqKXJZCloiIiOTMrl1m92BgoGk2WqqU1RW5NIUsERERyd7Bg3DPPVC4sGnXULGi1RW5PPXJEhERkZtLTTUBKy0N1q+HWrWsrsgtKGSJiIjIjZ06Zc4jPHjQzGA1bmx1RW5DIUtERESu788/oWtX2LEDvvoK2rSxuiK3ku2arISEw0RFdaFNmxbceWcoU6dOAeDEieP07Nmdli2D6dmzOydPngDAbrfz7LNjCQ1tQtu2Yfzyy3bn/g1ERETE8dLToWdP2LgRPvsMOnWyuiK3k23I8vPz48UXX2Hjxh+JjV3NtGkfsWfPbiZPnkR4eFu2bNlOeHhbJk+eBMDq1SvZt+8PtmzZzltvxTBu3Bin/yVERETEgTIzITra7CD88EPo08fqitxStiGrYsVKNG4cDECJEiWpU6cuSUmJLF++jD59+gLQp09fYmO/BiA29ht69/4HNpuN5s1DOXXqFCkpyU78K4iIiIjD2O0wbBjMmwdvvAFDhlhdkdvK1Zqsgwfj2bHjF0JCmpOamkrFipUAqFChIqmpqQAkJycSFFQl62uCgiqTlJSY9djLZs6czqxZM7K+Ji4uLh9/jZxJS0srkOfxFhpPx9OYOpbG0/E0po7lcuNpt3Pr++9TdcEC4vv2ZX+LFuBK9eWAK41pjkNWWloagwcPYOLE1yhZ8urmYzabDVsuzyyKjh5EdLQ5rbtTpwjatWuXq6/Pi7i4uAJ5Hm+h8XQ8jaljaTwdT2PqWC43ni+8AAsWwPDhVJ88mepueB6hK41pjpqRZmRkMHhwf3r06M2993YDICAgIOsyYEpKMuXLlwegUqUgEhMPZ31tYmICgYFBjq5bREREHOnf/4YXX4RBgyAmRgc+O0C2IctutzN69DDq1KnLY48Nz7o/MrIz8+bNBmDevNl07NgFgI4dOzF//hzsdjtbt/5AqVKlrrlUKCIiIi5kyhQYN84scP/oI/DRgTCOkO3lwi1bNrNgwVzq129IRERrAMaPf56RI8cwZMhAPv98JlWqVOPjj2cA0L59JKtWrSQ0tAnFihUjJmaKU/8CIiIikg+ffmoWunftCrNmga+v1RV5jGxDVqtWYRw5cvq6f7Zw4dJr7rPZbLz++tv5r0xERESca948GDwY2reH+fOhUCGrK/Iomg8UERHxRosWQb9+0Lo1fPklFClidUUeRyFLRETE2yxdatZfhYbCsmVQvLjVFXkkhSwRERFvEhtrjstp2tR8XLKk1RV5LIUsERERb7FqFURFQcOGsGIFlC5tdUUeTSFLRETEG6xdC926QZ06sHIl3HKL1RV5PIUsERERTxcXB126QM2aZjbrrwbi4lwKWSIiIp5s3borAWvNGqhQweqKvIZCloiIiKdatw46d4YaNUzAqljR6oq8ikKWiIiIJ7ocsKpXV8CyiEKWiIiIp1HAcgkKWSIiIp5kzRro1OlKwKpUyeqKvJZCloiIiKdYudIscr/1VrOjUAHLUgpZIiIinuCbb6BrV6hb1/TE0i5CyylkiYiIuLuvvoL77oPbbzeXCNUHyyV4T8iy262uQERExPEWLYIePSA42DQaLVvW6orkL94RshISoHVriv/xh9WViIiIOM7nn0Pv3tCiBXz7rY7KcTHeEbLOnoVDhwgeMwZ++MHqakRERPLvo49gwAAIDzcL3nXYs8vxjpBVpw5s2MDFkiXh7rtN/xARERF3FRMDjzxiWjUsWwYlSlhdkVyHd4QsgBo1+CkmBqpWhY4dYflyqysSERHJvVdfhdGj4f77YfFiKFrU6orkBrwnZAHp5cubWax69aBbN/PiFBERcQd2O4wfb279+sG8eVC4sNVVyU14VcgCICDAbG8NCYFeveCzz6yuSERE5OYuXYLhw80s1pAh8Omn4OdndVWSDe8LWWB2X6xcaRYLDhgA771ndUUiIiLXl5EB0dEwZQqMHQsffgi+vlZXJTmQbcgaNepxGjSoRXh4y6z73njjVRo3rktERGsiIlqzatWKrD+LiXmL0NAmhIU1Y82aVc6p2hFKljTdcbt3hxEjYOJE9dISERHXcv686YH1+edmFuv118Fms7oqyaFs5xofeKAfDz30CMOHD73q/qFDhzFs2Mir7tuzZzeLFy9kw4YfSE5OomfPbmze/BO+rpq4ixSBL76Ahx6C55+H48fhrbfAxzsn+ERExIWcOWPWD69bZ2axHnvM6ookl7JNE2FhrSlTJmfNzZYvX0ZUVA/8/f2pXr0GNWvWYtu2rfku0qn8/GD6dBg1Ct55xwSuixetrkpERLxZaircdRds2GDWDitguaU8r5qbNm0q8+fPITi4KS+++AplytxCUlIiISEtsh4TFFSZ5OSk6379zJnTmTVrBgDJyYnExcXltZQcS0tLu/HzdO9O9dOnqTl9Okd//51d//wnl/z9nV6TO7vpeEqeaEwdS+PpeBpTx7reeBZJTqbxuHH4HznCrokTORYUBBrzHHOl12ieQtbAgQ/z5JNPY7PZeO21l5kwYTwxMVNy9T2iowcRHT0IgE6dImjXrl1eSsmVuLi4mz9PRASEhFB+xAjCX37ZHLipIwpuKNvxlFzTmDqWxtPxNKaOdc14/vqrac/w55+wejWNWre2rDZ35Uqv0TwtPqpQoQK+vr74+PjQv/+D/PTTfwAIDAwiMTEh63GJiQlUqhTomEoLyrBhMHcubNlidh8mJlpdkYiIeINNm+DOO83HGzaAApbby1PISklJzvr4m2+WUq9efQAiIzuzePFCLly4QHz8Afbt20ezZs0dU2lB6t0bYmPhwAG44w747TerKxIREU+2bBl06AAVKpiwdfvtVlckDpDt5cKhQwexadNGjh8/RpMm9Rg37lk2bdrAzp07ABvVqlXjzTdjAKhXrz7du0fRpk0L/Pz8eP31N113Z2F27r7bXAPv1Mm8m1i2DEJDra5KREQ8zSefwNChEBxs3uAHBFhdkThItiHrww+nX3Nfv37RN3z8mDFjGTNmbP6qchUhIeYdRWSkWa81fz506WJ1VSIi4gnsdqp/+inMmGH+nVmwwPRwFI+hhlDZqV0bvvsO6tc3jUs//tjqikRExN1dvAhDh1Jzxgx48EFYulQBywMpZOVEpUrm0mGHDubMqAkT1B1eRETy5uxZiIqCjz4ivn9/06uxUCGrqxIn0OmSOVWihGnpMHQovPQSHDpkzo/SD4aIiOTUkSOmi/uPP8KUKeyvX5/qOibHY2kmKzcKFTILFCdMMO88unaF06etrkpERNzBnj3QqhX88gssXKgu7l5AISu3bDZ44QWzNmvVKtPT5PBhq6sSERFXtn49hIVBWhqsXQv33Wd1RVIAFLLy6qGH4JtvYP9+aNkSfvrJ6opERMQVzZ59pQfW5s3m3wzxCgpZ+XHPPabFg6+vmdH65hurKxIREVdht8Orr5pjcsLCzE71WrWsrkoKkEJWfjVqZN6Z1K1r1mhNyd0ZjiIi4oHS02HwYBg/3oSsFSugbFmrq5ICppDlCEFBsG6daVQ6bBiMGmV6oIiIiPc5etRcHpwxw6zhnTUL/P2trkosoJDlKCVKwOLF8MQTMHmymdU6dcrqqkREpCDt3m12EG7ZYtZiTZhgNkyJV1LIciRfX3jrLZg61ew8vOMOszBeREQ83+rVZu3VmTNmB+E//mF1RWIxhSxnGDIEVq6EpCRzqPTGjVZXJCIizvTBB9CxI1SpYmaxwsKsrkhcgEKWs0REmB+0smXhrrtg2jSrKxIREUfLyIDHHzeNRS/vOK9Rw+qqxEUoZDlT7dpm52G7dqav1ujRWhAvIuIpjh41wer992HcOHP0WqlSVlclLkQhy9luucX0zxozBmJioFMnOH7c6qpERCQ/fv3VLAf5/nuYORNef92syxX5G4WsguDnB2+/bS4Zrl9vfjB37bK6KhERyYslS8yaq/PnTfueAQOsrkhclEJWQRo0yOw4SUszW3y//NLqikREJKcuXTItGe67D+rVgx9/1BE5clMKWQXtjjvMD2a9ehAVBc8/b35wRUTEdZ06ZcLVSy/BwIGwYQNUrmx1VeLiFLKsULWquWw4aBBMnGgal548aXVVIiJyPbt3mxmr2Fh4912z9KNIEaurEjegkGWVIkXgk0/MWYcrV0KLFrBzp9VViYjI3y1ZYtbRHj9umo0OH64O7pJjCllWstlMb5XL67RatoT5862uSkRELl6E//kfc4mwbl34z38gPNzqqsTNKGS5gjZtzA9wkybQp4/pp5WebnVVIiLe6cgRiIw0bRmGDjWndlStanVV4oayDVmjRj1Ogwa1CA+/soPixInj9OzZnZYtg+nZszsnT54AwG638+yzYwkNbULbtmH88st251XuaYKCIC7OBKyYGNMxPiHB6qpERLzL5s3QrBl89x1Mn26Oy/H3t7oqcVPZhqwHHujH3LmLrrpv8uRJhIe3ZcuW7YSHt2Xy5EkArF69kn37/mDLlu289VYM48aNcU7VnqpQIZg0CebNg59/Nj/oa9ZYXZWIiOez2+G998wlwcKFTZPRgQOtrkrcXLYhKyysNWXK3HLVfcuXL6NPn74A9OnTl9jYrwGIjf2G3r3/gc1mo3nzUE6dOkVKSrITyvZwvXubNg/lykGHDvDqq2rzICLiLKdPm6UaI0aYy4T/+Q8EB1tdlXiAPK3JSk1NpWLFSgBUqFCR1NRUAJKTEwkKqpL1uKCgyiQlJTqgTC9Uvz788IP5wR8/Hjp3hr/GWUREHGT7dggJgUWLzBqsJUvMcWgiDuCX329gs9mw5WE768yZ05k1awZgwllcXFx+S8lWWlpagTyPQw0ZQmBgILXffZeMBg3Y9dxznGrSxOqqADcdTxenMXUsjafjecyY2u0Efv21+d1aujS7Jk3iVKNGpodhAfKY8XQhrjSmeQpZAQEBpKQkU7FiJVJSkilfvjwAlSoFkZh4OOtxiYkJBAYGXfd7REcPIjp6EACdOkXQrl27vJSSK3FxcQXyPA4XEQEDBuDfuzdNn3gCXn4Znn4afKzdHOq24+nCNKaOpfF0PI8Y09OnTfuc2bPhnnvw/+wzmgYEWFKKR4yni3GlMc3Tv9KRkZ2ZN282APPmzaZjxy4AdOzYifnz52C329m69QdKlSqVdVlR8ik4GLZuhV694NlnoVMnSNZ6NxGRXNm61WwqmjvXnLgRGwsWBSzxfNmGrKFDB9G5c3v27v2dJk3q8fnnMxk5cgzr1q2lZctg1q2LY+RIs4uwfftIqlevQWhoE558ciSvv/620/8CXqVUKZgzx2wpXr/e9NVascLqqkREXN+lS/D22+b82PR0WLcOnnvO8isC4tmyvVz44YfTr3v/woVLr7nPZrMpWDmbzWaa47VpYxbFd+wITz0Fr7xith2LiMjVjhwx7RhiY00H908+gbJlra5KvIAivLtq2NC0eXjsMXjzTWjdGvbutboqERHXsnKlmfVfs8b0wVq0SAFLCoxCljsrWtQcML1woQlYwcHmdHi73erKRESsdf48jBlj+l6VLQtbtsCwYTrcWQqUQpYnuP9++OUXc1L8Qw9Bz55w7JjVVYmIWOPXX83vw3fegeHDzWJ3F2l9I95FIctTVK0Kq1bBG2/A0qXQqJGZJhcR8RaXLsG770Lz5pCSAsuWmc+LFrW6MvFSClmexMcHxo41neLLlDHT5KNGwblzVlcmIuJchw6Z33kjR0L79rBjhzkpQ8RCClmeKDjYnL01YgRMngxNm5rgJSLiaex2+OwzM3v//ffw4YdmNr9CBasrE1HI8lhFi5qA9e23cPas6Q3zz3+a/jAiIp4gNdU0aB4wAG6/HX7+GR55RIvbxWUoZHm6y9Pm/fub43hatTKLQkVE3NmXX5rZq6VLzcHO69bBrbdaXZXIVRSyvEGZMjBjBixeDAkJ5kiJV16BjAyrKxMRyZ1jx6BvX4iKgsBA0y9w3Djw9bW6MpFrKGR5k/vuM7NYUVHmOImWLc30uoiIO1i8GBo0gAUL4MUXzVrTxo2trkrkhhSyvE1AAMybZxqYJiSYrc4vvKC1WiLiuo4eNbNX998PlSubvlfPPw+FClldmchNKWR5q/vvh127zPmHL74ILVqYaXcREVdht8Ps2VC/PnzxBbz0kuncrsai4iYUsrxZuXJm6/OSJeadYqtW5hiKtDSrKxMRbxcfD126QL9+ZkH7tm1mh7Rmr8SNKGQJdOtmZrWGDjXHUNx+OyxfbnVVIuKNMjNNl/aGDWH9eoiJgU2bzO8lETejkCVG6dLmsOkNG0yPrU6dTNuHI0esrkxEvMX27aan38iR0KaN2agzcqR2DorbUsiSq7VpY37RPf88zJ8PdevC1KnmTDAREWdIS4MnnzQbcQ4cMMsYYmOhRg2rKxPJF4UsuZa/v1kM//PPZoHp0KHQurXaPYiI4y1ZYtoyvP02PPQQ7N5t1mGpa7t4AIUsubH69WHtWpg5E/74A0JCzLvNM2esrkxE3N2+fWY96H33meUKmzaZcwdvucXqykQcRiFLbs5mM+eC7d4NDz8MkyaZS4iff262V4uI5Ma5czBhgpm9WrMG3njD7By84w6rKxNxOIUsyZmyZeGDD2DzZqhSBfr3J3j0aF1CFJGcsduvXBp86SVz8sSePTB2rNoyiMdSyJLcCQ01QevjjykWH2/OQRw+HI4ft7oyEXFV//0vdO5sLg0WK2ZmsObMMd3bRTyYQpbkno8PPPQQP8yaBY8/Du+/D7Vrw3vv6dBpEbnixAkYPdqcL/j99/DWW2b3ckSE1ZWJFAiFLMmziyVLmqaBP/0EwcEwYoTZjbhihdWliYiVLl688ubr3XfNrsHff4cnntClQfEq+QpZISG307ZtKyIiWtOhQ1sATpw4Ts+e3WnZMpiePbtz8uQJhxQqLqxxY1i1Cr780hw03bGjOQ7jv/+1ujIRKUh2u+lv1bSpmeVu1Mgsav/gA3M4vYiXyfdM1qJFy1i7dhPffrsOgMmTJxEe3pYtW7YTHt6WyZMn5btIcQM2G3TvDjt3wptvwsaN5hfso49CcrLV1YmIs23fDvfcY9ZenTtnDnRes0aHOYtXc/jlwuXLl9GnT18A+vTpS2zs145+CnFl/v6ml9bevTBsGHzyCdx2m2luqoOnRTzP4cMwcKDZBLNtmzlrcNcu6NFDDUXF69mOHDmd52ZHzZs3onTpMthsNqKjBxEdPYjbbqvK3r2HALDb7dSuXS3r87+bOXM6s2bNACA5OZG5c+fmtYwcS0tLo0SJEk5/Hm+Rk/EsmpBAzY8+osK6daTfcgsHBg4kqXNn7H5+BVSle9Fr1LE0no53eUz9Tp2i2pw5VF68GJvdzuEePTjYrx8XNd65oteo4xXEmI4e/UTWFbybyVfISkpKJDAwiNTUVHr16s6//vVvBgx44KpQVbt2NX7//eBNv0+nThFs3bo1r2XkWFxcHO3atXP683iLXI3n5s2mH87GjVCrlumT88ADOvj1/9Fr1LE0no63ITaWO7dtM01Ez5yB6Gh44QWdM5hHeo06XkGMaXBwsxyFrHxdLgwMDAIgICCAzp3vZdu2/xAQEEBKilmDk5KSTPny5fPzFOIpWrWC9eth2TIoWRL69zc7Er/6Sp3jRdzBhQvw3nu07NcPnnvOtGH45ReYMUMBS+QG8hyyzp49S1ramayP4+LWUL9+fSIjOzNv3mwA5s2bTceOXRxTqbg/m80sit22zTQiPH/eLJYPCzNtHxS2RFxPero5U7B2bRgxgj+rVoXvvjO7iW+/3erqRFxankNWauoR7r03knbt7qBjxwg6dIjkrrs6MHLkGNatW0vLlsGsWxfHyJFjHFmveAIfH3OpcNcumDoVkpJM24fWrWHlSoUtEVeQkQHTppmzSh991HRn//Zbtr/zjnljJCLZyvPq4xo1ahIX990195ctW46FC5fmqyjxEoUKwZAhZk3H9OnwyisQGWnC1gsvwN13a3eSSEFLT4dZs+Bf/4I//oDmzU1j0chI8/MYF2d1hSJuQx3fxXr+/uad8t698L//CwcOQIcO5t3y0qWa2RIpCOfPm5+/226Dhx+G0qXNmskffjAzzXrDI5JrClniOvz9TZfovXvNO+eUFOjWzSyQnzcPMjOtrlDE86SlwdtvQ82a5rD3qlVN1/atW6FrV4UrkXxQyBLXU6SImdn67Tf49FNz+eKBB6BBA/joI/OOW0Ty58gR+Oc/oVo100C4QQPToX3jRs1ciTiIQpa4rkKFzHqtX3+FBQtM64dHHjHbxV99FU7oXEyRXPvjDzNjXL26WQfZrh18/z2sXm3aMihciTiMQpa4Pl9f6NkTfvzRvNNu2hTGjzeXNUaPhn37rK5QxLXZ7bBpk/k5qlPHHHfVv785xH3RItPHTkQcTiFL3IfNZt5px8bCzz/D/fdfWagbFQXr1mmRvMjfZWTA7NkQGgpt2pg3KWPHwv795tJ73bpWVyji0RSyxD01bgwzZ5qdiM88Axs2mMsezZqZDtRatyXe7MgRcymwZk3o1w9On4YpU+DQIXjtNQgKsrpCEa+gkCXurXJl84/JoUPmnXlGBgwaBFWqwLhxupQo3sNuN2ur+vUzr//nnoN69eDrr81lwcceg+LFra5SxKsoZIlnKFrU9PbZsePKAt633zaXEjt3Nv/QqAWEeKIzZ8wbjJAQuOMO81p/9FETrFatgi5dzCkLIlLg9JMnnsVmg7vuMrsR4+NhwgSzfqtrV7MrccIEc4lRxJ3Z7WYjyCOPmEt/jzxiZnHffx8SEmDyZDOLJSKWUsgSz1W58pVQtXChOcx24kSoVcscEbJgAVy4YHWVIjl37Bi8955ZexgaCp9/Dr16mQObf/nFzGCVKGF1lSLyF4Us8XyFCpmdiLGxJnBNmGAupfTubWYBhg2DLVu0M1FcU0aGOV6qRw8IDIQRI8yM7fvvQ2KiOcQ5LEz9rURckEKWeJdq1UzI2r/fhK7ISPOPVKtWUL++aXIaH291lU73W69uJE1+K+tze2YmfwzuT8K/XgIg7cctHF/8hVXlXePc778R/9Qo4p8axYGnRnJmy/dWl+Rcly8HjhljFrF362Y6sQ8fDtu3w7ZtZtaqdGmrKxWRm/CzugARS/j6mqNDOnaEU6fgiy9MS4jx483tjjvMUT69e0PFilZX63A2/yKkHzzIpQsX8PH3589ftuNXtlzWn5do0RJatLSwwqv5V6tOtdffxubry8UTx4l/ahQlmodi8/W1ujTH2r0b5swxva327oXCheHhBtznAAAOJUlEQVTee2HgQPNaLVTI6gpFJBc0kyVSujQ89JBpZrpvn5nNOnMGRo40lxM7dICPP4bUVKsrdajizUI4u20rAKc3rqdkm/CsPzu1djUpH38AQPJ773Bk2lQOjh/H/mFDOPP9ppt+38yzZ9k/8jHSEw4DkPTOvzm5akW+avXx988KVPb0dPCkK2O//WZec02bmtnUl182R9588ok5JH3hQrNxQwFLxO1oJkvk72rWNM1Nn3kGdu6EuXPNzMKQITB0KLRta9Z3RUWZhfVurGTrOzm2YB7FQ1qQHn+A0ne159x/d173sRdPHKfqxNdITzhM4usvUzKsNQDxT42i+psxVz3Wt3hxKjz0CMn/G0OZLl3JTEujTPvIa77nkRkfc+7XHdetq2xUz2vuP/f7HlKmTCYjNZVKI8a47yyW3Q67dpnw9MUXpu0ImEvW77xjZk8DA62tUUQcQiFL5EYaNjS7EV96ybSBWLjQ3EaMMLdWrcwMQ9euZueimy089q9ek4zUI5zZtJ7izUJu+tgSoa2w+fjgX7UaF0+dyrr//wesy4o3aUra95s48vGHN3xMhYEP56reorXrUmPS/3Lh8CGS33uH4k1D8ClcOFffwzIXL5qzA7/6ytz27jWvlzZtICbGBPcqVayuUkQcTCFLJDs2GwQHm9vEiWZn4sKF8OWXV9Zw1ahhwta990J4OBQpYnXVOVKieSipM6dT9YVXyEw7c8PH2fz+dqkqB7sw7ZcukZ5wGB9/fy6lpUG58tc8JrczWZf5V6mKT5EipB+Kp8ittbOtxTLHjsG338I338CyZXD8uFljdddd8MQTcN99mrES8XAKWSK5Vb++ObLkuefMFvqvvzZb7D/6CN5913Sfb9fO7Fzs2BHq1HHZWa5Sd7XHp3hx/KvX4M+d1waevDqxbAmFK1ehXN8BJE+ZTLVX3sDmd/Wvm9zMZGWkJONXPgCbry8ZqUdIT0ygUICLbUjIzIStW2H5crNz9YcfTCAtV86E727d4J57oGRJqysVkQKikCWSH5e7bT/yCPz5J6xdCytWmNvo0eYx1aub2Yu77jLH/bjQWq5C5cpzS+euef76663JSk84zOnV31LtX2/iU7QYRes35NjC+ZTv0zfPz3Nu9385/uVEbL5+4GOj4sOP4luqVJ6/n0PY7WZWc/Vqc4uLMztVbTbTKPT556FTJ2je3OxmFRGvo5Al4ijFiplz4rp0MZ/v32/C1sqV5tLi9Onm/jp1TNgKDzdrcqpVK/BSa382/5r7ijVsRLGGjQAoHXE3pSPuBqDS8NE3/NrrrbcqXLkKNd6ZkvV5hYEP5bveUm0jKNU2It/fJ18uXYJff4UNG8xt/XpISjJ/VqMG9OwJd99tdqOWv/byqIh4H4UsEWepWdM0jHz0UfMP9M8/m5muNWtMH6QPPzSPq1rVhK3WrSnh52d6dLnLgm5PduaMaQi6ZYs5tmbTJjhxwvxZ5crmkvBdd5lgVbOmpaWKiGtyWshas+Zbxo9/mszMTPr3f5CRI59w1lOJuD4fH9MHqWlTs+g5M9Ns3d+40dzWrYM5c2gOMGrUlbPpWrSAkBCoXVuXnJzp3Dnz/+Onn0yw2rzZtFm4vMi/Th2zAzA8HO6808xcueg6OxFxHU4JWZmZmTz99JMsWLCEoKDK3HNPOyIjO1O3rk6FFwFMYLq8Y3H4cPOPeXw8O6dPp+HZs2bR9Ecfme39YC5FNm5sHt+0KTRqBA0a6FiV3LLbITnZXPb79VdzRM1PP5lAlZlpHlO2LLRsaQ5ebtnShN2yZa2tW0TcklNC1rZtW6lZsxY1apgp9KioHixfvkwhS+RGbDaoUYPUiAhzGQpMb6WdO00I2L7d3ObMgQ8+uPJ1VaqYsNWwIdSrZ2a8atc2C/J9vPhAh4wMOHCAst9/b8Zvzx4TpH799colP4BKlcysYbdu5r/NmpmNCpqlEhEHcErISk5OonLlK431AgOD2PbX8R0ikkN+ftCkibldZrfDgQMmLOzcaYLDzp0meJ07d+VxRYvCbbfBrbea0PD3W7Vqpq2AE0LYlz8l8O8Ve0g8eY6gMkUZG1mX+5o6YTdlRoaZkTp82BzovX+/GZcDB8zH+/fDxYs0vvz4MmVMEO3VyzSOvf1283mFCo6vTUTkL5YtfJ85czqzZs0AIDk5kbi4OKc/Z1paWoE8j7fQeDpejse0ZEnTcb5VK/P5pUv4HzlC0YQEiiUkUPTwYYomJFB0+3b8V6zA7+8BDLjk60t6uXKkly3LhfLlSS9blozSpckoVYqM0qW5+NfHF4sXJ7N4cS4WK8Ylf/+bzvB8l5jBjF/TSb9kPk84eY5xC7az67+7uCPoJufu2e34njuH39mz+Kal4Xf2LH5nzlD41CkKnTpFoZMnKXTyJIVPnqTwsWP4Hz1KoZMnsf2/pqjppUtzvlIlzgcFca5FC85VrcqxcuWgTh0ySpW6tvZdu8xNckU/946l8XQ8VxpTp4SsSpUCSfjrcFiApKREAgODrnpMdPQgoqMHAdCpUwTtLl8icaK4uLgCeR5vofF0PKeMqd1uLpHFx5vbwYP4JCVRJCmJIomJpg3B7t2mI/mlSzf+Pj4+JtwVK2Y62v/95ufH+JBHSfe/eo1Y+iVY9kMqz26dYtY8paebfmLnzl25nT178+ctXNi0RKhQwVwKbdvW7O4LCjL/rVEDqlencIkSFAb+3j1Lr1HH05g6lsbT8VxpTJ0Sspo2DWHfvn3Exx8gMDCIxYsX8sEHnzjjqUQkOzabWbhdtqxZNH8jly7ByZNw9Ki5HTsGp0+b25kzVz4+dw7On7/6lplJYuHrNwdN9C9lgpivrwlMRYuaoFa0qLkVL24W8P/9VqYMBASYcFWypNZIiYhbckrI8vPz47XX/k2fPlFkZmbSt+8A6tWr74ynEhFH8fG5Esbq1Mn1lwe9toaEk+euvf+WYrBqlSMqFBFxK05bk9W+fSTt20c669uLiIsZG1mXZxbt4FxGZtZ9RQv5MjayroVViYhYRx3fRcQhLu8iLJDdhSIibsB25Mhpe/YPc6569WpQtWp1pz/PsWNHKVdOZ4o5isbT8TSmjqXxdDyNqWNpPB2vIMb00KF4du8+kO3jXCJkFZQOHdry7bfrrC7DY2g8HU9j6lgaT8fTmDqWxtPxXGlMvbgltIiIiIjzKGSJiIiIOIHv2LHPvGB1EQWpSZOb9AmSXNN4Op7G1LE0no6nMXUsjafjucqYetWaLBEREZGCosuFIiIiIk7g8SHrq68Wc+edoVSsWJrt27dd9WcxMW8RGtqEsLBmrFmjjtR58cYbr9K4cV0iIloTEdGaVatWWF2SW1qz5lvCwpoRGtqEyZPftrocjxAScjtt27YiIqI1HTq0tboctzRq1OM0aFCL8PCWWfedOHGcnj2707JlMD17dufkyRMWVuherjee+h2adwkJh4mK6kKbNi24885Qpk6dArjWa9TjQ1a9eg2YPv1zwsJaX3X/nj27Wbx4IRs2/MDcuYt4+uknyMzMvMF3kZsZOnQYa9duYu3aTerynweZmZk8/fSTzJmzkI0bf2TRoi/Ys2e31WV5hEWLlrF27SaX2c7tbh54oB9z5y666r7JkycRHt6WLVu2Ex7elsmTJ1lUnfu53niCfofmlZ+fHy+++AobN/5IbOxqpk37iD17drvUa9TjQ1adOnW57bba19y/fPkyoqJ64O/vT/XqNahZsxbbtm21oELxdtu2baVmzVrUqFGTwoULExXVg+XLl1ldlghhYa0pU+aWq+5bvnwZffr0BaBPn77Exn5tRWlu6XrjKXlXsWIlGjcOBqBEiZLUqVOXpKREl3qNenzIupGkpESCgq4c9xEUVJnk5CQLK3Jf06ZNpW3bMEaNelyXDvIgOTmJypWrZH0eGBhEUlKihRV5BpvNRu/e99G+fTgzZ063uhyPkZqaSsWKlQCoUKEiqampFlfk/vQ7NP8OHoxnx45fCAlp7lKvUY84u7BHj26kpqZcc/8zzzxPp05dLKjIs9xsfAcOfJgnn3wam83Ga6+9zIQJ44mJmWJBlSJXW7p0BYGBQaSmptKrV3dq165zzbIByR+bzYbNZrO6DLem36H5l5aWxuDBA5g48TVKlix11Z9Z/Rr1iJC1cOFXuf6awMAgEhMTsj5PTEygUqVAR5blMXI6vv37P0j//r2dXI3nqVQpkISEw1mfJyUlEhgYZGFFnuHyGAYEBNC5871s2/YfhSwHCAgIICUlmYoVK5GSkkz58jp3Lz8qVKiQ9bF+h+ZeRkYGgwf3p0eP3tx7bzfAtV6jXnu5MDKyM4sXL+TChQvExx9g3759NGvW3Oqy3E5KSnLWx998s5R69epbWI17ato0hH379hEff4D09HQWL15IZGRnq8tya2fPniUt7UzWx3Fxa6hfX69NR4iM7My8ebMBmDdvNh076mpBfuh3aN7Z7XZGjx5GnTp1eeyx4Vn3u9Jr1OObkS5btpRnnx3LsWNHKVWqNLff3oj5878EYNKkfzN79iz8/Px4+eXXuPvueyyu1v08/vgQdu7cAdioVq0ab74Zk3UtXHJu1aoVPPfc/5CZmUnfvgMYM2as1SW5tQMH9jNwYD8AMjMvcv/9vTSmeTB06CA2bdrI8ePHCAiowLhxz9KpUxeGDBnI4cOHqFKlGh9/PINbbilrdalu4XrjuWnTBv0OzaPNm7+nW7dI6tdviI+PmTMaP/55mjVr7jKvUY8PWSIiIiJW8NrLhSIiIiLOpJAlIiIi4gQKWSIiIiJOoJAlIiIi4gQKWSIiIiJOoJAlIiIi4gQKWSIiIiJOoJAlIiIi4gT/BzeyF1yGVHTvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95df74f198>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-10,20,100)\n",
    "y = x**2 - 6*x + 5\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-')\n",
    "plt.plot([3],[3**2 - 6*3 + 5],'o')\n",
    "plt.ylim([-10,250])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "ax.text(3,\n",
    "        10,\n",
    "        'Min: x = 3',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['pale red'],\n",
    "       )\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Second approach\n",
    "\n",
    "To find the local minimum using **gradient descend**: you start at a random point, and move into the direction of steepest **descent** relative to the derivative:\n",
    "\n",
    "+ Start from a random $x$ value.\n",
    "+ Compute the derivative $f'(x)$ analitically.\n",
    "+ Walk a small step in the **opposite** direction of the derivative. \n",
    "\n",
    "In this example, let's suppose we start at $x=15$. The derivative at this point is $2×15−6=24$. \n",
    "\n",
    "Because we're using gradient descent, we need to subtract the gradient from our $x$-coordinate: $f(x - f'(x))$. However, notice that $15−24$ gives us $−9$, clearly overshooting over target of $3$. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAADGCAYAAADhV5X4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xlc1NX+x/HXAIq4p6Is7iUuuaAoShpKaYiWimvXhdQyb2kudbVf2W51q5sZ1s/KSktzy+uWGWoumFpaZpbp1RaXlE3cxTQR5/fHuaH+TNlm+M7yfj4e8wiGgflwGvDN+Z7zObbDh0/ZERERERGH8rG6ABERERFPpJAlIiIi4gQKWSIiIiJOoJAlIiIi4gQKWSIiIiJOoJAlIiIi4gR5hqyUlEPEx3elXbtW3HprJNOmTQXglVdepGnT+sTEtCUmpi2rV6/M/ZzExElERjYjKqoFa9eudl71IiIiIi7KllefrIyMdDIy0mnaNJysrNN07BjNhx/OZenSRZQpU5YRI0Zd8fg9e3YzfPhQVq5cR3p6Gr17d2Pz5u/w9fV16jciIiIi4krynMmqVi2Ipk3DAShbthxhYfVJS0u95uNXrFhOfHwv/P39qVWrNnXq1GXbtq2Oq1hERETEDRRoTdZvvx1gx44fiIhoCcD06dNo3z6K0aMf5MSJ4wCkpaUSEhKa+zkhIaGkp6c5sGQRERER1+eX3wdmZWUxdOggJk58iXLlyjN48H088sij2Gw2XnrpeZ5+egKJiVPz/cQzZ85g1qwPAPjpp93UqFGjwMUX1MWLF/Hx0Vp/R9F4Op7G1LE0no6nMXUsrx9Pu53Shw7he+4cv9eoQU6pUkX+ksUxpkeOHGH37v15Pi5fISs7O5uhQwfSq1df7ryzGwBVq1bN/fjAgfcwcGBfAIKDQ0hNTcn9WGpqCkFBwVd9zYSEISQkDAEgLi6GrVudf0kxOTmZDh06OP15vIXG0/E0po6l8XQ8jaljefV42u2QkAA//wzz50Pfvg75ssUxpuHhLfL1uDyjnt1uZ8yYEYSF1eeBB0bm3p+RkZ779mefLaNBg4YAxMZ2YfHihfzxxx8cOLCfvXv30qJFy4LWLyIiIp7shRfgo4/gueccFrBcTZ4zWVu2bGbBgnk0bHgzMTFtAZgw4SkWLfo3O3fuAGzUrFmTV19NBKBBg4Z07x5Pu3at8PPz4+WXX9XOQhEREblkzhx48kkYOBCeeMLqapwmz5DVpk0Uhw+fuur+jh1jr/k5Y8eOY+zYcUWrTERERDzPhg0wZAi0bw/vvQc2m9UVOY0Xr7YTERGRYvXTT9CjB9SpA4sWgb+/1RU5lUKWiIiION+RI9C1K/j4wPLlUKmS1RU5Xb5bOIiIiIgUyrlz0L07HDwI69bBjTdaXVGxUMgSERER57l4EQYPhi+/hI8/hqgoqysqNrpcKCIiIs7z6KOmD9Yrr0CfPlZXU6wUskRERMQ53nwTXn0VRoyAf/zD6mqKnUKWiIiION6SJTBqlFmLlZjo0a0arkUhS0RERBxr82b4298gMtI0HvXSpuQKWSIiIuI4v/wCd90FoaGwbBmULm11RZZRyBIRERHHyMiA2P+eCJOUBIGB1tZjMbVwEBERkaLLyjLNRtPTYe1aqFfP6oosp5AlIiIiRXP+PPTqBdu3w9Kl0Lq11RW5BIUsERERKTy7He67D1atgunTzWyWAFqTJSIiIkXx2GMwaxY8/zwMGWJ1NS5FIUtEREQKZ8oUePllePBBePxxq6txOQpZIiIiUnBz5sDo0dCzpwlbXthsNC8KWSIiIlIwK1bAPfdAhw4we7bXNhvNi0KWiIiI5N/mzWYnYZMmZidhqVJWV+SyFLJEREQkf3btMrsHg4NNs9Hy5a2uyKUpZImIiEjefvsN7rgDSpY07RqqVbO6IpenPlkiIiJyfZmZJmBlZcEXX0DdulZX5BYUskREROTaTp405xH+9puZwWra1OqK3IZCloiIiPy133+Hu+6CHTvgk0+gXTurK3Irea7JSkk5RHx8V9q1a8Wtt0YybdpUAI4fP0bv3t1p3Tqc3r27c+LEcQDsdjuPPz6OyMhmtG8fxQ8/bHfudyAiIiKOd/489O4NGzfCRx9BXJzVFbmdPEOWn58fzz77Ahs3fkNS0hqmT3+XPXt2M2XKZKKj27Nly3aio9szZcpkANasWcXevb+yZct2Jk1KZPz4sU7/JkRERMSBcnIgIcHsIHznHejXz+qK3FKeIatatSCaNg0HoGzZcoSF1SctLZUVK5bTr19/APr1609S0qcAJCV9Rt++f8Nms9GyZSQnT54kIyPdid+CiIiIOIzdDiNGwPz58MorMGyY1RW5rQKtyfrttwPs2PEDEREtyczMpFq1IACqVq1GZmYmAOnpqYSEVM/9nJCQUNLSUnMf+6eZM2cwa9YHuZ+TnJxchG8jf7KysorlebyFxtPxNKaOpfF0PI2pY7nceNrt3PjWW9RYsIAD/fuzr1UrcKX68sGVxjTfISsrK4uhQwcxceJLlCt3ZfMxm82GrYBnFiUkDCEhwZzWHRcXQ4cOHQr0+YWRnJxcLM/jLTSejqcxdSyNp+NpTB3L5cbzmWdgwQIYOZJaU6ZQyw3PI3SlMc1XM9Ls7GyGDh1Ir159ufPObgAEBgbmXgbMyEinSpUqAAQFhZCaeij3c1NTUwgODnF03SIiIuJI//oXPPssDBkCiYk68NkB8gxZdrudMWNGEBZWnwceGJl7f2xsF+bPnwPA/Plz6Ny5KwCdO8fx8cdzsdvtbN36NeXLl7/qUqGIiIi4kKlTYfx4s8D93XfBRwfCOEKelwu3bNnMggXzaNjwZmJi2gIwYcJTjBo1lmHDBjN79kyqV6/Je+99AEDHjrGsXr2KyMhmlC5dmsTEqU79BkRERKQIPvzQLHS/6y6YNQt8fa2uyGPkGbLatIni8OFTf/mxhQuXXXWfzWbj5ZdfK3plIiIi4lzz58PQodCxI3z8MZQoYXVFHkXzgSIiIt5o0SIYMADatoUlS6BUKasr8jgKWSIiIt5m2TKz/ioyEpYvhzJlrK7IIylkiYiIeJOkJHNcTvPm5u1y5ayuyGMpZImIiHiL1ashPh5uvhlWroQKFayuyKMpZImIiHiDdeugWzcIC4NVq+CGG6yuyOMV6FgdTxe0tAYNyzfIfT8+tDujwkZe5zOuFLGqNavaJ1HZv5IzyhMRETfx4uwTzFt7Bl8f03Jq6pgqbN51jmFdy1G6VMHmNz5ceZpOEQGEVCnCP9nJydC1K9SpY2az/ttAXJxLIesypXxLsS7mc6vLEBERN/bVrnMs33yWb94Kwb+kjSMnczifbaf/olMM6FiW0gXYxJeTY+fDlVncXLtk4UPW+vWXAtbatVC1auG+jhSYQlY+RKxqTd+afViV/jkXLl7gvVbvUK/cTRw7f4zhW0eQfi6dljdEYMdudakiImKx9KM5VKngg39JcyxNlQq+vLH4FKlHc+j4SDqVK/iyZlIQI14/yrpvg/GblULP6NI8c4+5fHfjgIP06VCGNd+eY0zv8nz703kS/plJgL+NjVOCCfAvwEzY+vXQpQvUrm0CVrVqTviO5Vq0Jusy53LOEbOuU+5tScrS3I9VLlmJNR1Wck+dQUz95W0AXt09mdaVI9lw2zq6BHfm0NkUq0oXEREX0allAIcyL9DwnkOMTDzK+u/P8VB8eUIq+7J6UhBrJpmj5iYOrcjUe9P4bloIX3x/jh/2ns/9GpXL+/LN2yEM6FiWiLCSzHwskG/fCS1cwKpVSwHLIprJusz1Lhd2DY4DoFmFpixPTQLgq6ObmRH5HgCdgjpSsUTF4ilURERcVtkAH76eGsKGHedI/v4c/Z8/zIv3Xb3IfMH6M0yeH0zA7FTSjuXwnwPnaVq3JAB9OxSxb5UClktQyMqnkj7+APjafMmx51hcjYiIuDJfXxsdwgPoEB5Akzolmbkq64qP70vL5rUFp3h1QAZ3xdVn6CuZnDt/aclJmVK2wj/52rVw552XLhEGBRX+a0mR6HJhEURVbsOiQ4sBWJOxlhPZJyyuSERErLbnYDY/H8rOfX/7L+epVc2PcgE+nP79IgCnfrdTppSNMqUuknE8hxVfn73m1ytX2ofTZy/m78lXrTKL3G+80ewoVMCylGayLvPnmqw/3VY1hidvfvyaj/9Hg7EM3zqCW9fG0KpSS6oHhBZHmSIi4sKyzl5k9JvHOJl1ET9fuDG0BG+Prcy8tWfo+lgGwZX9WDMpiPCbSjLkrVDCamVyS+Nrbzm8546yjHj9aN4L3z/7zDQabdhQbRpchELWZdK7H/zL+7+9Y0vu2+E3NGNJu38DUKlkJRbcMrdYahMREfcQEebPxinBV90/Mr48I+PL574/fXwgyck76dChwxWP+3V2jSve7xldhp7ReazR+uQTc1ROkybw+edQSf0aXYH3XC60q72CiIh4oEWLoFcvCA83M1gKWC7DO0JWSgq0bUuZX3+1uhIRERHHmT0b+vaFVq3MDJaOynEp3hGyzpyBgwcJHzsWvv7a6mpERESK7t13YdAgiI42C9512LPL8Y6QFRYGGzZwoVw5uP120z9ERETEXSUmwv33Q1wcLF8OZctaXZH8Be8IWQC1a/NdYiLUqAGdO8OKFVZXJCIiUnAvvghjxkDPnrB4MQQEWF2RXIP3hCzgfJUqZharQQPo1s28OEVERNyB3Q4TJpjbgAEwfz6ULGl1VXIdXhWyAAgMNB1wIyKgTx/46COrKxIREbm+ixdh5EgzizVsGHz4IfipC5Or876QBWb3xapVZrHgoEHw5ptWVyQiIvLXsrMhIQGmToVx4+Cdd8DX1+qqJB/yDFmjRz9Io0Z1iY5unXvfK6+8SNOm9YmJaUtMTFtWr16Z+7HExElERjYjKqoFa9eudk7VjlCunOmO2707PPQQTJyoXloiIuJazp0zPbBmzzazWC+/DLYinGsoxSrPuca77x7Avffez8iRw6+4f/jwEYwYMeqK+/bs2c3ixQvZsOFr0tPT6N27G5s3f4evqybuUqXg3/+Ge++Fp56CY8dg0iTw8c4JPhERcSGnT5v1w+vXm1msBx6wuiIpoDzTRFRUWypWzF9zsxUrlhMf3wt/f39q1apNnTp12bZta5GLdCo/P5gxA0aPhtdfN4HrwgWrqxIREW+WmQm33QYbNpi1wwpYbqnQq+amT5/Gxx/PJTy8Oc8++wIVK95AWloqERGtch8TEhJKenraX37+zJkzmDXrAwDS01NJTk4ubCn5lpWVde3n6d6dWqdOUWfGDI78/DO7nnySi/7+Tq/JnV13PKVQNKaOpfF0PI2pY/3VeJZKT6fp+PH4Hz7MrokTORoSAhrzfHOl12ihQtbgwffxyCOPYrPZeOml53n66QkkJk4t0NdISBhCQsIQAOLiYq46INMZkpOTr/88MTEQEUGVhx4i+vnnzYGbOqLgmvIcTykwjaljaTwdT2PqWFeN548/mvYMv/8Oa9bQpG1by2pzV670Gi3U4qOqVavi6+uLj48PAwfew3fffQtAcHAIqakpuY9LTU0hKOjqk8hd2ogRMG8ebNlidh+mplpdkYiIeINNm+DWW83bGzaAApbbK1TIyshIz337s8+W0aBBQwBiY7uwePFC/vjjDw4c2M/evXtp0aKlYyotTn37QlIS7N8Pt9wCP/1kdUUiIuLJli+HTp2galUTtho3troicYA8LxcOHz6ETZs2cuzYUZo1a8D48Y+zadMGdu7cAdioWbMmr76aCECDBg3p3j2edu1a4efnx8svv+q6Owvzcvvt5hp4XBwMbgPdKsO5TKhQHW5/Cpr2tbpCERFxU0u+S+FfK/eQcuIsoauXMm5JIj0aNTJ/4AcGWl2eOEieIeudd2Zcdd+AAQnXfPzYseMYO3Zc0apyFRER8NET8MWTcO6wue/kQVj239YVCloiIlJAS75L4bFFOzibnQNAygU/HusyCuIb00MBy6OoIVRedk67Oopmn4U1z1lSjoiIuLd/rdyTG7D+dNanBP9a/5tFFYmzKGTl5eShgt0vIiJyHaknzhbofnFfCll5qVD9r++/UMqcJyUiIpJfhw8Tcu7EX34opGJAMRcjzqaQlZfbn4IS//+F7wdLj8Jdd8GpU5aUJSIibmbPHmjThnHJHxLgc+VZuQElfBkXW9+iwsRZFLLy0rQv3DUFKtQAbOa/Pd+CMW/D6tWmp8khXToUEZHr+OILiIqCrCx6vDORf/ZpTuh/Z65CKwbwz55N6NE81OIixdEKfayOV2na9+qdhE2BGjWgd29o3Ro+/RSaN7ekPBERcWFz5sCQIVCnDnz2GdStSw+gR/NQl+pOLo6nmayiuOMO0zTO19fMaH32mdUViYiIq7Db4cUXzTE5UVHw5ZdQt67VVUkxUsgqqiZNYPNmqF/frNGaWrAzHEVExAOdPw9Dh8KECSZkrVwJlSpZXZUUM4UsRwgJgfXroWtXc/bh6NFw4YLVVYmIiBWOHDFH5HzwATzzDMyaBf7+VlclFlDIcpSyZWHxYnj4YZgyxcxqnTxpdVUiIlKcdu+GNm1gyxazFuvpp8Fms7oqsYhCliP5+sKkSTBtmtl5eMstsG+f1VWJiEhxWLPGrL06fRrWrYO//c3qisRiClnOMGwYrFoFaWkQGQkbN1pdkYiIONPbb0PnzlC9upnFioqyuiJxAQpZzhITY37QKlWC226D6dOtrkhERBwtOxsefBAeeODSjvPata2uSlyEQpYz1atndh526AD33gtjxmhBvIiIpzhyxASrt96C8ePhk0+gfHmrqxIXopDlbDfcYPpnjR0LiYkQFwfHjlldlYiIFMWPP5rlIF99BTNnwssvm3W5IpdRyCoOfn7w2mvmkuEXX5gfzF27rK5KREQKY+lSs+bq3DnTvmfQIKsrEhelkFWchgwxO06ysswW3yVLrK5IRETy6+JF05KhRw9o0AC++cYcqyZyDQpZxe2WW8wPZoMGEB8PTz1lfnBFRMR1nTxpwtVzz8HgwbBhA4TqQGe5PoUsK9SoYS4bDhkCEyeaxqUnTlhdlYiI/JXdu82MVVISvPGGWfpRqpTVVYkbUMiySqlS8P775qzDVaugVSvYudPqqkRE5HJLl5p1tMeOmWajI0eqg7vkm0KWlWw201vlz3VarVvDxx9bXZWIiFy4AP/zP+YSYf368O23EB1tdVXiZhSyXEG7duYHuFkz6NfP9NM6f97qqkREvNPhwxAba9oyDB9uTu2oUcPqqsQN5RmyRo9+kEaN6hIdfWkHxfHjx+jduzutW4fTu3d3Tpw4DoDdbufxx8cRGdmM9u2j+OGH7c6r3NOEhEBysglYiYmmY3xKitVViYh4l82boUUL+PJLmDHDHJfj7291VeKm8gxZd989gHnzFl1x35Qpk4mObs+WLduJjm7PlCmTAVizZhV79/7Kli3bmTQpkfHjxzqnak9VogRMngzz58P335sf9LVrra5KRMTz2e3w5pvmkmDJkqbJ6ODBVlclbi7PkBUV1ZaKFW+44r4VK5bTr19/APr1609S0qcAJCV9Rt++f8Nms9GyZSQnT54kIyPdCWV7uL59TZuHypWhUyd48UW1eRARcZZTp8xSjYceMpcJv/0WwsOtrko8QKHWZGVmZlKtWhAAVatWIzMzE4D09FRCQqrnPi4kJJS0tFQHlOmFGjaEr782P/gTJkCXLvDfcRYREQfZvh0iImDRIrMGa+lScxyaiAP4FfUL2Gw2bIXYzjpz5gxmzfoAMOEsOTm5qKXkKSsrq1iex6GGDSM4OJh6b7xBdqNG7HriCU42a2Z1VYCbjqeL05g6lsbT8TxmTO12gj/91PxurVCBXZMnc7JJE9PDsBh5zHi6EFca00KFrMDAQDIy0qlWLYiMjHSqVKkCQFBQCKmph3Ifl5qaQnBwyF9+jYSEISQkDAEgLi6GDh06FKaUAklOTi6W53G4mBgYNAj/vn1p/vDD8Pzz8Oij4GPt5lC3HU8XpjF1LI2n43nEmJ46ZdrnzJkDd9yB/0cf0Tww0JJSPGI8XYwrjWmh/pWOje3C/PlzAJg/fw6dO3cFoHPnOD7+eC52u52tW7+mfPnyuZcVpYjCw2HrVujTBx5/HOLiIF3r3URECmTrVrOpaN48c+JGUhJYFLDE8+UZsoYPH0KXLh355ZefadasAbNnz2TUqLGsX7+O1q3DWb8+mVGjzC7Cjh1jqVWrNpGRzXjkkVG8/PJrTv8GvEr58jB3rtlS/MUXpq/WypVWVyUi4vouXoTXXjPnx54/D+vXwxNPWH5FQDxbnpcL33lnxl/ev3Dhsqvus9lsClbOZrOZ5njt2plF8Z07wz/+AS+8YLYdi4jIlQ4fNu0YkpJMB/f334dKlayuSryAIry7uvlm0+bhgQfg1VehbVv45RerqxIRcS2rVplZ/7VrTR+sRYsUsKTYKGS5s4AAc8D0woUmYIWHm9Ph7XarKxMRsda5czB2rOl7VakSbNkCI0bocGcpVgpZnqBnT/jhB3NS/L33Qu/ecPSo1VWJiFjjxx/N78PXX4eRI81idxdpfSPeRSHLU9SoAatXwyuvwLJl0KSJmSYXEfEWFy/CG29Ay5aQkQHLl5v3AwKsrky8lEKWJ/HxgXHjTKf4ihXNNPno0XD2rNWViYg418GD5nfeqFHQsSPs2GFOyhCxkEKWJwoPN2dvPfQQTJkCzZub4CUi4mnsdvjoIzN7/9VX8M47Zja/alWrKxNRyPJYAQEmYH3+OZw5Y3rDPPmk6Q8jIuIJMjNNg+ZBg6BxY/j+e7j/fi1uF5ehkOXp/pw2HzjQHMfTpo1ZFCoi4s6WLDGzV8uWmYOd16+HG2+0uiqRKyhkeYOKFeGDD2DxYkhJMUdKvPACZGdbXZmISMEcPQr9+0N8PAQHm36B48eDr6/VlYlcRSHLm/ToYWax4uPNcRKtW5vpdRERd7B4MTRqBAsWwLPPmrWmTZtaXZXINSlkeZvAQJg/3zQwTUkxW52feUZrtUTEdR05YmavevaE0FDT9+qpp6BECasrE7kuhSxv1bMn7Nplzj989llo1cpMu4uIuAq7HebMgYYN4d//hueeM53b1VhU3IRCljerXNlsfV661Pyl2KaNOYYiK8vqykTE2x04AF27woABZkH7tm1mh7Rmr8SNKGQJdOtmZrWGDzfHUDRuDCtWWF2ViHijnBzTpf3mm+GLLyAxETZtMr+XRNyMQpYYFSqYw6Y3bDA9tuLiTNuHw4etrkxEvMX27aan36hR0K6d2agzapR2DorbUsiSK7VrZ37RPfUUfPwx1K8P06aZM8FERJwhKwseecRsxNm/3yxjSEqC2rWtrkykSBSy5Gr+/mYx/PffmwWmw4dD27Zq9yAijrd0qWnL8NprcO+9sHu3WYelru3iARSy5NoaNoR162DmTPj1V4iIMH9tnj5tdWUi4u727jXrQXv0MMsVNm0y5w7ecIPVlYk4jEKWXJ/NZs4F270b7rsPJk82lxBnzzbbq0VECuLsWXj6aTN7tXYtvPKK2Tl4yy1WVybicApZkj+VKsHbb8PmzVC9OgwcSPiYMbqEKCL5Y7dfujT43HPm5Ik9e2DcOLVlEI+lkCUFExlpgtZ771H6wAFzDuLIkXDsmNWViYir+s9/oEsXc2mwdGkzgzV3runeLuLBFLKk4Hx84N57+XrWLHjwQXjrLahXD958U4dOi8glx4/DmDHmfMGvvoJJk8zu5ZgYqysTKRYKWVJoF8qVM00Dv/sOwsPhoYfMbsSVK60uTUSsdOHCpT++3njD7Br8+Wd4+GFdGhSvUqSQFRHRmPbt2xAT05ZOndoDcPz4MXr37k7r1uH07t2dEyeOO6RQcWFNm8Lq1bBkiTlounNncxzGf/5jdWUiUpzsdtPfqnlzM8vdpIlZ1P722+ZwehEvU+SZrEWLlrNu3SY+/3w9AFOmTCY6uj1btmwnOro9U6ZMLnKR4gZsNujeHXbuhFdfhY0bzS/Yv/8d0tOtrk5EnG37drjjDrP26uxZc6Dz2rU6zFm8msMvF65YsZx+/foD0K9ff5KSPnX0U4gr8/c3vbR++QVGjID334ebbjLNTXXwtIjnOXQIBg82m2C2bTNnDe7aBb16qaGoeD3b4cOnCt3sqGXLJlSoUBGbzUZCwhASEoZw0001+OWXgwDY7Xbq1auZ+/7lZs6cwaxZHwCQnp7KvHnzCltGvmVlZVG2bFmnP4+3yM94BqSkUOfdd6m6fj3nb7iB/YMHk9alC3Y/v2Kq0r3oNepYGk/H+3NM/U6epObcuYQuXozNbudQr178NmAAFzTeBaLXqOMVx5iOGfNw7hW86ylSyEpLSyU4OITMzEz69OnOP//5LwYNuvuKUFWvXk1+/vm3636duLgYtm7dWtgy8i05OZkOHTo4/Xm8RYHGc/Nm0w9n40aoW9f0ybn7bh38+v/oNepYGk/H25CUxK3btpkmoqdPQ0ICPPOMzhksJL1GHa84xjQ8vEW+QlaRLhcGB4cAEBgYSJcud7Jt27cEBgaSkWHW4GRkpFOlSpWiPIV4ijZt4IsvYPlyKFcOBg40OxI/+USd40XcwR9/wJtv0nrAAHjiCdOG4Ycf4IMPFLBErqHQIevMmTNkZZ3OfTs5eS0NGzYkNrYL8+fPAWD+/Dl07tzVMZWK+7PZzKLYbdtMI8Jz58xi+ago0/ZBYUvE9Zw/b84UrFcPHnqI32vUgC+/NLuJGze2ujoRl1bokJWZeZg774ylQ4db6Nw5hk6dYrnttk6MGjWW9evX0bp1OOvXJzNq1FhH1iuewMfHXCrctQumTYO0NNP2oW1bWLVKYUvEFWRnw/Tp5qzSv//ddGf//HO2v/66+cNIRPJU6NXHtWvXITn5y6vur1SpMgsXLitSUeIlSpSAYcPMmo4ZM+CFFyA21oStZ56B22/X7iSR4nb+PMyaBf/8J/z6K7RsaRqLxsaan8fkZKsrFHEb6vgu1vP3N38p//IL/O//wv790KmT+Wt52TLNbIkUh3PnzM/fTTfBffdBhQpmzeTXX5uZZv3BI1JgClniOvz9TZfoX34xfzlnZEC3bmaB/PzGjno0AAAOmUlEQVT5kJNjdYUinicrC157DerUMYe916hhurZv3Qp33aVwJVIEClniekqVMjNbP/0EH35oLl/cfTc0agTvvmv+4haRojl8GJ58EmrWNA2EGzUyHdo3btTMlYiDKGSJ6ypRwqzX+vFHWLDAtH64/36zXfzFF+G4zsUUKbBffzUzxrVqmXWQHTrAV1/BmjWmLYPClYjDKGSJ6/P1hd694ZtvzF/azZvDhAnmssaYMbB3r9UVirg2ux02bTI/R2Fh5rirgQPNIe6LFpk+diLicApZ4j5sNvOXdlISfP899Ox5aaFufDysX69F8iKXy86GOXMgMhLatTN/pIwbB/v2mUvv9etbXaGIR1PIEvfUtCnMnGl2Ij72GGzYYC57tGhhOlBr3ZZ4s8OHzaXAOnVgwAA4dQqmToWDB+GllyAkxOoKRbyCQpa4t9BQ84/JwYPmL/PsbBgyBKpXh/HjdSlRvIfdbtZWDRhgXv9PPAENGsCnn5rLgg88AGXKWF2liFdRyBLPEBBgevvs2HFpAe9rr5lLiV26mH9o1AJCPNHp0+YPjIgIuOUW81r/+99NsFq9Grp2NacsiEix00+eeBabDW67zexGPHAAnn7arN+66y6zK/Hpp80lRhF3ZrebjSD3328u/d1/v5nFfestSEmBKVPMLJaIWEohSzxXaOilULVwoTnMduJEqFvXHBGyYAH88YfVVYrk39Gj8OabZu1hZCTMng19+pgDm3/4wcxglS1rdZUi8l8KWeL5SpQwOxGTkkzgevppcymlb18zCzBiBGzZop2J4pqys83xUr16QXAwPPSQmbF96y1ITTWHOEdFqb+ViAtSyBLvUrOmCVn79pnQFRtr/pFq0wYaNjRNTg8csLpK8XZ/Xg4cO9YsYu/WzXRiHzkStm+HbdvMrFWFClZXKiLXoZAl3snX1xwdMmcOpKfDe+9BtWqmyWnt2tC2Lbzxhjk/UaS47N5t/ggICzOXA6dONf2tPvkEDh0ymzmaNbO6ShHJJ4UskQoV4N57TTPTvXvNbNbp0zBqlLmc2KmTCWGZmVZXKp7op5/Ma655czOb+vzz5sib9983IX/hQrNxo0QJqysVkQJSyBK5XJ06prnpDz+YMxMff9xcWhw2DIKCzM7FN980O7hECsNuh5074bnnTFPd+vXNDGqpUvD662bGavVqGDoUKla0uloRKQKFLJFruflmsxvx55/hu+9M4EpPNwuPq1c3i41ffNH05tKiebmeCxfMTOkjj5hLgY0bwzPPmBCVmGia6X71FYwebRa3i4hH8LO6ABGXZ7NBeLi5TZxodiYuXAhLlpgZiD/Xcd11F9x5J0RHm1kJ8W5Hj8Lnn8Nnn8Hy5XDsGJQsaWZDH34YevRQoBLxcApZIgXVsKE5suSJJ8wW+k8/NVvs333XLJYPCDDnKMbGmsX1YWHaXu8NcnJg61ZYscLsXP36azPDWbmyCd/dusEdd0C5clZXKiLFRCFLpCj+7LZ9//3w+++wbh2sXGluY8aYx9SqZWYvbrvNHPcTGmptzeIYdruZ1VyzxtySk+HkSROoIyPhqacgLg5atjS7WUXE6yhkiThK6dLmnLiuXc37+/aZsLVqlbm0OGOGuT8szISt6GizPb9mTetqlvy7eNFshtiwwdy++ALS0szHateG3r3h9tvNbtQqVSwtVURcg0KWiLPUqWMaRv797+Yf6O+/NzNda9ea/lzvvGMeV6OGCVtt21LWz88c8luypLW1i2nj8c035jSAL7+ETZvg+HHzsdBQc0n4tttMsKpTx9JSRcQ1OS1krV37ORMmPEpOTg4DB97DqFEPO+upRFyfj4/pg9S8uVn0nJNjdiVu3Ghu69fD3Lm0BLPD7M+z6Vq1gogIqFdPl5yc6exZ8//ju+9MsNq8GXbturRrNCzMHM0UHQ233mpmrrTOTkTy4JSQlZOTw6OPPsKCBUsJCQnljjs6EBvbhfr1dSq8CGAC0587FkeONP+YHzjAzhkzuPnMGbNo+t13zfZ+MJcimzY1j2/eHJo0gUaNdKxKQdntpg3Hjz+a2/btJljt2mWCL0ClStC6tTl4uXVrE3YrVbK2bhFxS04JWdu2baVOnbrUrm2m0OPje7FixXKFLJFrsdmgdm0yY2LMZSgwvZV27jQhYPt2c5s7F95++9LnVa9uwtbNN0ODBmbGq149syDfx4vb4GVnw/79VPrqKzN+e/aYIPXjj5cu+YFpMNuihdn516KFudWqpVkqEXEIp4Ss9PQ0QkOr574fHBzCtm1bnfFUIp7Lz8+cU3f5WXV2O+zfb8LCzp0mOOzcaYLX2bOXHhcQADfdBDfeaELD5beaNU1bAXcOYdnZZkbq0CFzoPe+fWZc9u83b+/bBxcu0PTPx1esaIJonz6mEWjjxub9qlWt+x5ExONZtvB95swZzJr1AQDp6akkJyc7/TmzsrKK5Xm8hcbT8fI9puXKQZs25gZw8SL+hw8TkJJC6ZQUAg4dIiAlhYDt2/FfuRK/ywMYcNHXl/OVK3O+UiX+qFKF85UqkV2hAtnly5NdoQIX/vv2hTJlyClThgulS3PR3985Mzx2O75nz+J35gy+WVn4nTmD3+nTlDx5khInT1LixAlKnDhByRMnKHn0KP5HjlDixAls/6/L/vkKFTgXFMS5kBDOtmrF2Ro1OFq5MoSFkV2+/NW179plblIg+rl3LI2n47nSmDolZAUFBZOScij3/bS0VIKDQ654TELCEBIShgAQFxdDhz8vkThRcnJysTyPt9B4Op5TxtRuN5fIDhwwt99+wyctjVJpaZRKTTVtCHbvNh3JL1689tfx8THhrnRp09H+8pufn/m4r6/5r4+Ped6cHHPZMyfH3M6fN/3Ezp69dDtz5vrPW7KkaYlQtaq5FNq+vdndFxJi/lu7NtSqRcmyZSkJlL/sU/UadTyNqWNpPB3PlcbUKSGrefMI9u7dy4ED+wkODmHx4oW8/fb7zngqEcmLzWYWbleqZBbNX8vFi3DiBBw5Ym5Hj8KpU+Z2+vSlt8+ehXPnrrxdHqIuXjRv22wmfPn6miDm62sCU0CACWoBAeZWpoxZwH/5rWJFCAw04apcOa2REhG35JSQ5efnx0sv/Yt+/eLJycmhf/9BNGjQ0BlPJSKO4uNzKYyFhVldjYiI23PamqyOHWPp2DHWWV9eRERExKW58fYiEREREdflEsfq7Nu3j/DwFk5/nqNHj1C5ss4UcxSNp+NpTB1L4+l4GlPH0ng6XnGM6cGDB/L1ONvhw6fseT/MM3Tq1J7PP19vdRkeQ+PpeBpTx9J4Op7G1LE0no7nSmOqy4UiIiIiTqCQJSIiIuIEvuPGPfaM1UUUp2bNrtMnSApM4+l4GlPH0ng6nsbUsTSejucqY+pVa7JEREREiosuF4qIiIg4gceHrE8+Wcytt0ZSrVoFtm/fdsXHEhMnERnZjKioFqxdu9qiCt3bK6+8SNOm9YmJaUtMTFtWr15pdUluae3az4mKakFkZDOmTHnN6nI8QkREY9q3b0NMTFs6dWpvdTluafToB2nUqC7R0a1z7zt+/Bi9e3endetwevfuzokTxy2s0L381Xjqd2jhpaQcIj6+K+3ateLWWyOZNm0q4FqvUY8PWQ0aNGLGjNlERbW94v49e3azePFCNmz4mnnzFvHoow+Tk5NjUZXubfjwEaxbt4l16zapy38h5OTk8OijjzB37kI2bvyGRYv+zZ49u60uyyMsWrScdes2ucx2bndz990DmDdv0RX3TZkymejo9mzZsp3o6PZMmTLZourcz1+NJ+h3aGH5+fnx7LMvsHHjNyQlrWH69HfZs2e3S71GPT5khYXV56ab6l11/4oVy4mP74W/vz+1atWmTp26bNu21YIKxdtt27aVOnXqUrt2HUqWLEl8fC9WrFhudVkiREW1pWLFG664b8WK5fTr1x+Afv36k5T0qRWluaW/Gk8pvGrVgmjaNByAsmXLERZWn7S0VJd6jXp8yLqWtLRUQkJCc98PCQklPT3Nworc1/Tp02jfPorRox/UpYNCSE9PIzS0eu77wcEhpKWlWliRZ7DZbPTt24OOHaOZOXOG1eV4jMzMTKpVCwKgatVqZGZmWlyR+9Pv0KL77bcD7NjxAxERLV3qNeoSx+oUVa9e3cjMzLjq/scee4q4uK4WVORZrje+gwffxyOPPIrNZuOll57n6acnkJg41YIqRa60bNlKgoNDyMzMpE+f7tSrF3bVsgEpGpvNhs1ms7oMt6bfoUWXlZXF0KGDmDjxJcqVK3/Fx6x+jXpEyFq48JMCf05wcAipqSm576emphAUFOzIsjxGfsd34MB7GDiwr5Or8TxBQcGkpBzKfT8tLZXg4BALK/IMf45hYGAgXbrcybZt3ypkOUBgYCAZGelUqxZERkY6Varo3L2iqFq1au7b+h1acNnZ2QwdOpBevfpy553dANd6jXrt5cLY2C4sXryQP/74gwMH9rN3715atGhpdVluJyMjPfftzz5bRoMGDS2sxj01bx7B3r17OXBgP+fPn2fx4oXExnaxuiy3dubMGbKyTue+nZy8loYN9dp0hNjYLsyfPweA+fPn0LmzrhYUhX6HFp7dbmfMmBGEhdXngQdG5t7vSq9Rj29Gunz5Mh5/fBxHjx6hfPkKNG7chI8/XgLA5Mn/Ys6cWfj5+fH88y9x++13WFyt+3nwwWHs3LkDsFGzZk1efTUx91q45N/q1St54on/IScnh/79BzF27DirS3Jr+/fvY/DgAQDk5FygZ88+GtNCGD58CJs2beTYsaMEBlZl/PjHiYvryrBhgzl06CDVq9fkvfc+4IYbKlldqlv4q/HctGmDfocW0ubNX9GtWywNG96Mj4+ZM5ow4SlatGjpMq9Rjw9ZIiIiIlbw2suFIiIiIs6kkCUiIiLiBApZIiIiIk6gkCUiIiLiBApZIiIiIk6gkCUiIiLiBApZIiIiIk6gkCUiIiLiBP8HQQw44QpPXOkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95df712b00>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-10,20,100)\n",
    "y = x**2 - 6*x + 5\n",
    "start = 15\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-')\n",
    "plt.plot([start],[start**2 - 6*start + 5],'o')\n",
    "ax.text(start,\n",
    "        start**2 - 6*start + 35,\n",
    "        'Start',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['blue'],\n",
    "       )\n",
    "\n",
    "d = 2 * start - 6\n",
    "end = start - d\n",
    "\n",
    "plt.plot([end],[end**2 - 6*end + 5],'o')\n",
    "plt.ylim([-10,250])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "ax.text(end,\n",
    "        start**2 - 6*start + 35,\n",
    "        'End',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['green'],\n",
    "       )\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To fix this, we multiply the gradient by a step size. This step size (often called **alpha**) has to be chosen carefully, as a value too small will result in a long computation time, while a value too large will not give you the right result (by overshooting) or even fail to converge. \n",
    "\n",
    "In this example, we'll set the step size to 0.01, which means we'll subtract $24×0.01$ from $15$, which is $14.76$. \n",
    "\n",
    "This is now our new temporary local minimum: We continue this method until we either don't see a change after we subtracted the derivative step size, or until we've completed a pre-set number of iterations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Local minimum occurs at 3.000000.\n"
     ]
    }
   ],
   "source": [
    "old_min = 0\n",
    "temp_min = 15\n",
    "step_size = 0.01\n",
    "precision = 0.0001\n",
    "\n",
    "def f(x):\n",
    "    return x**2 - 6*x + 5\n",
    "    \n",
    "def f_derivative(x):\n",
    "    import math\n",
    "    return 2*x -6\n",
    "\n",
    "mins = []\n",
    "cost = []\n",
    "\n",
    "while abs(temp_min - old_min) > precision:\n",
    "    old_min = temp_min \n",
    "    gradient = f_derivative(old_min) \n",
    "    move = gradient * step_size\n",
    "    temp_min = old_min - move\n",
    "    cost.append((3-temp_min)**2)\n",
    "    mins.append(temp_min)\n",
    "\n",
    "# rounding the result to 2 digits because of the step size\n",
    "print(\"Local minimum occurs at {:3.6f}.\".format(round(temp_min,2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An important feature of gradient descent is that **there should be a visible improvement over time**: In this example, we simply plotted the squared distance from the local minima calculated by gradient descent and the true local minimum,  ``cost``, against the iteration during which it was calculated. As we can see, the distance gets smaller over time, but barely changes in later iterations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAADFCAYAAABnw+dWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl0VGWexvFvQQECYdNANhoCStgJEAgdAlkQJYA0okiUaJrYg4w6jdsIPeLa9Ki0K9hjK9pih0FAh2UGMQgIxaYmAgJiC0qzZwFkD0tCkpo/XhOCJgRqyU2qns85dSp1q+reXz3nhvy476332g4fPuVERERERDyqjtUFiIiIiPgiNVkiIiIiXqAmS0RERMQL1GSJiIiIeIGaLBEREREvUJMlIiIi4gVqskRERES8QE2WiIiIiBeoyRIRERHxArvVBQB07tyO8PBwr2/nzJkzNG7c2OvbqcmUgaEcDOWgDEopB0M5GMrh8hns2bOHHTv2VrmOGtFkhYeHs3HjRq9vx+FwkJCQ4PXt1GTKwFAOhnJQBqWUg6EcDOVw+Qx69ux9RevQcKGIiIiIF6jJEhEREfECNVkiIiIiXqAmS0RERMQL1GSJiIiIeEGVTdZDDz1Aly7tiYvr94vn3nzzDVq1asrRo0cBcDqdPPHE40RHRxIfH8O2bVs8X7GIiIhILVBlk3XnnSnMm7fwF8uzsw/icHxG69a/Klv22WfL2b37n2RmbuGVV6YzadIjnq1WREREpJaossmKiYmlefMWv1j+1FP/wdNPT8Vms5Uty8j4hDFj7sJms9GnTzQnT57k0KE8z1YsIiIiUgu4NBlpRsZSQkJC6Nat+yXL8/JyCA1tXfY4NDSM3NwcgoKCf7GO9PRZzJ79ftn7HA6HK6VcsXonT3K2pMTr26np8vPz/T4DUA6llIMyKKUcDOVgKAfPZHDVTdbZs2eZPv1lPvxwsVsbTk1NIzU1DYChQxO9O7Psd9/BK6+wafhwokaN8t52agHN4msoB0M5KINSysFQDoZy8EwGV91k7d27h/3795GYGAtATk42gwcPZNmy1QQHh5KTc7DstTk52YSEhLpVoEdcfz3Y7TTes8fqSkRERMRPXPUUDl26dOUf/9jNpk3b2bRpO6GhYaxcuY6goCCSkoby4YdzcTqdbNyYRdOmTSscKqx29etDhw403rvX6kpERETET1TZZE2YkMawYYPZtesHIiM7MWdOeqWvHTx4CG3bhhMdHcljj01k2rRXPVqsW7p1o2F2Npw/b3UlIiIi4geqHC58++1Zl31+06btZT/bbLaa1ViV17UrlJTA999Djx5WVyMiIiI+zn9mfO/UCWw22L696teKiIiIuMl/mqzGjTkfHAzffmt1JSIiIuIH/KfJAs60awc7dkBRkdWliIiIiI/zryYrPBwKC+Gf/7S6FBEREfFxftVknQ0PNz9oyFBERES8zK+arKKAAAgL08nvIiIi4nV+1WQB0K0b/OMfZjoHERERES/xvyarRw84c0bnZYmIiIhX+V+T1b27ud+2zdo6RERExKf5X5PVogW0aQNbt1pdiYiIiPgw/2uyACIjzTcML1ywuhIRERHxUVU2WQ899ABdurQnLq5f2bJnn32S/v2jiI+P4be/HcvJkyfKnps+/RWioyOJienNqlUrvVO1uyIjzXxZO3daXYmIiIj4qCqbrDvvTGHevIWXLIuPT2Tt2kzWrPmC66+/genTzUWhd+7cwaJFC1i3Lot58xYyefKjFBcXe6dyd3TrZq5jqCFDERER8ZIqm6yYmFiaN29xybLExBux2+0AREX1JScnG4Bly5YyatTtNGjQgLZtw2nXrj2bN2/0QtluatwYOnRQkyUiIiJeY3d3BXPnzmbkyNsAyM3NISqqb9lzoaFh5OXlVvi+9PRZzJ79PgB5eTk4HA53S6lSfn5+2XaC6tWj5fr1/GP5ckrq1/f6tmuK8hn4M+VgKAdlUEo5GMrBUA6eycCtJuu1116ibl07o0cnX/V7U1PTSE1NA2Do0EQSEhLcKeWKOByOi9tp3hy+/Za4666DqCivb7umuCQDP6YcDOWgDEopB0M5GMrBMxm4/O3CefPmsHz5Mv7613ex2WwAhISElg0dAuTkZBMcHOJWgV7TpQvUqwdbtlhdiYiIiPggl5qsVatW8Je/vM7s2fNp1KhR2fIhQ4axaNECCgoK2LdvL7t376Z37z4eK9aj6teHrl1h82arKxEREREfVOVw4YQJaWzYsJ5jx44SGdmJSZOeYPr0VygsLOSOO0YC5uT3l19+nU6dOjNy5CgGDOiL3W5n2rSXqVu3rtc/hMuiouBvf4Mff4TAQKurERERER9SZZP19tuzfrEsJSW10tc/8sjjPPLI4+5VVV1Km6zNm+Hmm62uRkRERHyIf874Xqp1a3MEa9MmqysRERERH+PfTZbNZo5mbdkCRUVWVyMiIiI+xL+bLDBN1tmzusSOiIiIeJSarB49oG5dDRmKiIiIR6nJatwYOndWkyUiIiIepSYLoHdv2L0bjh+3uhIRERHxEWqyAPr+dL3FjTXwYtYiIiJSK6nJAmjbFlq1gsxMqysRERERH6EmC8xUDv36wddfQ0GB1dWIiIiID1CTVSo6GgoLdcFoERER8Qg1WaW6dTPfNNSQoYiIiHiAmqxSdruZmDQrC0pKrK5GREREarkqm6yHHnqALl3aExfXr2zZ8ePHGD16JP369WT06JGcOGGmPnA6nTzxxONER0cSHx/Dtm21bOitXz84eVKzv4uIiIjbqmyy7rwzhXnzFl6ybMaM14iLiyczcwtxcfHMmPEaAJ99tpzdu/9JZuYWXnllOpMmPeKdqr0lKsrM/p6VZXUlIiIiUstV2WTFxMTSvHmLS5YtW7aU5OSxACQnjyUj42MAMjI+YcyYu7DZbPTpE83Jkyc5dCjPC2V7SePG5tysL74Ap9PqakRERKQWs7vypiNHjhAUFAxAq1ZBHDlyBIC8vBxCQ1uXvS40NIzc3Jyy15aXnj6L2bPfL3ufw+FwpZSrkp+fX+V2rg0IIHTVKn6YP5+C4F/WXdtdSQb+QDkYykEZlFIOhnIwlINnMnCpySrPZrNhs9mu+n2pqWmkpqYBMHRoIgkJCe6WUiWHw1H1dnr2hM8/J7C4GKqhpup2RRn4AeVgKAdlUEo5GMrBUA6eycClbxe2bNmybBjw0KE8AgMDAQgODiUn52DZ63JysgkJCXWrwGrXvLkZMtywwepKREREpBZzqckaMmQY8+d/AMD8+R+QlDQcgKSkoXz44VycTicbN2bRtGnTCocKa7zYWDhwAPbvt7oSERERqaWqbLImTEhj2LDB7Nr1A5GRnZgzJ52JEx9hzZrV9OvXkzVrHEycaL5FOHjwENq2DSc6OpLHHpvItGmvev0DeEX//uZSO+vXW12JiIiI1FJVnpP19tuzKly+YMGSXyyz2Wy1t7Eqr0UL6NrVNFljx1pdjYiIiNRCmvG9MqVDhgcOWF2JiIiI1EJqsipTOmS4dq3VlYiIiEgtpCarMtdeCz16wJo1mphURERErpqarMtJTITcXPj+e6srERERkVpGTdblxMRA/fqwerXVlYiIiEgtoybrcho1gn79zHlZRUVWVyMiIiK1iJqsqiQmwunT8PXXVlciIiIitYiarKr06gVNm2rIUERERK6Kmqyq2O0QFweZmXDmjNXViIiISC2hJutKDBoEhYWaM0tERESumJqsK3HDDdCuHSxfbnUlIiIiUku41WS99dZfGDgwmri4fkyYkMb58+fZt28vSUmJREdHMn78OAoLCz1Vq3VsNrj5Zti1C3bvtroaERERqQVcbrJyc3N49923Wb58DWvXZlJcXMLixQuYOvUZJkx4kKysrTRr1pw5c9I9Wa91EhKgXj0dzRIREZEr4taRrKKiIs6fP0dRURHnzp2lVasg1q9fw4gRtwKQnHwXGRkfe6RQywUEmOsZOhzm/CwRERGRy7C7+saQkFAeeOD39OrVlYYNryE+fhCRkb1o2rQZdrtZbWhoGHl5uRW+Pz19FrNnvw9AXl4ODofD1VKuWH5+vlvbadyiBe327+fgm29yomdPzxVWjdzNwFcoB0M5KINSysFQDoZy8EwGLjdZJ04cZ9myT9i48RuaNWvG736XyqpVK674/ampaaSmpgEwdGgiCQkJrpZyxRwOh3vbiY+HzEwC8/LM8GEt5HYGPkI5GMpBGZRSDoZyMJSDZzJwebhw7VoHbdq0JTAwkHr16jF8+AiysjI5deokRT9dgiYnJ5vg4BC3CqxRbDYYOhS+/Rb27rW6GhEREanBXG6ywsJas2nTV5w9exan08m6dWvo2LEjsbFxLFmyGID58+eSlDTcY8XWCIMHm4tGf+wj55qJiIiIV7jcZEVF9eWWW0YyePBA4uN/TUlJCffck8ZTTz3HW2/9hejoSI4fP0ZKSqon67VekyZmqHD1asjPt7oaERERqaFcPicLYPLkKUyePOWSZeHh7fj0U4c7q635hg83UzmsWAGjRlldjYiIiNRAmvHdFe3bQ5cu8MknUFJidTUiIiJSA6nJctUtt0BeHmzcaHUlIiIiUgOpyXJVTAwEBsKiRVZXIiIiIjWQmixX2e1w662wfTvs3Gl1NSIiIlLDqMlyx803Q+PGsHCh1ZWIiIhIDaMmyx0NG8KwYfDFF5Bb8eWDRERExD+pyXLXiBFQty4sXmx1JSIiIlKDqMlyV4sWMGgQrFwJx45ZXY2IiIjUEGqyPGH0aCgq0rlZIiIiUkZNlieEhEBiImRkwIkTVlcjIiIiNYCaLE8ZMwYuXNDRLBEREQHcbLJOnjzBvffeQ//+UcTG9uGrrzI5fvwYo0ePpF+/nowePZITJ457qtaaLTTUXDh66VIdzRIRERH3mqwpUyYzaNBgPv98E6tXf05EREdmzHiNuLh4MjO3EBcXz4wZr3mq1povOdkczVqwwOpKRERExGIuN1mnTp3kyy8/JyUlFYD69evTrFlzli1bSnLyWACSk8eSkfGxZyqtDcLCzDcNly6FI0esrkZEREQsZDt8+JTTlTd+8802/v3fJxIR0Ylvv91OZGRP/vSnaURGdmLXrgMAOJ1OOnRoU/a4vPT0Wcye/T4AeXk5zJs3z/VPcYXy8/MJCAjw6jbqnThBxKuvcrJHDw6OHu3VbbmiOjKoDZSDoRyUQSnlYCgHQzlcPoOHH36UFSvWVLkOu6sbLy4uYtu2rTz//EtERfVlypRJvPHGq5e8xmazYbPZKnx/amoaqalpAAwdmkhCQoKrpVwxh8NRLdvh2DFaLl7MDeHhEB7u/e1dhWrLoIZTDoZyUAallIOhHAzl4JkMXB4uDAkJIzQ0jKiovgCMGHEr27ZtpWXLlhw6lAfAoUN5BAYGulVgrXTHHdCoEaSnW12JiIiIWMTlJisoKIjQ0DB27foBgLVrHUREdGLIkGHMn/8BAPPnf0BS0nDPVFqbNGliJij96ivYts3qakRERMQCLg8XAjz//Evcf/+/UFhYSNu24cyY8SYlJSWMHz+OOXPSad26De+++76HSq1lfvMbMznpO+/A66+b6xuKiIiI33CryerevUeFJ34tWLDEndX6hvr14Xe/gxdegGXLYLgfHtETERHxY5rx3ZtiYiAyEv77v+H0aaurERERkWqkJsubbDa47z44exZmz7a6GhEREalGarK8rU0bGDHCDBnu3Gl1NSIiIlJN1GRVh5QUuO46eOMNKCqyuhoRERGpBmqyqkPDhnD//bBvHyxaZHU1IiIiUg3UZFWX6GiIjYW5cyEnx+pqRERExMvUZFWn++4zUzu8/jqUlFhdjYiIiHiRmqzqdO21MGECfPedhg1FRER8nJqs6paQAP37m7mz9u61uhoRERHxEjVZ1c1mgwcegMaN4dVX4cIFqysSERERL1CTZYVmzWDiRNizB2bNsroaERER8QI1WVaJjjYXkV6yBL74wupqRERExMPcbrKKi4sZNGgAKSl3ALBv316SkhKJjo5k/PhxFBYWul2kz0pLgw4dYPp0OHzY6mpERETEg9xusmbO/CsRERFlj6dOfYYJEx4kK2srzZo1Z86cdHc34bvsdpg0CZxOePFFUEMqIiLiM9xqsnJyslm58lNSUn4LgNPpZP36NYwYcSsAycl3kZHxsftV+rLgYHj0UfjhB/iv/zINl4iIiNR6dnfe/OSTf+Dpp/9Ifn4+AMeOHaNp02bY7Wa1oaFh5OXlVvje9PRZzJ79PgB5eTk4HA53Srki+fn51bIdV7Tq2ZNWH31E7tmzHI2N9dp2anIG1Uk5GMpBGZRSDoZyMJSDZzJwuclavjyDwMBAIiN7sWHDuqt+f2pqGqmpaQAMHZpIQkKCq6VcMYfDUS3bcUl8PFxzDYFZWTB8OPTs6ZXN1OgMqpFyMJSDMiilHAzlYCgHz2Tg8nBhVlYmn36aQVRUN+67L43169cyZcokTp06SVFREWCGE4ODQ9wq0G/YbGbYsE0beP55TVQqIiJSy7ncZD355LNs3bqDTZu2M3PmLAYMiOOtt/5GbGwcS5YsBmD+/LkkJQ33WLE+r2FDeOYZaNQInn0WfvzR6opERETERR6fJ+upp57jrbf+QnR0JMePHyMlJdXTm/BtgYGm0Tp7Fp57Dn46301ERERqF7dOfC8VGzuQ2NiBAISHt+PTTx2eWK3/atcOnnjCNFnPPgtTp5qjXCIiIlJraMb3mqpnT5g82UztMHWq5tASERGpZdRk1WS//jU88ghs3w5/+hMUFFhdkYiIiFwhNVk1XUKCuZj0li0Xz9USERGRGk9NVm0weDA8/jjs2AFPPQWnT1tdkYiIiFRBTVZtMXCgORl+925zf+KE1RWJiIjIZajJqk2io82QYW4uPPYY7N9vdUUiIiJSCTVZtU3PnvDCC3DhghlC3LTJ6opERESkAmqyaqMOHeDVVyEoyMyl9fHHVlckIiIiP6Mmq7YKDIQ//9kMIb79NsyYobm0REREahA1WbXZNdeYk+DHjIEVK8x5WgcPWl2ViIiIoCar9qtTB+65xwwbHjtmJi9dvdrqqkRERPyey01WdvZBRo0azoABfRk4MJqZM98E4PjxY4wePZJ+/XoyevRITpw47rFi5TJ69zZDhjfcYM7XevllzaclIiJiIZebLLvdznPP/Sfr139FRsZnvPfeO+zcuYMZM14jLi6ezMwtxMXFM2PGa56sVy7nuuvM5XdSUmD9erj/fvj8c6urEhER8UsuN1lBQcH06NETgICAJkREdCQ3N4dly5aSnDwWgOTksWRk6Jtv1apuXbjzTnjtNdN0vfACTJumyUtFRESqme3w4VNOd1eyf/8+Ro4cytq1X9KrV1d27ToAgNPppEOHNmWPy0tPn8Xs2e8DkJeXw7x589wto0r5+fkEBAR4fTs1ha24mMA1a2i1ejVOu51DN97I/u7dadysmdWlWc7f9oXKKAdlUEo5GMrBUA6Xz+Dhhx9lxYo1Va7D7oki7r33HqZOfZEmTZpe8pzNZsNms1X4vtTUNFJT0wAYOjSRhIQEd0upksPhqJbt1Cg33gjZ2fDOO7TKzOTar74i7LnnzKSmfswv94UKKAdlUEo5GMrBUA6eycCtbxdeuHCBe++9m9tvH8Mtt/wGgJYtW3LoUB4Ahw7lERgY6FaB4gFhYeZyPE8/ja242Fxk+pln4IcfrK5MRETEZ7ncZDmdTh5++EEiIjpy//3/VrZ8yJBhzJ//AQDz539AUtJw96sU99ls0LcvPzz8MNx7r2mwHn0Unn9e10AUERHxApeHCzMzv+Sjj+bRuXNXEhNjAZgy5WkmTnyE8ePHMWdOOq1bt+Hdd9/3VK3iAU67HUaNgiFD4H//FxYtgi+/hJgYuO026NjR6hJFRER8gstN1q9/HcPhw6cqfG7BgiUuFyTVpFEjuOsuuOUWWLwYli410z1062aarT59zNEvERERcYnbJ75LLdekiZkxfvRoWL7cNFx//KO5+HRSEtx0E+jbiCIiIldNTZYYDRvCyJEwfDh88QV88gn8/e8wZw707w+DBkFkJNi1y4iIiFwJ/cWUS9ntMHCguR04AMuWwapVsHatOaI1YAAkJJhztzScKCIiUik1WVK5X/0Kxo+HceNg82ZwOGDFCnP+VlCQOcIVHQ2dO5uZ5kVERKSMmiypWr160K+fuZ09a4YT166FJUvMtxMDAiAqyjzfq5d5LCIi4ufUZMnVadTIzCJ/441w7hx8/TVkZcFXX8GaNWYIsX176NHDnMPVpYs530tERMTPqMkS1zVsaIYM+/eHkhLYuRO2bjW30qNcderADTdAp07mPK5OnaBlS53PJSIiPk9NlnhGnTrm3KzOneHOO6GgAL77Dr75BrZvNyfQ/9//mdc2b24arnbtLt6Cg9V4iYiIT1GTJd7RoIG5CHXphaiLimDfPnO0a+dO+P57M8zodJrnr7kG2rY1DVebNtC6tbnmYmCgaeBERERqGTVZUj3sdrj+enMbNswsKygw103cuxf27DG3devgzJmL76tXD0JCTMMVFgatWplby5bm/pprLPk4IiIiVVGTJdZp0AA6dDC3Uk4nnDgB2dmQk2Pus7Ph4EFzcn1R0aXrCAi42HS1bAktWphb8+bmVvqzJlEVEZFqpr88UrPYbBcbpW7dLn2upASOH4fDh+HIkYv3R45AXp4596v8UbDyAgIuNlxNmhCWm2uOoAUEXHpr0uTiz40a6TwxERFxmdearFWrVjBlymSKi4u5++7fMnHio97alPiLOnXguuvMrXPnil9TWGiOhB0/ful9+Z8PHKDJrl3m6NiFC5Vvz2YzR9saNjS3a6659P7nPzdoYIY3S+/r1794+/nj8st0zpmIiE/ySpNVXFzM5MmP8dFH/0toaBg335zAkCHD6Nixkzc2J3JR/foXz9u6jB0OB8EJCaYpO30a8vPN/Zkzl/58/ryZD+zcuYs/nzoFhw5dXHb27MUT+F1Rt6652e2eubfZTONW/la6rPzzNhutdu40RwQv956KloH5ufRW/nFlP1f3666EzUbD7GzYtesXyyt7/dWs2+11XO163Fh3gyNHzH88XFmPD6l/5Ig5RaG61NCs6//4ozllozZq3NhcBq4G8EqTtXnzRtq1a094eDsARo26nWXLlqrJkpqnfv2LR8dc5XSaI2IFBaZpK3+7cOHifUHBxcc/f76oCIqLr/y+sLDi5SUl5lZcbOoqfVx6K7/M6aTV0aNmQlk/dv3RozBvntVlWK7D0aOQnm51GZaLUA7ATzn8/e9Wl+GaYcPg/vutrgLwUpOVl5dLWFjrsschIaFs3rzxktekp89i9uz3f3p9Dg6HwxulXCI/P79atlOTKQPDshxKhyAbNKj+bf+c00n+6dM0adwYSkqwOZ1lTZit/L3T+YvHZTfA9tO6SteJ04mt3DbKP1fRa11aR/nlFa3jCj8/wLlz52hY/qoEV3lUssLjEJWt42qPeFby+qs69nGFtZw7d44DlV2dwZ0jtbXMZXPwsJp5DMs4d+4cB2vp1ToKAgI454F/3z3xd8KyE99TU9NITU0DYOjQRBISEry+TYfDUS3bqcmUgaEcDIfDQbyf5+BwOIjy8wzA5NBfOSiHnzgcDmL8PAdP/J3wyhm3wcEhZGcfLHucm5tDSEioNzYlIiIiUiN5pcnq1SuK3bt3s2/fXgoLC1m0aAFDhgzzxqZEREREaiSvDBfa7XZefPElkpNHUVxczNix99CpUyVfuRcRERHxQV47J2vw4CEMHjzEW6sXERERqdE0C6KIiIiIF6jJEhEREfECNVkiIiIiXqAmS0RERMQLLJuMtLw9e/bQs2dvr2/n6NEfue66QK9vpyZTBoZyMJSDMiilHAzlYCiHy2dw4MC+K1qH7fDhU35zvYSbbopnxYo1VpdhKWVgKAdDOSiDUsrBUA6GcvBMBhouFBEREfECNVkiIiIiXlD38cf/41mri6hOkZG9rC7BcsrAUA6GclAGpZSDoRwM5eB+Bn51TpaIiIhIddFwoYiIiIgXqMkSERER8QK/aLJWrVpBTExvoqMjmTHjVavLqVZRUd2Ij/81iYmx3HRTPADHjx9j9OiR9OvXk9GjR3LixHGLq/S8hx56gC5d2hMX169sWWWf2+l08sQTjxMdHUl8fAzbtm2xqmyPqiiDP//5eXr06EhiYiyJibGsXPlp2XPTp79CdHQkMTG9WbVqpRUle0V29kFGjRrOgAF9GTgwmpkz3wT8a3+oLAN/2x/Onz/PkCEJJCT0Z+DAaKZN+08A9u3bS1JSItHRkYwfP47CwkIACgoKGD9+HNHRkSQlJbJ//5XNjVTTVZbD73//r/Tp071sf/jmm22Ab/5OlCouLmbQoAGkpNwBeH5f8Pkmq7i4mMmTH2Pu3AWsX/8VCxf+Dzt37rC6rGq1cOFSVq/eUDbfx4wZrxEXF09m5hbi4uKZMeM1iyv0vDvvTGHevIWXLKvsc3/22XJ27/4nmZlbeOWV6Uya9IgVJXtcRRkATJjwIKtXb2D16g0MHjwEgJ07d7Bo0QLWrcti3ryFTJ78KMXFxdVdslfY7Xaee+4/Wb/+KzIyPuO9995h584dfrU/VJYB+Nf+0KBBAxYs+BiH43NWrdrA6tUr2bgxi6lTn2HChAfJytpKs2bNmTMnHYA5c9Jp1qw5WVlbmTDhQaZOfcbiT+AZleUA8MwzU8v2h+7dewC++TtRaubMvxIREVH22NP7gs83WZs3b6Rdu/aEh7ejfv36jBp1O8uWLbW6LEstW7aU5OSxACQnjyUj42OLK/K8mJhYmjdvccmyyj53RsYnjBlzFzabjT59ojl58iSHDuVVe82eVlEGlVm2bCmjRt1OgwYNaNs2nHbt2rN580YvV1g9goKC6dGjJwABAU2IiOhIbm6OX+0PlWVQGV/dH2w2GwEBAQBcuHCBCxeKsNlsrF+/hhEjbgUgOfmusn3B7CN3ATBixK2sW+fA6az93xWrLIfK+OLvBEBOTjYrV35KSspvAXPEztP7gs83WXl5uYSFtS57HBISetl/XHyNzWZjzJhbGTw4jvT0WQAcOXKEoKBgAFojJc3JAAAEc0lEQVS1CuLIkSNWllhtKvvceXk5hIZe3EdCQ8N8eh95772ZxMfH8NBDD5QNkeXm5hAaGlb2mtDQMPLycq0q0Wv279/HN99sIyqqj9/uD+UzAP/bH4qLi0lMjKVLl+uJj08kPLw9TZs2w243V5kr/1nL//2w2+00adKUY8eOWVa7J/08h6iovgA8//wfiY+P4amn/kBBQQHgu78TTz75B55++o/UqWNaoWPHjnl8X/D5JsvfLVnyKZ99to65cxfw3nvv8MUXGy553mazXfZ/ML7KXz/3uHH/QlbWVlav3kBQUDDPPDPF6pKqTX5+Pvfeew9Tp75IkyZNL3nOX/aHn2fgj/tD3bp1Wb16A1u3fsfXX2/ihx++t7okS/w8h++++wdPPvksn3++ieXLHRw/fpw33vC9U0lKLV+eQWBgoNfnAvP5Jis4OITs7INlj3NzcwgJCbWwoupV+llbtmzJsGG3sHnzJlq2bFl2qPfQoTwCA/3jIqCVfe7g4FByci7uIzk52T67j7Rq1Yq6detSp04d7r77t3z99SbA7Cc5Odllr8vJySY4OMSqMj3uwoUL3Hvv3dx++xhuueU3gP/tDxVl4K/7A0CzZs2JjR3Ixo1ZnDp1kqKiIuDSz1r+70dRURGnT5/i2muvtaxmbyjNYdWqlQQFBWOz2WjQoAF33XV32f7gi78TWVmZfPppBlFR3bjvvjTWr1/LlCmTPL4v+HyT1atXFLt372bfvr0UFhayaNEChgwZZnVZ1eLMmTPk558u+9nhWEXnzp0ZMmQY8+d/AMD8+R+QlDTcyjKrTWWfOylpKB9+OBen08nGjVk0bdq0bBjJ15Q/j+KTT5bQqVNnwGSzaNECCgoK2LdvL7t376Z37z5WlelRTqeThx9+kIiIjtx//7+VLfen/aGyDPxtf/jxxx85efIEAOfOnWPNmtVEREQQGxvHkiWLAZg/f27ZvmD2kbkALFmymAED4n3iiGdFOXTo0KFsf3A6nWRkfEynTl0A3/ydePLJZ9m6dQebNm1n5sxZDBgQx1tv/c3j+4Ldex+hZrDb7bz44kskJ4+iuLiYsWPvKfuHxNcdOXKYceNSACguLuK22+5g0KCb6NmzN+PHj2POnHRat27Du+++b22hXjBhQhobNqzn2LGjREZ2YtKkJ5g48ZEKP/fgwUNYuXI50dGRNGrUiOnT37S2eA+pKIMNG9bx7bffADbatGnDyy9PB6BTp86MHDmKAQP6YrfbmTbtZerWrWvtB/CQzMwv+eijeXTu3JXExFgApkx52q/2h8oyWLjwf/xqfzh0KI/f//5fKS4uxuks4Te/GcXNNw8lIqITEyak8cILU+nePZKUlFQAUlJSefDB+4iOjqRFixa8/fYsiz+BZ1SWw2233cLRoz/idDrp2rU7L730OuCbvxOVeeqp5zy6L+iyOiIiIiJe4PPDhSIiIiJWUJMlIiIi4gVqskRERES8QE2WiIiIiBeoyRIRERHxAjVZIiIiIl6gJktERETEC/4f0siW9GyofIcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95df6a4278>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-10,20,100)\n",
    "y = x**2 - 6*x + 5\n",
    "\n",
    "x, y = (zip(*enumerate(cost)))\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-', alpha=0.7)\n",
    "plt.ylim([-10,150])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAADGCAYAAADhV5X4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd4VNXWx/HvmclkMumkkEoJSCC0hBZAEIiCNBUpghcQKXIRQQERvPYCcpVrC94XFRUQRAEFVESKCEHRC4iIICoWehrpPZl23j+OBJBOZphksj7PMw/JZCazsh3Dj733WVs5ebJQRQghhBBCOJTO1QUIIYQQQrgjCVlCCCGEEE4gIUsIIYQQwgkkZAkhhBBCOIGELCGEEEIIJ5CQJYQQQgjhBJcMWampJxg4sD9du3bghhsSWbBgPgBz586hdeumJCV1ISmpC5s3b6x8TnLySyQmxtO5c1u2bNnsvOqFEEIIIaop5VJ9sjIzM8jMzKB16wSKi4vo2bMb7777AZ98shofH18mTXrgrMcfPPgrEyaMZePGrWRkpDNkyG3s2PEDer3eqT+IEEIIIUR1csmZrLCwcFq3TgDA19eP2NimpKenXfDxGzasY+DAwRiNRho0aEhMTCP27NntuIqFEEIIIWqAK9qTdezYUfbv30e7du0BWLhwAd27d2bKlPvIz88DID09jcjIqMrnREZGkZGR7sCShRBCCCGqP4/LfWBxcTFjx97FrFnP4+fnz+jR9zB9+sMoisLzz8/mqaceIzl5/mW/8JIli1i6dDEAv/32K/Xq1bvi4q+U3W5Hp5O9/o4i4+l4MqaOJePpeDKmjlXrx1NV8T5xAn15OaX16mHz8qryt7wWY5qdnc2vvx655OMuK2RZLBbGjh3J4MFDueWW2wCoW7du5ddHjrybkSOHAhAREUlaWmrl19LSUgkPjzjne44aNYZRo8YA0LdvErt3O39JMSUlhR49ejj9dWoLGU/HkzF1LBlPx5MxdaxaPZ6qCqNGwe+/w4oVMHSoQ77ttRjThIS2l/W4S0Y9VVWZOnUSsbFNmThxcuX9mZkZlR9//vlamjWLA6B3736sWbOKiooKjh49wqFDh2jbtv2V1i+EEEIId/bcc/Dee/Dssw4LWNXNJWeydu7cwYcfLicurgVJSV0AeOyxJ1m9+iMOHNgPKNSvX58XX0wGoFmzOAYMGEjXrh3w8PDghRdelCsLhRBCCHHa++/DE0/AyJHw+OOursZpLhmyOnXqzMmThefc37Nn7ws+Z9q0GUybNqNqlQkhhBDC/Xz9NYwZA927w9tvg6K4uiKnqcW77YQQQghxTf32G9x+O8TEwOrVYDS6uiKnkpAlhBBCCOfLzob+/UGng3XrICjI1RU53WW3cBBCCCGEuCrl5TBgABw/Dlu3QuPGrq7ompCQJYQQQgjnsdth9Gj49ltYuRI6d3Z1RdeMLBcKIYQQwnkefljrgzV3Ltxxh6uruaYkZAkhhBDCOf77X3jxRZg0CR56yNXVXHMSsoQQQgjheB9/DA88oO3FSk5261YNFyIhSwghhBCOtWMH/OMfkJioNR6tpU3JJWQJIYQQwnH++ANuvRWiomDtWvD2dnVFLiMhSwghhBCOkZkJvf86EWb9eggNdW09LiYtHIQQQghRdcXFWrPRjAzYsgWaNHF1RS4nIUsIIYQQVWM2w+DBsHcvfPIJdOzo6oqqBQlZQgghhLh6qgr33AObNsHChdpslgBkT5YQQgghquKRR2DpUpg9G8aMcXU11YqELCGEEEJcnXnz4IUX4L774NFHXV1NtSMhSwghhBBX7v33YcoUGDRIC1u1sNnopUjIEkIIIcSV2bAB7r4bevSAZctqbbPRS5GQJYQQQojLt2OHdiVhq1balYReXq6uqNqSkCWEEEKIy/Pzz9rVgxERWrNRf39XV1StScgSQgghxKUdOwY33wyenlq7hrAwV1dU7UmfLCGEEEJcXFaWFrCKi+Grr6BRI1dXVCNIyBJCCCHEhRUUaOcRHjumzWC1bu3qimoMCVlCCCGEOL/SUrj1Vti/Hz79FLp2dXVFNcol92Slpp5g4MD+dO3agRtuSGTBgvkA5OXlMmTIADp2TGDIkAHk5+cBoKoqjz46g8TEeLp378y+fXud+xMIIYQQwvHMZhgyBLZvh/feg759XV1RjXPJkOXh4cEzzzzH9u3fsX79lyxc+BYHD/7KvHmv0K1bd3bu3Eu3bt2ZN+8VAL78chOHDv3Jzp17eemlZGbOnOb0H0IIIYQQDmSzwahR2hWEb74Jw4a5uqIa6ZIhKywsnNatEwDw9fUjNrYp6elpbNiwjmHDhgMwbNhw1q//DID16z9n6NB/oCgK7dsnUlBQQGZmhhN/BCGEEEI4jKrCpEmwYgXMnQvjx7u6ohrrivZkHTt2lP3799GuXXuysrIICwsHoG7dMLKysgDIyEgjMjK68jmRkVGkp6dVPvaUJUsWsXTp4srnpKSkVOHHuDzFxcXX5HVqCxlPx5MxdSwZT8eTMXWsajeeqkrj11+n3ocfcnT4cA536ADVqb7LUJ3G9LJDVnFxMWPH3sWsWc/j53d28zFFUVCu8MyiUaPGMGqUdlp3375J9OjR44qefzVSUlKuyevUFjKejidj6lgyno4nY+pY1W48n34aPvwQJk+mwbx5NKiB5xFWpzG9rGakFouFsWNHMnjwUG655TYAQkNDK5cBMzMzCAkJASA8PJK0tBOVz01LSyUiItLRdQshhBDCkf7zH3jmGRgzBpKT5cBnB7hkyFJVlalTJxEb25SJEydX3t+7dz9WrHgfgBUr3qdPn/4A9OnTl5UrP0BVVXbv3oW/v/85S4VCCCGEqEbmz4eZM7UN7m+9BTo5EMYRLrlcuHPnDj78cDlxcS1ISuoCwGOPPckDD0xj/PjRLFu2hOjo+rz99mIAevbszebNm0hMjMfb25vk5PlO/QGEEEIIUQXvvqttdL/1Vli6FPR6V1fkNi4Zsjp16szJk4Xn/dqqVWvPuU9RFF544eWqVyaEEEII51qxAsaOhZ49YeVKMBhcXZFbkflAIYQQojZavRpGjIAuXeDjj8HLy9UVuR0JWUIIIURts3attv8qMRHWrQMfH1dX5JYkZAkhhBC1yfr12nE5bdpoH/v5uboityUhSwghhKgtNm+GgQOhRQvYuBECAlxdkVuTkCWEEELUBlu3wm23QWwsbNoEdeq4uiK3JyFLCCGEcHcpKdC/P8TEaLNZfzUQF84lIUsIIYRwZ9u2nQ5YW7ZA3bqurqjWkJAlhBBCuKtt26BfP2jYUAtYYWGurqhWkZAlhBBCuKNTAatBAwlYLiIhSwghhHA3ErCqBQlZQgghhDvZsgX69j0dsMLDXV1RrSUhSwghhHAXmzZpm9wbN9auKJSA5VISsoQQQggHm7Msn9bjUmkzPpV2E1LZ+UsFyasKKC23X/H3endjEWnZ1ks/8PPP4dZboWlTrSeWXEXoch6uLkAIIYRwJ//7uZx1O8r47vVIjJ4K2QU2zBaV4asLGdHTF+8rOIfZZlN5d2MxLRp6Ehlykb+yP/1UOyqnVSv44gsICqr6DyKqrPbMZKmqqysQQghRC2Tk2AgJ0GH0VAAICdCz6utS0nJs9JyewU3TMwCY9GoO970TQetxqTz9bl7l8xuPOM6/3sqlw71pLN9awve/mRn17yzaTUilrOI8M2GrV8PgwZCQoDUalYBVbdSOkJWaCl264PPnn66uRAghhJvr1d7EiSwrcXefYHJyDtt+LOf+gf5EBuvZ/FI4X76k7ZOaNTaQ+ePS+WFBJF/9WM6+Q+bK7xHsr+e7NyIZ0dOXdrGeLHkklO/fjMJk/Ntf28uWwdCh0KGDNoMlR+VUK7UjZJWUwPHjJEybBrt2uboaIYQQbszXpGPX/EhenxZMSKCO4bNP8u7GonMe9+G2Eu59O4L296bx81ELvxw9HbKG9vC59Au99RbcdRd066ZteJfDnqud2hGyYmPh66+x+vnBTTdp/UOEEEIIJ9HrFXokmHj67jrMuz+Y1V+XnvX1w+kWXv6wkP+MyOSHt6Lo19FEufn0thYfL+XiL5CcDP/8p9aqYd068PV1xo8hqqh2hCyAhg35ITkZ6tWDPn1gwwZXVySEEMINHTxu4fcTlsrP9/5hpkGYB34mHUWl2p6qwlIVHy8FHy87mXk2Nuwqu+D38/PWUVR2xl6sOXNg6lQYNAjWrAGTyWk/i6iaWnV1oTkkRJvFuvlmuO02WLECBg50dVlCCCHcSHGZnSn/zaWg2I6HHhpHGXhjWjDLt5TQ/5FMIoI9+PKlcBKu82TM61HENsji+pYXvuTw7pt9mfRqDiZPhe3Bb2J6fhaMGAGLF4NHrfprvMapff91QkO1Drj9+sEdd2hv0pEjXV2VEEIIN9Eu1sj2eRHn3D95oD+TB/pXfr5wZigpKQfo0aPHWY/7c1m9sz4f1M2HQV1NcP/98Px8GD8eXn8d9Hqn1C8cp/YsF56pTh1tk2C3btqmwf/+19UVCSGEEOdnscCoUTB/PsyYAW++KQGrhrhkyJoy5T6aN29Et24dK++bO3cOrVs3JSmpC0lJXdi8eWPl15KTXyIxMZ7OnduyZctm51TtCH5+WnfcAQO0fx3MmiW9tIQQQlQv5eVaD6xly7S9WC+8AMolNsWLauOSy4V33jmCceP+yeTJE866f8KESUya9MBZ9x08+Ctr1qzi6693kZGRzpAht7Fjxw/oq2vi9vKCjz6CcePgySchNxdeegl0tXOCTwghRDVSVKTtH962TZvFmjjR1RWJK3TJNNG5cxcCAy+vudmGDesYOHAwRqORBg0aEhPTiD17dle5SKfy8IBFi2DKFHj1VS1wWS/jjCghhBDCWbKy4MYb4euv4b33JGDVUFe98X3hwgWsXPkBCQlteOaZ5wgMrEN6ehrt2nWofExkZBQZGennff6SJYtYunQxABkZaaSkpFxtKZetuLj4wq8zYAANCguJWbSI7N9/5+cnnsBuNDq9pprsouMproqMqWPJeDqejKljnW88vTIyaD1zJsaTJ/l51ixyIiNBxvyyVaf36FWFrNGj72H69IdRFIXnn5/NU089RnLy/Cv6HqNGjWHUqDEA9O2bdM7VFc6QkpJy8ddJSoJ27Qi5/366zZ6tHbgpRxRc0CXHU1wxGVPHkvF0PBlTxzpnPH/6SWvPUFoKX35Jqy5dXFZbTVWd3qNXtfmobt266PV6dDodI0fezQ8/fA9AREQkaWmplY9LS0slPPzcy1irtUmTYPly2LlTu/owLc3VFQkhhKgNvvkGbrhB+/jrr0ECVo13VSErMzOj8uPPP19Ls2ZxAPTu3Y81a1ZRUVHB0aNHOHToEG3btndMpdfS0KGwfj0cOQLXXw+//ebqioQQQrizdeugVy+oW1cLWy1buroi4QCXXC6cMGEM33yzndzcHOLjmzFz5qN8883XHDiwH1CoX78+L76YDECzZnEMGDCQrl074OHhwQsvvFh9ryy8lJtu0tbA+/bV/jWxbh0kJrq6KiGEEO7mnXdgwgRISND+gR8a6uqKhINcMmS9+eaic+4bMWLUBR8/bdoMpk2bUbWqqot27bR/UfTure3XWrkS+vd3dVVCCCFqmPT8Mn48kU9uiZkgH0/iowOJCPCiwbvvaieP9O4NH36o9XAUbkMaQl1Kkybw7bcQF6c1Ln37bVdXJIQQogZJzy/ji58zKTPbCPE1Uma28cVP6aTfO4WYxYvh7rth7VoJWG5IQtblCA/Xlg579dLOjHrqKekOL4QQ4rL8eCIfPy8P/LwM6BQFP8WG3+uv8ePW7zg6cqTWq9FgcHWZwglq3wHRV8vXV2vpMGECPPssHD+unR8l/2MIIYT4mzOXBw+kFdAqKgA/DFBQALNn4fP772Tfcx+HE+vTQI7JcVsSsq6EwaBtUKxfH555RmvvsHIl+Ptf+rlCCCFqhVPLg35eHoT4GvHU6/juSB4dTWaCnp8N+XmUzHiEoM6JkH3Q1eUKJ5LlwiulKPD009rerM2btZ4mJ064uiohhBDVxN+XB+MiAlAzMvhl/hLs5WUUPT2LohbxxEcHurpU4WQSsq7WuHHw+edw+DB07Ag//ODqioQQQlQDuSVmfIynF4qCvv8fiQtfpcLPn+xnn8cU15RezcOICDS5sEpxLchyYVXcfLPW4qF/f21Ga+VK6NfP1VUJIYS4xs7cg3U0p4Ryi436Qd7w4UewdCnG+LZ0e3AMfTo1cXWp4hqSmayqatUKduyApk3h1lth/pWd4SiEEKJm+3uLhnB/E98fzuHYa29hX7qUoqSbKJo6nfhm0a4uVVxjMpPlCJGRsG0bDB+unX148CC89BJ4yPAKIYS7O3MPFkA9gxXWvkdGZj5ed44i6M5BdIoOlOXBWkhSgKP4+sKaNTBzJrz8snbe4fLlEBDg6sqEEEI4waklwg0/pRPub6JRqA9B+Vnw7LNEZedgnDSV4ZOHuLpM4UKyXOhIer02g7VggXbl4fXXaxvjhRBCuJUzlwgjArwoqrCwd+cBch9/BsrKKHlmNkE9uri6TOFiErKcYfx42LQJ0tO1Q6W3b3d1RUIIIRzozCXCmBA/1AO/oKz9jEPRTSiaM5ei6IbSokFIyHKapCTYuROCguDGG2HhQldXJIQQogrS88vY8FM67+88yte/Z1FhtYHNStDSd0hY8Rb+kWFk3D4MU2SYtGgQgOzJcq4mTbQrD4cN0/pq7dsHL74oG+KFEKKGOW8X91/T6bh2KUE/7CZo8CAMdwwn3suDPi0jXF2uqCZkJsvZ6tTRmpZOmwbJydC3L+TmuroqIYQQV+CcLu72ItTPPuOXfCv2adMoGjaCIrNNlgjFWWRK5Vrw8NCuOGzVCu69V9un9emn0Ly5qysTQghxARc85HnnToJefonEgBD23TWR7IQmBHnq6dQoWJYIxVkkZF1LY8ZoTUsHDYJOnWDJErj9dldXJYQQ4m/Ouzx4OJeOP24naOV70OQ6jDP+Rbe6IbI8KC5Ilguvteuvh+++g2bNYOBAePJJsNtdXZUQQghOb25/bcvvHM4uxmKza8uDgQbUL7fwy44fsd90E0XPzKbIy0+WB8VFSchyhXr14KuvtJmtWbO043jy811dlRBC1Gpn9r7SKaCgsPd4Abl/HiHoyUdI3LWRih43kj1mAiZvL7mCUFySLBe6ipcXvPMOdOgADzyg/fnxx9CihasrE0KIWunMze3+JgMVVjumY4c58tmnBBUVYXz4X3Rr1VyWB8Vlk5ksV1IUmDgRtm6F4mLo2BFWrnR1VUIIUWuct/cV0LCOibLt36J+8gmFYZEUvfASRQ0ay/KguCISsqqDrl3h++8hPl7rqTV1KpjNrq5KCCHc2pnLg5Wb24/kkZueTdALz5GwbgVq69aoAwdiCg+V5UFxxS4ZsqZMuY/mzRvRrVvHyvvy8nIZMmQAHTsmMGTIAPLz8wBQVZVHH51BYmI83bt3Zt++vc6r3N1ERkJKihawkpO1jvGpqa6uSggh3NY5va8iAlCzsvgl+S3sv/6CYdw4YoYNYHLPZvRpGSEBS1yxS4asO+8cwfLlq8+6b968V+jWrTs7d+6lW7fuzJv3CgBffrmJQ4f+ZOfOvbz0UjIzZ05zTtXuymCAV16BFSvgxx+hbVvYssXVVQkhhFs5tUS44ad0DmYUkVtSAagEpWwi8a0XqTAYyX72BUw9e8jslaiSS4aszp27EBhY56z7NmxYx7BhwwEYNmw469d/BsD69Z8zdOg/UBSF9u0TKSgoIDMzwwllu7mhQ7U2D8HB0KsXzJkjbR6EEMIBzlwijAjwoqjCwt5DWeS+mAxvLsDYojndZtzD8CFdZPZKVNlVXV2YlZVFWFg4AHXrhpGVlQVARkYakZHRlY+LjIwiPT2t8rHiCsTFwa5d8M9/wmOPaS0fli6F0FBXVyaEEDXKmZ3bj+aUEObvhZ+XgZgQP/b+8AfKpo0cykrHcPcYim7qTadm8neWcIwqt3BQFAVFUa74eUuWLGLp0sWAFs5SUlKqWsolFRcXX5PXcajx44mIiKDJa69had6cnx9/nIL4eFdXBdTQ8azmZEwdS8bT8WramOaW2diTZcPbQ8FLDwcybRzUqTTy19Fg3/fEb97Gn2ENOXBjHwxNQokpOMTBvUc5eI3qq2njWRNUpzG9qpAVGhpKZmYGYWHhZGZmEBISAkB4eCRpaScqH5eWlkpEROR5v8eoUWMYNWoMAH37JtGjR4+rKeWKpKSkXJPXcbikJLjrLoxDh9LmwQdh9mx4+GHQufbi0Bo7ntWYjKljyXg6Xk0b0w0/pdMuzIaflwEAa0AuBQUleKV8SdQXn0CbNnhOnkLb4ECX9L+qaeNZE1SnMb2qv6V79+7HihXvA7Bixfv06dMfgD59+rJy5Qeoqsru3bvw9/eXpUJHSUiA3bvhjjvg0Uehb1/IkP1uQghxPuff3A4Ni7Kwf7SKrKOp2EeMoOhfj1FkMEn/K+EUl5zJmjBhDN98s53c3Bzi45sxc+ajPPDANMaPH82yZUuIjq7P228vBqBnz95s3ryJxMR4vL29SU6e7+z6axd/f/jgA21ma+pUra/WkiXQu7erKxNCiGrjzMOdIwK8KCy3sPdYHgm/7CLovUXERjUmY9hdZMc3JcjoQafGIbLBXTjFJUPWm28uOu/9q1atPec+RVF44YWXq16VuDBFgQkTtAamw4ZBnz7w0EPw3HPg6enq6oQQwmVObXD/6rcsjB464iL8tc3tv6ehfLmFQ7//giHxevSjxjK6fYwEK+F00vG9pmrRQmvzMHEivPgidOkCf/zh6qqEEMIlLnS4Mwf2kzB/Lv6HfiPjpn6YZk6nlwQscY3IAdE1mckE8+dDz54wbpy2b2vePBgzRpvxEkIIN3ah1gz+JgMVFRZM337NkV07aeupYJgynvjGMXK4s7imZCbLHQwaBPv2QWKiFraGDIGcHFdXJYQQTvP3cwdzS8z8nllMbkkFDcvzKVv+Ier3eyjs0Imif/+HotBI2dwurjkJWe6iXj3YvBnmzoW1a6FVK9i0ydVVCSGEQ526avC1Lb9zOLsYi82OTlEI9TOiU+DIhq8IemQGCcd+Qr19EGrXrph8vOR4HOESErLciU4HM2ZoneIDA7WrDqdMgbIyV1cmhBBVdqF9V7klFTRUKrB/9hlZ3+zCnhCP4bk5xHRoweSkJnI8jnAZ2ZPljhIS4PvvtYal8+bBxo1aq4fERFdXJoQQV+x8Vw36mwxUWO2YDDqObNtF26X/R6x3HTJuu4PspOsJ8vWkU3SghCvhUhKy3JXJpAWs227TNsJffz088gg88YS0ehBC1Bhn9rw6c/aqYbA3R1Kz8dqWQuGhIxQ1aoJ+7HhG39BSgpWoNiRkubuePWH/fq156ezZsG6dNqvVsqWrKxNCiAu6+OyVnvwfD5CwYjG/eIeidu6KaciNdKofJAFLVCuyJ6s2CAyExYthzRpITYW2bbXmpRaLqysTQohzXGjvVaDJk7LCUtT16yhcux6Dvx8xE0Yx+cEh9GkdJQFLVDsSsmqT22+Hn36CgQPh8cehY0f48UdXVyWEEMD5rxz0NxlQdGizV3t/IuH/5qD+9jtqh0RMzz9Hr55tJVyJaktCVm0TGgorVsCqVdqsVvv28PTTYDa7ujIhRC128dmrEm326rMNGAICiJlwN5Nn3kmf+GgJWKJak5BVWw0aBD//rJ1/+Mwz0KGDdkyPEEJcQxefvdKRv/sHEl6bg/rbH6iJiZj+PZtevWT2StQMErJqs+BgeO89+OQTyM6GTp1g2jQoLnZ1ZUKIWuCis1e5Bahr1lC44UsMocHETLybyTNk9krULBKyhNbm4eefYcIEePVV7crDDRtcXZUQwo2l55ex+NvD7E8t4PfMYhQFbfbKQyH/q/+RMG82amqq1rF99rP0uqmNhCtR40gLB6EJCNAOmx4+HMaPh759YcQIePllqFvX1dUJIdzAqbYMh7NKOJpbSlGZmQbBPlRY7RSV2VBzswjevoXC7FwM111HzMjR9OoaJ+FK1FgykyXO1rUr7N0LTz4JK1dC06awYAHY7a6uTAhRg525NFhYbkavg8JyKzklFZgUG8F7/kfAxytRi4pQe/bE9PgjErBEjSczWeJcRqO2Gf7OO2HiRG0ZcdEieOMNiI93dXVCiBrkfE1FiytsBJo8sQVC6u9H8P3fFox5ORS260yrW3vQq12MhCvhFiRkiQuLi4OtW7XN8dOnQ7t22oHTTz8Nfn6urk4IUU39fVkwNsz3rI3tep1KeU4uwVs2Y07PxWjSk3XnXQTFREvAEm5FQpa4OEWBu+6C/v3h0UfhlVfggw/gP/+ByEhXVyeEqGbOPGvw1LLgHydL0Ov+2tiuU7Hs/I6yvXspR6FuYlua3JJEuEWlV/MwCVjCrUjIEpcnKEhbLhw7FiZPhpEjSWjdWjsHUZYQhRCcvmIwt8RCqK+RzMJyIgK8KbfYsdhslP3+J17btmIvKadJw7ocvKEP/vVCMJk86dQkUAKWcDsSssSVSUyEHTtg0SK8p0/XzkGcOBGefVYLYkKIWuViVwxmFZkx6HWEVBRj3vI1Cbu+5JembVEH3ExEjwT6REuwEu5NQpa4cjodjBvHrrp16bppk9b64YMPtM3yEyaAweDqCoUQ18D5lgZPXTEY6mciyltH6o69eOz/Dj+7GcOdw4jpdiO9WkVKuBK1grRwEFfN6ucHr70GP/wACQlw//3a0uHGja4uTQjhZH9vJppZWE6gyZOoQG/S8soo272HoHffxv+3n7HFNcf/gUmYBtwiAUvUKlWayWrXriW+vr7odHo8PDz44ott5OXlMn78GI4fP0q9eg14++3FBAbWcVS9ojpq3Ro2b4ZPP9WuQuzTB/r1gxdf1K5QFEK4hfT8Mr7PsPLL1j8uvDSYmYr5m90YT/xJVuNmRHe/ntG3d5RgJWqlKs9krV69jq1bv+GLL7YBMG/eK3Tr1p2dO/fSrVt35s17pcpFihpAUWDAADhwQAtX27dDq1Zw772QkeHq6oQQVZCeX8YHO48y5/Nf+DXPSmp+ydmdWLm3AAAbm0lEQVTNRD31RNnLSNu8nfyPP6NuSQ5NRg6k1fh/SMAStZrDlws3bFjHsGHDARg2bDjr13/m6JcQ1ZnRqM1m/fEHTJoE77wD112n7deSg6eFqFHODFebDmRgMigowMHMYvSKoi0NZuZTtvYzgt5biH92OrauN+A/eSKm6zvRq3m4BCxRqyknTxaqV/vk9u1bERAQiKIojBo1hlGjxnDddfX444/jAKiqSpMm9Ss/P9OSJYtYunQxABkZaSxfvvxqy7hsxcXF+Pr6Ov11aovLGU9Taioxb71F3W3bMNepw5HRo0nv1w/VQ665OB95jzqWjOfVyy2zsSfLxsliOyhwosiOTlEIMtgotOlQzFZaHjlAcWoW4cV5HGkRj67pddzUxESQSe/q8msMeY863rUY06lTH6xcwbuYKv1Nt3btRiIiIsnKyuKOOwbQpEnsWV9XFAVFUc773FOhDKBv3yR69OhRlVIuS0pKyjV5ndrissdzxAjYsQPPGTOIfeUVYj/5RGv5cOedoJdfxmeS96hjyXheuVMtGXb/loUxUIev0UxEgDce2cWUVNgozcsmPi+Tn49lY888RlhsI2JvG0WEb6A0E70K8h51vOo0plVaLoyI0Dp+h4aG0q/fLezZ8z2hoaFkZmp7cDIzMwgJCal6laLm69QJvvoK1q3TjuQZOVK7IvHTT0G96slUIYSDnLk0uPNQDmVmKwoKWUVmsovLqetjwH70GB4/7EP/7XZivBVsQ4fhP/h2TJFhErCEOI+rDlklJSUUFxdVfpySsoW4uDh69+7HihXvA7Bixfv06dPfMZWKmk9RtKsO9+zR+mqVl2ub5Tt31to+SNgS4po7374rnaIju9hMidlClL8naT/9gX7RIiJ2bsNg8CB70DDajB3ClCEduS+pCX1aRkjAEuI8rnq5MCvrJKNHjwDAZrMyaNAd3HhjLxIS2jJ+/GiWLVtCdHR93n57saNqFe5Cp9OWCgcPhsWLYfZsre1D587a4dO9emmBTAjhNOn5ZaQcPMm3f+ZQXG4h1M+TnBI76QUVmDwNRAUYST3wB01++IYQs4q9ji/lPa6nUbCeCQNukFAlxGW46pDVsGEMKSnfnnN/UFAwq1atrVJRopYwGGD8eBg1ChYtgueeg969oUsXLWzddJOELSEc7GLhyqhXsNtUTu77lUbfbcNs1aMG+ODd43o69mxLfHQgB/fulIAlxGWSS7yE6xmNWj+tMWO0lg9z5mizWR07wmOPwS23SNgSogr+fr6gzWb7K1xVnBWuOHoc+6HDFFSYKTP5EdC5MzGJLc9qxXDQxT+LEDWJhCxRfRiNcN99MHastoz4wgtw221aR/lHH4UhQ+RqRCEuw5mhKjW/lPSCcuoHeWOx2dDrFP7MKiUu3J9Ak4GSsgo4fBz74cNYzFaivSCrXVuy60XQuXEQPZrKhnYhrpaELFH9eHlpM1vjxmkb5P/9b20PV2wsPPQQ3HWX9hghxFnOXAo0GfSUW2zklZqx2OyUVtg4lFNMXLg/AV4GUjMLqHfkF/48ehKL1U60j46s+HaUR4Vzs4QrIRxCQpaovgwGbb/WiBGwZg08/zz885/wxBPwwAMwcSLUkXMxhQD48VgeH3x3nKM5JZgMOnKLy7GjoKp2fI2eFFZYtHCVmk30Hz/x88lS9Om/EREbR1bTVpSHh3JzIwlXQjiShCxR/en12lLh4MGQkgJz52p7tebMgXvu0QJXo0aurlIIlzg1e7Vqzwm8PfXYVTs6xYPM4jLCfI2UWVUwqpRl5tDgt338XGTHmnuCpk2bYL95NOWePhKuhHASCVmi5lAUSErSbvv2aQdR/9//wbx5Wr+tqVOhW7datUne8+YjtIoxVH4+tIcPD/8j8LKf33jEcXbOjyQk4Oy9bqqq0mtGJqufqYunAXpMy8BsUbHaYFA3b56+W5tB/HJPGf9akIddVfHx0rFwZgjXRRl4cH4u234sA6C0XOVkvo2cTxpcVk3JqwoY398Pby/HHa26/5CZVz4qYOHMUId9T1c5334ru13Fbrfjqfcku9iMQWfFx6CnqMyMT0EeJXuP4p2dhb4sj5iENtj6dKd+vRAahfoSHx0o4UoIJ5GQJWqm1q1hyRJtv9b8+fDmm/Dxx1oX+SlTtD1ctWDflslT4fs3oxz+fT/fWUbrRp74++hQVZXNL4bja9Jhsap0m5pOnw4mOjX3YnJyDqufrUtcA09e/6SQOcvyWTgzlJfvC6r8Xv9dU8jeP8yX/drzVhcyoqcv3g78z9eqkScnsmwcy7RSP6xm/tq72H4rs81OuVWl3GIl2NtIdmExwdmZpOaVEJidSqABjPEtyW7UiM5N68qslRDXSM38bSPEKVFRWn+txx+HZcvg1Ve1VhAPPaRdpXjvvbVyKbHxiOPcdbMv6/5XhsWqsvzJUJrV9ySnwMaIOVmkZdvo1Nx4wSb7H3xZwj39tQNWFUXB16TNDlqsKlYrlWeSKgoUlmrfpKDETkTwub9Slm8t4am7z51dKymzc+esLFKzrdjs8NiIQDLzbaTl2Og5PYPgAD1fvhTOpt1lPPNuPmaLSqNID96ZEYKvSUfjEccZ0t2HjbvK8DIqvPdoKNdFGfhoWwmzluaj14FqCeeHHtrr3dLZmxUpJcwYFlDF0b12Ts1a7Tuez48nCigzWwn0Npyz3yq3pAJ/kwc5WflEZRwhICsPxWLGPzyKel06EBXfhEZ1/WTWSohrTEKWcA8mk7Y/a9w42LoVXn8dXn5ZW1Ls00drDdG3r9u1gCgzq7SbkFr5+cN3BjI0yQeAEH89370RyeufFPLyh4UsmB7CrKX5dGnpxRN3BbJuRykL1xef9/t+e6Cc16cFV35us6kk3pfGH6lWJg7wo2OcEYA3p4dw66OZmIwK/t46vnkt4qzvczTTypEMKzcmnDsttfG7MiKD9aydEwZAQbGdAF8dr35UwOaXwgkJ0JNdYGPOsnw2zQ3Dx6Rj7vICXvmokCfu0kJbgI+OvW9HsXRTMQ/Oz+XT58KYvTSfz58PIyrEg882fAU0A6BdrCdzlxdU65B1vqXAYB8jh7KL8PTQU1hhxWTQk1lsPr3fSm/BeDITjxPH8CmrQLVWoNRvQFRcIx7q1Yr4+nJxiBCuIiFLuBdFgRtv1G6pqfD227BgAdx6K0RHa7NbY8ZAw4aurtQhLrZcOPAGbwDaxhpZs70UgK/3l/PhU3UB6N/Jmzp+59/3lFtkx8/79Nf0eu118ottDH4qi58Om2kZ40nyqkLWzgmjY5yRF1cU8NAbuSyYfvpQ+BVbSxh8gzd6/bn75Fo28mTGm7n8661c+nfy5oZW5waxHb9U8MtRC92mpgNgtkCn5sbKr9/5V6C880Yfpr+eC8D1LY2MnZvNHd19CD3jZesG6knPsZ33560OTl0dWFRmoaDMQoVV62lVbraSU2yhYbABH4Oe4oq/9lvl5OOTlUFJZgbepSUEGyGraTw+jRvQU5YEhagWJGQJ9xUVBU89pTUyXbsW3noLZs3Sbr16aTNft92mNUF1Q0aDljD0OrDaruzwbQ+9gt2uotOdHY4CffX0SPBi43dlhNXRs+9Pc+Ws1tAePvR/JPOsx6/cWsK8B4I4n9hoA9+9Hsn6XWU8uSiPG9uYKmeoTlFV6NnOxLLHzr9h/cxrHE59PH9qCDt/qeDznaU8tTaSrl1sBAfoKTereHlWr4siTu2z2nk4h/0nCgj29cTboEOv05FXZiHM10h+uQU/Lw9yS8wEeyocP3aSsIwjpGIisDiHwPAgjLFtKa8TKv2thKhmJGQJ92cwwKBB2u3YMe2cxHfegaFDIShI2yQ/ahQkJrr9lYk3tPLigy0lPDYykPW7Sskrsp/3cbHRHhxKt3JdlIGsfBsGDy1glVXY2fx9GTPuDKCOn46CEju/nbAQG21g854ymtU/faXjr8fM5BXb6Nz8/CE2LdtKkL+OET19CfDVsfDzIgD8TDqKSu2EBOjpFGfkgXk5/JFq4booAyVldlJzbMRGa6+zMqWEh/8RyMqUksoZrj/TLHSMM9IxzsiHX57keJaV4AA9v5+w0PKMKzGvtb8vBZ4sKiejoAId4O2pQ1GgoNRKptVKoxBfbbaq3IqiqvgXF5B5Mp/A1ENE2m3YAwLxjwqlXov2RIUFylWCQlRTErJE7VK/vja79fjj8MUX2hWKCxdqVyg2bXq6+WmDy2s34Gp/35N1c3sT/x5//pkjgCfuCmTEnCxaj0ulc3Mj9euef49av07ebPuxnOuiDKTn2hj7QjY2u4pdhSHdfbilk7YU+eaDwQx9+iQ6HQT66nj7obOXCof28KncJP93+w9b+NeCXHQ6MOgV/jtF2wN2T38/+j+SSUSwB1++FM47M0MY+VwWFRZtNu7ZMXUqQ1ZesZ0241MxGhTe+2u26+EFefxxwoIKtIiuIL6xJwApP5bTt6P35QyrQ6Xnl/HJDyfY/OtJ7HYVq82OClRY7djtNlD05JWZ8TUaUIByC9qslaWUw+n5RKYfxmaxEmQ0Ym/YEP+G0fiFh/BQh/qy30qIak45ebLwytYRnKBv3yR2797t9NdJSUmhR48eTn+d2sJtxrOgAD76SAtcX32l3Xf99doM19ChEBZ2zUqpLmOanmNl9AvZbJwb7upSLuhCPb7OdGo8K8wqSQ+m81VyBB7n2R/maGcuAx7KKqGo3EyonxflFhsniyowGfQoiorFBkHenuSXmvE2GlAqyqnIy0eXX0BobgYWDwMN/TzIj2lK3euiiQr2dfmsVXV5j7oLGU/HuxZjmpDQli++2HbJx8lMlhABAdpViePGweHDsHy5dmbiAw9oDU5vvBGGDdManobW/GaWlyMi2IN7+vlRWGLH38dxTUFd5dhJK3PuqeO0gHUqVO1PLSCn2Ex6filWO+h0UG62UFxhQ6+zoNOp6HW6yv1udtUO5eV45eVjLSwkOCeDEqMXnj4+KM2b0651DAmxEbIUKEQNJSFLiDPFxMAjj2i3AwdOB67x42HCBOjeXdvbNXCgtrHejd3Rw8fVJVzUn8vqXfZjm0QbaBLtmP1YZwaq0gobZWYLR/NKKTerBPsayCmpoNRsw8egR9FpS7oGvZ4Kiw2Dhw4PBWzlZgxlxXgXFFJcXoan3UKwUYc9tglh0eHclFBfNrAL4QYkZAlxIS1aaFciPvss/PgjrFql3e6/X7t16qS1hrj1VmjZ0u03zddmp4JVysGT/JpRiNUG4QFGyi02MgorAJUgk4GMggrKrVY8dDrsKujR4aGzY1dVLGUVGEuLUIqKKVP0eJeXEKuUkBbVCEudIOpG1aFjjFwdKIQ7kZAlxKUoinZcT0KCFrp++UULWx9/rB1U/dhjWt+tW2+FW27Rzk+sBUf6uKsz91Ol5ZdRVK71rVLtYLXbAQUPvcKJ3DJUQAdYVQUbCigqCjpsdhWb1YaptBh9URF5ZpU6xQV4qWZUv0B8AvwIq9+YgLqBdIn0l2AlhJuSkCXElYqL065OfPxxSEuDzz473Yfrtde07vM9ekDv3lq3+dhYmeWqpi4WqHR6BVWFonILdlVFh4IdFZ1Oh7+HgXK71tjUoFNQ7HbMVjtGSwW2kjKs5RVYysrwLM3HhBVraDReDevTuFEYHZuGSagSopaQkCVEVURGwj//qd1KS7UjfTZu1G5Tp2qPadDgdBf6pCS338tVHf14LI8Pvz/O/tQCSiqsqKqdcoudogorOhSMfzUAPTNQAeh0yl/5WMGOtvynqipmqw29AnaLFSxmDOXl2MvL0JcVoRp98fHUEVDHG49GEdj9AhgaF8rtbepJsBKilpGQJYSjeHtD//7aDbQrFTduhE2btKXFRYu0+2NjtbDVrRt07ar17hJVdua+qT+ziykutVBSbkGfsoFyiw29Hox6PRa7SrnF/tdsFShAqdmGp4fyV4d7LVAB2FUVPQqnTtLWqTbsFisVJRa8S4uxqmBXFMLMhRj9fMkLaUgdfx+aRQcSG+7v8nYLQgjXkpAlhLPExMC992o3u13bPL91K2zZAu+/D2++qT2uXj0tbHXpgq+Hh9ajy9PTtbVXM6caem44kMGx3FJKzVbsfyUhnaKtxtrsnHWfTQUVsFttKIDOBhbFRmVmUqhslKqgYrGrGBVdZaBCVcGmotgteJitqDYLqs2GyWJGNRjw8DISaDLSICoQrzqB+Hp50FL2VwkhzuC0kLVlyxc89tjD2Gw2Ro68mwceeNBZLyVE9afTQZs22u3BB8Fmg/37Yft27bZtG3zwAe0BpkyBtm21Y346dIB27aBJE9BfuOlmdfXQ8t18tDfz0g90tL9yknLGDQXsfwWsU59zRitm1QaKtQIPmwXVYgO7Dftf4c3fZkZnMmL18cPT20izekEkNZNDmIUQF+eUkGWz2Xj44el8+OEnREZGcfPNPejdux9NmzZzxssJUfPo9aevWJw8Wfub/+hRDixaRIuSEti1S9tIn5ysPd7bG1q31h7fpg20agXNm2uNVKuRhCfWkW9xdRWnnbr6D/7KVOrpPz1UG3qbFbtNWzpUFDs2qxV/Sxk6oxGrtxeKp4lAfyO+Piai6pikxYIQ4oo4JWTt2bObmJhGNGwYA8DAgYPZsGGdhCwhLkRRoGFDspKStCsTAaxWrSHqDz/A3r3a7YMP4I03Tj8vOloLWy1aQLNm2oxXkybahnzdte3UXt0CFoCCDT3axKGCDcWmoqh2rDo9WCsw2q2oBgM2L0+8FE9CAoz4BvhKoBJCOIRTQlZGRjpRUdGVn0dERLJnj/PPJhTCrXh4QHy8djtFVeHIEfjpJy2A/fyz9ucbb0BZ2enHmUxw3XXQuLF2deOZt/r1ITjY4SHMdQHLdtYfoAUpVVG0TVqqHS+rBdXDgKIHT70ef09PFC8fVL2e8AAvkprK1X9CCMdz2cb3JUsWsXTpYgAyMtJISUlx+msWFxdfk9epLWQ8He+yx9TPT+s436mT9rndjvHkSUypqXinpmI6cQJTaiqmvXsxbtyIx5kBDLDr9ZiDgzEHBVEREoI5KAhLQAAWf38sAQFY//rY6uODzccHq7c3dqPRyf2+bJd117lObVS3o7fbUVBR9Xr0ehWDHgx6T0wGE+G+EF/XQOtQPUGmM/e3WYA0Du5N42DVfwi3J//fO5aMp+NVpzF1SsgKD48gNfVE5efp6WlERESe9ZhRo8YwatQYAPr2Tbomp5DLaeeOJePpeE4ZU1WFvDw4elS7HTuGLj0dr/R0vNLSID0dfv0VcnNPX553PjqdFu68vbWO9mfePDyg40OOrfti9CoeKOjQo0N7eQ8PHSZPD+oFeVfOTB3cu1Peow4m/987loyn41WnMXVKyGrTph2HDh3i6NEjREREsmbNKt544x1nvJQQ4lIUBYKCtFubNhd+nN0O+fmQna3dcnKgsFC7FRWd/risDMrLz77ZbASWF5DvdRUb8c+5aPL8V1F6Ktp5gf1bRzDq+kaXtbQnM1NCCFdySsjy8PDg+ef/w7BhA7HZbAwffhfNmsU546WEEI6i050OY7GxV/z0vVx68/v93RsyvW+Lq69RCCFqEKftyerZszc9e/Z21rcXQlRDe2f1d3UJQghRbVzba7yFEEIIIWqJanGszuHDh0lIaOv018nJySY4OMTpr1NbyHg6noypY8l4Op6MqWPJeDretRjT48ePXtbjlJMnC9VLP8w99OrVnS++2ObqMtyGjKfjyZg6loyn48mYOpaMp+NVpzGV5UIhhBBCCCeQkCWEEEII4QT6GTMeedrVRVxL8fEX6RMkrpiMp+PJmDqWjKfjyZg6loyn41WXMa1Ve7KEEEIIIa4VWS4UQgghhHACtw9Zn366hhtuSCQsLIC9e/ec9bXk5JdITIync+e2bNmy2UUV1mxz586hdeumJCV1ISmpC5s3b3R1STXSli1f0LlzWxIT45k372VXl+MW2rVrSffunUhK6kKvXt1dXU6NNGXKfTRv3ohu3TpW3peXl8uQIQPo2DGBIUMGkJ+f58IKa5bzjaf8Dr16qaknGDiwP127duCGGxJZsGA+UL3eo24fspo1a86iRcvo3LnLWfcfPPgra9as4uuvd7F8+WoefvhBbDabi6qs2SZMmMTWrd+wdes30uX/KthsNh5+eDoffLCK7du/Y/Xqjzh48FdXl+UWVq9ex9at31Sby7lrmjvvHMHy5avPum/evFfo1q07O3fupVu37syb94qLqqt5zjeeIL9Dr5aHhwfPPPMc27d/x/r1X7Jw4VscPPhrtXqPun3Iio1tynXXNTnn/g0b1jFw4GCMRiMNGjQkJqYRe/bsdkGForbbs2c3MTGNaNgwBk9PTwYOHMyGDetcXZYQdO7chcDAOmfdt2HDOoYNGw7AsGHDWb/+M1eUViOdbzzF1QsLC6d16wQAfH39iI1tSnp6WrV6j7p9yLqQ9PQ0IiOjKj+PjIwiIyPdhRXVXAsXLqB7985MmXKfLB1chYyMdKKiois/j4iIJD09zYUVuQdFURg69HZ69uzGkiWLXF2O28jKyiIsLByAunXDyMrKcnFFNZ/8Dq26Y8eOsn//Ptq1a1+t3qPV4lidqho8+DaysjLPuf+RR56kb185sLaqLja+o0ffw/TpD6MoCs8/P5unnnqM5OT5LqhSiLOtXbuRiIhIsrKyuOOOATRpEnvOtgFRNYqioCiKq8uo0eR3aNUVFxczduxdzJr1PH5+/md9zdXvUbcIWatWfXrFz4mIiCQtLbXy87S0VMLDIxxZltu43PEdOfJuRo4c6uRq3E94eASpqScqP09PTyMiItKFFbmHU2MYGhpKv363sGfP9xKyHCA0NJTMzAzCwsLJzMwgJETO3auKunXrVn4sv0OvnMViYezYkQwePJRbbrkNqF7v0Vq7XNi7dz/WrFlFRUUFR48e4dChQ7Rt297VZdU4mZkZlR9//vlamjWLc2E1NVObNu04dOgQR48ewWw2s2bNKnr37ufqsmq0kpISiouLKj9OSdlCXJy8Nx2hd+9+rFjxPgArVrxPnz6yWlAV8jv06qmqytSpk4iNbcrEiZMr769O71G3b0a6bt1aHn10Bjk52fj7B9CyZStWrvwYgFde+Q/vv78UDw8PZs9+nptuutnF1dY89903ngMH9gMK9evX58UXkyvXwsXl27x5I48//i9sNhvDh9/FtGkzXF1SjXbkyGFGjx4BgM1mZdCgO2RMr8KECWP45pvt5ObmEBpal5kzH6Vv3/6MHz+aEyeOEx1dn7ffXkydOkGuLrVGON94fvPN1/I79Crt2PE/brutN3FxLdDptDmjxx57krZt21eb96jbhywhhBBCCFeotcuFQgghhBDOJCFLCCGEEMIJJGQJIYQQQjiBhCwhhBBCCCeQkCWEEEII4QQSsoQQQgghnEBClhBCCCGEE0jIEkIIIYRwgv8Hw4zLJi0LWbsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95df5d4eb8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-10,20,100)\n",
    "y = x**2 - 6*x + 5\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-')\n",
    "plt.ylim([-10,250])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "plt.plot(mins,cost,'o', alpha=0.3)\n",
    "ax.text(start,\n",
    "        start**2 - 6*start + 25,\n",
    "        'Start',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['blue'],\n",
    "       )\n",
    "ax.text(mins[-1],\n",
    "        cost[-1]+20,\n",
    "        'End (%s steps)' % len(mins),\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['blue'],\n",
    "       )\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## From derivatives to gradient: $n$-dimensional function minimization.\n",
    "\n",
    "Let's consider a $n$-dimensional function $f: \\Re^n \\rightarrow \\Re$. For example: \n",
    "\n",
    "$$f(\\mathbf{x}) = \\sum_{n} x_n^2$$\n",
    "\n",
    "Our objective is to find the argument  $\\mathbf{x}$ that minimizes this function.\n",
    "\n",
    "The **gradient** of $f$ is the vector whose components are the $n$ partial derivatives of $f$. It is thus a vector-valued function. \n",
    "\n",
    "The gradient points in the direction of the greatest rate of **increase** of the function.\n",
    "\n",
    "$$\\nabla {f} = (\\frac{\\partial f}{\\partial x_1}, \\dots, \\frac{\\partial f}{\\partial x_n})$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.000000 [2.0000000001, 2.0000000001, 2.0000000001]\n",
      "3.000000 [2.0000009999, 2.0000009999, 2.0000009999]\n"
     ]
    }
   ],
   "source": [
    "def f(x):\n",
    "    return sum(x_i**2 for x_i in x)\n",
    "\n",
    "def fin_dif_partial_centered(x, \n",
    "                             f, \n",
    "                             i, \n",
    "                             h=1e-6):\n",
    "    '''\n",
    "    This method returns the partial derivative of the i-th component of f at x\n",
    "    by using the centered finite difference method\n",
    "    '''\n",
    "    w1 = [x_j + (h if j==i else 0) for j, x_j in enumerate(x)]\n",
    "    w2 = [x_j - (h if j==i else 0) for j, x_j in enumerate(x)]\n",
    "    return (f(w1) - f(w2))/(2*h)\n",
    "\n",
    "def fin_dif_partial_old(x, \n",
    "                        f, \n",
    "                        i, \n",
    "                        h=1e-6):\n",
    "    '''\n",
    "    This method returns the partial derivative of the i-th component of f at x\n",
    "    by using the (non-centered) finite difference method\n",
    "    '''\n",
    "    w1 = [x_j + (h if j==i else 0) for j, x_j in enumerate(x)]\n",
    "    return (f(w1) - f(x))/h\n",
    "\n",
    "def gradient_centered(x, \n",
    "                      f, \n",
    "                      h=1e-6):\n",
    "    '''\n",
    "    This method returns the gradient vector of f at x\n",
    "    by using the centered finite difference method\n",
    "    '''\n",
    "    return[round(fin_dif_partial_centered(x,f,i,h), 10) for i,_ in enumerate(x)]\n",
    "\n",
    "def gradient_old(x, \n",
    "                 f, \n",
    "                 h=1e-6):\n",
    "    '''\n",
    "    This method returns the the gradient vector of f at x\n",
    "    by using the (non-centered)ç finite difference method\n",
    "    '''\n",
    "    return[round(fin_dif_partial_old(x,f,i,h), 10) for i,_ in enumerate(x)]\n",
    "\n",
    "x = [1.0,1.0,1.0]\n",
    "\n",
    "print('{:.6f}'.format(f(x)), gradient_centered(x,f))\n",
    "print('{:.6f}'.format(f(x)), gradient_old(x,f))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function we have evaluated, $f({\\mathbf x}) = x_1^2+x_2^2+x_3^2$, is $3$ at $(1,1,1)$ and the gradient vector at this point is $(2,2,2)$. \n",
    "\n",
    "Then, we can follow this steps to maximize (or minimize) the function:\n",
    "\n",
    "+ Start from a random $\\mathbf{x}$ vector.\n",
    "+ Compute the gradient vector.\n",
    "+ Walk a small step in the opposite direction of the gradient vector."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> It is important to be aware that this gradient computation is very expensive: if $\\mathbf{x}$ has dimension $n$, we have to evaluate $f$ at $2*n$ points.\n",
    "\n",
    "\n",
    "### How to use the gradient.\n",
    "\n",
    "$f(x) = \\sum_i x_i^2$, takes its mimimum value when all $x$ are 0. \n",
    "\n",
    "Let's check it for $n=3$: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def euc_dist(v1,v2):\n",
    "    import numpy as np\n",
    "    import math\n",
    "    v = np.array(v1)-np.array(v2)\n",
    "    return math.sqrt(sum(v_i ** 2 for v_i in v))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by choosing a random vector and then walking a step in the opposite direction of the gradient vector. We will stop when the difference between the new solution and the old solution is less than a tolerance value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[-7, -5, -6]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# choosing a random vector\n",
    "\n",
    "import random\n",
    "import numpy as np\n",
    "\n",
    "x = [random.randint(-10,10) for i in range(3)]\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-0.0, -0.0, -0.0]\n"
     ]
    }
   ],
   "source": [
    "def step(x,\n",
    "         grad,\n",
    "         alpha):\n",
    "    '''\n",
    "    This function makes a step in the opposite direction of the gradient vector \n",
    "    in order to compute a new value for the target function.\n",
    "    '''\n",
    "    return [x_i - alpha * grad_i for x_i, grad_i in zip(x,grad)]\n",
    "\n",
    "tol = 1e-15\n",
    "alpha = 0.01\n",
    "while True:\n",
    "    grad = gradient_centered(x,f)\n",
    "    next_x = step(x,grad,alpha)\n",
    "    if euc_dist(next_x,x) < tol:\n",
    "        break\n",
    "    x = next_x\n",
    "print([round(i,10) for i in x])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Alpha\n",
    "\n",
    "The step size, **alpha**, is a slippy concept: if it is too small we will slowly converge to the solution, if it is too large we can diverge from the solution. \n",
    "\n",
    "There are several policies to follow when selecting the step size:\n",
    "\n",
    "+ Constant size steps. In this case, the size step determines the precision of the solution.\n",
    "+ Decreasing step sizes.\n",
    "+ At each step, select the optimal step.\n",
    "\n",
    "The last policy is good, but too expensive. In this case we would consider a fixed set of values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "step_size = [100, 10, 1, 0.1, 0.01, 0.001, 0.0001, 0.00001]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learning from data\n",
    "\n",
    "In general, we have:\n",
    "\n",
    "+ A dataset $(\\mathbf{x},y)$ of $n$ examples. \n",
    "+ A target function $f_\\mathbf{w}$, that we want to minimize, representing the **discrepancy between our data and the model** we want to fit. The model is represented by a set of parameters $\\mathbf{w}$. \n",
    "+ The gradient of the target function, $g_f$. \n",
    "\n",
    "\n",
    "In the most common case $f$ represents the errors from a data representation model $M$. To fit the model is to find the optimal parameters $\\mathbf{w}$ that minimize the following expression:\n",
    "\n",
    "$$ f_\\mathbf{w} = \\frac{1}{n} \\sum_{i} (y_i - M(\\mathbf{x}_i,\\mathbf{w}))^2 $$\n",
    "\n",
    "For example, $(\\mathbf{x},y)$ can represent:\n",
    "\n",
    "+ $\\mathbf{x}$: the behavior of a \"Candy Crush\" player; $y$: monthly payments. \n",
    "+ $\\mathbf{x}$: sensor data about your car engine; $y$: probability of engine error.\n",
    "+ $\\mathbf{x}$: finantial data of a bank customer; $y$: customer rating.\n",
    "\n",
    "> If $y$ is a real value, it is called a *regression* problem.\n",
    "\n",
    "> If $y$ is binary/categorical, it is called a *classification* problem. \n",
    "\n",
    "Let's suppose that our model is a one-dimensional linear model $M(\\mathbf{x},\\mathbf{w}) = w \\cdot x $. \n",
    "\n",
    "### Batch gradient descend\n",
    "\n",
    "We can implement **gradient descend** in the following way (*batch gradient descend*):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import random\n",
    "\n",
    "# f = 2x\n",
    "x = np.arange(10)\n",
    "y = np.array([2*i for i in x])\n",
    "\n",
    "# f_target = 1/n Sum (y - wx)**2\n",
    "def target_f(x,y,w):\n",
    "    return np.sum((y - x * w)**2.0) / x.size\n",
    "\n",
    "# gradient_f = 2/n Sum 2wx**2 - 2xy\n",
    "def gradient_f(x,y,w):\n",
    "    return 2 * np.sum(2*w*(x**2) - 2*x*y) / x.size\n",
    "\n",
    "def step(w,grad,alpha):\n",
    "    return w - alpha * grad\n",
    "\n",
    "def BGD_multi_step(target_f, \n",
    "                   gradient_f, \n",
    "                   x, \n",
    "                   y, \n",
    "                   toler = 1e-6):\n",
    "    '''\n",
    "    Batch gradient descend by using a multi-step approach\n",
    "    '''\n",
    "    alphas = [100, 10, 1, 0.1, 0.001, 0.00001]\n",
    "    w = random.random()\n",
    "    val = target_f(x,y,w)\n",
    "    i = 0\n",
    "    while True:\n",
    "        i += 1\n",
    "        gradient = gradient_f(x,y,w)\n",
    "        next_ws = [step(w, gradient, alpha) for alpha in alphas]\n",
    "        next_vals = [target_f(x,y,w) for w in next_ws]\n",
    "        min_val = min(next_vals)\n",
    "        next_w = next_ws[next_vals.index(min_val)]   \n",
    "        next_val = target_f(x,y,next_w)    \n",
    "        if (abs(val - next_val) < toler):\n",
    "            return w\n",
    "        else:\n",
    "            w, val = next_w, next_val"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.999641\n"
     ]
    }
   ],
   "source": [
    "print('{:.6f}'.format(BGD_multi_step(target_f, gradient_f, x, y)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6.53 ms ± 290 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "BGD_multi_step(target_f, gradient_f, x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def BGD(target_f, \n",
    "        gradient_f, \n",
    "        x, \n",
    "        y, \n",
    "        toler = 1e-6, \n",
    "        alpha=0.01):\n",
    "    '''\n",
    "    Batch gradient descend by using a given step\n",
    "    '''\n",
    "    w = random.random()\n",
    "    val = target_f(x,y,w)\n",
    "    i = 0\n",
    "    while True:\n",
    "        i += 1\n",
    "        gradient = gradient_f(x,y,w)\n",
    "        next_w = step(w, gradient, alpha)\n",
    "        next_val = target_f(x,y,next_w)    \n",
    "        if (abs(val - next_val) < toler):\n",
    "            return w\n",
    "        else:\n",
    "            w, val = next_w, next_val"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.000101\n"
     ]
    }
   ],
   "source": [
    "print('{:.6f}'.format(BGD(target_f, gradient_f, x, y)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "148 µs ± 7.56 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "BGD(target_f, gradient_f, x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stochastic Gradient Descend\n",
    "\n",
    "The last function evals the whole dataset $(\\mathbf{x}_i,y_i)$ at every step. \n",
    "\n",
    "If the dataset is large, this strategy is too costly. In this case we will use a strategy called **SGD** (*Stochastic Gradient Descend*).\n",
    "\n",
    "When learning from data, the cost function is additive: it is computed by adding sample reconstruction errors. \n",
    "\n",
    "Then, we can compute the estimate the gradient (and move towards the minimum) by using only **one data sample** (or a small data sample).\n",
    "\n",
    "Thus, we will find the minimum by iterating this gradient estimation over the dataset.\n",
    "\n",
    "A full iteration over the dataset is called **epoch**. During an epoch, data must be used in a random order.\n",
    "\n",
    "If we apply this method we have some theoretical guarantees to find a good minimum:\n",
    "+ SGD essentially uses the inaccurate gradient per iteration. Since there is no free food, what is the cost by using approximate gradient? The answer is that the convergence rate is slower than the gradient descent algorithm.\n",
    "+ The convergence of SGD has been analyzed using the theories of convex minimization and of stochastic approximation: it converges almost surely to a global minimum when the objective function is convex or pseudoconvex, and otherwise converges almost surely to a local minimum."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  0   0\n",
      "  1   2\n",
      "  2   4\n",
      "  3   6\n",
      "  4   8\n",
      "  5  10\n",
      "  6  12\n",
      "  7  14\n",
      "  8  16\n",
      "  9  18\n",
      "\n",
      "  3   6\n",
      "  9  18\n",
      "  1   2\n",
      "  5  10\n",
      "  7  14\n",
      "  6  12\n",
      "  0   0\n",
      "  4   8\n",
      "  8  16\n",
      "  2   4\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "x = np.arange(10)\n",
    "y = np.array([2*i for i in x])\n",
    "data = list(zip(x,y))\n",
    "\n",
    "for (x_i,y_i) in data:\n",
    "    print('{:3d} {:3d}'.format(x_i,y_i))\n",
    "print(\"\")\n",
    "\n",
    "def in_random_order(data):\n",
    "    '''\n",
    "    Random data generator\n",
    "    '''\n",
    "    import random\n",
    "    indexes = [i for i,_ in enumerate(data)]\n",
    "    random.shuffle(indexes)\n",
    "    for i in indexes:\n",
    "        yield data[i]\n",
    "        \n",
    "for (x_i,y_i) in list(in_random_order(data)):\n",
    "    print('{:3d} {:3d}'.format(x_i,y_i))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import random\n",
    "\n",
    "def SGD(target_f, \n",
    "        gradient_f, \n",
    "        x, \n",
    "        y, \n",
    "        toler = 1e-6, \n",
    "        epochs=100, \n",
    "        alpha_0=0.01):\n",
    "    '''\n",
    "    Stochastic gradient descend with automatic step adaptation (by\n",
    "    reducing the step to its 95% when there are iterations with no increase)\n",
    "    '''\n",
    "    data = list(zip(x,y))\n",
    "    w = random.random()\n",
    "    alpha = alpha_0\n",
    "    min_w, min_val = float('inf'), float('inf')\n",
    "    epoch = 0\n",
    "    iteration_no_increase = 0\n",
    "    while epoch < epochs and iteration_no_increase < 100:\n",
    "        val = target_f(x, y, w)\n",
    "        if min_val - val > toler:\n",
    "            min_w, min_val = w, val\n",
    "            alpha = alpha_0\n",
    "            iteration_no_increase = 0\n",
    "        else:\n",
    "            iteration_no_increase += 1\n",
    "            alpha *= 0.95\n",
    "        for x_i, y_i in list(in_random_order(data)):\n",
    "            gradient_i = gradient_f(x_i, y_i, w)\n",
    "            w = w - (alpha *  gradient_i)\n",
    "        epoch += 1\n",
    "    return min_w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w: 2.000000\n"
     ]
    }
   ],
   "source": [
    "print('w: {:.6f}'.format(SGD(target_f, gradient_f, x, y)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise: Stochastic Gradient Descent and Linear Regression\n",
    "\n",
    "The linear regression model assumes a linear relationship between data:\n",
    "\n",
    "$$ y_i = w_1 x_i + w_0 $$\n",
    "\n",
    "Let's generate a more realistic dataset (with noise), where $w_1 = 2$ and $w_0 = 0$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Once deleted, variables cannot be recovered. Proceed (y/[n])? y\n"
     ]
    }
   ],
   "source": [
    "%reset\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets.samples_generator import make_regression \n",
    "from scipy import stats \n",
    "import random\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnMAAADZCAYAAABLqLavAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XlcVXXi//HXZUlRU3CLe7FSK0XN1FSUNAFFQdMUaHE0SbNyprGcNLWc9mYmG2vKfjXT8m1sdJyxjItm7oq4lZqaZYta4srigqKiiIj398eZC5Ioi9x77oX38/HoEZx7uOdz5xP6nrO8P5bDh086EBERERGv5GP2AERERESk8hTmRERERLyYwpyIiIiIF1OYExEREfFiCnMiIiIiXkxhTkRERMSLmRbm0tMPEhd3Fz17duXOO8P44IO/X7KPw+FgypSJhIV1ICIinO++21b02pw5s+nWrSPdunVkzpzZ7hy6iIiIiMfwM+3Afn689NKfue22juTmniI6uhcREb1p3Tq0aJ+VK5eRlrabjRu3sWXL10ya9CRLlqzi+PFjvP76ayxfnorFYiE6OoLY2AEEBgaZ9XFERERETGHambnrrgvmtts6AlCv3rW0atWazMyMEvssXryI++77DRaLhS5dwjhx4gSHDmWxatVKIiKiCApqSGBgEBERUaSkrDDjY4iIiIiYyiPumdu/fx/bt39H585dSmzPysrAZmtW9L3NFkJmZgaZmZmEhIRctN1GZmam28YrIiIi4ilMu8zqlJuby0MPjeCVV6Zy7bX1q/z9Z86cwaxZHwOwa9cOrr/++io/xsUuXLiAj49HZGT5H82JZ9K8eB7NiWfSvJgnL8+X3Fw/Tp3y5/x5CwB16hRSt+45AgMLsVhcuyLq0aNH2bFjb5n7mRrmCgoKeOihB0hIuI+BA+++5PXgYBsZGQeLvs/ISMdqtWG1Wlm/ft1F2zPo0aNnqcdITBxFYuIoAPr3j2Lz5s1V/ClKSk1NJTIy0qXHkIrRnHgmzYvn0Zx4Js2L+xQUQGoq2O2QnAyHDsE110BsLCQkwKBB0KiR++akY8fby7WfaVHf4XDwhz/8nlatWvO7340tdZ/Y2P58+ul/cTgcbN68ifr163PddcFERfVh9eoUcnKOk5NznNWrU4iK6uPmTyAiIiLeLi8PPv8cHnwQrrsO+vWDWbOgVy+YMweOHIEFC2DkSCPIeSLTzsxt3LiBuXPn0KZNO6KiegDwxz8+z8GDxpm4kSNHEx0dw4oVywgL60CdOnWYPt2oLwkKasj48ZPo1y8SgAkTJhMU1NCUzyEiIiLe5dQpWLQIkpKMf58+DUFBcPfdEB8PfftCQIDZoyw/08Jc9+7hHD588or7WCwWXnvtb6W+NmzYCIYNG+GKoYmIiIiHm/dNOtOW7iQjJw9bYAATY1ozpFPIZffPzjbOwNntsHw55OcbZ+JGjDACXGQk+Pu7b/xVyfQHIEREREQqYt436Txj305eQSEA6Tl5PGPfDlAi0GVmGve+2e3GvXCFhXDjjfDYY0aACw8HX18zPkHVUpgTERERrzJt6c6iIOeUV1DItKU76RAYgt1uBLivvgKHA1q3hsmTjQB3++1gMR5MrfDZPU+lMCciIiJeJSMnr8T3547W48yuYDJ3BdPyGWNbp07w8stGgGvb9tL3KO/ZPW+gMCciIiKVYtaZLWuDAPbs9OfMLitndgZz/lg9AK69MYe/vA5xcdCy5ZXf40pn9xTmREREpNpz95mtCxfgyy+Ny6e/zOnFkUw/sFyg9g3HqN95L0Ftj/D6g60Y0imwXO/367N7l9teWmAt3xHcR2FOREREKswdZ7YuLvGdNw+ysowS3379/Gje+TgbC7dzpOBUpc4K2gIDSC8l0NkCiztJLhdYR7TxJfJqP1wVUpgTERGRCivvma2KysszqkPsdqNK5PhxqFsXBgww7n8bMADq1wcIAnpV+jgTY1qXCGoAAf6+TIxpXfT95QJr0q4LTKn0kauewpyIiIhUWHnObJWXs8TXboeFC40S38DA4hLffv2qvsTXeRbvSvf8XS6YZp917ZqsFaUwJyIiIhVWnjNbV5KdbSyTZbfDsmVGiW/TpvDAA8Y6qO4o8R3SKeSKl2YvF1gb1ba4clgVpjAnIiIiFVaeM1u/lplp3PuWlFRc4nvDDdAvIZc91+7kVP0stjcMILZxa/z9zX+i9HKBNaGVZzUNK8yJiIhIpZR1Zgtgzx5jFYakpOIS31atYNIk4wzcfks6U5I9s+/tcoE18MTPpo7r1xTmREREpEr99JNx+TQpCb75xtjWsWNxiW+bNsWrMDwx1bP73koLrKmpCnMAjBv3GMuXL6Fx4yasWbPxktffeWc6SUmfAlBYeJ5du3by009pBAU1pHPnW6lXrx4+Pr74+fmxfPlqdw9fRERE/sfhMEJbUpIR4nbsMLaHh8PrZZT4uuqp2JrEtDA3dOhwRo9+lLFjx5T6+tix4xg7dhwAS5cu5v333yUoqGHR63b7Qho1auSWsYqIiEhJFy4Yl02dAW7fPmPR+shIePxxGDIEbLay36cqn4qtqUwLc+HhPdi/f1+59k1Onktc3D0uHpGIiIhcSUEBrF5tBLiSJb7wwgswaBA0blyx97zap2IFLIcPnzStLGX//n088MB9pV5mdTpz5gwdO4ayceO2ojNzXbq0p0GDQCwWC4mJo0hMHHXZn585cwazZn0MQFZWBnPmzKnSz/Brubm51KtXz6XHkIrRnHgmzYvn0Zx4JrPn5dw5H77+Ooi1a5vw5ZeNOHXKn9q1C+nWLZtevY7SrVs2desWlv1GV/BlRgFJuwrIPuugUW0LCa38ucPm4l6Sq+CuOfnDH8aX61Yyj38AYtmyxYSFdS9xiXXBgqVYrTaOHDnCvfcO5pZbWhEe3qPUn7847PXvH0VkZKRLx5uamuryY0jFaE48k+bF82hOPJMZ83Jxie+iRZCba5T4xsU5S3x9CQhoCjStkuNFgketqFAWT/td8fgwl5ycdMklVqvVuAjfpEkTBgwYyNatWy4b5kRERKRsx44Zy2f9usR3+HAjwEVGGpdUxfN4dJg7efIEX321jr///cOibadPn8bhuEC9etdy+vRpUlNTeOqpySaOUkRExDs5S3ztdli1qrjE93e/MwLcHXcYDzWIZzMtzI0ZM4r169dx7Fg2HTqEMmnSFAoKCgAYOXI0AIsWfUFkZG/q1q1b9HNHjhxm5MjhgFFZEh9/L71793X/BxAREfFCe/ca4c1uhy+/LFniGx8PnTsXd8CJdzAtzL3//owy9xk6dDhDhw4vsa158xakpn7pqmGJiIhUO84SX7sdtm41tnXsCC+9ZAS4tm0V4LyZR19mFRERkYpzlvg6A9xPPxnbw8Nh2jQjwF2uxFe8j8KciIhINeAs8XUGuL17jfvdIiLg9783SnxDzF8dS1xAYU5ERMRLOUt87XZjMXtniW/fvvDcc3D33RUv8RXvozAnIiLiRc6eheXLjQD3+edGpUidOjBgACQkGP+uX9/sUYo7KcyJiIh4uFOnYPFiYxktZ4lvgwbGmbeEBGM5rQAtZVpjKcyJiIh4oGPHYMEC+OCDW9mypbjEd9gwI8CpxFecFOZEREQ8RFaWUeKblFRc4nvddfVU4itXpDAnIiJior17jYcXkpJKL/E9dWoDUVGRZg9TPJjCnIiIiJvt2GGEt4tLfDt0KL3ENzXVtGGKl1CYExERcTGHA7ZtKw5wvy7xjYuDm24yd4zivRTmREREXODCBdiwoTjA7d0LPj7Ggwsq8ZWq5GPWgceNe4y2bVvSq1e3Ul9fv34tN93UjKioHkRF9eD116cWvZaSspzw8NsJC+vA22//zV1DFhERuaKCAlixAh57zAhqPXrAO+9Au3bw0Udw6BCsXGmEOQU5qSqmnZkbOnQ4o0c/ytixYy67T/fu4cyePbfEtsLCQiZPnsDcufOx2ULo1y+SmJgBtG4d6uohi4iIXOJKJb7x8ca/GzQwe5RSnZkW5sLDe7B//74K/9zWrZtp0aIlzZu3ACAuLoElSxYqzImIiNs4S3ztdli4sGSJb3w8xMSoxFfcx6Pvmdu8eRORkXcQHBzMiy/+mdDQNmRlZRIS0qxoH6vVxtatmy/7HjNnzmDWrI8ByMrKINXFjwXl5ua6/BhSMZoTz6R58Tyakys7edKPL79sxNq1Tfj664YUFPgQFHSOyMij3HnnETp1ysHf3wHAxo1Vd1zNi+fxtDnx2DB3220d2LLlB+rVq8eKFUt58MHfsHHjtgq/T2LiKBITRwHQv38UkZGRVTzSklJTU11+DKkYzYln0rx4Hs3JpZwlvna7UeJ7/jxcf71xT1x8PPTocQ2+vjbA5rIxaF48j6fNiceGuWuvLV4lODo6hsmTJ5CdnU1wsJX09INFr2VmZmC1uu6XSEREahZnia/dDuvXG7Uit9wCTz1lLKPVuXNxB5yIJ/DYMHfo0CGaNm2KxWJh69bNXLhwgYYNG9KgQWfS0tLYt28vVquN5OQk3nvvI7OHKyIiXmzHDiO8JSWVLPF98UUjwF1c4iviaUwLc2PGjGL9+nUcO5ZNhw6hTJo0hYKCAgBGjhzNF1/M4+OPP8LX14+AgNq8//4MLBYLfn5+TJ06jfvvj6OwsJBhw0YQGtrGrI8hIiJeyFni6wxwzhLf7t3hr381LqGqxFe8hWlh7v33Z1zx9dGjxzB6dOm1JdHRMURHx7hiWCIiUk05S3ztduOfPXuMEt+ICOMeuLg4db+Jd/LYy6wiIiJXq6AA1qwxwltyMmRmgr8/9O0Lzz5rVIk0bmz2KEWujsKciIhUK2fPGqswJCWVLPHt39+4/00lvlLdKMyJiIjXy801SnyTkkov8e3Xzwh0ItWRwpyIiHil48dhwQIjwC1dCvn50LQpDBtmBLioKLjmGrNHKeJ6CnMiIuI1Llfi+9vfOkt8wdfX7FGKuJfCnIiIeLR9+4qfQP11iW98PHTpog44qdkU5kRExOPs3GlcPrXbYcsWY5uzxDc+Htq1U4ATcVKYExGppuZ9k860pTvJyMnDFhjAxJjWDOnkmUVqF5f42u3w44/GdmeJb1wc3HyzuWMU8VQKcyIi1dC8b9J5xr6dvIJCANJz8njGvh3AYwLdlUp8f/c7GDIEmjUze5Qink9hTkSkGpq2dGdRkHPKKyhk2tKdpoa58+dh9erSS3z/+EejSqRJE9OGJ+KVFOZERKqhjJy8Cm13JWeJr90O8+eXLPGNj4e77rp8ia83XSoWMYtpYW7cuMdYvnwJjRs3Yc2ajZe8/tlnn/D//t9bgIO6devx17++ya23tgegc+dbqVevHj4+vvj5+bF8+Wo3j15ExLPZAgNILyW4+VgstHh6ocuDkbPE1243SnxPnTIC26BBRoCLiSm7xNcbLhWLeALTwtzQocMZPfpRxo4dU+rrN9zQnPnzFxEYGMTKlct46qknWLJkVdHrdvtCGjVq5K7hioh4lYkxrUsEIadChwNwTTBylvja7UaJ79mzxiXToUONANe7d8VKfD31UrGIpzEtzIWH92D//n2XfT0srFvR1507dyUjI8MdwxIRqRacYcd5idLHYikKck5VEYyysoxLp3Y7pKQY98Q1awaPPmqsg3o1Jb6edKlYxJN5xT1zs2fPok+fvkXfWywW7rtvCBaLhcTEUSQmjjJxdCIinmlIp5CioNbi6YWl7lOZYLRvn/HwQlJScYnvzTfDhAlGgKuqEt/LXSq2BQZc/ZuLVCMeH+bWrVvDf/4zkwULlhZtW7BgKVarjSNHjnDvvYO55ZZWhIf3KPXnZ86cwaxZHwOQlZVBamqqS8ebm5vr8mNIxWhOPJPmxb0a1raQfdZR6nbnPFxpTvbvD2Dt2iasWdOEXbuuBaBly1wefPAIvXodpXnz01gscPq08bRqVbjrhkI+PgnnLhRvu8bH2F6T/tvR74rn8bQ58egw98MP3/Pkk2OZMyeJhg2L74+zWm0ANGnShAEDBrJ165bLhrmLz9z17x9FZGSkS8ecmprq8mNIxWhOPJPmxb2ea5B+yT10Af6+PDe4PZH/O3t38Zw4HPDtt8bl06Sk4hLfbt0uLvGtB9QDWrhkzJFAWz3Nqt8VD+Rpc+KxYe7gwQOMGjWcd9/9kJtuuqVo++nTp3E4LlCv3rWcPn2a1NQUnnpqsokjFRHxfL++h660YHThAnz1VfEyWs4S3169zCvxvfhSsYiUzrQwN2bMKNavX8exY9l06BDKpElTKCgoAGDkyNG88cZrHD9+nMmTxxsD/V8FyZEjhxk5cjgAhYXniY+/l969+172OCIiYigtGJ0/D2vWGAHuk0/Cyc5Wia+ItzEtzL3//owrvv7mm+/w5pvvXLK9efMWpKZ+6aphiYhUe/n5RolvUhJ8/jlkZxudb126nGTMmCZXLPEVEc/jsZdZRUSk6uTmwpIlRoC7XInvpk0/eNR9QCJSPgpzIiIewBXLVlV1ia+IeCaFORERk1XlslWHDsG8eaWX+MbHQ8+elS/xFRHPpDAnImKyq122ylnia7fDunUlS3zj46Fr16op8RURz1ThMPfjjz/wzTdbOHz4EPn5ZwkMDOKmm26ma9duBAYGuWKMIiLVWmWWrdq50whvdjts3mxsu+02eOEFI8DdeqsCnEhNUa4wt3fvHj7++COSkj7lyJHD+Pj40KBBA665phYnTpwgL+8MPj4+3HFHT4YPT2TIkAR8fHxcPXYRkWqhPMtWXVzia7fDDz8Y27t1g9deM0p8b7nlkrcQkRqgzMT15JNj6dWrG99/v50JEyazcuU6Dhw4wk8/7eHbb3ewd28mP/6Yxr///Qlt2rTl5Zefp0ePLmzY8JU7xi8i4vUmxrQmwL/kjWwB/r5M6Nuar76CiRONy6adOsGf/2w8xPD223DgAGzYAJMmKciJ1GRlnpmrXbs269dv5vrrb7jsPo0aNaJPn3706dOPl19+lc8/TyYrK6NKByoiUl1dvDpD+rGz1Dtu5YaTt/D43fXIyDBKfKOj4ZlnYPBglfiKSEllhrlXX329Qm/o4+PDkCEJlR6QiEhNk58P/hkhhP4cws75sC8b0gKgf3/j/reBA1XiKyKXV6EHIJ59djKvvDIVi+6qFRG5Ks4SX7sdvvjCKPGtX98o8U1IMEp869Qxe5Qi4g0qFObs9s/Yv38f7733T+qU8qfMypXL6NOnX5UNTkSkOjl+3AhuSUnFJb6NG8P99xsBTiW+IlIZFXrkdPHilezZk8bdd8dy6FBW0faUlBX079+b4cPvq/IBioh4s0OH4IMPjDNtTZtCYiJs2WKU+KamQmYmfPghxMYqyIlI5VQozN14Y3MWLlxOw4YNiYmJ4j//mcVdd/XlN79JoF69a5k3b1GFDj5u3GO0bduSXr26lfq6w+FgypSJhIV1ICIinO++21b02pw5s+nWrSPdunVkzpzZFTquiIgr7d8P06dDr15gtcKYMZCWZpT4btxolPxOnw4REeDngur2ed+k02NqCi2eXkiPqSnM+ya96g8iIh6jwn+M1K/fgMcee4KRI4czfvzjtGvXnkWLVtC5c9cKH3zo0OGMHv0oY8eOKfX1lSuXkZa2m40bt7Fly9dMmvQkS5as4vjxY7z++mssX56KxWIhOjqC2NgBKi0WEdPs2mXc/5aUVFzi2769+0t8q3JpMBHxDhU6M5eSspxBg2IYOjSerl3DSEwcxa5dO9izJ61SBw8P73HFALZ48SLuu+83WCwWunQJ48SJExw6lMWqVSuJiIgiKKghgYFBREREkZKyolJjEBGpDGeJ7/PPG0GtdWujOsTHxyjx3bULvvvOCHPt27tvNYYrLQ0mItVThc7M/eY399CzZy/mzVtM9+7hANx6622MG/cYu3f/wuTJf6zSwWVlZWCzNSv63mYLITMzg8zMTEJCQi7abiMzM7PU95g5cwazZn1c9H6pqalVOsZfy83NdfkxpGI0J57JG+flwgXYsaM+a9Y0Zu3aJmRkBODj4+C223J4/PGj9Ox5lKZN8wFITzf+cbfSVpJwbi/rf29vnJOaQPPieTxtTioU5ubNW0R4eI8S2xITR3HDDTfyyCMjSUv7hfffn1GlA7xaiYmjSEwcBUD//lFERka69HipqakuP4ZUjObEM3nLvJw/D2vXGpdPk5MpUeL70ktw990WmjYNAoIA85dhCNmQUmqgCwkMKPN/b2+Zk5pG8+J5PG1OKnSZ9ddBzikysjcLFixly5YtVTIop+BgGxkZB4u+z8hIx2q1YbVaSb/o//JmZGRgtVqr9NgiUnPl58PChTB6NAQHG5Uh//wndO8O//43HD4MixbBww8bT6h6ksstDTYxprVJIxIRV6tQmLuS0NA2LFmSUlVvB0BsbH8+/fS/OBwONm/eRP369bnuumCiovqwenUKOTnHyck5zurVKURF9anSY4tIzXL6NHz2GQwbZiyXNXCg8X1srHFW7sgR49/Dh0NgoNmjvbwhnUJ4Nb49IYEBWDDOyL0a314PP4hUY2VeZv300/+SkHAfvr6+Ze1K48aNAUhL283hw4fo3v2OK+4/Zswo1q9fx7Fj2XToEMqkSVMoKCgAYOTI0URHx7BixTLCwjpQp04dpk//OwBBQQ0ZP34S/fpFAjBhwmSCghqWOT4RkYs5S3ztdmM1hotLfOPjoU8f7+x+G9IpROFNpAYpM8y99967vPban7n33vsZOHAIt97avtT9jh3LJiVlBfPmJbFu3VqmT3+3zIOXdX+dxWLhtdf+Vuprw4aNYNiwEWUeQ0TkYocOwfz5RoBbudK4Jy4kBB55xAhwPXu6pvtNRMRVyvwja+7c+axbt5oPP3yPN998nbp169GqVSsaNmzENdfU4uTJE+zfv4+DBw8QGBjIPffcz7Rpb2G12twxfhGRMu3fbzy8YLcbDzM4HHDTTTB+vBHgunY1KkVERLxRmWGuXbubWLIkhS++WMaGDV+ya9dOvv12G0eOHOLMmdM0adKE7t3vICysOz163Im/v787xi0ickXOEl+7Hb7+2tjWvr3RC5eQ4L4SXxERVyszzNWuHcC5c8Z9bEOGDGDx4pVFVR8iIp7C4TBKep0B7vvvje1hYTB1qnEG7hbzm0NERKpcmWEuNDSUmTP/ScuWN+FwONwxJhGRcrlwATZtKg5wu3cbl0vvvBPefhuGDIHrrzd7lCIirlVmmHvhhT8xevQIPvvsEywWCy+++Cxdu3ajbdt2tG17K7fc0qpcT7qKiFQFZ4mv3W7cB5eebpT49ukDTz8Nd9/ted1vIiKuVGaYCw/vwbff7mTdujUMHRpPrVq1SElZwfvvv0tBQQH+/v7cfHMr2rZtR7t27XnsscfdMW4RqUHy840nT+1240nUo0chIMDogEtIgLvu8uzuNxERVyrXA/j+/v5ERfUhLKw7L7zwJ9q1u5XCwkJ+/nkXP/74PT/88D0//vg9H3zwd4U5kRpq3jfpTFu6k4ycPGyBAUyMaX1VXWenTxvdb0lJRhfcqVNQv75R5puQADExULduFX4AEREvVaE2pQULlhZ97evrS2hoG0JD2xAff2+VD0xEvMe8b9J5xr6dvIJCwFjU/Rn7doAKBbqcHCO4JSWVXuLbuzfUquWSjyAi4rVUjSkiV23a0p1FQc4pr6CQaUt3lhnmDh82Lp0mJanEV0SkMvRHpIhctYycvAptP3AAPvsshBdegHXrjKdSPaXEt6ovF4uIuJrCnIhcNVtgAOmlBDdbYEDR1z//bJx9Ky7xvYX27eG554wA1769+SW+VXW5WETEnUwNcykpy/njHydTWFjIAw88yBNPjC/x+nPPPc26dWsByMs7w9GjR/nllwMABAcH0qZNOwCaNWvGrFmfuHfwIlJkYkzrEiEIoLafL/e2aMcLL5Re4muzbWTEiG4mjbh0V3O5WETELKaFucLCQiZPnsDcufOx2ULo1y+SmJgBtG4dWrTPK69MLfr6//7vPbZv/67o+9q1A1i1ar1bxywipXMGnb8u3smen2rhs78Z+WkhPHnQr6jEd/p0iIsrLvFNTS39EqyZKnq5WETEE5gW5rZu3UyLFi1p3rwFAHFxCSxZsrBEmLuY3f4ZkyZNcecQRaQcnCW+K+0h7E8OIeuiEt/452HwYO8p8S3P5WIREU9j0i3GkJWVSUhIs6LvrVYbmZkZpe574MB+9u/fx513RhRty88/S9++EfTv35tFi75w+XhFpFh+PixaBA8/DFarURny0UfGJdRZs4wnVBcvNp5I9ZYgB8bl4gD/kivaBPj7MjGmtUkjEhEpm1c8AJGcnMSgQYNLLBu2desPWK029u7dQ0LCINq0aUuLFi0v+dmZM2cwa9bHAGRlZZCamurSsebm5rr8GFIxmpOqkZfnw6ZNDVm7tgkbNjTi9Gk/6tY9T/fu2dx55xHCwo4REHABgG3byn4/T5yXQGBEG1+Sdl0g+6yDRrUtJLTyJfDEz6Sm/mz28FzOE+dENC+eyNPmxLQwFxxsJT39YNH3mZkZWK22UvedNy+JqVPfKLHNuW/z5i24446efP/9d6WGucTEUSQmjgKgf/8oIiMjq+gTlC41NdXlx5CK0ZxUnrPE1243Snzz8qBRIxg61HgCtU8fP2rVug64rsLv7anzEgnU1Bs6PHVOajrNi+fxtDkxLcx16tSZtLQ09u3bi9VqIzk5iffe++iS/X7+eRcnTuTQtWtY0bacnOMEBNShVq1aZGdns2nTBsaO/YM7hy9SbTlLfO12o8S3oMAo8R092lhGSyW+IiKexbQ/kv38/Jg6dRr33x9HYWEhw4aNIDS0DVOn/omOHW8nNnYAAMnJnzFkSAKWiwqodu3axcSJ47BYfHA4LvDEE+Mv++CEiJTtwAFITjYC3Nq1Rolvy5bwhz8YAc7MEl8REbkyU///dXR0DNHRMSW2Pf30syW+L+0J1rCwbqxevcGlYxOp7n7+2QhvSUnOEl+49VZ49lkjwHlCia+IiJRNF0tEvFBllpxyOGD79uIA5yzx7drVKPGNi4NWrdwweBERqVIKcyJepiJLTl24YJx1s9uNf375xTjb5izxHTIEbrjB7R9D32V4AAAUh0lEQVRBRESqkMKciJcpa8mp8+eNxeudAS493XhgoU8fmDQJ7r4brqv4w6ciIuKhFOZEvExpS0s5zvvwy5Z6PPyw8STq0aMQEACxsUaFyMCBEBhowmBFRMTlFOZEvIxzyakL53w5u6cJp3cFk/dLUxzn/Pn0Whg0yAhwsbFQt67ZoxUREVdTmBPxIjk50PV8R76bd47Tu5vgOO+LT8A56rfN4vGHAnj20cbUqmX2KEVExJ0U5kQ83KUlvg1p2KSQel0zcNyYTstb85g0oBVDOjU2e6giImIChTkRD1SyxNfBhQsW/AJP0zT8KE88FMDEEU3x8bkeuN7soYqIiMkU5kQ8hLPE126HTZuMbTfcXEBQj31cc3MG/k1OYbHAx7/40vrb9mX2yomISM2gMCdikotLfO1242swSnxffdV4iGGUfS3pv3p69eIaEleqTDGxiIi4n8KciBtdqcT3rbeMVRguLvEtrYbkSturSkWKiUVExFwKcyIudnGJb3IyHDxYXOI7cSIMHnz5El9nDUlp212prGJiERHxHD5mHjwlZTnh4bcTFtaBt9/+2yWvz5kzmzZtWhAV1YOoqB78+9//KvFat24d6datI3PmzHbnsEXKlJ8PixfDI4+A1QpRUfDhh9ClC8ycaTyhumQJPProlVdjmBjTmgB/3xLbAvx9mRjT2qXjN+uMoIiIVJxpZ+YKCwuZPHkCc+fOx2YLoV+/SGJiBtC6dWiJ/QYPjmfq1DdKbDt+/Bivv/4ay5enYrFYiI6OIDZ2AIGBQe78CCIlnD4NS5caZ+AWLICTJ+Haa43VFxISKlfi6zwL5u5718w6IygiIhVnWpjbunUzLVq0pHnzFgDExSWwZMnCS8JcaVatWklERBRBQQ0BiIiIIiVlBfHx97p0zCK/duIEfPGFEeAWL4a8PGjUCO65xwhwffpw1SW+QzqFuP3S5sSY1iXumQP3nBEUEZGKMy3MZWVlEhLSrOh7q9XG1q2bL9nviy8+56uvvuSmm27mlVdeJSSkGZmZmYSEFP/lZrPZyMzMLPU4M2fOYNasj/93zAxSU1Or9HP8Wm5ursuPIRVT1XOSk+PP+vWNWbu2MVu2BHH+vA+NG+cTE3OUXr2OcNttJ/D1dQDw1VdVdli3CgRGtPEladcFss86aFTbQkIrXwJP/Exq6s9Vcgz9rngezYln0rx4Hk+bE49+AKJfv1ji4u6hVq1a/Otf/+Txx3+L3f5Fhd4jMXEUiYmjAOjfP4rIyEgXjLRYamqqy48hFVMVc3LwoPHwQlISrF1rPJXasiU8+aRRIRIWVgsfnxCg+jwcEAlMceH763fF82hOPJPmxfN42pyY9gBEcLCV9PSDRd9nZmZgtdpK7NOwYSNq/e8a1QMPPMi3324DwGq1kp6eXrRfRkYGVqvVDaOWmuSXX+Cvf4Vu3eD66+GJJ+DoUXj2Wdi2rfj17t3Bx9RHiUREpCYz7a+gTp06k5aWxr59ezl37hzJyUnExAwosc+hQ1lFXy9ZsohWrVoBEBXVh9WrU8jJOU5OznFWr04hKqqPW8cv1Y+zxPell+C22+CWW2DyZOMs3Kuvwo4d8P33xusdOhj9cCIiImYz7TKrn58fU6dO4/774ygsLGTYsBGEhrZh6tQ/0bHj7cTGDuDDD99j6dJF+Pr6ERQUxNtvvwdAUFBDxo+fRL9+kQBMmDC56GEIkYpwOIwS36Sk8pX4VoZWUhAREVcy9Z656OgYoqNjSmx7+ulni75+9tkXefbZF0v92WHDRjBs2AhXDk/cyJ2Bp7DQKPFNSqp4iW9FaSUFERFxNY9+AEJqBncEnnPnYOVK4+zb/Plw5AjUrm10v/3lL0YXXJALagq1koKIiLiawpyYzlWB58wZY5WFf/yjDZs2lSzxjY83gly9elc7+ivTSgoiIuJqCnNiuqoMPKWV+Nav35B77jECXJ8+xhk5d9FKCiIi4moKc2K6qw08/0rJ5JV3c8jY1oiz+xrjKPTBZoOHHjICnMPxJX36RFT1sMtFKymIiIirKcyJ6SoTeJwlvh/MzOf7LcHgsOLX4AzXdt5DUNsj/G3s9cR3Ni7RpqY6XP4ZLsestVVFRKTmUJgT05U38Pzyi3H51G6HjRuNbQHXFdAgfD91Wmfi3+RUUffbG8vPFIU5s5mxtqqIiNQcCnPiEUoLPA6HUdLrDHDffWds79LFeAI1Ph5iZ6ymtPNuesBARERqCoU58SjOEl9ngPv5Z6PEt2dPePNNo8T3xhuL99cDBiIiUtMpzInpnCW+zgDnLPHt3RueeurKJb56wEBERGo6hTkxxblzkJJihLd584pLfGNiKlbiqwcMRESkplOYE7c5cwaWLjUC3IIFRidcvXpGcEtIqHyJrx4wEBGRmszUMJeSspw//nEyhYWFPPDAgzzxxPgSr//jH+8we/a/8PX1o3Hjxrz11rtcf72x6nlwcCBt2rQDoFmzZsya9Ynbxy9lO3ECFi401kF1lvg2bGg8vJCQ4P4SXxERkerGtDBXWFjI5MkTmDt3PjZbCP36RRITM4DWrUOL9mnf/jaWLVtNnTp1mDHj/3j55ef58MOPAahdO4BVq9abNHq5kiNH4PPPjQC3YgUUFIDVWlzi26uXcU+ciIiIXD3T/krdunUzLVq0pHnzFgDExSWwZMnCEmGuZ89eRV936dKVzz7T2TdPdfCgce9bUhKsWQMXLkCLFjBunBHgunUDHx+zRykiIlL9mBbmsrIyCQlpVvS91Wpj69bNl91/9uyZ9OnTt+j7/Pyz9O0bgZ+fL48/Pp4BAwaW+nMzZ85g1qyP/3fMDFJTU6tk/JeTm5vr8mN4ivT0ANasaczatU346af6ANx442mGDz9Cr15HuemmXCwWyM83Ap5ZatKceBPNi+fRnHgmzYvn8bQ58YqLXXPnzuHbb79h3rzFRdu2bv0Bq9XG3r17SEgYRJs2bWnRouUlP5uYOIrExFEA9O8fRWRkpEvHmpqa6vJjmMXhgB9+MM6+lVbiGxcHoaF1gbpAcxNHWlJ1nhNvpnnxPJoTz6R58TyeNiemhbngYCvp6QeLvs/MzMBqtV2y3+rVq3jrrdeZN28xtWrVKtru3Ld58xbccUdPvv/+u1LDnFwdhwM2by4OcGWV+IqIiIh7mRbmOnXqTFpaGvv27cVqtZGcnMR7731UYp/t27/lqafG8ckndpo0aVK0PSfnOAEBdahVqxbZ2dls2rSBsWP/4O6PUG1dXOKbnAwHDhSX+E6YYJT4BgebPUoREREBE8Ocn58fU6dO4/774ygsLGTYsBGEhrZh6tQ/0bHj7cTGDuDFF5/j9OnTjB79IFBcQbJr1y4mThyHxeKDw3GBJ54YX+LBCam4K5X4/ulPMGhQ+Up8RURExL1MvWcuOjqG6OiYEtuefvrZoq+Tkj4v9efCwrqxevUGl46tJrhSiW98PPTvX7kSXxEREXEfr3gAQqqOs8TXbjdKfM+cKS7xjY+H6GiV+IqIiHgThbkawFnia7cbJb7nzhklviNHGqswqMRXRETEe+mv8GoqPd14eMFuh9Wri0t8H3/cCHAq8RUREakeFOaqkd27jfBmt8OG/91S2LYtTJliBLgOHYxaEREREak+FOa8mLPE1243euCcJb6dO8Of/2zcAxeqh3xFRESqNYU5L+Ms8XUGOGeJb48eKvEVERGpiRTmvEBhIaxfX3wJ1VniGxWlEl8REZGaTmHOQ507B6tWGWffnCW+tWoVl/gOHGhUioiIiEjNpjDnQc6cgWXLjACnEl8REREpD4U5k508aZT4JiWpxFdEREQqTmHOBEePwvz5pZf4xscbJb7+/qX/7Lxv0pm2dCcZOXnYAgOYGNOaIZ1C3Dp+ERER8Rym1sampCwnPPx2wsI68Pbbf7vk9fz8fB55ZCRhYR2IjY1i//59Ra9Nn/4GYWEdCA+/nZSUFe4cdqWkp8M770Dv3nDddfDww/Djj0aJ7/r1cPAgvPsu9Olz5SD3jH076Tl5OID0nDyesW9n3jfpbv0sIiIi4jlMOzNXWFjI5MkTmDt3PjZbCP36RRITM4DWrYuL0WbPnkmDBoFs2vQtycmf8corL/Dhhx+zc+cOkpOTWLt2E1lZmdxzz91s2PANvr6+Zn2cUl2pxDc+Hjp2rFiJ77SlO8krKCyxLa+gkGlLd+rsnIiISA1l2pm5rVs306JFS5o3b8E111xDXFwCS5YsLLHPkiULuf/+3wAwaNAQ1q5NxeFwsGTJQuLiEqhVqxY33ticFi1asnXrZjM+RgkOB+zZU5eXXzaC2s03w6RJUFBglPj+9JNR8vvKK9CpU8VXY8jIyavQdhEREan+TDszl5WVSUhIs6LvrVbbJYHs4n38/Py49tr6HDt2jMzMDDp37lq0n80WQlZWpnsGfhkOB9xxB2zY0LWoxPdvfzNKfJs3r5pj2AIDSC8luNkCA6rmACIiIuJ1qv0DEDNnzmDWrI8B+PnnXXTseLtLj9es2VEaNWrMqVPwr38Z/1SVE2cLyDpxFsdF2yyApUFtOs65zI12Qna2MSfiWTQvnkdz4pk0L57HXXNy4MC+snfCxDAXHGwlPf1g0feZmRlYrbZS97HZQjh//jynTp2kYcOGWK02MjKKb/rPyEgnONha6nESE0eRmDjKNR+iFH37RrB8+Wq3HU/KpjnxTJoXz6M58UyaF8/jaXNi2j1znTp1Ji0tjX379nLu3DmSk5OIiRlQYp+YmAF88sl/AViwYB49e0ZgsViIiRlAcnIS+fn57Nu3l7S0NG6/vYsZH0NERETEVKadmfPz82Pq1Gncf38chYWFDBs2gtDQNkyd+ic6dryd2NgBDB+eyO9//yhhYR0ICgri/fdnABAa2obBg+Po2bMrfn5+vPba6x73JKuIiIiIO5h6z1x0dAzR0TEltj399LNFX9euXZuPPppZ6s8++eREnnxyokvHVxkjRow0ewjyK5oTz6R58TyaE8+kefE8njYnlsOHTzrK3k1EREREPJGpK0CIiIiIyNVRmKukq1mKTFyjrDn5xz/eoWfPrkREhJOQMIgDB/abMMqap6x5cVqwYD5Nm9Zn27atbhxdzVSeOZk/307Pnl25884wfvvbh9w8wpqprHk5ePAAcXF30bt3TyIiwlmxYqkJo6w5xo17jLZtW9KrV7dSX3c4HEyZMpGwsA5ERITz3Xfb3DzCYgpzleBciuy//01i3bqvsds/Y+fOHSX2uXgpsjFjfs8rr7xg0mhrhvLMSfv2t7Fs2WpWr/6KgQMH8/LLz5s02pqjPPMCkJt7ig8//AedO+updFcrz5ykpf3C9Ol/44svlrF27SZeeeU1k0Zbc5RnXt58cxp33x1HSso6PvhgBpMnTzBptDXD0KHDmTPHftnXV65cRlrabjZu3MYbb0xn0qQn3Ti6khTmKuFqliIT1yjPnPTs2Ys6deoA0KVL1xJdheIa5ZkXgKlT/8Tjj/+BWrVqmzDKmqU8czJr1r946KFHCAwMAqBJkyZmDLVGKd/vioVTp04BcPLkCa67Ltj9A61BwsN7FP0OlGbx4kXcd99vsFgsdOkSxokTJzh0KMuNIyymMFcJpS1FlpmZcdl9Ll6KTFyjPHNysdmzZ9KnT193DK1GK8+8fPfdNtLT0+nbN9bdw6uRyjMnu3f/wu7dv3DXXX3p3783KSnL3T3MGqc88zJp0jMkJX1Chw6hDBt2L6++Os3dw5SLZGVlYLMVz5nNFnLFv3dcqdov5yXya3PnzuHbb79h3rzFZg+lxrtw4QLPPz+Ft9/+h9lDkYsUFp4nLW038+YtIiMjncGD+7N69Vc0aBBo9tBqNLv9M+6/fziPPfY4X3+9kd///lHWrNmIj4/Oy9R0+i+gEiqyFBlQYikycY3yzAnA6tWreOut15k58xNq1arlziHWSGXNS27uKXbs+JG4uLvo3PlWtmz5mhEjhuohCBcqz++K1RpCbOwA/P39ufHG5tx0082kpe1291BrlPLMy3/+M5PBg+MA6Nq1G2fP5pOdne3WcUqx4GAbGRnFc5aRkV7q3zvuoDBXCVezFJm4RnnmZPv2b3nqqXHMmjVH9wC5SVnzUr9+A3bs2MuWLd+zZcv3dO7clVmz5tCx4+0mjrp6K8/vyoABd7F+/VoAsrOz2b37F268sbkJo605yjMvISHNWLvWWA90166d5OefpXFj1y/2LqWLje3Pp5/+F4fDwebNm6hfv75p9zHqMmslXM1SZOIa5ZmTF198jtOnTzN69IMANGvWjFmzPjF55NVbeeZF3Ks8cxIVFc2qVSn07NkVX19fXnjhFRo2bGT20Ku18szLSy/9hfHjH+e9997FYrHw9tv/0EkCFxozZhTr16/j2LFsOnQIZdKkKRQUFAAwcuRooqNjWLFiGWFhHahTpw7Tp//dtLFqBQgRERERL6bLrCIiIiJeTGFORERExIspzImIiIh4MYU5ERERES+mMCciIiLixRTmRERERLyYwpyIiIiIF1OYExEREfFiCnMiIhX05ZfraNq0PitWLC3atm/fXtq2bcmUKRNNHJmI1ERaAUJEpBLi4weSn5/PwoXLOXnyBHfd1ZcbbriRmTPn4Ovra/bwRKQG0Zk5EZFKmDRpCl9/vZFVq1by8MMP4ufnz/vvz1CQExG305k5EZFKuueewWzevJH69RuwZEkKNluI2UMSkRpIZ+ZERCqpRYuWnDlzhkmTpijIiYhpFOZERCph5swZ/Pe/s2jXrj2zZ880ezgiUoMpzImIVFBqagpPPz2BN954m2nT3mTLlq9ZuXKZ2cMSkRpK98yJiFTAjh0/MXBgP0aPfoRnnnkeMO6dy809yZIlq0wenYjURDozJyJSTkeOHOGBB+4jMrI3Tz/9XNH2CRMmsXXrFpYvX2Li6ESkptKZOREREREvpjNzIiIiIl5MYU5ERETEiynMiYiIiHgxhTkRERERL6YwJyIiIuLFFOZEREREvJjCnIiIiIgXU5gTERER8WIKcyIiIiJe7P8DFdSR1ifGI+MAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95df5b2ba8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# x: input data\n",
    "# y: noisy output data\n",
    "\n",
    "x = np.random.uniform(0,1,20)\n",
    "\n",
    "# f = 2x + 0\n",
    "def f(x): return 2*x + 0\n",
    "\n",
    "noise_variance =0.1\n",
    "noise = np.random.randn(x.shape[0])*noise_variance\n",
    "y = f(x) + noise\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.xlabel('$x$', fontsize=15)\n",
    "plt.ylabel('$f(x)$', fontsize=15)\n",
    "plt.plot(x, y, 'o', label='y')\n",
    "plt.plot([0, 1], [f(0), f(1)], 'b-', label='f(x)')\n",
    "plt.ylim([0,2])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Complete the following code in order to:\n",
    "+ Compute the value of $w$ by using a estimator based on minimizing the squared error.\n",
    "+ Get from SGD function a vector, `target_value`, representing the value of the target function at each iteration.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "ename": "IndentationError",
     "evalue": "expected an indented block (<ipython-input-58-f770ca9d719f>, line 6)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-58-f770ca9d719f>\"\u001b[0;36m, line \u001b[0;32m6\u001b[0m\n\u001b[0;31m    def gradient_f(x,y,w):\u001b[0m\n\u001b[0m      ^\u001b[0m\n\u001b[0;31mIndentationError\u001b[0m\u001b[0;31m:\u001b[0m expected an indented block\n"
     ]
    }
   ],
   "source": [
    "# Write your target function as f_target 1/n Sum (y - wx)**2\n",
    "def target_f(x,y,w):\n",
    "    # your code here\n",
    "\n",
    "# Write your gradient function\n",
    "def gradient_f(x,y,w):\n",
    "    # your code here\n",
    "\n",
    "def in_random_order(data):\n",
    "    '''\n",
    "    Random data generator\n",
    "    '''\n",
    "    import random\n",
    "    indexes = [i for i,_ in enumerate(data)]\n",
    "    random.shuffle(indexes)\n",
    "    for i in indexes:\n",
    "        yield data[i]\n",
    "\n",
    "# Modify the SGD function to return a 'target_value' vector\n",
    "def SGD(target_f, \n",
    "        gradient_f, \n",
    "        x, \n",
    "        y, \n",
    "        toler = 1e-6, \n",
    "        epochs=100, \n",
    "        alpha_0=0.01):\n",
    "    \n",
    "    # Insert your code among the following lines\n",
    "    \n",
    "    data = zip(x,y)\n",
    "    w = random.random()\n",
    "    alpha = alpha_0\n",
    "    min_w, min_val = float('inf'), float('inf')\n",
    "    iteration_no_increase = 0\n",
    "    epoch = 0\n",
    "    while epoch < epochs and iteration_no_increase < 100:\n",
    "        val = target_f(x, y, w)\n",
    "        if min_val - val > toler:\n",
    "            min_w, min_val = w, val\n",
    "            alpha = alpha_0\n",
    "            iteration_no_increase = 0\n",
    "        else:\n",
    "            iteration_no_increase += 1\n",
    "            alpha *= 0.95\n",
    "        for x_i, y_i in in_random_order(data):\n",
    "            gradient_i = gradient_f(x_i, y_i, w)\n",
    "            w = w - (alpha *  gradient_i)\n",
    "        epoch += 1\n",
    "    return min_w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'SGD' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-59-2c273941b2f4>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Print the value of the solution\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget_value\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mSGD\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtarget_f\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgradient_f\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'w: {:.6f}'\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'SGD' is not defined"
     ]
    }
   ],
   "source": [
    "# Print the value of the solution\n",
    "\n",
    "w, target_value = SGD(target_f, gradient_f, x, y)\n",
    "print('w: {:.6f}'.format(w))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'w' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-60-a683d115b3ae>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'o'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m't'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'b-'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'f(x)'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0malpha\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0.5\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'r-'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'fitted line'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0malpha\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0.5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlinestyle\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'--'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      8\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mxlabel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'input x'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      9\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mylabel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'target t'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'w' is not defined"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl4lOW9//F3JAESIYQ9C8qisogS1ggSBYSQhEWJKCAIAv6UVuvP9liw2uu09eg50otTe7S2av1ZLdRdkCJbCKBhUZaEXSUKkW2yAFlIgBCSyfz+uA/QSJKZJDPzZGY+r+vykmSe5Pk+xHy8537u53sHnTxZ4kBERPzKNVYXICIi7qdwFxHxQwp3ERE/pHAXEfFDCncRET+kcBcR8UMKdxERP6RwFxHxQwp3ERE/FGzVifv06U63bt2sOr2IiE/64YcfOHjwiNPjLAv3bt26kZGRYdXpRUR8Uv/+A106TtMyIiJ+SOEuIuKHFO4iIn5I4S4i4ocU7iIifshpuNtsJ0hJGU98/BDuuCOOv/71L1cd43A4ePbZ+cTFxTJixDD27dvjkWJFRMQ1TpdCBgcH89xz/0m/fv05e7aUMWPuZMSIu+jVq/flYzZsWEd29mG2b99DZuZOFiz4BWvXfu7RwkVEpHZOR+6dO0fSr19/AFq1ak3Pnr3Izc2pdsyaNauZMuUBgoKCGDw4jjNnzpCfn+eZikVEfJTdDlu2gM3m+XPV6yGmY8eOsn//PgYNGlzt83l5OURHd7n8cXR0DLm5OXTuHFntuMWL32bJkncAKCoqaGDJIiK+58QJ+OwzyM+H+HiIifHs+VwO97NnzzJ37kyef34hrVuHN+hks2bNYdasOQAkJ49q0PcQEfEl5eWwcSPs2AGtW8O0adC7t/OvayyXwr2iooK5cx9k8uQpTJhw91WvR0ZGk5Nz4vLHOTk2oqKi3VeliIgPysqCVaugtBSGDIHRo6FFC++c2+mcu8Ph4Oc/f5yePXvx05/+rMZjkpKS+eij93E4HGRk7CA8PPyqKRkRkUBRWgoffQTvvw8tW8LDD8O4cd4LdnBh5L59+zY+/vgD+vTpy6hRwwH49a9/w4kTZqQ+e/bDjBmTyPr164iLiyUsLIyXX756uaSIiL9zOCAzE9LSzM3T0aPh9tuhWTPv1+I03IcOHcbJkyV1HhMUFMTvf/+S24oSEfE1J0+aG6bHj0P37jBhArRvb109lrX8FRHxFct321iUmkVOcRnREaHMT+zFpAFmuUtlJWzebJY4Nm8OkyZBbCwEBVlbs8JdRKQOy3fbeGbZfsoq7ADYist4Ztl+APq3jeGzz6CgAPr1g8REuPZaK6u9QuEuIlKHRalZl4P9knPnHTzzxwKm9oihbVuYORNuuMG8Vtco35sU7iIidcgpLrv8Z4cDKk63oiy7E6WV1zB8JowcCSEh5vW6RvneDnh1hRQRv7V8t43hCzfS/VerGL5wI8t31/+5/+iIUACqLgRz7ptozmdFcU2LCm688xQJCVeCHWoe5ZdV2FmUmtWo62gIhbuI+KVLo2hbcRkOroyi6xvwTyX0gvz2lO7qhr0klNDup+g4OIdf39f9qmP/dZTvyuc9SeEuIn7JHaPo3Fw4lRHD0OY30zG6kvCBR+nRt5yFk2+tcZrl0ij/xyLCQhr9DqK+NOcuIn6pMaPoixfhiy/gq6/M6pd//7/h3HxzX4KC+tb5dfMTe1WbcwcIaRbE2QuVFJ2vALw3D69wFxG/FB0Riq2GIK9tdH3J99+bfjDFxTBoEIwZA6F1f8lll8L6X1fLnCuvpLisotpxl95BKNxFROqpplF0aEgz5if2qvH4s2chNRX274cOHWDOHOjatf7nnTQgplpod//VqhqP8/Q8vMJdRPxSTaPomtacOxywZw+sW2emY0aONP3Wg92Ujg19B9FYCncR8Vs/HkX/WEGB6Qdz5IgZpU+YAB07ureG+r6DcBeFu4gEHLsdtm6FTZvMCH3iRDgWZGPSW+5/stTVdxDupnAXkYBy/DisWAGnTkHfvpCcDOu/t/GsB58sdfYOwhMU7iISEC5cgA0bICMDwsNh+nTo2dO8VteaeCv6wriDwl1E/N6338Lq1WZFzG23wV13mfa8lzSlJ0vdReEuIn6rpMSE+sGDEBlpNqeOqWEgbtWKFk9SuIuI36mqMtMvGzaYPyckwNChtW93Z9WKFk9SuIuIX8nPN8sbT5wwPdYnTIC2bev+GqtWtHiSwl1E/EJFhVnauHUrtGwJ994Lt97q+nZ3Vqxo8SSFu4j4vOxsWLkSCguhf38YOxbCwqyuyloKdxHxWefPm7YBe/ZAu3bw0EPQ/eo26wFJ4S4iPsfhMA2+1q4169fvuAPuvLP6rkiBTuEuIj6lqMhMwRw+DF26mNYBnTtbXVXTo3AXEZ9gt8O2bWYTjWuugXHjYPBg82e5msJdRJo8m80sb8zLg969TbCHh1tdVdOmcBeRJuviRdi4EbZvh1atYOpU6NPH6qp8g8JdRJqk774z292VlJjpl9Gjzfp1cY3CXUSalLNnYc0a+Ppr6NQJ5s6F666zuirfo3AXkSbB4YBduyAtDSorTefG4cNr7wcjdVO4i4jlTp0yyxuPHoVu3czyxvbtra7KtyncRcQylZWwZQts3mz6q99zj2kf4Go/GKmdwl1ELHH0qFneePq0afCVlATXXmt1Vf5D4S4iXlVWBuvXQ2YmRETAgw/CjTdaXZX/UbiLiFc4HPDNN2YlzLlzcPvtMHJk9e3uxH0U7iLicWfOmDXr330HUVEwY4b5t3iOwl1EPKaqCnbsME+ZOhyQmGg2qFY/GM9TuIuIR+TlwYoVkJMDN90E48ebOXbxDoW7iLhVRYXp3PjVVxAaCvfdB337anmjtzkN9yeffIy0tLV06NCRTZu2X/X61q2bmTXrAa6/visA48dP5Je//JX7KxWRJu/wYfMwUlERDBwICQkm4MX7nIb7tGkzePjhR/nZz+bVeszQocN4992P3VqYiPiOc+cgNRX27TNPls6ebZ40Fes4Dfdhw4Zz7NhRb9QiIj7G4YC9e02wX7wII0aYLe+CNeFrObf8CDIydjBy5O1ERkbyu9/9J71719xwefHit1my5B0AiooK3HFqEbFIYaF5wvSHH+D6600/mI4dra5KLml0uPfrF0tm5te0atWK9etTeeihB9i+fU+Nx86aNYdZs+YAkJw8qrGnFhEL2O3w5ZeQnm46Nk6YAIMG6YZpU9Po1aatW4fTqlUrAMaMSaSyspKCAo3KRfzRiRPwxhuwYQP07Ak/+5nZSEPB3vQ0euSen59Pp06dCAoKYteuDKqqqmjXrp07ahORJqK83AT6zp3QujU88AD06mV1VVIXp+E+b94ctm7dQmFhAbGxvVmw4FkqKioAmD37YVauXM4777xFs2bBhIa25I033iZI/xsX8RsHD8Lq1VBaCnFxZhONFi2srkqcCTp5ssRhxYmTk0eRkZFhxalFxAUlJabJ17ffQufO5oZply5WVyX9+w8kLS3d6XFasCQi1TgckJFh2vLa7TBmDAwbpu3ufI3CXUQuO3nSLG88fhx69DArYXQLzTcp3EWEykrYtAm2bjXz6Skp0K+fVsH4MoW7SIA7csSM1gsKIDbWtOUNC7O6KmkshbtIgCorg3XrYPduaNsWZs0yUzHiHxTuIgHG4YADB2DtWhPw8fGmJ0xIiNWViTsp3EUCSHGxacl76BDExMDMmRAZaXVV4gkKd5EAUFUF27bB55+bm6TJyTBkiLa782cKdxE/l5Njbpjm5pqWAePGQZs2VlclnqZwF/FTFy+akfq2bXDttTBlCvTpo+WNgULhLuJFy3fbWJSaRU5xGdERocxP7MWkATFuP8/338OqVWaOffBg85Rpy5ZuP400YQp3ES9ZvtvGM8v2U1ZhB8BWXMYzy/YDuC3gz541q2AOHDAbZ8ydazbSkMCjcBfxkkWpWZeD/ZKyCjuLUrMaHe4Oh1mvvm4dVFTAqFEwfLi2uwtk+tGLeElOcVm9Pu+q06fN8sYjR6BrV9O9sUOHK697aypImhaFu4iXREeEYqshyKMjQhv0/ex22LLF9IQJCYG774YBA6rfMPXGVJA0TVrlKuIl8xN7ERpydd/cc+WVLN9tq9f3OnYMXn/drIbp08dsdzdw4NUrYeqaChL/ppG7iJdcGik/99nXFJ2vuPz54rIKl0fTFy6YPusZGWat+vTpZi/T2nhqKkiaPo3cRbxo0oAYwppfPaZyNpp2OOCbb+DPf4bMTLN5xuOP1x3sUPuUT0OngsR3KNxFvKy+o+kzZ+CDD+Cjj8zDSI88YtryNm/u/Fw1TQWFhjRjfqJ2t/Z3mpYR8TJXb6xWVcHOnbBhgxm5jx0LQ4fWrx/MpWkerZYJPAp3ES+bn9ir2goWuHo0nZ8PK1aAzQY33gjjx5ue6w0xaUCMwjwAKdxFvKyu0XRFBaSnw5dfQmgoTJ4Mt9yifjBSfwp3EQvUNJrOzjYPIxUWmvXqCQna7k4aTuEuYrHz5yE1FfbuhXbt4KGHoHt3q6sSX6dwF7GIwwH795tGXxcuwJ13wh13aLs7cQ+Fu4gFCgtNS97Dh6FLF9M6oFMnq6sSf6JwF2mghjTkstvN5hlffGGWNI4fb/qt64apuJvCXaQBGtKQy2Yz293l5Zl+MMnJEB7utZIlwCjcRRqgPr3Zy8tNg6/t26FVK5g61YS7iCcp3EUawNUWAllZsHo1lJTAkCFw113a7k68Q+Eu0gDOWgiUlppVMF9/bW6Uzp0L113n7SolkCncRRqgthYCvxzbi8xMSEuDykoYPRpuvx2aXd3GXcSjFO4iDVBTC4FH4/pQtCeKvcfMQ0gTJkD79hYXKgFL4S7SQJdaCFRWmu3uNm82bXgnTYLYWC1vFGsp3EUa4ehRs7zx9Gno18/0Wb/2WqurElG4izRIWZnZ7i4zEyIi4MEHTWtekaZC4S5SD5e2u1uzxjT8Gj4cRoxwbVckEW9SuIu4qLjYrFn/7juIjoYZMyAqyuqqRGqmcBdxoqoKduyAjRvNx0lJEBdXv+3uRLzNabg/+eRjpKWtpUOHjmzatP2q1x0OB7/+9QLWr19HaGgYf/rTa/Tr198jxYp4W26uuWGakwM33WQafUVENOx7NaTRmEhDOR17TJs2gw8+WFbr6xs2rCM7+zDbt+/hD394mQULfuHWAkWsUFFhHkR6803TOuD++2H69MYF+zPL9mMrLsPBlUZjy3fb3Fq3yCVOR+7Dhg3n2LGjtb6+Zs1qpkx5gKCgIAYPjuPMmTPk5+fRuXOkWwsV8ZZDh0yv9aIiGDQIxowx+5k2Rn0ajYm4Q6Pn3PPycoiO7nL54+joGHJzcxTu4nPOnTPb3e3bBx06wJw50LWre763q43GRNzFqzdUFy9+myVL3gGgqKjAm6cWqZXDYfYvTU2Fixdh5EiIj4dgN/52OGs0JuJujb7fHxkZTU7Oicsf5+TYiIqKrvHYWbPmkJaWTlpaOh07dmzsqUUaraAAFi+G5cuhY0f4yU9MuLsz2ME0GgsNqd49LDSkGfMTe7n3RCL/q9H/CSclJfPWW38lJeU+MjN3Eh4erikZafLsdvjyS0hPN0E+cSIMHOi5fjA1NRrTahnxJKfhPm/eHLZu3UJhYQGxsb1ZsOBZKioqAJg9+2HGjElk/fp1xMXFEhYWxssv/8XjRYs0xokTsGIFnDwJffuadeutW3v+vJcajYl4g9Nwf+ONt+t8PSgoiN///iW3FSTiKeXlsGED7NxpwvyBB6CXZkXET+kJVQkIBw+a1gGlpXDbbTBqFLRoYXVVIp6jcBe/VlJimnx9+y107mw2p47RzIgEAIW7+CWHAzIyTFteux0SEmDoUG13J4FD4S5+5+RJ0w/m+HHo0cNsd9eundVViXiXwl38RmUlbNpktrxr2RJSUszuSNruTgKRwl38wg8/wMqV5qGk2Fiz3V1YmNVViVhH4S4+7fx5071x924z9TJrlpmKEQl0CnfxSQ4HHDgAa9ea/Uzj4812dyEhVlcm0jQo3MXnFBWZlryHDplljbNmmWWOInKFwl0sU9+diaqqYNs2+Pxzc5M0ORmGDNF2dyI1UbiLJS7tTHRpA4tLOxMBNQZ8To5Z3piba1oGjBsHbdp4tWQRn6JwF0u4ujPRxYtmpL5tG7RqBVOmQJ8+Wt4o4ozCXSzhys5E339vljeeOQODB5vt7lq29FaFIr5N4S6WqGtnorNnzSqYAwfMBhpz58L117vv3PWd6xfxRboVJZaoaWeilsHNuPf6W3j1VdPoa9QomDfP/cH+zLL92IrLcHBlrn/5bpv7TiLSBGjkLpb48c5EHUPCua3ZLZR+15auXc3OSB06uP+8rs71i/g6hbtYZtKAGCbcGsPWraYnTEgIjB0LAwZ47oapK3P9Iv5A4S6WOXbMLG88dQpuucVsd9eqlWfPWddcv4g/0Zy7eN2FC2YVzN/+BhUVMGMG3Hef54Mdap7rDw1pxvxE7bcn/kUjd/Eah8PcKF29Gs6dg2HDzE3T5s29V8OP5/q1Wkb8lcJdvOLMGRPqWVkQFQXTp0N0tDW1TBoQozAXv6dwF4+qqoKdO2HDBjNyHzvWbHenfjAinqVwF4/Jz4cVK8BmgxtvhPHjoW1bq6sSCQwKd3G7igpIT4cvv4TQUJg82ayGUT8YEe9RuItbZWeblTCFhWa9+tixJuBFxLsU7uKUK71Yzp+H1FTYuxfat4eHHoLu3S0qWEQU7lI3Z33XHQ7Yt88E+4ULcOed5p9g/ZclYin9Ckqd6urFcmfXGFauNFMx111n+sF06mRRoSJSjcJd6lRTzxVHFWQfaMlf/gLNmplVMIMH64apSFOicJc6/bgXS2VpC8oOdSbM3oqbJpp9TMPDLSxQRGqkR0mkTpd6sTgqr6HscEfO7r2e4KoQnn4sjKlTFewiTZVG7lKnSQNisB0J5r//XwkXzziIvrGM537agftv62x1aSJSB4W71Kq0FNasgVPfdGb++M7cfTd06WJ1VSLiCoW7XMXhgMxMWL8eKith9Gi4/XZz81REfIPCXao5dcpsoLFuewmZhScojz7Bmq3BzG+ltrgivkThLoAZoW/eDFu2wPenz7C96gBVPYq5JghsxRXVHlwSkaZP4S4cOWL6wZw+Df36waoDu3GEnuNfl603dBNpV1oXiIj7KdwDWFkZpKXBrl2mFe/MmXDDDfDUjnM1Hl/fTaSdtS4QEc9RuAcghwO+/hrWrjUNv4YPh5EjISTEvO6uTaTral2gcBfxLJceYtq4MY1hwwYSFxfLK6+8dNXrH3zwLn36dGfUqOGMGjWcf/zj724vVNyjuBjeew8++cQ8gPToo5CQcCXYwX2bSNc20q/vOwARqT+nI3e73c7TTz/Fxx//k+joGMaOHUli4jh69epd7bh77rmXhQv/4LFCpXGqqmD7dti40fSASUqCuLiat7tz1ybS7noHICL15zTcd+3KoHv3HnTrZppzp6RMZu3aVVeFuzRdublmeWNODvTsaRp9tWlT99e4YxPp+Ym9qs25Q8PeAYhI/TkN97y8XGJirjyWGBUVza5dGVcdt3LlCr766ktuuOFGnn/+xWpfI9a4eBG++AK2bYOwMLj/frj5Zu91b3TXOwARqT+33FAdOzaJlJT7aNGiBX//+9944omfsGzZyquOW7z4bZYseQeAoqICd5xaanHokFneWFwMgwbBmDHWbHfnjncAIlJ/Tm+oRkZGYbOduPxxbm4OUVHR1Y5p1649LVq0AODBBx9i7949NX6vWbPmkJaWTlpaOh07dmxM3VKLc+dg6VL4xz/Mbkhz5phNNLSPqUhgcTpyHzBgENnZ2Rw9eoSoqGg+/XQpr7/+VrVj8vPz6Nw5EoC1a1fTs2dPz1QrtXI4YM8eWLfOTMeMHAnx8druTiRQOf3VDw4OZuHCRUydmoLdbmf69Jn07t2HhQtfoH//gSQljePNN18nNXU1zZoF07ZtW1555XVv1C7/q6DATMH88ANcf70ZqeuNkUhgCzp5ssRhxYmTk0eRkXH1jVlxnd0OW7fCpk1mhJ6QAAMHars7EX/Wv/9A0tLSnR6nN+0+6vhxs7zx5Eno29esW2/d2rWvVb8XEf+ncPeyxgbrhQuwYQNkZJgnTKdPN2vX63N+9XsR8X8Kdy9qbLB++y2sXg1nz8Jtt8Fdd0Hz5vWrQf1eRAKDwt2LGhqsJSUm1A8ehMhImDYNYhqYw+r3IhIYFO5eVN9graoy0y8bNpg/JyTA0KGN2+5O/V5EAoNLXSHFPWoL0Jo+n58Pf/ubGbF36QKPPWZa8zZ2H1N3dXwUkaZN4e5FrgTrJzts9P0/u+l9XxYvLD1ERJ98HnzQbKbhDpMGxPDivbcSExFKEBATEcqL996q+XYRP6NpGS9y1kjrjZV5/OZPJZSfu5bmnUqo6n6KNw4E0a2ne8NX/V5E/J/C3ctqCtbz503bgBdeLaOCKq69JZeQCDMvXlaBVrKISL0p3C3kcMD+/Wa7uwsXoKJDPq2vKySoWfWHhrWSRUTqS+FukaIi0w/m8GFzw3TiREh9+zy24qu7QWgli4jUl8Ldy6qq4KuvzCYa11wD48bB4MHmz9q5SETcReHuRTk5sGIF5OVB794m2MPDr7yunYtExF0U7l5w8aLZmHr7dmjVCqZOhT59aj5WK1lExB0U7h723XewapVpITB4MIweDS1bWl2ViPg7hbuHnD0La9bA119Dp04wdy5cd53VVYlIoFC4u5nDAbt3m3XrlZWmc6M72gaIiNSHwt2NTp82G2gcPQrdupnlje3bW12ViAQihbsbVFZe2e6ueXO45x7o31/b3YmIdRTujXTsmBmtnzoFt95qtru79lqrqxKRQKdwb6ALF2D9etNvPSICHnwQbrzR6qpERAyFez05HFe2uzt3Dm6/HUaOrP92dyIinqRwr4czZ0yoZ2VBVBTMmGH+LSLS1CjcXVBVBTt3mu3uHA5ITDQbVF+jrU5EpIlSuDuRl2dumNpscNNNMH68mWMXEWnKFO61qKiA9HT48ksIDYX77oO+fbW8UUR8g8K9BocPm17rRUUwcCAkJJiAFxHxFQr3f3HunGkbsHevebJ09mzzpKmIiK9RuGNuku7bB6mpUF4OI0bAHXdAsP52RMRHBXx8FRaaKZjsbNO1ceJE08VRRMSXBWy42+1Xtrtr1gwmTIBBg3TDVET8Q0CG+4kTZnljfj7cfDMkJ0Pr1lZXJSLiPgEV7uXlZru7HTtMmE+bZvYyFRHxNwET7llZZru70lKIizObaLRoYXVVIiKe4ffhXlpqtrv75hvo3BmmTIEuXayuSkTEs/w23B0OyMyEtDRz83TMGBg2TNvdiUhg8MtwP3nS3DA9fhx69DArYdq1s7oqERHv8atwr6yEzZthyxYzn56SAv36aXmjiAQevwn3I0fMaL2gAGJjYezYq7e7W77bxqLULHKKy4iOCGV+Yi8mDYixpF4REU9yqSP5xo1pDBs2kLi4WF555aWrXi8vL+eRR2YTFxdLUtIojh076vZCa1NWBitWwDvvmL7rM2eaEXtNwf7Msv3YistwALbiMp5Ztp/lu21eq1VExFuchrvdbufpp5/i/feXsmXLTpYt+4SsrIPVjnn33cW0aRPBjh17mTfvcZ5//rceK/gShwMOHIBXX4U9eyA+Hh57DG64oebjF6VmUVZhr/a5sgo7i1KzPF6riIi3OQ33Xbsy6N69B926dad58+akpExm7dpV1Y5Zu3YVU6c+AMDEiZPYvPkLHA6HZyoGiovhvffgk0/MxhmPPmpWw4SE1P41OcVl9fq8iIgvczrnnpeXS0zMlYXhUVHR7NqVUesxwcHBtG4dTmFhIe3bt3dzuXDwICxdam6SJifDkCGubXcXHRGKrYYgj45Qo3YR8T9evaG6ePHbLFnyDgBFRQUN+h5RUdCzp7lh2qaN6183P7EXzyzbX21qJjSkGfMTezWoDhGRpszpmDcyMgqb7cTlj3Nzc4iKiq71mMrKSkpLS2hXw8LyWbPmkJaWTlpaOh07dmxQwW3awP331y/YASYNiOHFe28lJiKUICAmIpQX771Vq2VExC85HbkPGDCI7Oxsjh49QlRUNJ9+upTXX3+r2jGJieP48MP3GTLkNj77bDnx8SMI8tLi8vosb5w0IEZhLiIBwWm4BwcHs3DhIqZOTcFutzN9+kx69+7DwoUv0L//QJKSxjFjxiwef/xR4uJiadu2LW+88bY3ar+8vPHSVMul5Y2AQlxEAlrQyZMlnlvWUofk5FFkZGQ4P7AOwxdurPEmaUxEKFt/dVejvreISFPUv/9A0tLSnR7n0kNMTZWWN4qI1Mynw722ZYxa3igigc6nw31+Yi9CQ6r38NXyRhERH28cdummqZqBiYhU59PhDlreKCJSE5+elhERkZop3EVE/JDCXUTEDyncRUT8kMJdRMQPWbZa5ocffqB//4EN+tqCgtO0b9/BzRU1bbrmwKBrDgyNuebjx13bxtSy3jKNkZAwwqXeCv5E1xwYdM2BwRvXrGkZERE/pHAXEfFDzebPf+Z3VhfRELGxA6wuwet0zYFB1xwYPH3NPjnnLiIiddO0jIiIH2rS4b5xYxrDhg0kLi6WV1556arXy8vLeeSR2cTFxZKUNIpjx1xbItSUObvm1157lfj4IYwYMYzJkydy/PgxC6p0L2fXfMlnn/2TTp3C2bNnlxer8wxXrvmf/1xGfPwQ7rgjjp/8ZK6XK3Q/Z9d84sRxUlLGc9dd8YwYMYz161MtqNJ9nnzyMW6+uQd33nlbja87HA6efXY+cXGxjBgxjH379rj1/E023O12O08//RTvv7+ULVt2smzZJ2RlHax2zLvvLqZNmwh27NjLvHmP8/zzv7WoWvdw5ZpvvbUf69alk57+FRMm3MN//MdvLKrWPVy5ZoCzZ0t5883XGDRosAVVupcr15ydfYiXX36JlSvXsXnzDp5//vcWVeserlzzH/+4iLvvTmHjxi389a9v8/TTT1lUrXtMmzaDDz5YVuvrGzasIzv7MNu37+EPf3iZBQt+4dbzN9lw37WCDWh9AAADtklEQVQrg+7de9CtW3eaN29OSspk1q5dVe2YtWtXMXXqAwBMnDiJzZu/wOHw3VsIrlxzfPydhIWFATB48BBycmxWlOo2rlwzwMKFL/DEEz+nRYuWFlTpXq5c85Ilf2fu3EeIiGgLQMeOHa0o1W1c+zkHUVpaCkBJyRk6d470fqFuNGzY8Ms/v5qsWbOaKVMeICgoiMGD4zhz5gz5+XluO3+TDfe8vFxiYrpc/jgqKprc3JxajwkODqZ163AKCwu9Wqc7uXLN/+rddxczenSCN0rzGFeued++PdhsNhISkrxdnke4cs2HDx/i8OFDjB+fQHLyXWzcmObtMt3KlWtesOAZli79kNjY3kyffj8vvrjI22V6VV5eDtHRV/5OoqNj6vx9ry+f36wjUH388Qfs3bub5cvXWF2KR1VVVfGb3zzLK6+8ZnUpXmW3V5KdfZjly1eTk2PjnnuSSU//ijZtIqwuzWOWLfuEqVNn8NhjT7Bz53Yef/xRNm3azjXXNNkxaJPWZP/WIiOjsNlOXP44NzeHqKjoWo+prKyktLSEdu3aebVOd3LlmgHS0z/nf/7nv1m8+ENatGjhzRLdztk1nz1bysGD35CSMp5Bg24hM3MnM2dO8+mbqq78nKOiYkhKGkdISAhdu3bjhhtuJDv7sLdLdRtXrvm99xZzzz0pAAwZchsXLpRTUFDg1Tq9KTIympycK38nOTm2Gn/fG6rJhvuAAYPIzs7m6NEjXLx4kU8/XUpi4rhqxyQmjuPDD98H4LPPlhMfP4KgoCArynULV655//69/PKXT7JkyQc+Pw8Lzq85PLwNBw8eITPzAJmZBxg0aAhLlnzQ4KZzTYErP+dx48azdetmAAoKCjh8+BBdu3azoFr3cOWaY2K6sHmz6bfy3XdZlJdfoEMH/20olpSUzEcfvY/D4SAjYwfh4eFuvc/QZKdlgoODWbhwEVOnpmC325k+fSa9e/dh4cIX6N9/IElJ45gxYxaPP/4ocXGxtG3bljfeeNvqshvFlWv+3e/+nXPnzvHwww8B0KVLF5Ys+dDiyhvOlWv2N65c86hRY/j8843Exw+hWbNm/Pa3z9OuXXurS28wV675uef+i3/7tyd4/fU/ExQUxCuvvObTg7V58+awdesWCgsLiI3tzYIFz1JRUQHA7NkPM2ZMIuvXryMuLpawsDBefvkvbj2/nlAVEfFDTXZaRkREGk7hLiLihxTuIiJ+SOEuIuKHFO4iIn5I4S4i4ocU7iIifkjhLiLih/4/oi+aYD0PJ/EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95df5908d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualize the solution regression line\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x, y, 'o', label='t')\n",
    "plt.plot([0, 1], [f(0), f(1)], 'b-', label='f(x)', alpha=0.5)\n",
    "plt.plot([0, 1], [0*w, 1*w], 'r-', label='fitted line', alpha=0.5, linestyle='--')\n",
    "plt.xlabel('input x')\n",
    "plt.ylabel('target t')\n",
    "plt.title('input vs. target')\n",
    "plt.ylim([0,2])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'target_value' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-61-d882cf7e284f>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mfig\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0max\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msubplots\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mfig\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset_facecolor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'#EAEAF2'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0marange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtarget_value\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msize\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget_value\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'o'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0malpha\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0.2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mxlabel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Iteration'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mylabel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Cost'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'target_value' is not defined"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEVBJREFUeJzt3H9MVff9x/HX1SsuJsOfVC6QCLbLqN91IMhtSKl8sTJ+WMtuTMVqZEXT2miMW62Smsza6rbbuNpKtlnbLXbc+KsKaqyVicUvU9NYL0xtu6GTW61yofhF469GsVe+f3wzGFN7rsC9WD/Px38n53PPefuJeebmXu6xtbRcahcA4L7Xr68HAACEB8EHAEMQfAAwBMEHAEMQfAAwBMEHAENYBn/BgrkaM2a0xo9/9Lbn29vbtWTJIjmdScrMTNexY0d6fUgAQM9ZBn/atBnatKnijuc/+miPfL4GHTp0RG+8sVqLF/+iVwcEAPQOy+Cnpz+mIUOG3vH87t0faurUZ2Sz2TRunFMXL17UV1819+qQAICes/f0As3NfsXExHUcx8TEqqnJr5Ejo29ZW1a2Th7Pe5KkkydPKDExsae3BwCjfPHFF6qvP9Wt1/Y4+HejqKhYRUXFkqS8vCx5vd5w3h4AvvOSk1O6/doe/5VOdHSM/P6zHcd+f6McjpieXhYA0Mt6HPzc3Dy9//5Gtbe3y+v9RJGRkbf9OAcA0LcsP9KZM6dYBw8e0PnzrUpKStTixUt048YNSdKzz87WxIk52rt3j5zOJA0aNEirV/8h5EMDAO6eZfDXrl33redtNptef31Vrw0EAAgNfmkLAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYIKvjV1VVKT0+R05mk0tJVt5w/e/aMXK5JmjAhQ5mZ6dq79y+9PigAoGcsgx8IBFRSslAbN5brwIHDqqjYquPH67usefPNlXrqKZeqqw/onXfWqaRkYcgGBgB0j2Xw6+q8SkgYrfj4BEVERMjlmqLKyl3/scqmy5cvS5IuXbqokSOjQzErAKAH7FYLmpubFBsb13HscMSors7bZc3ixS9r6tSf6k9/Wquvv/5aW7fuuO21ysrWyeN5T5J04UJrD8YGANytXvnStqJiqwoLZ+jo0Xpt2LBF8+Y9r5s3b96yrqioWFVVNaqqqlFUVFRv3BoAECTL4EdHO9TYeLbjuKnJL4cjpsuaDRvKVFDgkiSlpT2qa9euq7WVd/AAcC+xDP7Ysany+Xw6ffqU2tratG1buXJy8rusiY2N0/79NZKkEyeO6/r1axoxYkRoJgYAdIvlZ/h2u11u90oVFroUCAQ0ffpMJSY+LLd7hZKTU5Sbm69XX/21Xnxxvt5++/ey2WwqLV0jm80WjvkBAEGytbRcau+LG+flZcnr9VovBAB0SE5OUVVVTbdeyy9tAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADBFU8Kurq5SeniKnM0mlpatuu2bHjgplZKTp8cedeuGFWb06JACg5+xWCwKBgEpKFmrLlh2KiYnVT37y38rJydcPf5jYscbnO6nVq1fpgw/2aMiQoTp37lxIhwYA3D3Ld/h1dV4lJIxWfHyCIiIi5HJNUWXlri5rPJ4/a9as5zRkyFBJUlRUVGimBQB0m+U7/ObmJsXGxnUcOxwxqqvzdlnT0HBSkjRpUrZu3gxo0aKXNWFC9i3XKitbJ4/nPUnShQutPZkbAHCXLIMfjEDgG/l8Ddq+/UP5/Y0qKMhTTc3HGjx4SJd1RUXFKioqliTl5WX1xq0BAEGy/EgnOtqhxsazHcdNTX45HDFd1jgcscrNzdeAAQM0alS8HnzwIfl8Db0/LQCg2yyDP3Zsqnw+n06fPqW2tjZt21aunJz8Lmvy8yfp4MH9kqTW1lY1NJzUqFHxIRkYANA9lh/p2O12ud0rVVjoUiAQ0PTpM5WY+LDc7hVKTk5Rbm6+srImat++amVkpKl///565ZXlGjZseDjmBwAEydbScqm9L26cl5clr9drvRAA0CE5OUVVVTXdei2/tAUAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQwQV/OrqKqWnp8jpTFJp6ao7rtu5c4ceeCBSR47U9dqAAIDeYRn8QCCgkpKF2rixXAcOHFZFxVYdP15/y7orVy7r3XfXKDV1XEgGBQD0jGXw6+q8SkgYrfj4BEVERMjlmqLKyl23rHO7V2j+/J9r4MDvhWRQAEDPWAa/ublJsbFxHccOR4yamvxd1hw7dkSNjY3Kzs791muVla1TdnamsrMzde7cuW6ODADoDntPL3Dz5k0tXbpEpaVrLNcWFRWrqKhYkpSXl9XTWwMA7oLlO/zoaIcaG892HDc1+eVwxHQcX7lyWfX1f5fLNUmpqT9Sbe1hzZw5jS9uAeAeYxn8sWNT5fP5dPr0KbW1tWnbtnLl5OR3nI+MHKz6+lOqrf1MtbWfKTU1TR7PJiUnp4R0cADA3bEMvt1ul9u9UoWFLj322DgVFLiUmPiw3O4Vqqz8MBwzAgB6ga2l5VJ7X9w4Ly9LXq+3L24NAN9Zyckpqqqq6dZr+aUtABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIYIKfnV1ldLTU+R0Jqm0dNUt59es+Z0yMtKUmZmuKVMm68yZL3t9UABAz1gGPxAIqKRkoTZuLNeBA4dVUbFVx4/Xd1nzyCM/1p49Naqp+VhPPlmg115bGrKBAQDdYxn8ujqvEhJGKz4+QREREXK5pqiycleXNRkZ4zVo0CBJ0rhxafL7G0MzLQCg2+xWC5qbmxQbG9dx7HDEqK7Oe8f169eX6Yknsm97rqxsnTye9yRJFy603uWoAICesAz+3diyZZOOHv2btm/ffdvzRUXFKioqliTl5WX15q0BABYsgx8d7VBj49mO46YmvxyOmFvW1dTs01tv/Vbbt+/WwIEDe3dKAECPWX6GP3Zsqnw+n06fPqW2tjZt21aunJz8Lms+/fSoXnppgTyeTYqKigrZsACA7rN8h2+32+V2r1RhoUuBQEDTp89UYuLDcrtXKDk5Rbm5+Vq27Je6evWqZs/+mSQpLi5OHs/mkA8PAAieraXlUntf3DgvL0te752//AUA3Co5OUVVVTXdei2/tAUAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQwQV/OrqKqWnp8jpTFJp6apbzl+/fl3PPfesnM4k5eZm6csvT/f6oACAnrEMfiAQUEnJQm3cWK4DBw6romKrjh+v77Jm/foyDR48RJ98clRz5szT8uWvhGxgAED3WAa/rs6rhITRio9PUEREhFyuKaqs3NVlTWXlLhUWPiNJmjz5p9q//3/U3t4emokBAN1it1rQ3Nyk2Ni4jmOHI0Z1dd47rrHb7fr+9yN1/vx5DR8+vMu6srJ18njekyT94x+fKzk5pafz3xdaW/9Xw4eP6Osx7gnsRSf2ohN70emf/zzR7ddaBr83FRUVq6ioWJKUnZ2pqqqacN7+nsVedGIvOrEXndiLTtnZmd1+reVHOtHRDjU2nu04bmryy+GIueOab775RpcvX9KwYcO6PRQAoPdZBn/s2FT5fD6dPn1KbW1t2ratXDk5+V3W5OTka/PmjZKknTu3KyMjUzabLTQTAwC6pf+iRS8v+7YF/fr10+jRozV37nP64x/X6umnCzV5coHc7hW6cuWKHnroBxoz5r9UXv6+fvWrZfr880+1cuVbGjJkqOXNk5LG9ta/4zuPvejEXnRiLzqxF526uxe2lpZL/DkNABiAX9oCgCEIPgAYIuTB57EMnaz2Ys2a3ykjI02ZmemaMmWyzpz5sg+mDA+rvfiXnTt36IEHInXkSF0YpwuvYPZix44KZWSk6fHHnXrhhVlhnjB8rPbi7NkzcrkmacKEDGVmpmvv3r/0wZSht2DBXI0ZM1rjxz962/Pt7e1asmSRnM4kZWam69ixI0FdN6TB57EMnYLZi0ce+bH27KlRTc3HevLJAr322tI+mja0gtkLSbpy5bLefXeNUlPH9cGU4RHMXvh8J7V69Sp98MEe7d//iZYvf72Ppg2tYPbizTdX6qmnXKquPqB33lmnkpKFfTRtaE2bNkObNlXc8fxHH+2Rz9egQ4eO6I03Vmvx4l8Edd2QBp/HMnQKZi8yMsZr0KBBkqRx49Lk9zf2xaghF8xeSJLbvULz5/9cAwd+rw+mDI9g9sLj+bNmzXqu4y/foqKi+mLUkAvu/4VNly9fliRdunRRI0dGh3/QMEhPf+xb/9Jx9+4PNXXqM7LZbBo3zqmLFy/qq6+aLa8b0uDf7rEMTU3+O67598cy3G+C2Yt/t359mZ54Ijsco4VdMHtx7NgRNTY2Kjs7N9zjhVUwe9HQcFINDSc1aVK28vImqLq6KtxjhkUwe7F48csqL9+spKRETZ/+tH7zm5XhHvOe0NzsV0xM517FxMR+a0/+JayPVkBwtmzZpKNH/6bt23f39Sh94ubNm1q6dIlKS9f09Sj3hEDgG/l8Ddq+/UP5/Y0qKMhTTc3HGjx4SF+PFnYVFVtVWDhDc+fO1+HDhzRv3vP6618PqV8//v4kGCHdJR7L0CmYvZCkmpp9euut36qsbLMGDhwYzhHDxmovrly5rPr6v8vlmqTU1B+ptvawZs6cdl9+cRvM/wuHI1a5ufkaMGCARo2K14MPPiSfryHco4ZcMHuxYUOZCgpckqS0tEd17dp1tba2hnXOe0F0dIz8/s698vsbb9uT/xTS4PNYhk7B7MWnnx7VSy8tkMez6b79nFay3ovIyMGqrz+l2trPVFv7mVJT0+TxbLovn64azP+L/PxJOnhwvySptbVVDQ0nNWpUfB9MG1rB7EVsbJz27///h6idOHFc169f04gR5j1FMzc3T++/v1Ht7e3yej9RZGRkUN9nhPQjHbvdLrd7pQoLXQoEApo+faYSEx+W271Cyckpys3N14wZRZo373k5nUkaOnSo1q5dF8qR+kwwe7Fs2S919epVzZ79M0lSXFycPJ7NfTx57wtmL0wRzF5kZU3Uvn3VyshIU//+/fXKK8s1bNhw64t/xwSzF6+++mu9+OJ8vf3272Wz2VRauua+fIM4Z06xDh48oPPnW5WUlKjFi5foxo0bkqRnn52tiRNztHfvHjmdSRo0aJBWr/5DUNfl0QoAYAi+6QAAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQ/wfpruRaQt91DMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95df617048>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualize the evolution of the target function value during iterations.\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(np.arange(target_value.size), target_value, 'o', alpha = 0.2)\n",
    "plt.xlabel('Iteration')\n",
    "plt.ylabel('Cost')\n",
    "plt.grid()\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mini-batch Gradient Descent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In code, general batch gradient descent looks something like this:\n",
    "\n",
    "```python\n",
    "nb_epochs = 100\n",
    "for i in range(nb_epochs):\n",
    "    grad = evaluate_gradient(target_f, data, w)\n",
    "    w = w - learning_rate * grad\n",
    "```\n",
    "\n",
    "For a pre-defined number of epochs, we first compute the gradient vector of the target function for the whole dataset w.r.t. our parameter vector. \n",
    "\n",
    "**Stochastic gradient descent** (SGD) in contrast performs a parameter update for each training example and label:\n",
    "\n",
    "```python\n",
    "nb_epochs = 100\n",
    "for i in range(nb_epochs):\n",
    "    np.random.shuffle(data)\n",
    "    for sample in data:\n",
    "        grad = evaluate_gradient(target_f, sample, w)\n",
    "        w = w - learning_rate * grad\n",
    "```\n",
    "\n",
    "Mini-batch gradient descent finally takes the best of both worlds and performs an update for every mini-batch of $n$ training examples:\n",
    "\n",
    "```python\n",
    "nb_epochs = 100\n",
    "for i in range(nb_epochs):\n",
    "  np.random.shuffle(data)\n",
    "  for batch in get_batches(data, batch_size=50):\n",
    "    grad = evaluate_gradient(target_f, batch, w)\n",
    "    w = w - learning_rate * grad\n",
    "```\n",
    "\n",
    "Minibatch SGD has the advantage that it works with a slightly less noisy estimate of the gradient. However, as the minibatch size increases, the number of updates done per computation done decreases (eventually it becomes very inefficient, like batch gradient descent). \n",
    "\n",
    "There is an optimal trade-off (in terms of computational efficiency) that may vary depending on the data distribution and the particulars of the class of function considered, as well as how computations are implemented."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loss Funtions\n",
    "\n",
    "Loss functions $L(y, f(\\mathbf{x})) = \\frac{1}{n} \\sum_i \\ell(y_i, f(\\mathbf{x_i}))$ represent the price paid for inaccuracy of predictions in classification/regression problems.\n",
    "\n",
    "In classification this function is often the **zero-one loss**, that is, $ \\ell(y_i, f(\\mathbf{x_i}))$ is zero when $y_i = f(\\mathbf{x}_i)$ and one otherwise.\n",
    "\n",
    "This function is discontinuous with flat regions and is thus extremely hard to optimize using gradient-based methods. For this reason it is usual to consider a proxy to the loss called a *surrogate loss function*. For computational reasons this is usually convex function. Here we have some examples:\n",
    "\n",
    "### Square / Euclidean Loss\n",
    "\n",
    "In regression problems, the most common loss function is the square loss function:\n",
    "\n",
    "$$ L(y, f(\\mathbf{x})) = \\frac{1}{n} \\sum_i (y_i - f(\\mathbf{x}_i))^2  $$\n",
    "\n",
    "The square loss function can be re-written and utilized for classification:\n",
    "\n",
    "$$ L(y, f(\\mathbf{x})) = \\frac{1}{n} \\sum_i (1 - y_i f(\\mathbf{x}_i))^2  $$\n",
    "\n",
    "\n",
    "### Hinge / Margin Loss (i.e. Suport Vector Machines)\n",
    "\n",
    "The hinge loss function is defined as:\n",
    "\n",
    "$$ L(y, f(\\mathbf{x})) = \\frac{1}{n} \\sum_i \\mbox{max}(0, 1 - y_i f(\\mathbf{x}_i))  $$\n",
    "\n",
    "The hinge loss provides a relatively tight, convex upper bound on the 0–1 Loss.\n",
    "\n",
    "\n",
    "<img src=\"images/loss_functions.png\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Logistic Loss (Logistic Regression)\n",
    "\n",
    "This function displays a similar convergence rate to the hinge loss function, and since it is continuous, simple gradient descent methods can be utilized. \n",
    "\n",
    "$$ L(y, f(\\mathbf{x})) = \\frac{1}{n} log(1 + exp(-y_i f(\\mathbf{x}_i))) $$\n",
    "\n",
    "\n",
    "### Sigmoid Cross-Entropy Loss (Softmax classifier)\n",
    "\n",
    "Cross-Entropy is a loss function that is very used for training **multiclass problems**. We'll focus on models that assume that classes are mutually exclusive. \n",
    "\n",
    "In this case, our labels have this form $\\mathbf{y}_i =(1.0,0.0,0.0)$. If our model predicts a different distribution, say  $ f(\\mathbf{x}_i)=(0.4,0.1,0.5)$, then we'd like to nudge the parameters so that $f(\\mathbf{x}_i)$ gets closer to $\\mathbf{y}_i$.\n",
    "\n",
    "C.Shannon showed that if you want to send a series of messages composed of symbols from an alphabet with distribution $y$ ($y_j$  is the probability of the  $j$-th symbol), then to use the smallest number of bits on average, you should assign  $\\log(\\frac{1}{y_j})$  bits to the  $j$-th symbol. \n",
    "\n",
    "The optimal number of bits is known as **entropy**:\n",
    "\n",
    "$$ H(\\mathbf{y}) = \\sum_j y_j \\log\\frac{1}{y_j} = - \\sum_j y_j \\log y_j$$\n",
    "\n",
    "**Cross entropy** is the number of bits we'll need if we encode symbols by using a wrong distribution $\\hat y$:\n",
    "\n",
    "$$ H(y, \\hat y) =   - \\sum_j y_j \\log \\hat y_j $$ \n",
    "\n",
    "In our case, the real distribution is $\\mathbf{y}$ and the \"wrong\" one is $f(\\mathbf{x}_i)$. So, minimizing **cross entropy** with respect our model parameters will result in the model that best approximates our labels if considered as a probabilistic distribution. \n",
    "\n",
    "Cross entropy is used in combination with **Softmax** classifier. In order to classify $\\mathbf{x}_i$ we could take the index corresponding to the max value of $f(\\mathbf{x}_i)$, but Softmax gives a slightly more intuitive output (normalized class probabilities) and also has a probabilistic interpretation:\n",
    "\n",
    "$$ P(\\mathbf{y}_i = j \\mid \\mathbf{x_i}) = - log \\left( \\frac{e^{f_j(\\mathbf{x_i})}}{\\sum_k e^{f_k(\\mathbf{x_i})} } \\right) $$\n",
    "\n",
    "where $f_k$ is a linear classifier. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Advanced gradient descend\n",
    "\n",
    "\n",
    "### Momentum\n",
    "\n",
    "SGD has trouble navigating ravines, i.e. areas where the surface curves much more steeply in one dimension than in another, which are common around local optima. In these scenarios, SGD oscillates across the slopes of the ravine while only making hesitant progress along the bottom towards the local optimum.\n",
    "\n",
    "<img src=\"images/ridge2.png\">\n",
    "\n",
    "Momentum is a method that helps accelerate SGD in the relevant direction and dampens oscillations. It does this by adding a fraction of the update vector of the past time step to the current update vector:\n",
    "\n",
    "$$ v_t = m v_{t-1} + \\alpha \\nabla_w f $$\n",
    "$$ w = w - v_t    $$\n",
    "\n",
    "The momentum $m$ is commonly set to $0.9$.\n",
    "\n",
    "### Nesterov\n",
    "\n",
    "However, a ball that rolls down a hill, blindly following the slope, is highly unsatisfactory. We'd like to have a smarter ball, a ball that has a notion of where it is going so that it knows to slow down before the hill slopes up again.\n",
    "\n",
    "Nesterov accelerated gradient (NAG) is a way to give our momentum term this kind of prescience. We know that we will use our momentum term $m v_{t-1}$ to move the parameters $w$. Computing \n",
    "$w - m v_{t-1}$ thus gives us an approximation of the next position of the parameters (the gradient is missing for the full update), a rough idea where our parameters are going to be. We can now effectively look ahead by calculating the gradient not w.r.t. to our current parameters $w$ but w.r.t. the approximate future position of our parameters:\n",
    "\n",
    "$$ w_{new} = w - m v_{t-1} $$\n",
    "$$ v_t = m v_{t-1} + \\alpha \\nabla_{w_{new}} f $$\n",
    "$$ w = w - v_t $$\n",
    "\n",
    "### Adagrad\n",
    "\n",
    "All previous approaches manipulated the learning rate globally and equally for all parameters. Tuning the learning rates is an expensive process, so much work has gone into devising methods that can adaptively tune the learning rates, and even do so per parameter. \n",
    "\n",
    "Adagrad is an algorithm for gradient-based optimization that does just this: It adapts the learning rate to the parameters, performing larger updates for infrequent and smaller updates for frequent parameters.\n",
    "\n",
    "$$ c = c + (\\nabla_w f)^2 $$\n",
    "$$ w = w - \\frac{\\alpha}{\\sqrt{c}} $$ \n",
    "\n",
    "\n",
    "### RMProp\n",
    "\n",
    "RMSProp update adjusts the Adagrad method in a very simple way in an attempt to reduce its aggressive, monotonically decreasing learning rate. In particular, it uses a moving average of squared gradients instead, giving:\n",
    "\n",
    "$$ c = \\beta c + (1 - \\beta)(\\nabla_w f)^2 $$\n",
    "$$ w = w - \\frac{\\alpha}{\\sqrt{c}} $$ \n",
    "\n",
    "where $\\beta$ is a decay rate that controls the size of the moving average.\n",
    "\n",
    "\n",
    "<img src=\"images/g1.gif\">\n",
    "\n",
    "(Image credit: Alec Radford) \n",
    "\n",
    "<img src=\"images/g2.gif\">\n",
    "\n",
    "(Image credit: Alec Radford) \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Once deleted, variables cannot be recovered. Proceed (y/[n])? y\n"
     ]
    }
   ],
   "source": [
    "%reset\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets.samples_generator import make_regression \n",
    "from scipy import stats \n",
    "import random\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAFpCAYAAABXkHk0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnW+Ibed13tfaM3OvJMeWcKUibLm1SY1BNaYBkTa00D8xVElDQ1IC8YeWkMCl0EACgRBX0HwogYAhFNqUIrDJF5P0g2NsWpfYhoBbqEMcY1LbsoMIMXEIDY7xX0n3zpz99sPcFM3dz3PvXtr7nD3vzO8HBt931ln7PWfOWbN1nme9K1trAQAAfTBsvQEAAJgPRRsAoCMo2gAAHUHRBgDoCIo2AEBHULQBADqCog0A0BEUbQCAjqBoAwB0BEUbAKAjjre46BsffUN76sknLqy1NspY2WZvWu9bqBwmVuZYIVauu6MCpuurHCqQleAVriiux+EI22B/9Yt/IYU31X7eUvfZw3Q9K7G5n9jzVXFfbHJ84cWvfK219oT84avYpGg/9eQT8dH//KsX1s7OXpGx4+50srY7uy1jd+N0fTdOHx8RMYp19XiXQz0+ImJsIjZ0bGu76dpg/nil+CSY94+Kbe6/qVRsqejrfcj9Fh7vKO9NXm9hVVlhE6UtVF5Kt7dSDvV4U5TE27WyBxeb4/QNm3kkY4c4ma7ldC0iYhim60di7Xz95qzHV3McHU/XIiLe8c9+5ivyB/fuYU4QAABcDijaAAAdQdEGAOgIijZMOH3d2dZbALiUOC3rkGwiRO52t+Ob3/yTe9a0EFkRF1WsEwHVehuMI0SsN6076BwrCIaVvNKV4vIaRUwV7tGIWcoxo+VUTSVvBZd3XwwL9UnneKjktXqzyG3zlgRDfbXZecc1BM5p7Sjl3ZnYU/GaCdHzfutSiDSi5Vy40wYA6AiKNgBAR1C0AQA6gqINADATp5Edkm2EyPFOfOvlP7mwZrsGlQh4ZATDk2XCnhcixZoR8FQOJ8rthDKzMy36SlTbGaFtVELktPnS7s3uQafQsfaoARVbyHtFGuSHQhuoEwzV8mDbrNUeNEcihxU4xeXcczsSSQbd5Cj3cOSEWrUvJ0QKMVOLqTrH+dr0w5SjNlLk+J3pmhE+58KdNgBAR1C0AQA6gqINANARFG0AgI6gaMOEr53Sxg6gcGaFQ7KJe6TlGHceuqi2WueGcIpUXB474/KoODfOpHPD5BX1zudVe5Ch0jVRco+YWLWHCF24ncuj4iqRLe+Fz8H2H5l1qPgHvHtkmVOk4ko5Nq6UkntkYazbg3SamOemYo9NFaw4WI4q540bZ8tcuNMGAOgIijYAQEdQtAEAOoKiDQAwE6WFHZpthMgh4vSRi0KXezFOhVCmhMGIiDOhzJ25vEL9csKe2psT8FSOisDpzpBWeV2ruIw1eeX8+oLAeb6PeXnPY+e/6SsCp3uNt0aJWY7qHZRqWXc5lOjoREAtLpq8IocTAdV+XawSWY8L4qLLq8RMKy6K5ROj3h6bHMci3gmfc+FOGwCgIyjaAAAdQdEGAOgIijYAQEdQtAEAZqIMDIdmE/fIWbT4xtnFg8SVS8Stu9hKW7h0jxScG949Mn8PKodzTOjnZl4H2So+3z3iBzHMz2GdLXJVsy+nySGp3BVVBhhERKj5AS6HbmOf3xbuXSnCHWHdGMoRMn+/vo193rVcDtvyLp7HyZDyTVxxq5wUf8/3wp02AEBHULQBADqCog0A0BEUbQCAmVxfIbK1+Muzi23s7sU4LbSFK4HSnnstlu8UBM7TpqUvHTtf2HMCp8pRaRWv7KHagr4TZwmPlfsBcxZxK506PT/v3jBnt89+uDvXopB3ML+9I5FjDdFSiWqVvE6UUwJeZQ9OtDzJ6S5c7A0nWkohcv7eKkcbKLjTBgDoCIo2AEBHULQBADqCog0A0BEUbQCAmbju5kOyiXtk1yK+cfra29jdEITb41Q5d0MQZGxpD8ZpIvLW2u61sjy2adOyc1c0EStH1UdEk9dzeZ3qPc3tYwvKecH94a+3LVlxlJTdJ9N4f73p+9LHFvKKceOpRpCHdscMeSYitdvFO0KEe2TQ7/dKW7mKvenymve1infDHObCnTYAQEdQtAEAOoKiDQDQERRtAICZKC3s0GwkRLb4+j3nad82beEVwXCpuHjH/EJOlQhohb2T6dqoTj6OUKKcFBHd9Vz7t1h3efUe5guONr4iIu5JnPTXW3avkmuc3l0QHW17eyGvFhKNYChjXd5pT7cXLZXA6fagTl43exjUHlze6fqJvFbEDSEiVkTLCCdEbtzGnplvyczfzcwvZuYXMvPnluYEAADNGnfaZxHxC621z2bm6yPiDzLzE621L66QGwAAXsXiO+3W2p+31j579/9/OyJeiIg3L80LAABTVhUiM/OtEfF9EfF7a+YFAIBzVivamfk9EfGhiPj51tq3xM9vZeZnMvMzL3379lqXBQA4GK67+ZCs4h7JzJM4L9gfbK39topprT0fEc9HRDz+tsfa107vXPj57RUcIbdH1U5tHA/j9Kl7R0jBPSL3UHCEOIeGyltqYy8MGrCuFPc3vpBjb23sl9O96lwMOrhWEKSrpNSabvamXB6FAQ3a+aFz5GBHboi8850mLq/KsTP7va1a9Afddu/2dnM4naxdBvdIRsT7I+KF1tqvLc0HAACeNW5P/n5E/MuI+CeZ+bm7//vhFfICAMA9LP56pLX2v6J0dBsAALxWLucXgQAAlxClhR2aTXZw2lp87fTiF/Sn5sUYC23hrRXERSkY6j1IIVI83l3PCpGqhdy1vKu87j9wCm3stfO05wuRpfOt9zaN/cD3JBXRUT18hWnstTOy3X7nn6ctRUcXK4VTIxiq1vRSK73LqwROI5yKs75zGCN2N2dfbydz6OvNhTttAICOoGgDAHQERRsAoCMo2gAAHUHRBgCYiTMgHJJN3COtDfHK6SP3rBnnhnBTOCfEWIit5C05QlSsdbuoN8D81nTn0NADE4zbZY0WdPmcjYuh0Klt5mLo2O2PhJBUOpazWg9E7hzMBVVLdqU13cZW2s3nt7xX3C61IQgLY4ddhDLMmOcxyL3hHgEAuDZQtAEAOoKiDQDQERRtAICZ+CHdh2MzIXJ3dlGIVCJiRO3c61G1sVsRULWm70mIdO3UhVi9Xy0ONaFzWKFOaDAu1oqL4szzijBoBceF4qLd756wIuDsBC5vIYVrYxc5UkwKP88x7/EuNk1dU6+PbyEXb4rChPVVhEgnIioh0rSmD6qNfeFxB9xpAwB0BEUbAKAjKNoAAB1B0QYA6AiKNgDATJwB4ZBs5B7JODt76OKaG4Ig3SNrtKZXBhsUprGr2J1W9EclOBthedyJH9hYtTEdqxwWVTeHiq+4VRytYEGptLzvDzVtfP6jszqlWzlCKg4UN6xAOk1cYrEtV9eG6fWGI/MCCWeLy5tHy1wpNrY0MEGvj7SxAwBcbyjaAAAdQdEGAOgIijYAQEdQtK8Jp9+dr9SdffeSHk7dIaffnv9ann7nUqip146z0+l0dce1dY9EG2J39vDFJXPux77cI6N0eZjD/9VZHkYAVi4PFyvzVs4TMZ9xdz1VuF0OVbjt9dS6ex57GoJQcaUclGF+4c6hVrj1GSH6PawdISaxjDVOE7EJf/aIWDtyDpZCXpHDuVJSrOeRLtwyb+4i4mQaax0ouEcAAK41FG0AgI6gaAMAdARFGwBgJkoLOzTbtLHHELt72tiV4Bjhppu72OkLOp7OF/ZG024uY11e1dJdECLdHpTQNk7PV7/7AyEiuhZ0kaMkOJrrOeTrcxUExyqV9nY3gECuuyEIQtgr5E1TKeTQBSOGDiqH62IXtdELkWLNCKfDyXyBc1Dt8UcRoyibw8lUnIyISHGmRA7ugzsP7rQBADqCog0A0BEUbQCAjqBoAwB0BEUbJpx9hzb2LeB1v/w4U8Eh2aiNPSfuETdUYNxN151rQg0bKLlHzgot5LaNXTg33H7F9eQAg4jl7hHnCDE5VAFxLfalwQa4R84xt0vqda+5R9z1VN5Cy7tzj6g9OEdIyT0i9nts9ivyKudHhK4H1j0i8tpY8zlSbfPD8Q0dPBPutAEAOoKiDQDQERRtAICOoGgDAMzk2gqRrWXszi5e2olqSjiwImApVqy5NnbV6m1ESyUkWiFSiXJODK20m4sctbOwC7FhBMqKOOna/JeKjocWLRfeAlnBsXLchRUtVRu7a3lXj3d5xZqpKmMlVoiO9kxvKUS6WJHX7EG10g8nEaFqhzE8qLZ5MzpgNtxpAwB0BEUbAKAjKNoAAB1B0QYA6AiKNkxgGjuARhkYDs1G7pGI3e2La059PaQjxLa8q9hCa/oa7hF5PeOOkO34xcntsnAX3CMl58ee3COL3SdFSm3llccX3AallveKI8TtQcTKdvVirHJ02OcmYpX75Px6qj1e521irkEea4eYa5tXR2uU3ECCVe60M/MDmfkXmfn5NfIBAIBmra9HfiMinl0pFwAAGFYp2q21T0XE19fIBQAAHoRIAICZOH3qkBxMiMzMWxFxKyLi+PWPxu6Vi1/Qu7OslQBXEgwPLC6q5+HP0573+PMfiCWnZMszqwvT403edcTFhaLlFT9PW+HFRSF+WZGrcE63EuAKexjtJPTpmmptjzBnWRdESydwNpnXiIjis5jHIdvYlWgZYURL0dpe4WB32q2151trz7TWnjl++JFDXRYA4ErB1yMAAB2xluXvNyPif0fEOzLzq5n5M2vkBQCAi6zynXZr7T1r5AEAgPvD1yMwYfcybewAimvlHnk1523sFwtDZbDBeGf+AILxjtnDJXCEKJeGb3kXeyi4POwAA+OukYV7X4MN3JT3Qt7eqA02cFPTF7ZIO0fIbbFmHA9yDyavem+7ifBNtZsbh8Yg3B/q8RErOU3kQBLjmCkcPzEX7rQBADqCog0A0BEUbQCAjqBoAwDMxB2dfEg2ESKjTcU5JxjK87SdYCgESjvlXcQeWlyUOZwgq3LY2GXnadu2ctcKX8ph1mfmrdDdedqFFvSIqE1NV7mtwCnWXLGSZ2SbNm3xHlbnW9vYs0sgWt7IaMoI4d5rN8T12rLCz502AEBHULQBADqCog0A0BEUbQCAjqBoAwDMxHVjH5LN2tjvdYvYaezCVWInlotYO1xB5pWh8npuAIFsjzex+3KEyL3t0fmxhgNF5j2g02RfVNrKy+4T4f6ouUeMa6I0jV04OtzvWO1tZ5wm6npisnlERIr3SY4mr9ibi1V5hxumcLvrqeETNnYe3GkDAHQERRsAoCMo2gAAHUHRBgCYievcPiQbtbG3ifBYOffatrwX2tiV6GgFTnG+cKmNvXCWtRVDldjn9iBjC3mrbezq/O6KMFj4JLR4qZDY5fjuosdnvG7+tcxrmTEdbm0l2uGGyz7NawRDKVBW2thdXtECPrrnLFrIR/OeUi3kY8wXDNOJoUIEHOwrL4TeY/N2tQJ75Xrz4E4bAKAjKNoAAB1B0QYA6AiKNgBAR1C0AQBmcm2nsUebPvlVWtOlI8RsQblShEvEXc9OQhfrbr+qLdy2xyunSaHd3OVVqrd1uzhHiPglVVwe3s2xzOWx1CXi82pKrhK5qh+f4/y8MU5dKRERTThQ8mh+u3kzrdfqrs86ZkQbupvyrvOaiedu6IJglK+824PY73HqoSjugvKoANrYAQCuDRRtAICOoGgDAHQERRsAYCbXdhp7a1PR0IpfqkXaKbgy1oiWst3c/EIKrd4yR+HcayU4uutZMVRdzzw3ffa2bit34mJN8JvGVh6/Rhv7vqjsTbWxe+HVCZHTdfdaSjGzIFoOJ+79M/9M71AT0t05+qr920xY15850/KuPsvmXHH13IYTXVPSHAkgj8tYeKvMnTYAQEdQtAEAOoKiDQDQERRtAICOoGgDAMzEdhYfkM3a2O91M9h280KbtWw3t63X6lour4h1Lg8Va/ewrDW9MqzAt7HPb0GvtJvvzxHyrUKsu94XFj0+42+vsIc1mL7GrpW+cj01VGA81YMYlKukFYYV2CMB1GfDTW6XFFwpbsCDGq5wrD/P7qiKQbhKrEttJtxpAwB0BEUbAKAjKNoAAB1B0QYAmMn1PU877iOi3YtsyV5+HSkGVCaIFyidT12Knd/yXmFf51uf51ai43xxcamIuAZr7KEiZvrzu3UbukaJxcvzqvdamsxKSHTSony/m1tMNyl+NuZzr9rb8zjNczYsrF8K7rQBADqCog0A0BEUbQCAjqBoAwB0BEUbAGAm13YIgmKp42GvVIYg7MmBsi/WGCqwr6nnUMcOQShNip++JzJ0G/tloOQ0UbGFKpiFye/7YpU77cx8NjO/nJkvZuYvrZETAACmLC7amXkUEb8eET8UEU9HxHsy8+mleQEAYMoad9rfHxEvttb+uLV2JyJ+KyJ+dIW8AABwD2sU7TdHxJ++6t9fvbsGAHCluFZCZGbeiohbERFHD79h+nPz52P7lygiRJusOmvXxV5mkVW1LFeFRSVyIU5uQ0Vw9Dkq7fHbo9rY7UT4hS3vV0WI/LOIeMur/v3U3bULtNaeb60901p7ZrjR15sCAOCysEbR/v2IeHtmvi0zb0TET0bER1fICwAA97D465HW2llm/mxE/E6c/8fHB1pr2x/FBgBwBVnlO+3W2sci4mNr5AIAAA9t7AAAM7lW7pF7uVfdtS+F+rNS+FNjVWQ1JfnI7GLhtIqKku2cJkqzbqOZei1ezUp7fX2i93yniHarzGedSejbT2OPmDqo/PXWEO6nv9NVnCby82nel8q5URhsYD9HS3HT2MXzyGrFXFi/5qYEAIBLCkUbAKAjKNoAAB1B0QYA6AiK9mskT+a3s1amRVfElkrscFLIe6nbmOcLeDWx7wcuwR4u8+s+n329h0ufo8rns7CHttCUsAbbuEcyJgpzHmsPQQqHRIrR9hERgzgXYBx1rHJpuF/eIF6l8cy9MYRz40y/4aT7Y9fkPuSz2Ok9N+U1GZss3OOpONx+vFMqIGpv3pmw7EyS8wP6lxVNV6APWbjd66vW/Ws53/2hctjf8TB9T9g/+s4pUnBNLHWgOEeH+nyqz7Lfg1gyr4OqPS6vcqVU4E4bAKAjKNoAAB1B0QYA6AiKNgDATMbTrXewkRCZORUERte+rb7IN7uW4qIRCAYhn42mLVy1t9shCKo93gmnShiUDesRcSpyGFFE7W08Na/DiRBORzN5e6fXc1St6W7KuxLPnDg5Xa9NFT/sIIblbeH68ZW8FXHRuTEqwqAU5gqCo3N51PIqs4KOlQNNCs/NDkFwexO1amk7PnfaAAAdQdEGAOgIijYAQEdQtAEAOoKiDRMuw8Tp68gQT2y9BXgA13cIQk5VVdWuHqFdHnokQESIlnVn8lA59LV07Fg4un84Mq4U4Qhxf0WbcqWYN5AaeGDz7kTeURdud3yAdJsYp4lCuU/OmRYx70oRec36UlfJKsMDjMtDFm7h/LB5V3CESJeHy6vcETZWvKcqeY3TRLtSdF7Vbj7cdHuYPwTBtcfL683/deqcyx4OAACHhKINANARFG0AgI6gaAMAzOQan6edMUxEBS1yqbbuimAYRjzTAmVBDDUijmpZb+a8AvXcXIvrKETHNAKnOh8hzaR5JVq6ifDh2vxLOdQ+tDIjjyVwsaVp84/ND17IvgZgRIR8D7ocpUnoBSFS7cGfvS2WnFNJXM+ehV1oN6+0pivR0YmI7nmo+KXuLO60AQA6gqINANARFG0AgI6gaAMAdARFGyYMN2ljB1CMd7bewZZDECaqqm86nqzYacbzXR46r4lUOVxLtxyCYLagHB3OBaGcIi5W7a3o/JCF2022X8GBMjdvBbuHPbH0cPuK0yQi5C1XzT2yvI19X44QHTvf5VFpY7fT3KV7xLhEXBu7iHfXmwt32gAAHUHRBgDoCIo2AEBHULQBAGYy3rnG52nfKwik0S6UKDKmeeGUsGLUXpXXnSsgW8jPjLAnXlH1+IiIFBPS3R5STU037fHq3GEnyqkzvfcpIsrcRuCs5O2Nkui4hmAo8+rlytR0GevyqhZy99wOKS5WBM5Cu3pExKAEVSNmzoU7bQCAjqBoAwB0BEUbAKAjKNoAAB1B0YYJRw/Txg6guLbT2DMjju5pk3YOC6mcO3VaOC/sBHHh/hidc0OsH9ppopwbo/vtCYdGcy3ool29nerCXWoLt+4R5VYp/JE4cGv63ijcLlk3hvpsrDB0Qbs8CnswsXI4gou9DI4QGav3MB3qcp+9mdi5cKcNANARFG0AgI6gaAMAdARFGwBgJtd3GvsQcfTQPW3sriVbCIlDZQq5eYbqxU/R0n0eux/RUgl7gxU4RaxpnZViZuU8bZPXiYBaXDQ51LnplZb3At2dp23PXV+eoyIY7u08bRlr9qCESLffA4qLNtbUJDn9vXpu+r05lzw4M38iM7+QmWNmPrNsKwAA8CCWfj3y+Yj48Yj41Ap7AQCAB7Do65HW2gsREemO6AMAgFVBiAQA6IgH3mln5icj4knxo+daax+Ze6HMvBURtyIiTt7w6OwNwuE5fl3G2Xe3b9cFuGy0s+WDeZfywMu31t69xoVaa89HxPMREY+86U3t6ObFn6dRX5XS6g7CTzHdfNjtx2kymHbzUVyv4h5xscpV4twRlVg9lOC8cC/NMZfSwIQKh25535N7ZJ3BBvPdGKUp7yrWtZDLWJN3oXtEDR+weQuOENmKH/53JB0oW7pHAADgsCy1/P1YZn41In4gIv57Zv7OOtsCAADFUvfIhyPiwyvtBQAAHgBfjwAAzERpYYdmo/O0WxwdX1TLhiP992M8nq67FnL1pf9oWtOVoODyylZvJ3DuREu3yyvETCVknv9ALLlzEOR52m4PhVg3Nb0ypb2wN32tQuxlZpXztJddz05CrwiGag8mr2xZd+dpC7HOCpxSiJxvbLAiohItC4Kji3et+3PhThsAoCMo2gAAHUHRBgDoCIo2AEBHULQBAGbiuncPyTZd9Nni6PiVC0ujGS2ew/TvynCkY1ubSrVuYrl68UfXTq1inStFOClsm7Zwiij3yfkPxFLFPWLS1twjztky/5wS3CN3Obh7pNAWXnGPqLfEKu6R+c4N6QixE+ELjpDCHqx7JKcf/nSTTmbCnTYAQEdQtAEAOoKiDQDQERRtAICZXFshMmOcCJHDqL/hVwJlM1/kN5FjMAfrjkK0dGdv64nlhTb2gsBpRcBSrBBDjRJZOQu7JFAacdIJopXrSS6rQLmC4GjjCyKgFBfXOE9bbMIKhoUp7/L874IQuUYbuzyfXwiLEVpwjIjIQQiRJnYu3GkDAHQERRsAoCMo2gAAHUHRBgDoCIo2AMBMlIHh0GzXxn500T3SjMsjlXvEvHDKPeJiB7HeRMt8REQ7VnlNrMprWtNl27xpFS+5UpSTohlnjHB5WNeGa4WXbfPzhzk4nONl7h4uAxVHiHJi3BflxjAp5D4KsW5ggspRmSrvXB7qudkJ66rd3Lo8pm8U7/woxOIeAQAABUUbAKAjKNoAAB1B0QYAmInTyA7JRtPYxzg+efnC2mja2AcpRJrJ7U3EmrxSMLSx0+tZMVTFmknzRzJvQeB0reKq5d1peur8bys4zj/re5V2dSdmzs5b2MQKWLFudgKXt5DCbaEyjV22x8+/Xqk1vSAYhlozsfLx5no2tiAiqtiIiCGnR264682FO20AgI6gaAMAdARFGwCgIyjaAAAdQdEGAJjJtXaPHB2/dGFtEM6PiFprunKgrNHyvtg9UnClOAuBdI+41nSV17lS1PVcXmdvkM/ZWVD0sgwttbyr1YVujhWodKZXhyDoFnJnQREOC3emQE5fzHS/uJJzQ+S1Ld0qtuLyWME9Iiep19rYB9rYAQCuNxRtAICOoGgDAHQERRsAYCZOnzokmwmRD51cFCJPRbt6hD503L1wSsy0beGjEheNGCr34NrNC6KlUJLsm0LltcKgak13YqjKURA4Tbw9T1smLgqfs/Me+J5kYXuyF/vmq7dK7DtnvrCnY3VeKaq5WPX8KuKieX1qremVM7JFC7p4/H2vJ3MgRAIAXBso2gAAHUHRBgDoCIo2AEBHULQBAGbiDAiHZBP3yElmPH5ycmHttjk1//Z4Olk7a3dMbGGoQGG4gm4hr7hSKm3shbyFlveSQ6PSHh8R0m2yhiOk4EDxe9uW0oH3BZdIhHNjrOAeKbWxz29NVzmcG0NN1qi1m1cGJhQmtw9TN8j99nZT7OO4craB4HK+0wEAQELRBgDoCIo2AEBHULQBAGaitLBDs8kOjjPj8ZMbF9Zum8OTb4/T9TMrWqpYnfe03Z6s3RGPP48tCJGF87R1+3dBtLSC4fy8tRZ095znC5Hy8YV2/DVoC+9V7DnUS6m0f1dzyPX55167g9B1C3lFtJwvLto9lM7Tnq6fGCHyxjB9n5wYEdGJizeHk8kaQiQAwDViUdHOzPdl5pcy8w8z88OZ+dhaGwMAgClL77Q/ERHvbK29KyL+KCLeu3xLAADgWFS0W2sfb639ldv80xHx1PItAQCAY83vtH86Iv7HivkAAC4Vp8YEcUge6B7JzE9GxJPiR8+11j5yN+a5iDiLiA/eJ8+tiLgVEfHo44/EG48vuhlOjRtDvUhn5nWT7hGjOFdcKXoPOvZMtN2fmrZ7lWNnHBNyGERlWIF1u1SGIMx3lfjYfbWxbz95XeGHEqjgakGYP6xAt4VXWt5XmMYu8g7GuXEkrudcF8rRcSycH+c5lCNEO6vU9W66vOZ9reKPF75VH1i0W2vvvt/PM/OnIuJHIuIHW/N/hlprz0fE8xERb/7eN27/5woAoEMW+bQz89mI+MWI+IettZceFA8AAMtY+p32f4qI10fEJzLzc5n5X1bYEwAAGBbdabfW/tZaGwEAgAdDR+Q14fU3pm37jkdu8E3XFpycfGfrLVxL/vo9R2rcD2VgODSbnD1ylBGPndzjHhmdc2O6trMuj+nfIBe7E8t3nCNErJ+aM010rM47FlwpKsdonDFjmx7Uftp04VZ7GEMXbhUboR2L2sGoAAAIpElEQVQvY+V+YI2BCYW8e6Ps/pgiC3ch72DOE1FujMG4MeRojoJzo5L3JHUJUs6Nyh6800S/L1XhviFyHJm31FFhby52LtxpAwB0BEUbAKAjKNoAAB1B0QYAmInSwg7NZkMQ/trxxUs7sU6tu1jdFq73oITPnRH2VA4nGCrh0+1B5XDatH5u5nUQz8OJiOp6Lm8lhxUt5arG5Zi7B3do/iGp3BU5oc21/qvm68G0ZFfERSWUueehcriWbpXXCYbqei5WiYNO7FM5rLgonsfJMH8P7npukMJcuNMGAOgIijYAQEdQtAEAOoKiDQDQERRtmPBkoa0X4DrhOrcPyTbukch47J4hCM65URmCcCZeUDcEQbpHTF61N+8eUWvO7TJdawUHi28rF7Gu5V2staYLt2+bn5f3PHapI0TjXuOtqbQsV++glHPDuzzU9fTe1JZtXpHDOSnUfl1sirxueIB6jStuDuUScTmce8Q5Zo5F/NIhCNxpAwB0BEUbAKAjKNoAAB1B0QYAmMm1FSJzjDh56eKljwf9Ytw4mq43EysGlsfOnEWsxMWKYFg5p9vnnS+GKhGwEut0OrUHO8/bXm8+SmitfA62/8isQ0WLMtqXFOtsjlLeKa6FXIuW84W9SqxvY1927nVJ4HT7NWe3q8H0uaONHQDg2kDRBgDoCIo2AEBHULQBADqCog0THj/ZRJ8GuPS4zu1Dso17pA1x45WHLqyNcSpjlVOkCUeJiz02f5ZkXudKETmacaWovTl3RcXBohwWa7hH3BAEVbgrLhHXrq5WK+4R10rfG841IWMLLg8/SEHtQVMZgrDYPbKCc0PtLUfj5hAujzSTOVQOm9d8OGQO3CMAANcHijYAQEdQtAEAOoKiDQAwE6d7HZJNhMij4Ua84eG3Xljb7V6Rsbvx9nRtp0XL3ek01gmcat0LkUq0lKE6h9EdlJi5Rl4l96kWf7cHl9cJhrI13W1tYd4Khz4mwolqc3Ft6ZW8VjAUuW1e9ZYotGn7N5B6fKH92+xB7reS18YqQfZExrr1o+GmWNOxc+FOGwCgIyjaAAAdQdEGAOgIijYAQEdQtGHCyXdpYwdQOGPDIdnGPXJ0Mx599K0X1s7OtHtkFE6R3dnUJRJhnCajcY+IdfV4l2M8NXmbiHUt+m3aP9sG7buouDwqrpRQsVks3GIfrs1/7uMdzkBQorK3PW2itIXKS1lwWPgc6vF7cnnYvNM3bKa2QCnnxpDG5SGcG0dHxvlxMnV+qMdHeEeIdI8cT9cqcKcNANARFG0AgI6gaAMAdARFGwBgJkoLOzTbCJEnD8WjTz19YW13+2UZu7szXT8TaxER49md6eNPTawSOJ1oKWLVWoQWIitiaGtaiFR5Xay6XjvThwarHC3mx7r4ZhrZlfjqaO6QYsUaAuU+KAmAtXsoJcyluQ/LELG5QqzYgxPlVA4nGMrYgghohUghOqo1m9eKlg/r6x3fmKwd39Cxc+FOGwCgIyjaAAAdQdEGAOgIijYAQEdQtAEAZuIMCIdkG/fIzYfi0e99x4W18c7U+RERcfby1P2xe8U4QkQO6zQ5ncYq94lbd7G7Quy4m647d4Ua/ODeQNIRMlbcIzXnh8oxFmIdJadJIe8hca4LHWsmVRRyDyaHinXXUw4U6x4Zpjmcw0LlcC3kam/D0dSJEaEdGkdizcWqNRt7omOdI2S4Ifb20Ibukcz895n5h5n5ucz8eGa+adFuAADgviz9euR9rbV3tdb+TkT8t4j4dyvsCQAADIuKdmvtW6/65+ui1EoAAABVFn+nnZm/EhH/KiK+GRH/ePGOAAAuKePpHfu99qHI1u5/c5yZn4yIJ8WPnmutfeRVce+NiIdaa79s8tyKiFt3//nOiPj8a9rx5efxiPja1pvYI1f5+fHc+uSqPLe/2Vp74kFBDyzac8nMvxERH2utvXNG7Gdaa8+scuFLxlV+bhFX+/nx3PrkKj83xVL3yNtf9c8fjYgvLdsOAADcj6Xfaf9qZr4jIsaI+EpE/OvlWwIAAMeiot1a+xev8aHPL7nuJecqP7eIq/38eG59cpWf24TVvtMGAID9w9kjAAAdsVnRvsot8Jn5vsz80t3n9+HMfGzrPa1FZv5EZn4hM8fMvBKKfWY+m5lfzswXM/OXtt7PmmTmBzLzLzLzyllsM/Mtmfm7mfnFu+/Jn9t6T4dgyzvtq9wC/4mIeGdr7V0R8UcR8d6N97Mmn4+IH4+IT229kTXI85OJfj0ifigino6I92Tm0/d/VFf8RkQ8u/Um9sRZRPxCa+3piPh7EfFvrtjvTrJZ0b7KLfCttY+31s7u/vPTEfHUlvtZk9baC621L2+9jxX5/oh4sbX2x621OxHxW3FuX70StNY+FRFf33of+6C19uettc/e/f/fjogXIuLN2+5q/2xyNOtfcU1a4H86Iv7r1psAy5sj4k9f9e+vRsTf3Wgv8BrJzLdGxPdFxO9tu5P9s9ei/aAW+NbacxHx3N0W+J+NCNkCfxmZ096fmc/F+X/CffCQe1vK3KMLAC4Dmfk9EfGhiPj5e/4L/kqy16LdWnv3zNAPRsTHoqOi/aDnlpk/FRE/EhE/2DrzVRZ+b1eBP4uIt7zq30/dXYMOyMyTOC/YH2yt/fbW+zkEW7pHrmwLfGY+GxG/GBH/vLX20tb7gfvy+xHx9sx8W2beiIifjIiPbrwnmEFmZkS8PyJeaK392tb7ORSbNddk5oci4kILfGvtStzhZOaLEXEzIv7y7tKnW2tXosU/M38sIv5jRDwREd+IiM+11v7ptrtaRmb+cET8h4g4iogPtNZ+ZeMtrUZm/mZE/KM4Pwnv/0bEL7fW3r/pplYiM/9BRPzPiPg/Ef9/Rt6/ba19bLtd7R86IgEAOoKOSACAjqBoAwB0BEUbAKAjKNoAAB1B0QYA6AiKNgBAR1C0AQA6gqINANAR/w882vmMOGw1OQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95e188f2e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# the function that I'm going to plot\n",
    "def f(x,y):\n",
    "    return x**2 + 5*y**2\n",
    "\n",
    "x = np.arange(-3.0,3.0,0.1)\n",
    "y = np.arange(-3.0,3.0,0.1)\n",
    "X,Y = np.meshgrid(x, y, indexing='ij') # grid of point\n",
    "Z = f(X, Y) # evaluation of the function on the grid\n",
    "\n",
    "plt.pcolor(X, Y, Z, cmap=plt.cm.gist_earth)\n",
    "plt.axis([x.min(), x.max(), y.min(), y.max()])\n",
    "plt.gca().set_aspect('equal', adjustable='box')\n",
    "plt.gcf().set_size_inches((6,6))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "def target_f(x):\n",
    "    return x[0]**2.0 + 5*x[1]**2.0\n",
    "\n",
    "def part_f(x, \n",
    "           f, \n",
    "           i, \n",
    "           h=1e-6):\n",
    "    w1 = [x_j + (h if j==i else 0) for j, x_j in enumerate(x)]\n",
    "    w2 = [x_j - (h if j==i else 0) for j, x_j in enumerate(x)]\n",
    "    return (f(w1) - f(w2))/(2*h)\n",
    "\n",
    "def gradient_f(x, \n",
    "               f, \n",
    "               h=1e-6):\n",
    "    return np.array([round(part_f(x,f,i,h), 10) for i,_ in enumerate(x)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 8.73043864e-04 -4.00026038e-12]\n",
      "Val: 0.000001, steps: 475\n"
     ]
    }
   ],
   "source": [
    "def SGD(target_f, \n",
    "        gradient_f, \n",
    "        x, \n",
    "        alpha_0=0.01,\n",
    "        toler = 0.000001):\n",
    "    alpha = alpha_0\n",
    "    min_val = float('inf')\n",
    "    steps = 0\n",
    "    iteration_no_increase = 0\n",
    "    trace = []\n",
    "    while iteration_no_increase < 100:\n",
    "        val = target_f(x)\n",
    "        if min_val - val > toler:\n",
    "            min_val = val\n",
    "            alpha = alpha_0\n",
    "            iteration_no_increase = 0\n",
    "        else:\n",
    "            alpha *= 0.95\n",
    "            iteration_no_increase += 1\n",
    "        trace.append(x)\n",
    "        gradient_i = gradient_f(x, target_f)\n",
    "        x = x - (alpha *  gradient_i)\n",
    "        steps += 1\n",
    "    return x, val, steps, trace\n",
    "\n",
    "x = np.array([2,-2])\n",
    "x, val, steps, trace = SGD(target_f, gradient_f, x)\n",
    "print(x)\n",
    "print('Val: {:.6f}, steps: {:.0f}'.format(val, steps))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " [1.90552394e-05 1.27080227e-05]\n",
      "Val: 0.000000, steps: 241\n"
     ]
    }
   ],
   "source": [
    "def SGD_M(target_f, \n",
    "        gradient_f, \n",
    "        x, \n",
    "        alpha_0=0.01,\n",
    "        toler = 0.000001,\n",
    "        m = 0.9):\n",
    "    \n",
    "    alpha = alpha_0\n",
    "    min_val = float('inf')\n",
    "    steps = 0\n",
    "    iteration_no_increase = 0\n",
    "    v = 0.0\n",
    "    trace = []\n",
    "    while iteration_no_increase < 100:\n",
    "        val = target_f(x)\n",
    "        if min_val - val > toler:\n",
    "            min_val = val\n",
    "            alpha = alpha_0\n",
    "            iteration_no_increase = 0\n",
    "        else:\n",
    "            alpha *= 0.95\n",
    "            iteration_no_increase += 1\n",
    "        trace.append(x)\n",
    "        gradient_i = gradient_f(x, target_f)\n",
    "        v = m * v + (alpha *  gradient_i)\n",
    "        x = x - v\n",
    "        steps += 1\n",
    "    return x, val, steps, trace\n",
    "\n",
    "x = np.array([2,-2])\n",
    "x, val, steps, trace2 = SGD_M(target_f, gradient_f, x)\n",
    "\n",
    "print('\\n',x)\n",
    "print('Val: {:.6f}, steps: {:.0f}'.format(val, steps))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f95df61df60>,\n",
       " <matplotlib.lines.Line2D at 0x7f95df652a58>]"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlUAAADFCAYAAAB9/wfYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl8VPW9//HXd7Zksi+EEJKQyCKLCIIgaC1aaRWrrXWrXvf+bG1te9XWarW2vV5vvb1t7WZrbXFpq221qHXDBRdwqSCrgAIiAgkECATIvs32/f1xhpBAINuQmeD72c7jzHzPmXM+yVF58z3f8z3GWouIiIiI9I0r3gWIiIiIHA0UqkRERERiQKFKREREJAYUqkRERERiQKFKREREJAYUqkRERERiQKFKREREJAYUqkRERERiQKFKREREJAY88TjooEGDbGlpaTwOLSIiItIjy5cv322tzetqu7iEqtLSUpYtWxaPQ4uIiIj0iDGmvDvb6fKfiIiISAwoVImIiIjEQJ9DlTGm2BizwBiz1hizxhhzYywKExERERlIYjGmKgTcbK1dYYxJB5YbY1611q6Nwb5FREREBoQ+91RZa3dYa1dE39cD64DCw32noq6qr4cVERERSSgxHVNljCkFJgGLO1l3nTFmmTFmWW2win+seiOWhxYRERGJq5iFKmNMGvAUcJO1tu7A9dba2dbaKdbaKcZ6+L8Vt7OltiJWhxcRERGJq5iEKmOMFydQ/d1a+6+uts9PKSRiQ3xt3rdpCjbFogQRERGRuIrF3X8GeAhYZ639VXe+k5OSSkrNVWxv2sSP3vkR1tq+liEiIiISV7HoqfoUcCVwhjFmZfT1+cN9wQBXnTCLll2zeKX8FR54/4EYlCEiIiISP32eUsFa+2+cnNQjs8YXcM8rM5g8spnfvfc7hmcO57Mln+1rOSIiIiJxEbcZ1UfkpVKam4qv5hIm5E3gtrdvY+WulfEqR0RERKRP4haqjDHMHJvP4o31/PzU3zAkdQjfnv9tNtdujldJIiIiIr0W12f/nTFmMIFwhPXbI9z/2ftxGzfXv3Y9u5t3x7MsERERkR6La6iaPCwbr9uwePNeitOL+cPMP7C3ZS/ffO2bNAYb41maiIiISI/ENVT5fW4mFmWxZPNeAI4bdBz3nHYPH1V/xM1v3EwwHIxneSIiIiLdFtdQBXDSMTm8X1FLUyAEwIyiGfz45B/zzvZ3uO3t2whFQnGuUERERKRrCRGqQhHLivKatrYLRl3ALVNu4ZXyV/jxOz8mYiNxrFBERESka32ep6qvTizJxhhYsaWaU0cNamu/6riraAm38Lv3fkeSJ4kfT/8xzuTtIiIiIokn7qEqPdnLyLw0Vm2tOWjddROuoznUzIPvP0iyO5lbp96qYCUiIiIJKe6hCmBicRZvrN+Ftfag0HTDpBtoCbXwt3V/I8mdxI2Tb1SwEhERkYQT9zFV4ISq3Q0BttU0H7TOGMOtU2/l4mMv5qEPHuIXy36hBzCLiIhIwkmInqoTirIAWLW1lqLslIPWG2P40fQf4XP7eHTto7SGWrlj+h24TEJkQhEREZHECFWjh6Tj87hYVVHDORMKOt3GGMP3p36fZHcyD33wEC3hFu465S7cLnc/VysiIiJysIQIVT6PizFD0lm7ve6w2xljuHHyjSR7krlv5X0EwgH+99P/i9fl7adKRURERDqXEKEKYFxBBq+s3dnpYPX2jDF8Y+I38Hv83LPsHuqD9fzqtF+R4j34sqGIiIhIf0mYQUljCzLY2xhgZ11rt7a/+rirufPkO3l3+7t8Zd5X9BBmERERiauECVXjhmYAsG7H4S8BtnfhsRdy7xn3srl2M1e8eAVltWVHqDoRERGRw0uYUDVmSDoAa3sQqsB5VuBDZz5Ec6iZK1+6kpW7Vh6J8kREREQOK2FCVXqyl5LclC4Hq3fm+LzjefTsR0n3pfPVV77Ky5tfPgIVioiIiBxawoQqgLFDMnrcU7XPsIxhPHr2o4zLHcctb93CvSvu1YOYRUREpN8kVKgaNzSDsj2NNLSGevX9XH8uD575IBeMuoAH3n+AGxfcSEOgIcZVioiIiBwssUJVQQbWwvrKOgg0wb+ug58MgVWPd3sfPrePO0++k9tPup23K97mihevYGvd1iNYtYiIiEiChaqx0TsA1+6oh/Uvwup/gtsLS2b3aD/GGC4bexl/+tyf2N2ym0teuIT5W+YfiZJFREREgAQLVUMzk8n0e53B6pvfgqRM+PR3Ydty2L2hx/ubVjCNx895nOL0Ym5ccCO/WPoLgpHgEahcREREPukSKlQZYxhXEB2sXvY2lJwCEy4F44LVc3q1z6L0Ih49+1EuHX0pj6x9hGtevoYdDTtiXLmIiIh80iVUqAJnZvWays2wdxMc82nIKIChk2DLol7v0+f2ccf0O/jFab9gY81GLp57MQu2LIhh1SIiIvJJl4ChKp2J4bXOh9JTnWXBRKhcDdb2ad+zSmfxz3P/SUFqATcsuIE7F95JU7CpjxWLiIiIJGSoymCEawcWFwwe5zQOmQAttVBT3uf9l2SU8PfP/51rx1/Lvzb8iwufu1CzsIuIiEifxSRUGWMeNsbsMsZ80Nd9jRycxjCzi3rfYOfOP4CCCc5yx6q+7h5wLgfedOJN/HnWn7FYrn75au5dcS/BsAaxi4iISO/EqqfqL8CsWOwo2etmlG8PO1z5+xsHHwfGDTtWx+IQbU7MP5Env/Ak5404jwfef4Avz/2yeq1ERESkV2ISqqy1bwF7Y7EvgCJTxcfB3P0N3mTIG+OMq4qxNF8ad33qLu6beR8NwQaueukqfrr4pzQGG2N+LBERETl69duYKmPMdcaYZcaYZVVVVYfeMNhCVmg361tyqGtpdzkufxxUfXjE6ptRNINnznuGS8dcymMfPsb5z57PWxVvHbHjiYiIyNGl30KVtXa2tXaKtXZKXl7eoTesdR4ps9Xmsb6yfn97zgio2QrBliNWY6o3lR9M+wGPnP0Ifo+fb73+LW6YfwMV9RVH7JgiIiJydEi4u/+odu7w22rz+HBH3f723BGAheqyI17CCYNP4IkvPMFNk2/i3R3vct4z53HfyvtoDjUf8WOLiIjIwJR4oaqmDIDapKGsO7CnCmDvxn4pw+f2ce3x1/L8l55nZslM/rjqj3zpmS8xr2weto/zZYmIiMjRJ1ZTKjwGLAJGG2MqjDHX9npn1eXg9pE7ZBjrOvRUDXeWe/onVO2Tn5rPz2f8nIfPephUXyrfe/N7XPHSFSzfubxf6xAREZHEFqu7//7DWltgrfVaa4ustQ/1emf1lZBewJihWayvrCcSifYK+bMhJRf2fByLknts6pCpPHHuE9x1yl1UNlRyzcvX8J/z/5NNNZviUo+IiIgklsS7/NdYBal5jC1IpykQZmt1u8fI5IxwngkYJ26Xm/NHnc/cC+Zy4+QbWVq5lPOfO587/n0HW+q2xK0uERERib/EDFVpgxkzJAOAdTvajavKHdHvl/864/f4+erxX+XFC17k8rGXM69sHl985ovc8e87KK/r+6N0REREZOBJzFCVOohj89NxGTqOq8oZDvXbIZgYd+HlJOdw69RbefnCl7l87OW8UvaKwpWIiMgnVGKFqkgEGndDah5+n5vSQal8WNkuVGUNc5Y1W+NT3yEM8g/ilqm38NKFL3Hl2CvbwtXNb9zM6qrYzwIvIiIiiSexQlVzNdgwpA4GYOyQDD5sP61CVomzrEnM8UuD/IP43tTv8dKFL3HNcdewaPsiLn/xcq566Spe3/I64Ug43iWKiIjIEZJYoapxl7NMHQTAmCHplO9porE15LRn7wtVZf1fWw8M8g/iOyd+h1cvfpXvT/0+u5p2cdOCm/jiM1/k8Q8f13MFRUREjkIJFqqizwRMc3qqxhQ4g9XbeqvShoDb1zbreqJL9aZyxbgrmHv+XO457R4ykzK5e/HdnDHnDP5n0f+wfu/6eJcoIiIiMeKJdwEdNOzrqXKeDTi2IB1wBqufWJINLhdkFifs5b9D8bg8nFV6FmeWnMnq3auZs34Oz258ljkfzWFi3kS+PPrLnFlyJsme5HiXKiIiIr2UYD1Vu51ldExVYZafTL+XNdvbDVbPLoGagdFTdSBjDBPzJnL3qXfz+sWvc+vUW6ltreWOf9/BGU84vVcrd63UY3BEREQGoMTqqWrcBcbtzJ6OE0LGF2bwwbba/dtklcD2lXEqMHYykzK5ctyVXDH2CpZULuHpj5/muY3PMeejOQxLH8YXRnyBL4z4AoVphfEuVURERLohwUKVM0cVrv0daOMLM/nzv8sIhCL4PC6np6p5L7TWQ1J6HIuNDWMM0wqmMa1gGo3TG3ml7BWe3/Q89628j/tW3sfkwZM5q/QsPlfyOfJS8uJdroiIiBxC4l3+S+0YHMYPzSQQjvDRzuhg9QSdqyoWUr2pnD/qfB4+62HmXTiPb5/wbeoCdfx0yU+Z+cRMrn7pav6x7h9UNVXFu1QRERE5QOKFqpTcDk3HF2YC7L8EmFHkLOu29Wdl/W5o2lC+PvHrPH3e0zxz3jNcP/H6DgHrqpeu4sH3H2RD9QaNwRIREUkAiXX5r7kaMjuOISrJTSE92cP722q5FCAzGqpqj76eqkMZkTWC60+4nutPuJ5NNZuYVz6PBVsW8NsVv+W3K35LYVohM4pmcFrRaUwZMoUkd1K8SxYREfnESbxQFR2kvo8xhvFDM/f3VKUPcQaz1x7dPVWHMjxrONdnXc/1E69nZ+NO3t72Nm9WvMnTG57msQ8fw+/xM23INKYPnc70gukMzxyOMSbeZYuIiBz1EidUWdtpqAIYX5jBXxeVEwxH8LrdkF4AtRWxPX44CB/8y7kDsegkGDYttvs/AvJT87no2Iu46NiLaAm1sKRyCW9ufZOF2xfyRsUbAOT585hWMI3pBdOZVjCNIalD4lu0iIjIUSpxQlVrvfPcv05DVSaBUIQNOxsYNzTDuQQYyzFVtdvg7xfBrrX728Z+Ac6fDb6U2B3nCEr2JDOjaAYzimYAUFFfweIdi1m8YzELty9k7qa5AJRklDBp8CQmD57MpMGTKMkoUU+WiIhIDCROqGqpcZbJWQetaj9Y3QlVhbBtRWyOG2yBf17h3E146T+geDos/zMsuBsePR+uega8/tgcqx8VpRdRlF7EhcdeSMRG2FC9gXd3vMuynctYsHUBz3z8DAA5yTmckHcCk/MnMzFvImNyxmhmdxERkV5InFDVXO0sO+mpKs1NJS3JGaz+5anFkFEI656HSKTDnFa9svBe2L4CLvk7jDnHaZvxPcgZDk/+P3juBrhgNgzg3hyXcTE6ZzSjc0Zz9XFXE7ERymrLWLFrBe/teo8VO1cwf+t8ANzGzciskYwfNJ5xueMYP2g8o7JG4XV74/xTiIiIJLYBEapcLsO4oRms3jdYPbMYwgFo2t328OVeH3Ph72HMuTD23I7rxl8AezbCgp/AiM/ACZf1/jgJxmVcDM8azvCs4Vx07EUAVDVVsXr3atbsXsOaPWt4tfxVntrwFAA+l4/ROaMZlzuOY7OP5djsYxmVPYpUb2o8fwwREZGEMiBCFcCkYVk8/O/NtATDJO+bdqG2om+havGfoLUWTr+98/Wfvhk2vg4v3wbDPwMZBb0/VoLLS8lj5rCZzBw2EwBrLRUNFazZs6YtaM3dNJfGYGPbdwrTCtsC1r5lcXoxXpd6tURE5JNnwISqKSU5/OnNTayuqOWkjHahqnBy744XicB7f4MRM2HI+M63cbngvPvg/lPghe86Y64G8GXAnjDGUJxeTHF6MbNKZwFO0NreuJ0N1Rv4qPqjtuWbFW8SsREAPMZDUXoRpZmlHJNxDKWZpZRmlFKaWUp2UrYGxYuIyFErAUPVwQPVAU4sccLWsvK9nDS12Gnsyx2AZW85E4h+7r8Pv13uCDjjh/DKD+GDp+D4i3p/zAHOGENhWiGFaYWcXnx6W3truJVNNZvYULOBzbWbKasto6yujHe2vUMwEmzbLsOXwTGZxzAsfRhF6UUUphW2LQenDMZlEmuCfxERkZ5IrFDl8R/yTrucVB/D81JZXlYNp40AT3Lf5qpa+RgkZ8Loc7redvo3Yc3TzmXAkTMP2Zv2SZXkTmJs7ljG5o7t0B6OhNneuL1D0CqrK2PpzqXM3TQXy/7H63hd3rbAVphWSGF6IUNShpCfmk9+Sj6DUwbjc/v6+0cTERHptgQKVTWH7KXaZ0pJNq+s3UnEgiujsPehKhyE9S85c1F5uzF9gMsN5/4aZp8Or9/lvJcuuV3utkuI++bP2icQDrCjcQfb6rdR0VBBRUNF2/sP9nxAbWvtQfvLSc4hPyW/LWgNSR1Cfko+eSl55CbnkuvPJSspSz1eIiISFwkUqjqfTb29KSU5zFlWwabdDYzMLOz95b8ti5wB6qPP7v53CibCtG/Au/fDxMugeGrvji0A+Nw+SjJKKMko6XR9Q6CBXU27qGyqZGfjTnY2RV+NO9nesJ0VO1dQF6g76Htu4yY7OZtB/kFtQWvfMic5h0H+QWQnZ5PpyyQzKRO/x69xXiIiEhMJFKpqugxVk6PjqpaXVzMysxg2zu/dsda/BO4kZ6qEnvjMD2DNMzD3O3DdG+BOnF/f0SbNl0aaL43hWcMPuU1TsIldTbuoaq5iT8se9jRHX+3eb6zdyJ7mPR3GdrXnc/nISsoiIymDrKQsspKyyExyAlf7z+m+dNK8Tk37lrrLUURE2kucVNBcDTnHHHaTEXmpZKd4WVZWzSW5hVBf6VzK6+nElB+9DMfMAF8P51lKSoez/w/mXAVL/gQnf6tn35eYSvGmOHcXZpYedjtrLfXB+ragVdNaQ21rbafLzbWbqQ0470OR0GH3m+xO3h+yokEr3ZdOqjeVNG9aWxBL8aaQ4knB7/Hj9/qdZfS1rz3Zk6zLliIiA1xMQpUxZhbwW8ANPGit/b8e76S5GvyTujoOJ5Zks7y8GoYXAhbqd0DWsO4fp3Yb7N0EU7/W4xIBGPtFGHUmzL8bRn++yyC4TzhiWVVRw5LNe/loZz0bdzUQrt/F1JaFjI18TI6po9Uks9NTyIbUyVRmTmRQRirDclIYlptCcU4Kw3JSyE316XJVDxljyPBltN192B3WWppCTdS01lDTWkNjoJH6YD0NgQYagg1ty/pAPY3B/euqmqqc9cGGDnN6dUf7sNU+cPk9fpI8SSS5k/C5fW3LZHdy2+f2r8Nts++91+3FYzy4Xe7e/EpFRKQTfQ5Vxhg3cB/wOaACWGqMec5au/bw3zxAS02nz/070IklOby2bhe1vnwywRms3pNQtWWRsyw5uUfltTEGzvmVM3fV09+Ar7zoDGTvhLWWpWXV/HPpVl7/cCc1Tc4lqGPTg9zqe4LTA/PwmCCNSdk0+HLxhirJCizCVTeHhvpU3mIKj7WezK8j44ng9GJkJHsYMTiNkXlpjBicxoi8NEYOTqM424/HfZiejnAIAg1Ozx7WufPRk9S738EngDGGVG8qqd5UCtMKe7WPcCRMY6iRpmATzaFmmkPNHd43h5ppCrX7HDzgc3T7mtYaApEAgXCAllALgXCA1nArgUigzz+ny7jwurx4XV48Ls/B76Phy+s+xPpOtvW4PLiMC4/Lg9u4cbvcuI27Q9u+9y7jwm3c+79jom0ut/Pe1cX66Pt9+zQYZ2mcpYt276Pr3S53h3aD0V9URCQmYtFTdRLwsbV2E4Ax5nHgPKD7oSochGBTt0LVtOE5ALxXm8rp4PQ89UT5O+BLh/zje/a99rKK4fP3wNPXOc8OPPU7HVYHgiGeW7KZR95Yz/adteS4w1xUmsFJY9OYbNfhX/QbaG2kZfgs7IgzIa0AfziCDYeob2nEVq7BblvFyTtWc3JwKWFfBjVZ46nwj2VLUyZV77eyp6GVtS1B1gIG8BjITfUyKNXHUF8jeZEqsoNVpLbswhuoxRWoBxudwmDfTAZurzONhS8FfGnO5dC2ZfS9x0fEuIngJowhHI4QCYeIRMLOKxzGhkPYcBAbCUMkBOEQkUgQImFMJISNhDGRICa63mKJ4CKCIWINEQzWuLAYLC6scZZt9bazr8XQ/mextP8j0bSbqmH/+47bHM6h/3zt/h+8JnrEA7/hBtIMpPV4/y4gOfpyOL9HS/igZaTt/b72MJEOn+1B74l+K0SEJiJYrKHtO7bd/pxtD2yHMJZgu/0NNMYY2v4X/YfgwLZ9Z7St/YA2FwbatTvLtiPsf3/A9zrU0Un7gW0H/pNlMDj/Nwe1Hbh9l/vqpL1920FHNs6yY2tnuv735+B/9w7+TndaurXFIX7vh99PN/4b0MkmR2rf3fqO9Fn+9NO6vW0sQlUhsLXd5wpg2oEbGWOuA64DGDbsgJ6lluhdXMkZXR5sQmEm6UkeFuzwRUPV1i6+cYDyRTBsWrcHmdtQiFBVFcHKSkI7dxKsrCS8ezfh2jrCa44jvOD3hNPnEW4KEGlsJNTUgisY4DjgZ53srwoAn/OavxxYfpijJ0VfAGvJZy353f5BIQzsvz8u/TBbNkVfsdO+z+zAf+0NTrjQhScREUl0m4Pd/2tivw1Ut9bOBmYDTJkypWMXREuNs0zO7HI/HreL6SNymb+5jv/2Z0PNlu4X0bgHqtZ1Oiu6tZZgeTnNq1fT+tFHtG7cROumjQS3VjiPtGnHeL24s7JwpWfiNtV4A+WEx81kyR7Y3BAmMzON6eMKGVmci9vvxwRqMO89hKt2I2bMLJh+HSbJj3G7we3BeNzgdmM8nv1tbtf+y4oGTEstfPwqbFsG21c4lz1tyAksxg2puZAzHAaPwQ4eR0POcexILqGqMcKu+lYaAmGaA2EaA2GaghGaAxEixkZ/9g6/CDJNI1mRWrJsNX7bQrIJ4jMhfCaMy+3B5fHhcntxeby4PD6MNxmXJxnjTcbtTcblTcLl8+P2JeP2JePx+vF4XHjdLrwug8ftwuMyeN0u3C7T4djs6+2KhJ2/SXZIZAf+Nc1EfzntlxzcdiQv7Rxm39Ye/Pvd32YP6KE4uFcuEQ20vxe73IruItI3Y4yBn/+jW9vGIlRtA4rbfS6KtnVfS3Six26EKoBTRw7i1bU7aR1WTFJPQlXbeKpPARAoL6d+wQIaFy6kZdVqwrVOHcbrxVdaSvKYsWScfTbegqF4C4bgyR+Cd0g+royMtj8QA9tWwUNnsS38Kv/I/QGXnnUaV0wvccJCOATLHoJX/wtyk+CaP8Fx53e/3g4Gw7BR+z9aC6EW55WU6TynsJ2s6KvjHOfSnwZaABERkb6JRahaCowyxhyDE6YuBS7r0R5a913+62aoGjUIgEqTT0nN5u4fZ8siWhv91M75N/Xz/4fAZue7vuHDSfvcZ/FPnIh/4kSShg/HeLr+1Sz8eDc/fLaarObv86j/l7zo+j6m9hpYdbzTg/b+HOdOwxEznQczZxR0v9auGOM80ucQj/URERGR/tXnUGWtDRljvg3Mwxkm87C1dk2PdrKvpyqp6zFVAMMHpVKSm8Ka5mxKGt90Ls+5Dn3nm41EaJg/nz0/f57mbdng/iup06aRfdllpJ1+Gr7i4kN+tzN7Glr5yQvrePq9bQzLSeGGay4ndfAF8NqdsPQB5/IVwLBT4MyfOFMv6O4iERGRo1pMxlRZa18EXuz1Dnp4+c8Yw8wx+Sxeksbn3QFoqISMoZ3VRf1rr7H73ntp3fAx3tQQg780hcyb78WTl9erUhes38UtT6ymtjnAf54xkm99ZiTJ3ui4jYv/DMEW5/E5qXndGngvIiIiR4fEmFG9h6EKYObYwcxelOf0jVWXHxSqWjduZOfdd9O4cBG+ESMYevPVZGz9Kebqr0EvAlUwHOHuF9bxl4VljM5P59FrT2JsQSehyZsMuSN6vH8REREZ2BLjuRgtdWBczrxI3XTSMTlU+6JjlGrK29qttex95FE2f+l8mj9YQ/4Pf8jwZ58hc3jQubOu6KQel7e7oZXLH1zMXxaWcc0ppTz77U91HqhERETkEytxeqqS0g87LupAXreL8eOOg7UQ3LMZLxCurWX7bbfTsGABaaefTsHdP8GTm+t8oXwhFEyEpO4HN4APttXy9UeXs7uhld9ccgJfmtS72bVFRETk6JY4oaoHl/72+fyk4exck0WwfD35lZVs/drXaC0rJ/8HPyD7yiv2zwMUbIFty+Gknj3vb96aSm58/D1yUnw8+Y1TOL6o5zWKiIjIJ8OADlUnj8hlpRlKxofrKHvkMiJ1dQx7YDap06d33HD7Cgi3Qskp3d73Y0u2cMfT7zOhKIsHr57CoDQ9J09EREQOLTFCVWudM4FlD7ldBndyKZFnVmBTvJT+7VGSx3Yy3WX5O85yWNcPUbbW8vv5H/PLVz/i9NF5/OHyyaT4EuPXJCIiIokrQQaq966nKlRdTcqLmyEMS669qfNABc7z/gaPg5Scw+4vErHc+dwafvnqR1wwqZAHrpqiQCUiIiLdMmBDVSQQoOIb1xOpaaJ4xl7e2baVpkDo4A3DIdi6uMteqtZQmBsef4+/LirnuhnDuefiiXjdifHrERERkcSXGKmhF6Gq6le/pnnVKobeeTspeQHyWsv527vlB2+4830INBx2PFVdS5CrH17C3NU7uP3sMfzg82NxuTQDuoiIiHRf/ENVJAKt9T2afbzhrbfY+5e/kH355WScfyn40pmRvZc/vrmJ2qZgx43LFzrLQ4SqytoWvvzHRSwrq+bXl0zk66dp4k4RERHpufiHqtY6wHa7pypUVcX2224nafRoBt96i/NMvUGjOCVzN7XNQX760rqOX9j8NmQf0+ljbD7aWc+F9y9k694m/vyVqZw/qSgGP5CIiIh8EsU/VPXwETU7f/ZzIg0NFP7ql7iSotMcDB5H2t51fPXUUh5fupUX39/htIcCUPY2jDjjoP08tbyC837/DoFwhH9+/WQ+Pap3zwIUERERgUSYUqG1zll2I1Q1LV1K3dy5DPrm9SSNaHeZbtg0WPk3vjvZxdKyLL7zz5WkJXmY4VvvjKdqF6r2Nga4+4V1PLWigmnH5HDvf0wiPyM51j+ViIiIfMLEP1Tt66lKOvyCMgZ9AAALI0lEQVSYKhsKUfmTu/EUFJD7tQNmRi92JvtM2r6EB666hMsfXMw1f17CX4e9zKnGTU3+yWytqOHlDyr527vlNAbC/OcZI7lx5ig8usNPREREYiBxQlUXPVXVc+bQun49hb/5NS6/v+PKQaPAnwNbFpM7+Sqeuv4UfvLCWvJWLmApI/nyz98FnOFXZ40bwnfPPJZj89OPxE8jIiIin1ADIlRFWlvZ/Yf7SZk6lfSzzjp4A2Ng2HTYsgiA1CQPPz0ZWL2V1RN+yI8Gj2NwehLTh+eSl67HzYiIiEjsJUCo6npMVc1TTxHevZtB99yz/yHJBzpmBqx/ESrfhyHHw6rHweVlwqxrmdDFTOoiIiIifRX/AUVdjKmywSB7H3wI/8SJpEw76dD7mXgpeFPg3fuhcTesegxGz+ry0TQiIiIisZAAPVW14EsDd+el1L7wAsHt28n/0Q8P3UsF4M+GEy6HFX+FnR9AoBFO+/4RKlpERESko8QIVYe49GcjEfbMfoCk0aNJO/30rvf16e9C1YdQ/g6c80vnMqCIiIhIP4h/qGo9dKhqWryYwKZNDP35zw7fS7VPxlC4Zq4z6afHF+NCRURERA4tMcZUHWI8Vc0TT+LKzOz8jr/DUaASERGRfpYYoaqTnqpwTQ31r71G5rnn7n8cjYiIiEiCSthQVfv8XGwgQNZFF8ahKBEREZGeSYBQVXdQqLLWUvPkkySPG0fy2LFxKkxERESk++IbqqyN9lR1HFPV8sEaWtevJ+vii+JUmIiIiEjPxDdUBRrBhg/qqaqb+zzG6yXjnHPiVJiIiIhIz8Q3VHXy3D8biVA37xVSTz0Vd0bndwWKiIiIJJo+hSpjzMXGmDXGmIgxZkqPd9Aafe5fuykVWlavJlRZScasHk6jICIiIhJHfe2p+gC4AHirV9/upKeqbt4r4PWSdsYZfSxNREREpP/0aUZ1a+06oHuznXemLVRltTU1LFhA6rRpuNPT+1KaiIiISL/qtzFVxpjrjDHLjDHLqqqqnMbmamfpd0JV6+bNBMrKuvecPxEREZEE0mVPlTHmNWBIJ6vusNY+290DWWtnA7MBpkyZYgFornFW+rMBaHjzTQCFKhERERlwugxV1trPHrGjt0RDVXRMVcObb5I0aiS+osIjdkgRERGRIyG+Uyo0Vzt3/rncRJqbaV62nNRTPx3XkkRERER6o69TKpxvjKkATgZeMMbM69EOmmvaxlM1LVuODQZJPeWUvpQkIiIiEhd9vfvvaeDpXu+gubrtzr/GRYswXi8pU07sS0kiIiIicRHnGdX391Q1LlyIf/JkXH5/XEsSERER6Y04j6mqAX824ZoaWj/8kNTp0+JajoiIiEhvxX+genIWTStWAJAydWpcyxERERHprfiFKmvbLv81LV2G8flIPv74uJUjIiIi0hfxC1XBZggHwJ9N07Jl+CdMwJWUFLdyRERERPoifqEq+oiaCCm0rF2Lf+qUuJUiIiIi0lfxC1XR2dSbK+ohHCZl0qS4lSIiIiLSV3HsqYqGqrI9APgnTIhbKSIiIiJ9FffLf80bKvCVluLOyopbKSIiIiJ9FdfLf9ZC87qN+CdOjFsZIiIiIrEQ156qYKOb8N5qkifq0p+IiIgMbPELVY1VtNQ4j6TxT1BPlYiIiAxscQxVu2lpyACvl6RjR8WtDBEREZFYiF+oathFS7WXpFEjcfl8cStDREREJBbiFqpswy5aqiIkjxsXrxJEREREYsYTrwOHdlURbkKhSkRERI4KceupaqmoA8CvUCUiIiJHgfiEqkiYlj0GjCFp9Oi4lCAiIiISS3EKVSFaaz34huTg8vvjUoKIiIhILMUpVAVprfWSdExxXA4vIiIiEmvxCVWhIIF6t+anEhERkaNGXEJVpKUFMCSNHR+Pw4uIiIjEXFxClW1tBSD5+BPjcXgRERGRmItPT1UgiHFbvCWl8Ti8iIiISMzFp6cqECIp14Nxu+NxeBEREZGYi0+oCkXwFeTE49AiIiIiR0R8QlUYkko1nYKIiIgcPeL2mBrfsWPjdWgRERGRmOtTqDLG/MIY86ExZrUx5mljTFZ3v5t0/El9ObSIiIhIQulrT9WrwHhr7QTgI+D2bn3LWHzHn9LHQ4uIiIgkjj6FKmvtK9baUPTju0BRd75n3AbjT+vLoUVEREQSSizHVP0/4KVDrTTGXGeMWWaMWWbjNpJLRERE5MjwdLWBMeY1YEgnq+6w1j4b3eYOIAT8/VD7sdbOBmYDnDh+rO1VtSIiIiIJqstQZa397OHWG2OuAc4FZlpruxWWTHJqt4oTERERGSi6DFWHY4yZBdwKnGatbYpNSSIiIiIDT19HN/0eSAdeNcasNMb8MQY1iYiIiAw4feqpstaOjFUhIiIiIgOZ7sMTERERiQGFKhEREZEYMN28YS+2BzWmHljf7weWI2kQsDveRUjM6bwefXROj046r0dWibU2r6uN+jSmqg/WW2unxOnYcgQYY5bpnB59dF6PPjqnRyed18Sgy38iIiIiMaBQJSIiIhID8QpVs+N0XDlydE6PTjqvRx+d06OTzmsCiMtAdREREZGjjS7/iYiIiMSAQpWIiIhIDPRrqDLGzDLGrDfGfGyMua0/jy19Y4x52BizyxjzQbu2HGPMq8aYDdFldrTdGGPujZ7n1caYyfGrXA7FGFNsjFlgjFlrjFljjLkx2q7zOoAZY5KNMUuMMaui5/W/o+3HGGMWR8/fP40xvmh7UvTzx9H1pfGsXw7NGOM2xrxnjJkb/axzmmD6LVQZY9zAfcDZwDjgP4wx4/rr+NJnfwFmHdB2G/C6tXYU8Hr0MzjneFT0dR1wfz/VKD0TAm621o4DpgPfiv47qfM6sLUCZ1hrJwInALOMMdOBnwG/jj6ztRq4Nrr9tUB1tP3X0e0kMd0IrGv3Wec0wfRnT9VJwMfW2k3W2gDwOHBePx5f+sBa+xaw94Dm84C/Rt//FfhSu/ZHrONdIMsYU9A/lUp3WWt3WGtXRN/X4/zHuhCd1wEten4aoh+90ZcFzgCejLYfeF73ne8ngZnGGNNP5Uo3GWOKgHOAB6OfDTqnCac/Q1UhsLXd54pomwxc+dbaHdH3lUB+9L3O9QATvTwwCViMzuuAF71MtBLYBbwKbARqrLWh6Cbtz13beY2urwVy+7di6YbfALcCkejnXHROE44GqktMWGduDs3PMQAZY9KAp4CbrLV17dfpvA5M1tqwtfYEoAjnKsGYOJckfWCMORfYZa1dHu9a5PD6M1RtA4rbfS6KtsnAtXPf5Z/ocle0Xed6gDDGeHEC1d+ttf+KNuu8HiWstTXAAuBknMu1+5732v7ctZ3X6PpMYE8/lyqH9yngi8aYMpyhM2cAv0XnNOH0Z6haCoyK3q3gAy4FnuvH40vsPQdcHX1/NfBsu/aroneLTQdq211OkgQRHWPxELDOWvurdqt0XgcwY0yeMSYr+t4PfA5nvNwC4KLoZgee133n+yJgvtWs0AnFWnu7tbbIWluK82fnfGvt5eicJpx+nVHdGPN5nOvCbuBha+3d/XZw6RNjzGPA6cAgYCfwX8AzwBxgGFAOfNlauzf6h/Xvce4WbAK+Yq1dFo+65dCMMacCbwPvs3+cxg9wxlXpvA5QxpgJOIOU3Th/cZ5jrb3LGDMcp5cjB3gPuMJa22qMSQYexRlTtxe41Fq7KT7VS1eMMacD37PWnqtzmnj0mBoRERGRGNBAdREREZEYUKgSERERiQGFKhEREZEYUKgSERERiQGFKhEREZEYUKgSERERiQGFKhEREZEY+P+0yy8hUXlBVwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95df63a9e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x2 = np.array(range(len(trace)))\n",
    "x3 = np.array(range(len(trace2)))\n",
    "plt.xlim([0,len(trace)])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.plot(x3, trace2)\n",
    "plt.plot(x2, trace, '-')"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}