08-softmax-one-vs-rest.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "%config IPCompleter.greedy=True\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import sklearn.preprocessing, sklearn.datasets, sklearn.model_selection\n",
    "from progressbar import progressbar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At the end of the previous notebook, I briefly showed one-vs-rest classification technique and normalization of the distribution. We may put this normalization after the sigmoid activation functions - that exactly, what softmax is suppose to do."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Softmax\n",
    "\n",
    "Formally, we may define the softmax as follow:\n",
    "\n",
    "$$\n",
    "softmax(\\pmb{x})_i = \\frac{e^{x_i}}{\\sum_k e^{x_k}}\n",
    "$$\n",
    "\n",
    "In other words, we apply exponential to each input of the softmax and then normalize the probability distribution.\n",
    "\n",
    "Note that softmax is just a generalization of the sigmoid activation.\n",
    "\n",
    "$$\n",
    "softmax(x, 0) = \\frac{e^x}{e^x + e^0} = \\frac{e^x}{e^x + 1} \\cdot \\frac{e^{-x}}{e^{-x}} = \\frac{e^{x-x}}{e^{x-x} + e^{-x}} = \\frac{e^0}{e^0 + e^{-x}} = \\frac{1}{1+e^{-x}} = \\sigma(x)\n",
    "$$\n",
    "\n",
    "Now, when we have the formula, it is easy to implement the sigmoid in NumPy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def softmax(x):\n",
    "    return np.exp(x) / np.sum(np.exp(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But why bother with sigmoid and not just normalize the outputs of sigmoids? It turns out, we may get better numerical properties if we join sigmoids and normalization into one \"layer\". Thanks to the following equation.\n",
    "\n",
    "$$\n",
    "softmax(\\pmb{x}+c)_i=\\frac{e^{x_i+c}}{\\sum_k e^{x_k+c}} = \\frac{e^c}{e^c} \\cdot \\frac{e^{x_i}}{\\sum_k e^{x_k}} = softmax(\\pmb{x})_i\n",
    "$$\n",
    "\n",
    "The danger in the softmax is the exponents when the divisor can become too big for float values and as a result, become infinity. However, we may set $c=max(\\pmb{x})$ and as all the scalars are negative or zero, there would be no overflow. Cases, where the divisor becomes zero are much more unlikely and even if some scalars are so small, that they are round to zero because of the float precision, it doesn't matter for users to distinguish between their actual value and zero. \n",
    "\n",
    "This way, we have our modified softmax function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def softmax(x):\n",
    "    x = x - np.max(x)\n",
    "    return np.exp(x) / np.sum(np.exp(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradient\n",
    "\n",
    "Before we move on, we need to know how to compute the gradient. That is not that easy as with softmax, as we have multiple inputs and multiple outputs. You may try it by hand if you wish.\n",
    "\n",
    "$$\n",
    "\\frac{\\partial softmax(\\pmb{x})_i}{\\partial x_j} = \\frac{\\partial \\frac{e^{x_i}}{\\sum_k e^{x_k}}}{\\partial x_j}\n",
    "$$\n",
    "\n",
    "There are three indices! Moreover, this is derivative only with respect to one input variable. We need to compute the gradient of every output with respect to every input. I don't want to dig too much into the math, you can see [[1]](#Bibliography) if you wanna know more. In the end, the gradient pops out, so we may implement it.\n",
    "\n",
    "$$\n",
    "\\frac{\\partial softmax(\\pmb{x})_i}{\\partial x_j} = \n",
    "\\begin{cases}\n",
    "    softmax(\\pmb{x})_j - softmax(\\pmb{x})_j \\cdot softmax(\\pmb{x})_i & \\text{if } i = j \\\\\n",
    "    0 - softmax(\\pmb{x})_j \\cdot softmax(\\pmb{x})_i & \\text{if } i \\neq j \\\\\n",
    "\\end{cases}\n",
    "$$\n",
    "\n",
    "You may notice something strange - the gradient is a table. Remember, we compute the gradient of each output with respect to each input. That need's to be a table. But we want only gradients with respect to the inputs - we need to sum the gradients over the outputs, the same way, as we sum gradients of multiple examples. I won't show the implementation just yet, as I believe it would be more helpful to view the code and the explanation in the bigger picture."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I mentioned **layer** a while ago. The fact is, the softmax is really a layer, how we understand them in the context of neural networks. We will implement our first simple neural networks in the very next notebook. It took us eight notebooks, but we finally have enough knowledge to do that. However, we need some refactoring of the code before we can do that."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Refactoring\n",
    "\n",
    "Let's start with the **layer** implementation itself. Maybe I should tell first, what the layer is. During the neural network training (we may think about the logistic regression as about a simple neural network) we are exploiting the chain rule. You may remember from school the chain rule.\n",
    "\n",
    "$$\n",
    "\\frac{\\partial f(g(h(x)))}{\\partial x} = \\frac{\\partial f(g(h(x)))}{\\partial g(h(x))} \\cdot \\frac{\\partial g(h(x))}{\\partial h(x)} \\cdot \\frac{\\partial h(x)}{\\partial x}\n",
    "$$\n",
    "\n",
    "This is not different from our model, remember that loss of logistic regression was written as:\n",
    "\n",
    "$$\n",
    "\\mathcal{L}(\\sigma(\\pmb{x}\\pmb{w})\n",
    "$$\n",
    "\n",
    "And we need to update the weights.\n",
    "\n",
    "$$\n",
    "\\frac{\\mathcal{L}}{\\partial \\pmb{w}} = \\frac{\\partial \\mathcal{L}}{\\partial \\sigma} \\cdot \\frac{\\partial \\sigma}{\\partial \\pmb{x}\\pmb{w}} \\cdot \\frac{\\partial \\pmb{x}\\pmb{w}}{\\partial \\pmb{w}}\n",
    "$$\n",
    "\n",
    "Each part of the gradient will be one layer. We will use the same layers for neural networks later on. We already implemented some - for example the loss function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CategoricalCrossEntropyLoss:\n",
    "    def __call__(self, target, predicted):\n",
    "        indices = np.arange(len(target))\n",
    "        return -np.log(np.maximum(predicted[indices,target], 1e-15))\n",
    "    \n",
    "    def gradient(self, target, predicted):\n",
    "        grad = np.zeros((len(target), 10))\n",
    "        indices = np.arange(len(target))\n",
    "        grad[indices,target] = -1 / np.maximum(predicted[indices,target], 1e-15)\n",
    "        return grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice, that the layer accepts the output of the previous layer (in this case in shape `(batchsize, classes)`) and it returns its gradient with respect to its input - again in the shape `(batchsize,classes)`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We may use the same approach - let's now implement the sigmoid layer. It takes inputs and applies the sigmoid activation function on each of them. It is simple to implement it right away, but there is one problem. If you carefully look at the chain rule, the gradients are multiplied by the gradients of the following layer. In other words, by computing the gradient of the sigmoid, we need to compute this:\n",
    "\n",
    "$$\n",
    "\\frac{\\mathcal{L}}{\\partial \\pmb{w}} = \\frac{\\partial \\mathcal{L}}{\\partial \\sigma} \\cdot \\frac{\\partial \\sigma}{\\partial \\pmb{x}\\pmb{w}}\n",
    "$$\n",
    "\n",
    "As a result, the gradient function doesn't accept only the layer's input, but gradient from the following layer as well. Now we may implement the sigmoid activation layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class SigmoidLayer:\n",
    "    def __call__(self, inputs):\n",
    "        return 1 / (1 + np.exp(-inputs))\n",
    "    \n",
    "    def gradient(self, inputs, gradients):\n",
    "        outputs = self(inputs)\n",
    "        my_gradient = outputs * (1 - outputs)\n",
    "        return my_gradient * gradients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's look at the softmax layer. This case is a bit complicated, as the gradient is not a vector, but a table. In general, most layers that have multiple inputs and multiple outputs will generate gradient table (note that for each example), as it needs to compute gradient of each output with respect to each input. The sigmoid activation layer, and activation layers in general, are exceptions. Each output corresponds to one input and as such the inputs don't interfere. You may look at it from the other side, the gradient of output $i$ in respect to input $j$ is zero if $i \\neq j$. That results in a table full of zeros except for diagonals.\n",
    "\n",
    "But let's return back to the softmax layer. We already implemented it above, but only for a single example. We need to generalize it for a batch of examples. The only thing we need to make sure about is to divide inputs in the correct axis - the generalization is simple."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "def batch_softmax(inputs):\n",
    "    inputs = inputs - np.max(inputs)\n",
    "    return np.exp(inputs) / np.sum(np.exp(inputs), axis=-1)[:,np.newaxis]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's look at the gradient, the formula is above. It would be simpler to rewrite it in the matrix form, so we may implement it in the numpy. Notice that the only difference is on the diagonal (when $i - j$) where we add additional term. In a matrix notation it results in following equation (as you may see in [[1]](#Bibliography) as well):\n",
    "\n",
    "$$\n",
    "\\frac{\\partial softmax(\\pmb{x})}{\\partial \\pmb{x}} = \n",
    "\\begin{pmatrix}\n",
    "o_1 & 0 & \\dots & 0 \\\\ \n",
    "0 & o_2 & \\dots & 0 \\\\\n",
    "\\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "0 & 0 & \\dots & o_k\n",
    "\\end{pmatrix}\n",
    "-\n",
    "\\begin{pmatrix}\n",
    "o_1 o_1 & o_1 o_2 & \\dots & o_1 o_k \\\\ \n",
    "o_2 o_1 & o_2 o_2 o_2 & \\dots & o_2 o_k \\\\\n",
    "\\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "o_k o_1 & o_k o_2 & \\dots & o_k o_k\n",
    "\\end{pmatrix}\n",
    "$$\n",
    "\n",
    "The vector $\\pmb{o}$ is the output of softmax $softmax(\\pmb{x})=\\pmb{o}$. Notice the symmetry of the matrices, we will exploit it in a minute. If we have variable `outputs` that contains the output of the softmax function, we may implement the matrix notations to the NumPy very simply."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "def softmax_gradient(inputs):\n",
    "    outputs = softmax(inputs)\n",
    "    return np.diag(outputs) - outputs @ outputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are three things we need to deal with before we may continue. Firstly, the problem is the matrix of the gradient - remember we want the gradient with respect to the inputs of the softmax, so we may pass it to the previous layer (remember the chain rule I showed you before). Now we have a matrix - gradient of each output with respect to each input. All we need is to sum up the gradients over the outputs. As a result, we get a vector, that represents the \"accumulated\" gradient with respect to each input. I may formulate it in other way around - we receive a gradient of how much we want to shift each output. Each output tells us, how much it wants to shift it's input so the output is shifted the correct way (and therefore we get the matrix). In the end, we want to satisfy all the outputs - we sum the effect of each output on the inputs and as a result, we get the vector that represents the gradient with respect to the inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "def softmax_gradient(inputs):\n",
    "    outputs = softmax(inputs)\n",
    "    return np.sum(np.diag(outputs) - outputs @ outputs, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second problem that I didn't account for yet is the gradient from the next layer. Once again, remember the chain rule. The gradient of the softmax need's to be multiplied with the gradient of the following layer (for example the cross-entropy loss). We just need to make sure, we multiply the gradient along a correct axis. We think about the matrix above as a gradient of outputs with respect to the inputs, in other words at index $i,j$ is gradient of output $j$ with respect to input $i$, we need to multiply each row and then sum the gradients over columns. As I pointed out, the matrix is in this case symmetric and as such doesn't matter in which way we multiply it - we just need to make sure, we multiply it in the different axis that we are summing it later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "def softmax_gradient(inputs, gradients):\n",
    "    outputs = softmax(inputs)\n",
    "    my_gradient = np.diag(outputs) - outputs @ outputs\n",
    "    return np.sum(gradients[np.newaxis,:] * my_gradient, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, the third and the last problem - we didn't count with the batches. The code above won't work for multiple examples, as with softmax, we need to generalize it. We can't rely on vector and matrix multiplication, as we have higher dimensional objects. The code is bellow and it gets a bit nasty, I add comments about the shapes of NumPy arrays, so you may better understand it. I wish I could write it more readable, but this is as far as I could get.\n",
    "\n",
    "There are two more attributes - `params` and `grads`. We don't need them yet and I will talk about them in a while, for now just ignore them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "class SoftmaxLayer:\n",
    "    def __init__(self):\n",
    "        self.params = []\n",
    "        self.grads = []\n",
    "    \n",
    "    def __call__(self, inputs):\n",
    "        inputs = inputs - np.max(inputs)\n",
    "        return np.exp(inputs) / np.sum(np.exp(inputs), axis=-1)[:,np.newaxis]\n",
    "    \n",
    "    def gradient(self, inputs, gradients):\n",
    "        outputs = self(inputs)  # examples, classes\n",
    "        examples, classes = outputs.shape\n",
    "        diag = np.zeros((examples, classes, classes))  # examples, classes, classes\n",
    "        diag[:, np.arange(classes), np.arange(classes)] = outputs # set the diagonal of each example\n",
    "        my_gradient = diag - outputs[:,:,np.newaxis] * outputs[:,np.newaxis,:]  # examples, classes, classes\n",
    "        return np.sum(gradients[:,np.newaxis,:] * my_gradient, axis=2) # examples, classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dense layer\n",
    "\n",
    "Now it is time for our last layer. If you remember, the logistic regression had vector of weights, that was multiplied with the example vector (or matrix when we had batch). For the one-vs-rest approach, we had multiple logistic regressions and when we were processing the examples, we need to pass it through every neuron. As turns out, we can join the weights of all the neurons together into the weights matrix.\n",
    "\n",
    "$$\n",
    "\\mathbb{X}\\mathbb{W}\n",
    "$$\n",
    "\n",
    "Moreover, we may export the bias explicitly and get rid of the additional $1$ that we needed to pad the examples with. As a result, we may express the whole loss of multiclass logistic regression in the following formula.\n",
    "\n",
    "$$\n",
    "\\text{CrossEntropy}(\\text{softmax}(\\mathbb{X}\\mathbb{W}+\\mathbb{b}))\n",
    "$$\n",
    "\n",
    "The layer, that is responsible for the linear combination (the $\\mathbb{X}\\mathbb{W}+\\mathbb{b}$) is called **dense layer** or **fully-connected layer**. That is exactly what we will now focus on.\n",
    "\n",
    "We have the formula for calculating the output of the layer. Now we need to compute the gradient. As we have only linearities, the gradients are simple as well.\n",
    "\n",
    "$$\n",
    "\\frac{\\partial \\mathbb{X}\\mathbb{W}+\\mathbb{b}}{\\partial \\mathbb{W}} = \\mathbb{X} \\\\\n",
    "\\frac{\\partial \\mathbb{X}\\mathbb{W}+\\mathbb{b}}{\\partial \\mathbb{b}} = sign(\\mathbb{b}) \\\\\n",
    "\\frac{\\partial \\mathbb{X}\\mathbb{W}+\\mathbb{b}}{\\partial \\mathbb{X}} = \\mathbb{W}^T\n",
    "$$\n",
    "\n",
    "But now we have three gradients - with respect to weights, to bias, and to inputs. All previous layers don't have learning parameters, so we need only derivatives with respect to the inputs. Now, we need also derivatives with respect to the weights and bias. That's why I defined the `params` and `grads` attributes. Attribute `params` is list of learnable parameters and `grads` attributes contain gradients of these parameters. We will use them during the optimization process.\n",
    "\n",
    "From the implementation of view, both the forward and backward pass is very simple, we just need to make sure the axis align. As with the softmax, I put comments about the implementation and about the resulting shape of the variable. Let's see it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "class DenseLayer:\n",
    "    def __init__(self,inputs, outputs, random_seed=None):\n",
    "        self._random_state = np.random.RandomState(random_seed)\n",
    "        self._W = self._random_state.uniform(-2,2,size=(inputs, outputs))\n",
    "        self._b = self._random_state.uniform(-2,2,size=(outputs,))\n",
    "        self.params = [self._W, self._b]\n",
    "        self.grads = [np.zeros_like(self._W), np.zeros_like(self._b)]\n",
    "    \n",
    "    def __call__(self, inputs):\n",
    "        return inputs @ self._W + self._b[np.newaxis,:]\n",
    "    \n",
    "    def gradient(self, inputs, gradients):\n",
    "        # gradient in respect to W\n",
    "        w_grad = inputs[:,:,np.newaxis] * gradients[:,np.newaxis,:]  # examples, inputs, outputs\n",
    "        np.add(self.grads[0], np.sum(w_grad, axis=0), out=self.grads[0])  # inputs, outputs\n",
    "        # gradient in respect to b\n",
    "        b_grad = gradients  # examples, outputs\n",
    "        np.add(self.grads[1], np.sum(b_grad, axis=0), out=self.grads[1])  # outputs\n",
    "        # gradient in respect to inputs\n",
    "        in_grad = self._W[np.newaxis,:,:] * gradients[:,np.newaxis,:] + np.sign(self._b)[np.newaxis, np.newaxis, :]  # examples, inputs, outputs\n",
    "        return np.sum(in_grad, axis=2) # examples, inputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is a small space for optimization. We may tell NumPy to use existing memory instead of creating ita gain and again at every pass. This optimization comes handy, as the dense layer is the most computationally demanding layer. The following code is equivalent to the previous one but uses cache memory in the size of *(examples,inputs,outputs)* to speed up the computation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "class DenseLayer:\n",
    "    def __init__(self,inputs, outputs, random_seed=None):\n",
    "        self._random_state = np.random.RandomState(random_seed)\n",
    "        self._W = self._random_state.uniform(-2,2,size=(inputs, outputs))\n",
    "        self._b = self._random_state.uniform(-2,2,size=(outputs,))\n",
    "        self.params = [self._W, self._b]\n",
    "        self.grads = [np.zeros_like(self._W), np.zeros_like(self._b)]\n",
    "        self._cache = None\n",
    "    \n",
    "    def __call__(self, inputs):\n",
    "        return inputs @ self._W + self._b[np.newaxis,:]\n",
    "    \n",
    "    def gradient(self, inputs, gradients):\n",
    "        # create the cache\n",
    "        if self._cache is None or self._cache.shape[0] != inputs.shape[0]:\n",
    "            self._cache = np.ndarray((inputs.shape[0],inputs.shape[1], gradients.shape[1]))\n",
    "        # gradient in respect to W\n",
    "        w_grad = np.multiply(inputs[:,:,np.newaxis], gradients[:,np.newaxis,:], out=self._cache)  # examples, inputs, outputs\n",
    "        np.add(self.grads[0], np.sum(w_grad, axis=0), out=self.grads[0])  # inputs, outputs\n",
    "        # gradient in respect to b\n",
    "        b_grad = gradients  # examples, outputs\n",
    "        np.add(self.grads[1], np.sum(b_grad, axis=0), out=self.grads[1])  # outputs\n",
    "        # gradient in respect to inputs\n",
    "        in_grad = np.multiply(self._W[np.newaxis,:,:], gradients[:,np.newaxis,:], out=self._cache)  # examples, inputs, outputs\n",
    "        return np.sum(in_grad, axis=2) # examples, inputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have all the layers we need. Last thing we need to refactor is the gradient descent algorithm."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gradient descent optimizer\n",
    "\n",
    "This refactoring will be very fast. We still don't have the `Model` class, but I may tell you in advance that it would have `layers` property, that would be a list of individual layers. As each layer has `params` and `grads` attributes (that's why they are even in the softmax layer), the optimizer can use that. The optimizer will have `optim` method, that accepts the model and perform one step of the gradient descent algorithm.\n",
    "\n",
    "We will have better optimizers later on, that's why I wanted to export it outside of the model itself. Our gradient descent algorithm can look like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "class SGD:\n",
    "    def __init__(self, learning_rate):\n",
    "        self.learning_rate = learning_rate\n",
    "        \n",
    "    def optim(self, model):\n",
    "        for layer in model.layers:\n",
    "            for params, grad in zip(layer.params, layer.grads):\n",
    "                np.add(params, - self.learning_rate * grad, out=params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model\n",
    "\n",
    "We are almost finished. Now we will define `Model` class, which would represent the model itself. It will contain layers in the `layers` attribute, as I mentioned before. Other than that, It will have standard methods we have seen in the logistic regression.\n",
    "\n",
    "The prediction is done layer by layer, the code can look something like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "zero_grad()\n",
    "current_grad = loss.gradient(batch_target, outputs[-1])\n",
    "for layer, layer_input in zip(self.layers[::-1], outputs[-2::-1]):\n",
    "    current_grad = layer.gradient(layer_input, current_grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will remember the outputs of layers (there are inputs to the following layers) in the `outputs` variable and use them during the backpropagation phase. Firstly, we will compute the gradient of the loss and then pass it to the `gradient` method of the last layer. The gradient will propagate through the layers (from the last to the first layer) until we have gradients of weights. As our code accumulates the gradients of the weights (notice the `np.add` call), we need to zero the gradients first. The code can look similar to this: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "self.zero_grad()\n",
    "current_grad = loss.gradient(batch_target, outputs[-1])\n",
    "for layer, layer_input in zip(self.layers[::-1], outputs[-2::-1]):\n",
    "    current_grad = layer.gradient(layer_input, current_grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the end, we call our optimizer to update the weights. The rest of the code should be straightforward, as it is just copied from the previous notebooks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MulticlassLogisticRegression:\n",
    "    def __init__(self, input_dim, classes, loss, metrices = [], random_seed = None):\n",
    "        self._rand = np.random.RandomState(random_seed);\n",
    "        self.loss = loss\n",
    "        self.metrices = metrices\n",
    "        self.layers = [\n",
    "            DenseLayer(input_dim, classes, self._rand.randint(100000)),\n",
    "            SoftmaxLayer()\n",
    "        ]\n",
    "        \n",
    "    def zero_grad(self):\n",
    "        for layer in self.layers:\n",
    "            for grad in layer.grads:\n",
    "                grad.fill(0)\n",
    "    \n",
    "    def fit(self, X, y, optimizer, Xtest=None, ytest=None, epochs=100, batch_size=32, progress=False):\n",
    "        # get how many data we have\n",
    "        n_train_data = len(X)\n",
    "        n_test_data = 1 if ytest is None else len(ytest)\n",
    "        # store gradients and losses\n",
    "        train_losses = np.zeros((epochs,))\n",
    "        test_losses = np.zeros((epochs,))\n",
    "        train_metrices = np.zeros((len(self.metrices), epochs))\n",
    "        test_metrices = np.zeros((len(self.metrices), epochs))\n",
    "        # decide whatever to log progress\n",
    "        epoch_counter = progressbar(range(epochs)) if progress else range(epochs)\n",
    "        # Learning\n",
    "        outputs = [None] * (len(self.layers) + 1)\n",
    "        for epoch in epoch_counter:\n",
    "            # shuffle the data\n",
    "            permutation = self._rand.permutation(n_train_data)\n",
    "            # for each batch\n",
    "            for batch_start in range(0, n_train_data, batch_size):\n",
    "                # get batch\n",
    "                batch_data = X[permutation[batch_start:batch_start+batch_size]]\n",
    "                batch_target = y[permutation[batch_start:batch_start+batch_size]]\n",
    "                # forward pass\n",
    "                outputs[0] = batch_data\n",
    "                for layer, i in zip(self.layers, range(len(self.layers))):\n",
    "                    outputs[i+1] = layer(outputs[i])\n",
    "                # backward pass\n",
    "                self.zero_grad()\n",
    "                current_grad = self.loss.gradient(batch_target, outputs[-1])\n",
    "                for layer, layer_input in zip(self.layers[::-1], outputs[-2::-1]):\n",
    "                    current_grad = layer.gradient(layer_input, current_grad)\n",
    "                # update the weights\n",
    "                optimizer.optim(self)\n",
    "                # store loss\n",
    "                train_losses[epoch] += np.sum(self.loss(batch_target, outputs[-1]))\n",
    "                # compute the metrices\n",
    "                for metric in self.metrices:\n",
    "                    metric(batch_target, outputs[-1])\n",
    "            # store train metrices\n",
    "            for num_metric, metric in enumerate(self.metrices):\n",
    "                train_metrices[num_metric, epoch] = metric.summary()\n",
    "                \n",
    "            # evaluate on the test set\n",
    "            if Xtest is not None and ytest is not None:\n",
    "                # for each batch\n",
    "                for batch_start in range(0, n_test_data, batch_size):\n",
    "                    # get batch\n",
    "                    batch_data = Xtest[batch_start:batch_start+ batch_size]\n",
    "                    batch_target = ytest[batch_start:batch_start + batch_size]\n",
    "                    # predict the data\n",
    "                    prediction = self.predict(batch_data)\n",
    "                    # store loss\n",
    "                    test_losses[epoch] += np.sum(self.loss(batch_target, prediction))\n",
    "                    # compute the metrices\n",
    "                    for metric in self.metrices:\n",
    "                        metric(batch_target, prediction)\n",
    "                # store test metrices\n",
    "                for num_metric, metric in enumerate(self.metrices):\n",
    "                    test_metrices[num_metric, epoch] = metric.summary()\n",
    "          \n",
    "        results = {\n",
    "            \"train_loss\": train_losses / n_train_data, \n",
    "            \"test_loss\": test_losses / n_test_data,      \n",
    "        }\n",
    "        results.update({f\"train_{metric.name}\": train_metrices[num_metric] for num_metric in range(len(self.metrices))})\n",
    "        results.update({f\"test_{metric.name}\": test_metrices[num_metric] for num_metric in range(len(self.metrices))})\n",
    "        return results\n",
    "    \n",
    "    def predict(self, X):\n",
    "        for layer in self.layers:\n",
    "            X = layer(X)\n",
    "        return X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I had some doubts about which parameters should be in the constructor and which one in the `fit` method, so don't be afraid of the differences with the model in the previous notebook. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training\n",
    "\n",
    "Now let's put out model in action. First, we need the data. Notice that I don't pad them by 1 this time - the bias term in the dense network cares of it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = sklearn.datasets.fetch_openml('mnist_784', version=1, return_X_y=True)\n",
    "y, X = y.astype(int), X.reshape(-1, 784)\n",
    "X[X < 128] = 0\n",
    "X[X > 0] = 1\n",
    "train_data, test_data, train_target, test_target = sklearn.model_selection.train_test_split(X, y, test_size=0.3, random_state=47)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will also need our accuracy metric. As we modified the output of the model a little bit (we return probability distribution instead of class), the accuracy is a bit different than in the previous notebook. I ass `np.argmax` call to get the predicted class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "class AccuracyMetric:\n",
    "    def __init__(self, name=None):\n",
    "        self.name = name or \"accuracy\"\n",
    "        self.correct = 0\n",
    "        self.num = 0\n",
    "    def __call__(self, target, predicted):\n",
    "        self.correct += np.sum(np.argmax(predicted,axis=1) == target)\n",
    "        self.num += len(target)\n",
    "    def summary(self):\n",
    "        acc = self.correct / self.num\n",
    "        self.correct = 0\n",
    "        self.num = 0\n",
    "        return acc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we may create the optimizer, model, and train it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (400 of 400) |######################| Elapsed Time: 0:27:09 Time:  0:27:09\n"
     ]
    }
   ],
   "source": [
    "optimizer = SGD(learning_rate=0.001)\n",
    "model = MulticlassLogisticRegression(784, 10, loss=CategoricalCrossEntropyLoss(), metrices=[AccuracyMetric()], random_seed=42)\n",
    "result = model.fit(train_data, train_target, optimizer, test_data, test_target, epochs=400, batch_size=128, progress=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you remember, the running time of 10 logistic regression were around 11 minutes in the previous notebook. It may be surprising the 27 minutes in this case. The fact is, that we compute something, we don't need to right now. When we call the `gradient` method on the dense layer, it returns the gradient for the previous layer - but we don't have any. This computation is the most demanding one and accounts for the additional 15 minutes of the training. I could just ignore it and do not implement it, but we are going to need it in the following notebook.\n",
    "\n",
    "Let's now look at the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train accuracy: 0.9334285714285714, Test accuracy: 0.9087142857142857\n"
     ]
    }
   ],
   "source": [
    "print(f\"Train accuracy: {result['train_accuracy'][-1]}, Test accuracy: {result['test_accuracy'][-1]}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs8AAAHSCAYAAAAT0iZvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdeXiV5Z3/8fedhYQECFvYdwVkB0HFqlW6uNZabadjq12cttaZ6TJOa9X+psuMM2On21hbq6Wt3WxtbauOu9ZWRVs3UBQQlFUICIQtEAJku39/3EEQUY4kJ4dw3q/r4jI5z3PO+Z5Ae32eO9/ne4cYI5IkSZIOrCDXBUiSJEkdheFZkiRJypDhWZIkScqQ4VmSJEnKkOFZkiRJypDhWZIkScpQUa4LeCt69+4dhw0blusyJEmSdJibM2fOhhhj5b6Pd6jwPGzYMGbPnp3rMiRJknSYCyG8vL/HbduQJEmSMmR4liRJkjJkeJYkSZIy1KF6niVJkrRHQ0MDVVVV7Ny5M9eldFilpaUMGjSI4uLijM43PEuSJHVQVVVVdO3alWHDhhFCyHU5HU6MkY0bN1JVVcXw4cMzeo5tG5IkSR3Uzp076dWrl8H5IIUQ6NWr11tauTc8S5IkdWAG59Z5qz+/VoXnEMKKEMK8EMLcEMLrBjCHEI4KITweQtgVQvjiXo8PDiE8FEJYGEJYEEL4fGvqkCRJUvvbsmULP/zhDw/quWeeeSZbtmzJ+Pyvf/3rfPvb3z6o92pLbbHyPCPGODnGOG0/xzYBnwP2/aSNwBdijGOA6cA/hxDGtkEtkiRJaidvFp6bmpre9Ln33HMP3bt3z0ZZWZXVto0Y4/oY49NAwz6PvxJjfKbl623AQmBgNmuRJElS27riiitYunQpkydP5rLLLuPhhx9mxowZfPjDH2bChAkAvO9972Pq1KmMGzeOmTNnvvrcYcOGsWHDBlasWMGYMWP41Kc+xbhx4zj11FPZsWPHm77v3LlzmT59OhMnTuTcc89l8+bNAFx77bWMHTuWiRMncv755wPwyCOPMHnyZCZPnsyUKVPYtm1bqz5za6dtROCBEEIEfhRjnHmgJ+wrhDAMmAI82cpaJEmS8ta/37mAF9ZsbdPXHDugG187e9wbHv/GN77B/PnzmTt3LgAPP/wwTz31FPPnz391esWNN95Iz5492bFjB8cccwzvf//76dWr12teZ/Hixdx88838+Mc/5oMf/CB//OMfufDCC9/wfT/60Y/y/e9/n5NPPpmvfvWr/Pu//zvXXHMN3/jGN1i+fDklJSWvtoR8+9vf5rrrruOEE06gtraW0tLSVv1MWrvyfEKM8WjgDFLrxdvfypNDCF2APwL/EmPc7992COHiEMLsEMLs6urqVpYrSZKkbDr22GNfM/bt2muvZdKkSUyfPp1Vq1axePHi1z1n+PDhTJ48GYCpU6eyYsWKN3z9mpoatmzZwsknnwzAxz72MWbNmgXAxIkTueCCC7jpppsoKkprxCeccAL/+q//yrXXXsuWLVteffxgterZMcY1Lf9dH0K4DTgWmJXJc0MIxaTg/OsY461v8h4zgZkA06ZNi62pV5Ik6XD1ZivE7am8vPzVrx9++GEefPBBHn/8ccrKyjjllFP2OxaupKTk1a8LCwsP2LbxRu6++25mzZrFHXfcwVVXXcWCBQu44oorOOuss7jnnnuYPn06Dz74IEcdddRBvT60YuU5hFAeQui6+2vgVGB+hs8NwE+BhTHG7x5sDZIkScqdrl27vmkPcU1NDT169KCsrIxFixbxxBNPtPo9Kyoq6NGjB48++igAv/rVrzj55JNpbm5m1apVzJgxg29+85ts2bKF2tpali5dyoQJE7j88suZNm0aixYtatX7t2bluS9wW8tsvCLgNzHG+0IIlwDEGG8IIfQDZgPdgOYQwr8AY4GJwEeAeSGEuS2v9+UY4z37e6MQwtnA2UceeWQrypUkSVJb6tWrFyeccALjx4/njDPO4KyzznrN8dNPP50bbriBiRMnMnr0aKZPn94m7/uLX/yCSy65hLq6OkaMGMHPfvYzmpqauPDCC6mpqSHGyKWXXkr37t35yle+wkMPPURhYSFjx47ljDPOaNV7hxg7TifEtGnT4uzZrxsnnVUrNmznqeWbOHNif7qUuJu5JEk6dCxcuJAxY8bkuowOb38/xxDCnP2NYnaHwQOY8/JmvvTH59lYuyvXpUiSJCnHDM8HUFiQtmxsau44K/SSJEnKDsPzAewOz80dqL1FkiRJ2WF4PoDd4bnRlWdJkqS8Z3g+gIJg24YkSZISw/MBFO1u22jOcSGSJEnKOcPzAexp2zA9S5Ik7W3Lli388Ic/POjnX3PNNdTV1e332CmnnEJ7jyjOhOH5AAq8YVCSJGm/shmeD1WG5wMofLXnOceFSJIkHWKuuOIKli5dyuTJk7nssssA+Na3vsUxxxzDxIkT+drXvgbA9u3bOeuss5g0aRLjx4/nd7/7Hddeey1r1qxhxowZzJgx403f5+abb2bChAmMHz+eyy+/HICmpiY+/vGPM378eCZMmMD//u//AnDttdcyduxYJk6cyPnnn9/mn9kt8w7AOc+SJKlDuPcKWDuvbV+z3wQ44xtvePgb3/gG8+fPZ+7cuQA88MADLF68mKeeeooYI+9973uZNWsW1dXVDBgwgLvvvhuAmpoaKioq+O53v8tDDz1E79693/A91qxZw+WXX86cOXPo0aMHp556KrfffjuDBw9m9erVzJ8/H0ir4LtrWr58OSUlJa8+1pZceT4Aw7MkSVJmHnjgAR544AGmTJnC0UcfzaJFi1i8eDETJkzgwQcf5PLLL+fRRx+loqIi49d8+umnOeWUU6isrKSoqIgLLriAWbNmMWLECJYtW8ZnP/tZ7rvvPrp16wbAxIkTueCCC7jpppsoKmr7dWJXng+gsOXyosmeZ0mSdCh7kxXi9hJj5Morr+TTn/70647NmTOHe+65hyuvvJJTTz2Vr371qxm/5v706NGD5557jvvvv5/rrruOW265hRtvvJG7776bWbNmcccdd3DVVVexYMGCNg3RrjwfQGFB+hE1u/IsSZL0Gl27dmXbtm2vfn/aaadx4403UltbC8Dq1atZv349a9asoaysjAsvvJAvfvGLPPPMM/t9/v4cd9xxPPLII2zYsIGmpiZuvvlmTj75ZDZs2EBzczPvf//7ueqqq3jmmWdobm5m1apVzJgxg29+85ts2bLl1VraiivPB7D7hkF3GJQkSXqtXr16ccIJJzB+/HjOOOMMvvWtb7Fw4UKOP/54ALp06cJNN93EkiVLuOyyyygoKKC4uJjrr78egIsvvpgzzjiD/v3789BDD+33Pfr378/VV1/NjBkziDFy5plncs455/Dcc89x0UUX0dwyTvjqq6+mqamJCy+8kJqaGmKMXHrppXTv3r1NP3N4o6XwQ9G0adNie8/7W7CmhrOufYwbLpzK6eP7tet7S5IkvZmFCxcyZsyYXJfR4e3v5xhCmBNjnLbvubZtHEDR7raNDnSRIUmSpOwwPB/AqzcM2rYhSZKU9wzPB1AQHFUnSZKkxPB8AM55liRJh7KOdP/aoeit/vwMzwfwanj2H6YkSTrElJaWsnHjRgP0QYoxsnHjRkpLSzN+jqPqDsCVZ0mSdKgaNGgQVVVVVFdX57qUDqu0tJRBgwZlfL7h+QAK7XmWJEmHqOLiYoYPH57rMvKKbRsHsHvl2VF1kiRJMjwfwO7w3NhkeJYkScp3hucDKHDlWZIkSS06RHgOIZwdQphZU1PT7u9d5A2DkiRJatEhwnOM8c4Y48UVFRXt/t6vbpLiyrMkSVLe6xDhOZdeHVVnz7MkSVLeMzwfQKErz5IkSWpheD6AgoJACNBsz7MkSVLeMzxnoDAEGg3PkiRJec/wnIGCgmDbhiRJkgzPmSgqCLZtSJIkyfCcCds2JEmSBFkKzyGE0hDCUyGE50IIC0II//4G550SQpjbcs4j2ailLRS48ixJkiSgKEuvuwt4R4yxNoRQDDwWQrg3xvjE7hNCCN2BHwKnxxhXhhD6ZKmWViuy51mSJElkKTzHGCNQ2/JtccuffdPnh4FbY4wrW56zPhu1tIWCgkBTc66rkCRJUq5lrec5hFAYQpgLrAf+FGN8cp9TRgE9QggPhxDmhBA+mq1aWqswBJqaTc+SJEn5LmvhOcbYFGOcDAwCjg0hjN/nlCJgKnAWcBrwlRDCqH1fJ4RwcQhhdghhdnV1dbbKfVOFrjxLkiSJdpi2EWPcAjwMnL7PoSrgvhjj9hjjBmAWMGk/z58ZY5wWY5xWWVmZ7XL3q7Ag0GzPsyRJUt7L1rSNypYbAgkhdAbeBSza57T/A04KIRSFEMqA44CF2aintQoLHFUnSZKk7E3b6A/8IoRQSArot8QY7wohXAIQY7whxrgwhHAf8DzQDPwkxjg/S/W0SkHAUXWSJEnK2rSN54Ep+3n8hn2+/xbwrWzU0JaKCgpoMjxLkiTlPXcYzECBc54lSZKE4TkjhQW48ixJkiTDcyYKbduQJEkShueMFAYcVSdJkiTDcyYKCwKNTYZnSZKkfGd4zkChNwxKkiQJw3NGCguCc54lSZJkeM5EQXCHQUmSJBmeM1JYELxhUJIkSYbnTBQVBEfVSZIkyfCciYJgeJYkSZLhOSOFrjxLkiQJw3NGHFUnSZIkMDxnxFF1kiRJAsNzRgodVSdJkiQMzxlx5VmSJElgeM6IPc+SJEmCDhKeQwhnhxBm1tTU5OT9CwoCTc05eWtJkiQdQjpEeI4x3hljvLiioiIn718YAk3NpmdJkqR81yHCc64551mSJElgeM5IYUHA7CxJkiTDcwYKCwKNtm1IkiTlPcNzBtKoulxXIUmSpFwzPGegMDiqTpIkSYbnjBS03DAYDdCSJEl5zfCcgaKCAOBNg5IkSXnO8JyBwpbw7Lg6SZKk/GZ4zkBB2L3ybHiWJEnKZ4bnDOxu22h05VmSJCmvGZ4zUGDbhiRJkjA8Z6QwZWeaDc+SJEl5zfCcgULbNiRJkkQrwnMIYXAI4aEQwsIQwoIQwuf3c84pIYSaEMLclj9f3etY9xDCH0IIi1pe4/iDrSXbCgvSj8kbBiVJkvJbUSue2wh8Icb4TAihKzAnhPCnGOML+5z3aIzxPft5/veA+2KMHwghdALKWlFLVhW2XGLY8yxJkpTfDnrlOcb4SozxmZavtwELgYGZPDeE0A14O/DTlufXxxi3HGwt2bZ7VJ3hWZIkKb+1Sc9zCGEYMAV4cj+Hjw8hPBdCuDeEMK7lsRFANfCzEMKzIYSfhBDK26KWbCgqNDxLkiSpDcJzCKEL8EfgX2KMW/c5/AwwNMY4Cfg+cHvL40XA0cD1McYpwHbgijd4/YtDCLNDCLOrq6tbW+5BeXXl2Z5nSZKkvNaq8BxCKCYF51/HGG/d93iMcWuMsbbl63uA4hBCb6AKqIox7l6p/gMpTL9OjHFmjHFajHFaZWVla8o9aLunbTiqTpIkKb+1ZtpGIPUsL4wxfvcNzunXch4hhGNb3m9jjHEtsCqEMLrl1HcC+95oeMhwh0FJkiRB66ZtnAB8BJgXQpjb8tiXgSEAMcYbgA8A/xhCaAR2AOfH+Grvw2eBX7dM2lgGXNSKWrLKGwYlSZIErQjPMcbHgHCAc34A/OANjs0Fph3s+7enV9s27HmWJEnKa+4wmIEC2zYkSZKE4TkjRd4wKEmSJAzPGSm051mSJEkYnjOyu23DOc+SJEn5zfCcgd1tG648S5Ik5TfDcwYKDM+SJEnC8JyR3T3PjqqTJEnKb4bnDOye89zYZHiWJEnKZ4bnDLhJiiRJksDwnJHCV3uec1yIJEmScsrwnIGCsHuHQdOzJElSPjM8Z6DItg1JkiRheD6wpQ/R944P05sa2zYkSZLynOH5QOo20nnlw3QL22l2zrMkSVJeMzwfSHFnADpTT6PhWZIkKa8Zng+kqBSAEuppsudZkiQprxmeD6S4DIDOYZdtG5IkSXnO8Hwgtm1IkiSpheH5QHavPOPKsyRJUr4zPB9Iy8pzabDnWZIkKd8Zng9kd3imniZXniVJkvKa4flAXu153mV4liRJynOG5wMp2nPDoOFZkiQpvxmeD6SgAIpKKQv1NNvzLEmSlNcMz5koKqVzaHBUnSRJUp7rEOE5hHB2CGFmTU1NbgooLnOTFEmSJHWM8BxjvDPGeHFFRUVuCijuTFmw51mSJCnfdYjwnHMtK8+2bUiSJOU3w3MmiksppcEbBiVJkvKc4TkTxZ2d8yxJkiTDc0aKy+jsqDpJkqS8Z3jORHFnStlFY5PhWZIkKZ8ZnjNRXEYJ9TS58ixJkpTXDM+ZKCqlNNY751mSJCnPZSU8hxBuDCGsDyHMf4PjFSGEO0MIz4UQFoQQLspGHW1md9uG4VmSJCmvZWvl+efA6W9y/J+BF2KMk4BTgO+EEDplqZbWKy6jhF3srG/KdSWSJEnKoayE5xjjLGDTm50CdA0hBKBLy7mN2ailTRR3poDIrl11ua5EkiRJOVSUo/f9AXAHsAboCvx9jLE5R7UcWHFnAOp3Gp4lSZLyWa5uGDwNmAsMACYDPwghdNvfiSGEi0MIs0MIs6urq9uzxj1awnPTzu25eX9JkiQdEnIVni8Cbo3JEmA5cNT+TowxzowxTosxTqusrGzXIl9VXAZAU70rz5IkSfksV+F5JfBOgBBCX2A0sCxHtRxYy8pzs+FZkiQpr2Wl5zmEcDNpikbvEEIV8DWgGCDGeANwFfDzEMI8IACXxxg3ZKOWNlGUwnNB404ampopLnQ8tiRJUj7KSniOMX7oAMfXAKdm472zomXluTTUs31XI93LDt2pepIkScoel1Az0RKeO7OL2l2H7kQ9SZIkZZfhORMtNwx2pt7wLEmSlMcMz5nYvfIcdlG70/AsSZKUrwzPmdjd80w921x5liRJyluG50zsFZ63G54lSZLyluE5E0V73TBo24YkSVLeMjxnorCIWNiJzsEbBiVJkvKZ4TlTRaWp59mVZ0mSpLxleM5QKC6ja2GDK8+SJEl5zPCcqeLOdC30hkFJkqR8ZnjOVElXKsJOR9VJkiTlMcNzpsp60SNsc9qGJElSHjM8Z6qsF93Zas+zJElSHjM8Z6qsF12bt9rzLEmSlMcMz5kq60V5cy11O3bmuhJJkiTliOE5U2U9ASjctSXHhUiSJClXDM+ZKusFQEnDZmKMOS5GkiRJuWB4zlRLeK5o3sauxuYcFyNJkqRcMDxnqiU89wjb3KJbkiQpTxmeM9USnnuGbY6rkyRJylOG50y13DDYg23U7GjIcTGSJEnKBcNzpopKaCruQs+wjQ3bduW6GkmSJOWA4fktiC1bdG+oNTxLkiTlI8PzW1BQ3oueGJ4lSZLyleH5LSgo701l4TY21NbnuhRJkiTlgOH5rSjrRa9QS7U9z5IkSXnJ8PxWlPWiO1uptm1DkiQpLxme34qynpTGnWzdtjXXlUiSJCkHDM9vRctGKY21G3NciCRJknLB8PxWtITnkl2b2NnQlONiJEmS1N4Mz29Ft4EADAwb2LjdiRuSJEn5xvD8VnQfCqTw7MQNSZKk/GN4fivKetJU1JlBYYNbdEuSJOWhVoXnEMKNIYT1IYT5b3D8ghDC8y1//hZCmLTXsUtDCAtCCPNDCDeHEEpbU0u7CIHmboMYGDa4y6AkSVIeau3K88+B09/k+HLg5BjjROAqYCZACGEg8DlgWoxxPFAInN/KWtpFYY8hDAzVhmdJkqQ81KrwHGOcBWx6k+N/izFubvn2CWDQXoeLgM4hhCKgDFjTmlraS0GPoQwq2GjPsyRJUh5qz57nTwD3AsQYVwPfBlYCrwA1McYH2rGWg1cxmB5sY9vWmlxXIkmSpHbWLuE5hDCDFJ4vb/m+B3AOMBwYAJSHEC58g+deHEKYHUKYXV1d3R7lvrnuQwBo2vxyjguRJElSe8t6eA4hTAR+ApwTY9y9Nd+7gOUxxuoYYwNwK/C2/T0/xjgzxjgtxjitsrIy2+UeWMXg9N+aVbmtQ5IkSe0uq+E5hDCEFIw/EmN8aa9DK4HpIYSyEEIA3gkszGYtbaZ7Cs9dd77iLoOSJEl5pqg1Tw4h3AycAvQOIVQBXwOKAWKMNwBfBXoBP0wZmcaWVeQnQwh/AJ4BGoFnaZnEccjr0o/mUMSgsIGqzXUc2adrriuSJElSO2lVeI4xfugAxz8JfPINjn2NFLY7loIC6rsMZNCWalZt2mF4liRJyiPuMHgQQs8RDAtrqdpcl+tSJEmS1I4MzwehU/8xHBlWU7WpNtelSJIkqR0Zng9CqDyK0tBA3frluS5FkiRJ7cjwfDAqjwKgaNNLBzhRkiRJhxPD88GoHA1At23LclyIJEmS2pPh+WB07s72TpUMblrJ1p0Nua5GkiRJ7cTwfJDquo/kyFBF1aYduS5FkiRJ7cTwfJAK+oxmZFjN8monbkiSJOULw/NB6jZ4AuVhF2tXLcl1KZIkSWonhueDVNxvLAANa+bluBJJkiS1F8Pzweo7DoDOm17IcSGSJElqL4bng1XajU0lg+hb9xJNzTHX1UiSJKkdGJ5bYXvPcYxhBas21eW6FEmSJLUDw3MrFAyYyNCC9SyveiXXpUiSJKkdGJ5boceIqQDUrHgmx5VIkiSpPRieW6FsyNHpi1eez20hkiRJaheG59bo2pctBT3osmVhriuRJElSOzA8t9KGrmMYtnMh9Y3NuS5FkiRJWWZ4bqX6QdM5Mqxm2cvLc12KJEmSsszw3Erdx8wAYOOCh3JciSRJkrLN8NxK/UZPpy6WULjyr7kuRZIkSVlmeG6lguJOLC4ZR//Nc3JdiiRJkrLM8NwGNlYey9Cml2ncVp3rUiRJkpRFhuc2UDTiJADWPv+XHFciSZKkbDI8t4GB495GXSyhbvHDuS5FkiRJWWR4bgMj+vbg+TCKLmufzHUpkiRJyiLDcxsIIbC6+1QG7FwKdZtyXY4kSZKyxPDcVoaeCMD2xY/muBBJkiRli+G5jQwY+zZ2xmI2LfhzrkuRJElSlhie28jEYX15Jo6itMrNUiRJkg5Xhuc2Ul5SxAvlx1FZtwS2rMx1OZIkScoCw3Mbqhv2bgCaFt2T40okSZKUDYbnNjRy7BSWNvdn+/N35boUSZIkZYHhuQ0dO7wnf2qeSvkrj8POrbkuR5IkSW0sa+E5hHB6COHFEMKSEMIV+zneI4RwWwjh+RDCUyGE8dmqpb306lLCSxUnUhgbYfEDuS5HkiRJbSwr4TmEUAhcB5wBjAU+FEIYu89pXwbmxhgnAh8FvpeNWtpbt5FvY23sSfPzt+S6FEmSJLWxbK08HwssiTEuizHWA78FztnnnLHAnwFijIuAYSGEvlmqp90cd0Qfbm96G2HJn2H7xlyXI0mSpDaUrfA8EFi11/dVLY/t7TngPIAQwrHAUGDQvi8UQrg4hDA7hDC7uro6S+W2neNG9OKO5hMIsREW3JrrciRJktSGshWew34ei/t8/w2gRwhhLvBZ4Fmg8XVPinFmjHFajHFaZWVl21faxnqWd6Ko/wRWFg2Deb/PdTmSJElqQ9kKz1XA4L2+HwSs2fuEGOPWGONFMcbJpJ7nSmB5luppVyeNquS3O6fDqidh02HxkSRJkkT2wvPTwMgQwvAQQifgfOCOvU8IIXRvOQbwSWBWjPGwmO/29pGV3N74tvSNq8+SJEmHjayE5xhjI/AZ4H5gIXBLjHFBCOGSEMIlLaeNARaEEBaRpnJ8Phu15MLRQ3tQ06kvy8onw/O3QNy3Y0WSJEkdUVG2XjjGeA9wzz6P3bDX148DI7P1/rlUXFjA8Uf05pZVx3PF9uthzTMwcGquy5IkSVIrucNglrx7bB9+UzuF5sISeOZXuS5HkiRJbcDwnCXvGtOX2tCFF3qdmlo3dtbkuiRJkiS1kuE5S3p1KWHasJ7csH0GNGyH536X65IkSZLUSobnLDptXD/u2tiPXX0mw9M/gebmXJckSZKkVjA8Z9GpY9Nu44/1+gBseBFeui/HFUmSJKk1DM9ZNLhnGeMHduP6DZOg+1B49DuOrZMkSerADM9ZdtrYfsxetY2tU/8ZVs+GFY/muiRJkiQdJMNzlp02vh8AdxbMgC590+qzJEmSOiTDc5aN7NOF4b3LuW/RZjj+n2HZw7B6Tq7LkiRJ0kHoEOE5hHB2CGFmTU3Hm5UcQuD08f3429KNbBxzIZRWwKPfzXVZkiRJOggdIjzHGO+MMV5cUVGR61IOyrlTBtLUHLlj4VY47hJYdJerz5IkSR1QhwjPHd2ovl0Z278btz27Gt72WSivhAe+4uQNSZKkDsbw3E7OO3ogz1fVsKQmwClXwst/hRfvyXVZkiRJegsMz+3kvZMGUBDg9mdXw9Efg54jYNa3XH2WJEnqQAzP7aRPt1JOHFnJbc+upjkUwgmfhzXPwvJHcl2aJEmSMmR4bkfnThnA6i07mP3yZpj0IejSz8kbkiRJHYjhuR2dNq4fZZ0Kue3ZKigqSXOflz8CK5/IdWmSJEnKgOG5HZV1KuL0cf246/lX2FHfBMd8Mu06+Oer7H2WJEnqAAzP7ezvjxnMtp2N3PX8GuhUBid9EV5+DJb+JdelSZIk6QAMz+3s2OE9ObJPF3795Mr0wNSPQY/hcNe/wM6Ot4OiJElSPjE8t7MQAhccN4S5q7Ywf3VN6n0+bybUrIa7v5jr8iRJkvQmDM85cN6UQZQWF/Cbp1pWnwcfCyd/CebdAosfzG1xkiRJekOG5xyoKCvm7IkD+L9nV1O7qzE9eOKlaeOU+6+EpobcFihJkqT9MjznyAXTh7K9vintOAipfeO0/4YNL8GTN+S2OEmSJO2X4TlHJg2qYNyAbtz0xMvE3WPqRp0Oo85Io+vWvZDbAiVJkvQ6HSI8hxDODiHMrKk5fKZRhBC4cPpQFq3dxhPLNu1+EN57LZR2g1s/BQ07c1ukJEmSXqNDhOcY450xxosrKipyXUqbOnfKQHqVd+Injy7b82CXPnDOD2HdfPjzf+SuOEmSJL1OhwjPh6vS4kI+cvxQ/rxoPUvW1+45MIIE11AAACAASURBVOpUOOZT8MR1sOTPuStQkiRJr2F4zrGPTB9KSVEBP31s+WsPnHoV9B4Nd34e6rfnpjhJkiS9huE5x3p1KeG8owfxx2eq2FC7a8+B4s5w9vegZhU8fHXuCpQkSdKrDM+HgE+cOJz6xmZ+9fjLrz0w9Hg4+mPw+A9h1dO5KU6SJEmvMjwfAo7s04V3jenDr554mR31Ta89+O7/gIqB8IeLoG5TbgqUJEkS0MrwHEI4PYTwYghhSQjhijc455QQwtwQwoIQwiP7HCsMITwbQrirNXUcDj598hFs2l7Pzbu37N6tc3f4u5/DtrVwy0ehvi4n9UmSJKkV4TmEUAhcB5wBjAU+FEIYu8853YEfAu+NMY4D/m6fl/k8sPBgazicHDOsJ9NH9OSGR5ays2Gf1eeBU+GcH8CKx+Dmv4eGHbkpUpIkKc+1ZuX5WGBJjHFZjLEe+C1wzj7nfBi4Nca4EiDGuH73gRDCIOAs4CetqOGw8rl3jGT9tl38fvaq1x+cdD6cewMsfxTu+Czs3pVQkiRJ7aY14XkgsHfKq2p5bG+jgB4hhIdDCHNCCB/d69g1wJeA5lbUcFg5/oheTBvag+sfXkp9435+LJPOh3f8G8z7PTz2v+1foCRJUp5rTXgO+3ls3+XQImAqaYX5NOArIYRRIYT3AOtjjHMO+CYhXBxCmB1CmF1dXd2Kcg99IQQ++86RrKnZyR+fqdr/SSd9AcZ/IO0++OK97VugJElSnmtNeK4CBu/1/SBgzX7OuS/GuD3GuAGYBUwCTgDeG0JYQWr3eEcI4ab9vUmMcWaMcVqMcVplZWUryu0Y3j6yN5MGVXDdQ0toaNrP6nMIqf95wGT44ydh3YL2L1KSJClPtSY8Pw2MDCEMDyF0As4H7tjnnP8DTgohFIUQyoDjgIUxxitjjINijMNanveXGOOFrajlsBFC4F/eNYqqzTteP3ljt+LOcP5voKQr/PqDsPWV9i1SkiQpTx10eI4xNgKfAe4nTcy4Jca4IIRwSQjhkpZzFgL3Ac8DTwE/iTHOb33Zh7dTRlcyfURPvvfgYrbtbNj/Sd0GwId/Bzs2pwkcbuEtSZKUdSF2oKkN06ZNi7Nnz851Ge1iXlUNZ//gMf55xhFcdtpRb3ziS/fDzefDyNPg/F9DQWH7FSlJknSYCiHMiTFO2/dxdxg8RE0YVME5kwfwk0eX80rNm8x1HnUanPFNeOle+PXfuQuhJElSFhmeD2FfPHU0McK373/pzU889lNw9vdgxaMw8xR45fl2qU+SJCnfGJ4PYYN7lvHxE4Zx67NVvLBm65ufPPXj8PF7oKkBfnoqPPe7dqlRkiQpn3SI8BxCODuEMLOmpibXpbS7fz7lSLqVFnP1vRnsYj74GPj0IzDwaLjtYnjgK9kvUJIkKY90iPAcY7wzxnhxRUVFrktpdxVlxXzunSN5dPEGHnkpg01iuvSBj/4fTL0I/nYtzL81+0VKkiTliQ4RnvPdR6YPZUjPMq6664X9b5yyr8JiOPNbMHAq3HUp1KzOfpGSJEl5wPDcAXQqKuCr7xnLkvW13PjY8syeVFgM5/0YmhvhV++D2vXZLVKSJCkPGJ47iHeN7cu7xvThe39e/Oaj6/bW6wj48C1QUwU/Pws2LMlukZIkSYc5w3MH8rWzx9HUHLnqrhcyf9KwE+CCP0DdxjTG7umfpokckiRJessMzx3I4J5lfGbGkdwzby2zMrl5cLdhJ8DFj0C/CXD3v8INJ9rGIUmSdBAMzx3MxSePYHjvcv7t9vnU1Tdm/sTug+Gie+D838CWlWk3wl212StUkiTpMGR47mBKigq5+rwJrNxUx3ceOMDOg/sKAY46C/7u57B2Hlx3LMz5BcSYlVolSZION4bnDmj6iF58ZPpQbvzrcua8vPmtv8Co0+Bjd0K3gXDn5+D3H4Nd29q+UEmSpMOM4bmDuvyMoxhQ0Zkv/eE5djY0vfUXGHYCfOIBOPU/YeGd8ON3QvVbXMmWJEnKM4bnDqpLSRFXnzeBpdXbufbPiw/uRUKAt3027UhYtxF+/I4UpCVJkrRfhucO7O2jKvngtEH8aNYy5lXVHPwLDX87fPoRqBwFv7sQ7v4CbN/YdoVKkiQdJgzPHdz/O2ssvco7cdkfnqO+MYOtu99IxSC46F447hKY/TO4dgosuL3tCpUkSToMGJ47uIrOxfz3uRNYtHYb33ngxda9WFEJnPE/8I9/g94j042ED3wFmlsRyiVJkg4jhufDwLvG9uWC44bwo1nL+Muida1/wT5HpVXoaZ+Av10Lt10MG5e2/nUlSZI6OMPzYeIr7xnLmP7d+MItz7Fmy47Wv2BRJzjrO/COf4N5v4fvH502Vqmva/1rS5IkdVCG58NEaXEhP7zgaOobm/nczc/S0NQGrRYhwNsvg8/NTSF68Z/gNx90pJ0kScpbhufDyPDe5fz3eROY/fJmvvunNgy4PYenEH3eTFj5BFx3DPzsTFj1VNu9hyRJUgdgeD7MnDN5IB86djDXP7yUh19c37YvPvGDcOmCtLHKxiXw03fDvZdD4662fR9JkqRDlOH5MPS1s8dxVL+uXPq7uaza1MY9yl37po1VPvsMHPeP8OQN8KO3w/O3QPNB7HQoSZLUgRieD0OlxYVcf+FUmpojn/rlbLbvamz7NynpAmd8Az702/T9rZ+CG0+DtfMgxrZ/P0mSpENAhwjPIYSzQwgza2pasYtenhneu5zvf/hoXlq3jS/+/jlitgLt6DPgHx+Hc2emVo4bToTvTYInfwRNWQjtkiRJOdQhwnOM8c4Y48UVFRW5LqVDOXlUJVeeMYZ756/lB39Zkr03KiiASX8Pn5kNZ34bKgbDvV+CmSfDy49n730lSZLaWYcIzzp4nzxpOOdOGch3/vQSDyxYm903K+8Nx34KPn4XfPCXsGML/Ox0uPXTsK0NNm+RJEnKMcPzYS6EwNXnTWDioAou/d1cXly7rT3eFMaeA595Ck76Aiy4Fa6dDHddClVz7ImWJEkdluE5D5QWFzLzI9MoLyniYzc+RdXmdtolsFM5vPOr8E9PwPjz4Nlfw0/ekYL0wjsN0ZIkqcMxPOeJfhWl/OIfjmV7fSMf/elTbKxtx9nMvY6Ac66DL74I77sBisvhdxfCj2fAM790xJ0kSeowQtamMGTBtGnT4uzZs3NdRof29IpNXPiTJxndryu/+dR0upQUtX8RTY3wzM/h6Z/C+hdgyNtg6PFQUwXHfhoGTW3/miRJkvYSQpgTY5z2uscNz/nnL4vW8alfzuG44T352UXHUFJUmJtCYoTnboZ7vgQNddCpC+yqgZGnpekdY98HBTmqTZIk5TXDs17j1meq+NdbnuP0cf247oKjKSwIuStmVy3Q8u/wb9+HZ34F29bAgCnwnmtgwOTc1SZJkvLSG4XnVvU8hxBODyG8GEJYEkK44k3OOyaE0BRC+MBej60IIcwLIcwNIZiI29l5Rw/iK+8Zy30L1vJvt8/L3iYqmSjpAiVd058ZX4ZLF8B5P4Ga1akv+u4vwvJHYefW3NUoSZIEHHTDawihELgOeDdQBTwdQrgjxvjCfs77H+D+/bzMjBjjhoOtQa3ziROHs2n7Lq57aCk9yztx2WlH5bqkpKAAJv4djHw3/OUqePon8PSP07HKo+Ds78GQ6bmtUZIk5aXW3C12LLAkxrgMIITwW+Ac4IV9zvss8EfgmFa8l7Lki6eOZtP2Bq57aCk9yjrxyZNG5LqkPTp3h7O+A6dcCWuehTVzYe6v4ab3w5nfgs49YfhJaSSeJElSO2hNeB4IrNrr+yrguL1PCCEMBM4F3sHrw3MEHgghROBHMcaZrahFBymEwH++bzxb6ur5z7sXEkLgEycOz3VZr1XeO61Cj3w3TLkQfvEeuP0f07EufWHqx6H7EBh9JpT1zGmpkiTp8Naa8Ly/O8z2bZy9Brg8xtgUwutOPyHGuCaE0Af4UwhhUYxx1uveJISLgYsBhgwZ0opy9UYKCwLXnD8ZfjuXq+56gW07G/j8O0eyn7+z3OvWHz79KFQvgh2b4NHvwiP/k46VdINjPgFHvAN6jYQufZzWIUmS2tRBT9sIIRwPfD3GeFrL91cCxBiv3uuc5ewJ2b2BOuDiGOPt+7zW14HaGOO33+w9nbaRXY1NzVxx6zz+MKeKT5w4nH87a8yhGaD31bADNrwEj3wTFt3Nq9dwpRVpVfrYT0PFwFxWKEmSOpg3mrbRmpXnp4GRIYThwGrgfODDe58QY3z19/8hhJ8Dd8UYbw8hlAMFMcZtLV+fCvxHK2pRGygqLOCb759Il5IifvrYcmp3NvLf503I7Ri7TBR3hv6T4Pxfw84aWPU01KyEZQ+n0XePXwdHvQeGngBHnQkVg3JdsSRJ6qAOOjzHGBtDCJ8hTdEoBG6MMS4IIVzScvyGN3l6X+C2llXNIuA3Mcb7DrYWtZ2CgsDXzh5Lt9Iirv3LEmrrG/nfD06mU1EH2cm9tAJGvit9Pe0fYPMKeOIGWHArvHA73P/lNKlj7TwYcXLaLrxTWU5LliRJHYebpOgN/XjWMv7rnoWcMrqS6y+YSudOHbh/OEbYtAyeuB5WPAaVo+CFO6DveBg4JfVIjzwV+hwi4/okSVJOucOgDsrNT63k/902j0mDu/PTjx1Dz/JOuS6p7bzwf/CX/4QdW2D7+vTYiFNg/Aeg+2AYcjwUleSyQkmSlCOGZx20++av5fO/fZaB3Tvzi384lsE9D8M2h5rVMO/3qT96d5Du3BMm/F36U9QJynp746EkSXnC8KxWmb1iE5/4xWw6FRXws48fw/iBFbkuKTuaGqCmCqpfhOd/m6Z3NNXvOT74uDRburwy9UyPOt3VaUmSDkOGZ7Xa4nXb+NiNT1Gzo4HrL5zK20dV5rqk7KvblKZ2FBan2dIv3gv1dSlg12+DisEw+YI0T3r0mdBvfK4rliRJbcDwrDaxtmYnH//ZU7y0bhtXnjGGT540vGPMgm5rTQ0pVD/0X2nrcABCWonudQQMPBpGzHDHQ0mSOijDs9pM7a5GLvv9c9w7fy1nTxrA/7x/AmWdWjMyvAOLERp3po1a/noNLLwTtq5Jj0HaNnzoiTD6jLTzYUmX3NYrSZIyYnhWm4oxcsMjy/jW/YsY1bcrP/rIVIb2Ks91WYeGpkZY80waiffKc2mFeucWKCyBniOgc4+0It25O3TplzZuGXA05OMKviRJhyjDs7Ji1kvVfPbmZ4kx8t0PTuZdY/vmuqRDT1MjrHoCXrovbdqyY0vqpd6xCbZXQ3MjFJel6R5lPaByDIx9b2oBKSxOr7GzJrWKlPfO6UeRJClfGJ6VNSs31nHJTXN44ZWtXHTCMK444yhKijrwhirtacfm1OpR/WIK1HUboerpFKy7DYIx70ktIfP+AE274OiPwYwvG6IlScqyDh2eQwhnA2cfeeSRn1q8eHGuy9F+7Gps4up7FvHzv61g3IBuXPuhKRxRaX/vQWlqhCUPwuM/SDcjNjellehO5fDML6G0O5x4aRqZ16kcSrqm0Xld+qRjBR1kK3VJkg5hHTo87+bK86HvwRfWcdkfnmNnQzNfPXss5x8zOD+ncWTLuhfg9n+EV+bu/3hBUdrMZdgJaafE/pPSKnVBsaFakqS3wPCsdrNu606+cMtzPLZkA6eO7cvV502gVxc3Emkzzc1Quxbqt6c/O2tS7/T2aqhdn6Z9LL4/tYTsVtQZxp4Dw06E4s6wcQn0HgXjzk03Ku7cCmufh0HHpt0UJUnKc4Zntavm5shPH1vON+9fRNfSYr76nrGcM3mAq9DtpXEXVM2G9S/Arq2w+WVYcFv6em99J6SV6aqnob42hedzb0hTQfy7kiTlMcOzcuLFtdu4/I/PM3fVFk4ZXcl/nTuBgd0757qs/NRYD9teSavVPYbCgtth9o1AhD5joN9E+PN/pBBd1iu1gBQUpV7qETPSnOrSijSrurwy9VrvLcY0OWT3hBBJkjoww7Nypqk58svHV/Ct+1+kIAT+31lj7IU+VG1eAS/el1asIYXhLSvh5b9BbNrrxAB9xkKPYamXetNy2Lg0rVa/7XMw6XzoNtAWEElSh2V4Vs6t2lTHl/7wPI8v28hJI3vz3+dOYHDPslyXpUzUVsO6eXv6rDe/nFo9tq6B5gboMTxtS15TBQvvaHlSSKvWnbq0tICElh0Xj0/n9xmTZlrvvogKAdbMhdictjeXJCmHDM86JDQ3R3791EquvmchTc2RS04+gktOPoLOnZwLfdh45Tl45XnYuhpqVqU51TGmUFz9IlQv3HNup67QuCPd0FjeGzYvT4+PPjO1iRSXwfb1aeb1wKPtxZYktRvDsw4pa7bs4Op7F3Hnc2sY2L0zXz5zDGdO6GcrRz7YuTUF69XPpDnWJV32rGaPfHe6qfGxa15/cyNAxeDUSlK7HkIB9JuQdmIcdWqaHhIKoXYdbFubpo10Kk8r4t0GpOevnQernoLx56Vt0ht3QZGTYCRJr2d41iHpyWUb+fqdL7Dwla0cN7wnX3/vOMb075brspRrzc0pBDfuSHOra1alvusVj6Y2kC59U7vIyifSVBEO8P9jZS07MtZtSP+tGJLaRhbfn1a53/bZdJPkuvkpgB91FqxbkHZ8HDHDGdmSlIcMzzpkNTVHbn5qJd954EVqdjTw4eOG8IV3j6ZHuTebKQO11bDsoTRJpKkhBeuu/dPKcv02WL8w3QAZCtJNjr1HwR2fSyvbY85OI/zqa1/7miUVsKsmfd1/EvSfTAroAQYfB8PfngJ996Gpr3v5rLS63WfMa18nRttMJKmDMjzrkLelrp5rHlzMr554mS4lRXzh1FF8+NghFBW66qc21rATiGnDmO0bUgvJjk1QORq2rYPnfweDjoHO3eGv34O6TSl8N9Wn8/ZW0m1Pi0m/CVC3GRp3pvN3bIKKQWmDmi59U5vIzi3p3LqNaZJJxZA9O0I21afX6to/9Yg37oJOb3JT7YrHYPmjcPw/pTGCkqQ2Y3hWh/Hi2m38+50L+NvSjYzs04XLThvNu8f2tR9auRdj6ple+3xadV43L43oG30mbHgRlj6UVqA7lUNzU1r9fuU5WP5ICsMAhSVpNbpTlzRve8vKtDtk554pODc3pkDeUJe+7tIPBh8Lg6al76tfhA2L02r5hpfSa/YYBmd+B46YAQUtN9/u3JraXAo7pZVye7sl6S0xPKtDiTFy/4K1fPO+F1m2YTtHD+nO5acfxXEjeuW6NOmta6xPYbiw0+tXkmOEpX+Bub9Oo/y6DkhBvLQiTSHZuCS1hWxbk87v0je1nxR3hmEnQd9xcPs/wdaq1G5SUJBW1ht37HmPTl1h0NQ0vWTj0jTZpLkR1i+CfuNhwJR0Xl3LqvqgaWllfOsa6NovXRB0HZD+261/ugBYNz9dHPQ6Mq2mxwil3VP7THFnKOu55/23b0zfewEsqQMxPKtDamxq5vdzqrjmwZdYt3UXM0ZX8pl3jGTq0B65Lk1qPzHCzpq0e2Nx2etDaMNOeOk+WPZw2hWyuDQF6aHHp1GBL96T5nI3NaSwu31Deo3eo1LLysbFqc2kc8/UclK3AQgpHO/bprKvTl326hkPvHrzZo/haWW9pipdAFQMhsqj0rl9x6Wxgzs2p68rhsArc1Pt5ZVp5X7BrbBpGcz4NxhyXHrN5ub0OTa8BOPel3a5bG5OFybrFsD6BemCorR7+nrQMem1JOkgGJ7Voe2ob+IXj6/g+oeXUrOjgaOHdOfit4/g3WP7UVjgapbUZmJMO02W9YLSbnu2dd/2SlqJ3romjRbsNz5NRFn3QloxLyhMK9fdBqT2kzVzU3AubQnxq59JIwqLy9Ic8Ibtb15HYUkK77XrUtBubkztLQ116XhZb6gYmMYP7m6J2VdpBQyenurZWZPaYUa+C3ZsSeMMi0vT52lqSHPFG3emzx4K0+fZvUX94ONS33phMax8PP0mYMQp6TcA26th9Zx0kdJtIBDThUKfo/ZfU211CvS7fwOxa1u6mOk5/C3/VUnKLsOzDgvbdzXy+9mr+Olfl7Nq0w6G9irjEycO5wNTB1HWqSjX5UnKRGN9WoEu6ZqC57a1qXUkFKQwumNTmnBSVJJmfm9ekY6VV6bNcrr2h79ekwLroGlppbnHsBRml/45PV45Gub/EapfSjd+llak/vK1z6dg3q1/WrHv2i/1p6+bl96j26BUY3NDCuwNO9PUlk5d03bzdRsz+4wDjk6Bfde2dHHRY1j6nC8/lo73GJYmuSx7OJ03+sw9K+iDjk2tNQ116bOUVqTX6FSeLk56jUyzzus2pAuHI96RLjRWPZlagLashKLSFMh7DE+hf8fmdANqWa/UQlNbnXYD7dI3zVfvPSr18TfUpVX/wpLUK1/cOb3W7rC/aXn6TcaY90L3wen7VU+mv6uj3gMEaNp14BX/GNOUnM490t994y4oKG7bsZBNjenvrrO/qdTBMTzrsNLUnHqiZ85axtxVW+heVsyFxw3lo28bSp+upbkuT9KhavvGtKJeWPzax2vXpxaUfXvSm5tTIH/xXohNKdQfdVYKqbXrUxDtPykF3K1rUgBf+UQagdhtQApuW1bClpdTz/v4D6RzXpmbNgkaeHRqZ3niBijvlVpNqmanEFtclsJr3SaoXfvGnykUpIDbUJdWyisGp1X0ba+8+c+i64DUr757Nb+gKAXYvfvldysuh95HptGPTfWptu5DoHrRnnPK+6SLhcYd6euhx6dwvuKv6edUXpkuhPqMTb95WHBbel6PYbBlVRr7eNR70oVPc2O6UOk+NF1orZ2XLjJ6DEu/BVg9J9U9/JT024ktK9N59bUpuPedAA/9V7rB9sRL030EL96bLpRGnQanXJE+R1Njqrdqdvr7Gvu+1/4bqJqTLuYGTEnnFxSli77Ff0r1jDot1b3ir/B//wQjT4MZV8LWV9JnL2+5T2fbunSBNvTE9BuPvcWYWpsKitLnO5h7A5oaYcWs9LPt2i+z5+zOfyGknwvsueEY0r/n8sr0v5X67XnZAmV41mEpxsiclzczc9Yy/rRwHcUFBZw7ZSCfPGk4I/t2zXV5kpSZ5uYUYt4oODU1pFXozcvTTZ9d+6fQ1rBjz6zy4SfDsBPTxQGkcLf1lRREO3dPAbtuU1qxLuyUVseb6lOI37Q0vW79dpj492kFeNXTaWW7oS4F1OpFKdxN+Qg8cT1sX59WzIe/PYXf536TAnl57/Rayx5KK95Djk+tM9s3pMC9Zm4Kz6d8OQXJFY+lGekbFsOSP+8/vHfply5ENi5JvxV4IwXF6fMSU/AbMAUWP5AujCb+fQqHz96054Lhdc9v+Q1meZ+WjZPm7eekvXr7Cek+gs0rWlb017321O5DoHJMavVp3JHuKxhxcrrYevlvsKu2pSVpfTq/64CWlp+QLn46ladV+S0r099Flz7pAmvjktR+1Hdc+vt++fF070JJBUz7ePrNQmxOn6HvOOjaNwXkDS+lSTzb18Oiu9Pff7dB6e+/oDhdzA06BtY8ky4QizqnGuo2pPapEafsuU9hx+b0m5NO5enPpqXp59ZnTPp8m5an92vcmVqmegxNF3YbXkwXQ6POSBeVi+6CSR+CqReln8GGF9NFWHllqr+8cs+/6XZmeNZhb/mG7fz0sWX8fnYVuxqbmTG6kk+9fQTHj+jlmDtJam8xpsBWuE9LXcOOFBq7VL7+Obt3Fy3s1NJq83Jafe82IB2vXZ9C3cBpaQX45b+mXvNeR6Tvi0pSSK+anUJgea8U1LoO2LMKXFOVXmP3qmoohAGTU//+4gfSSn5NVQrE49+fwuC6+amOpoZU39ATUpB96T5Y/Wyq9fSrU9vN8lkpUG9bk3r9182HgVPTqvqC21Iw3b4RhkzfE2oHH5dC9Kon00UEpBnx9bUp1PYYmv67eXn6bL2OSPWvW9DSb98fjvkkzL05tQZ17Z8+W231fi5GQgq7R74rBf6aqtQK1LQrjeJctyB9nuM+nQJ6fW0KvVVPp/p2z7WHFG6bGtPFUI9h6e/n1eOhpd2oSwra214BYrpA6T4k3RAcCtJF3Oo3yXYVg+HS+Rn9k2trhmfljU3b6/nV4y/zy8dXsHF7PeMHduOTJ47g9PH9KC0uPODzJUnqkGJMYXp360lzUwqpdZvSbzV6HfnaMZL7U1+XVuj3Nxs+xnTj79ZX0jn9J6fXjc3p+90XP7u2pfaRvVeMG3eloF7WK/0mZO38VGfPESm0r3mWtJJ/RPotQ93GPZOBJp3fZj+it8LwrLyzs6GJ255dzY8fXcay6u1UdC7m3CkD/3979x4k2Vned/z7nNP3ufRc9zY7w+qyAiSCtGLBONiSHUMQJliiEgxOQlwpJzJVFjGVUAmmKoSUKxXHFTvmD8eUDKRwAgZisBGCcClDpLiQhVbSalerlXZXQtqdvWguO/eZvp1+8sc5M+pdTc/2aGZ3Znd+n6qu6X77dPfpp97a/c0773lfPrB/N7fs0m5sIiIi0pzCs2xZ9brz4+fH+eqBU3zv6XNUojpvGujkg/sH+ZXbBijm05d+ExEREdlSFJ5FgMn5Cn/15Gm+emCYo2enyaYC3vOmHfzq/kHefn0vgdaMFhERES5TeDazu4DPACHwOXf/vYuevxv4XaAO1ICPufvfmNkg8GfAjuS5+939M5f6PIVnWS/uztOnp/nagVP81cHTzJRqDPUU+MBbdvP+2wfY3V249JuIiIjINWvdw7OZhcAx4F3AMPAY8Gvu/kzDMe3AnLu7mb0Z+Jq7v8HMdgI73f0JM+sAHgfuaXztchSe5XIoVSO++/Q5vvrYKR55Id4A4ZZdnXzwrYPcs2+AzpymdYiIiGw1zcLzWrZkextwwt1fSD7gK8DdwFIAdvfZhuPbSBZGdPezwNnk/oyZHQUGGl8rcqXk0iH37Bvgnn0DnByfnxxsiwAAGG9JREFU57tHzvKtp87yqW8e4XcffIa3X9/LB986yF237CAVruPuVyIiInLVWUt4HgBONTweBn7m4oPM7P3Afwa2Ae9d5vk9wD7g0eU+xMzuBe4FGBoaWsPpilzaUG+Be++4gXvvuIFDw5N8+/BZvnP4LPd9+Ul62jLceVM/v/iGbdy5t59iQSPSIiIiW81apm18AHi3u/+L5PGHgbe5+0ebHH8H8Cl3f2dDWzvwEPCf3P0bl/pMTduQjRDVnR89O8KDh87w0LFRJuarpALj5/f28d437+JdN2/Xih0iIiLXmMsxbWMYGGx4vBs40+xgd3/YzG4wsz53HzOzNPB14EutBGeRjRIGxjtv3s47b95OVHcOnprk+0fO8eChs/zofz9FGBi3D3Vxx95+7ripn78zUNSqHSIiIteotYw8p4gvGPwl4DTxBYP/2N2PNBxzI/B8csHg7cC3iEM2wBeB8+7+sVY/UyPPspm4x0H6r4+O8NCxUQ6fngKgu5Dm55MgfcfePrZ15jb4TEVERGS11n3k2d1rZnYf8D3ipeq+4O5HzOwjyfOfBf4h8M/MrAosAB9MgvTPAR8GDpvZweQtP+nu33mt5yNypZkZ+4a62TfUzcff/XrGZ8v8zYkxHnpulIePj/HAU/EfYt64s5M7burjzr39vGVPN9mUtggXERG5WmmTFJHLoF53jp6b5qFjozx8bJTHX5qgGjmFTMjPXt8bj0rf1M+e3gJmmuIhIiKy2WiHQZENNFuu8cjz4zx8bJSHj4/y0vg8AIM9ee68qZ+3X9/Lrbu72N2dV5gWERHZBBSeRTaRF8fmePh4PCr94+fHma9EAOzuzi/Nld431M12zZcWERHZEFd1eDaz9wHvu/HGG//l8ePHN/p0RNZVpVbn2MszPHFygv93fIxHnh9ntlwDoL8jy5sHitw22MW+oW5uHSzSoR0PRURELrurOjwv0sizbAXVqM6h4UkODU9x+PQUh4anODESb9ZpBq/f3sG+oThM3z7UzfV9bVoaT0REZJ1djnWeReQySIcBb3ldD295Xc9S29RClYOnJnny5ARPnJzk24fO8uc/iTf4LObT3DbYxe1D3ewb6uK2oS46NTotIiJyWSg8i1wFivk0d97Uz5039QPxah4vjM3yxEuTPHlqgidemuSP/voY7vHo9HW9bdy8q5NbdhW5ZVcnt+zqpLc9u8HfQkRE5Oqn8CxyFQoC48ZtHdy4rYNffWu80edMqcpTp6Z48uQEh09P8eTJSR48dHbpNTuLOW7Z1cnNDYF6oEure4iIiKyGwrPINaIjl+bn9vbxc3v7ltom5ys8c2aaI2emOXJmiiNnpvnhsyPUk0sdOnMp9vS18YYdHfzdG/q4eVcne3rbyKSCDfoWIiIim5suGBTZYhYqEUfPxYH62bPTnDw/z1OnJpkuxSt8hIHxup4CN25rX7rt3dbBDdvaKGT0+7aIiGwNumBQRADIZ0JuT1bqWBTVnWfPTXP85VlOjMS34yMz/PDZEWr1V37BHujKc9tgF7cOFtnVlY9vxTz9HVlCrfghIiJbgMKziBAGllxcWLygvRrVeWl8Lg7TL8/y3MszPPHSBN8+fPaC41KBLU3/eOPOTgZ7CmzvyHLjtnZdqCgiItcUhWcRaSodBksXJt71plfap+arnJla4OzUAmcmS5yZXOD4yCxPDV94kSJAT1uGvdva2bu9nT29bezuzjPQVWCgO093Ia0LFkVE5Kqi8Cwiq1YspCkW0rxxZ+ernpspVXl5usTpyVIyYj3D8ZFZvnnwDDPJvOpFhUzITds7uL6vjb6OLH3tGQa6Cty0vZ2B7rzmWIuIyKaj/5lEZF115NJ05NLcuK1jaV1qAHdnaqHK8MQCpycXOD2xwMnz8xw9O82jPz3P6GyZSq1+wXsV82l2FnMM9hS4rq+NPb1t7OzKsb0jx/bOLD1tGY1ci4jIFaXwLCJXhJnRVcjQVcjwpoHiq553d2bKNU6dn+fEyCynJxc4NxVPCXlxbI6Hjo2+KlxnUgEDXXl2deXYVcyzs5hjZ1eeHcUcA115hnoK5NLhlfqKIiKyBSg8i8imYGZ05tLLXrgI8Yog56ZLnJsqMTJd4tx0ibNTpaVR7IePjzIyU6Zx9c3AoKuQoTOX4o07O+lrz1KrO4M9eW7ob+eG/ja2deboyKY0gi0iIi1ReBaRq0IYGANdeQa68k2PqUZ1RmbKnJtaYHhigRdG5zg/V2F8rszh01PMlmoEZozPVS54XSYV0N+eZaArXnYvlw7ZWcwlFzVm6C6k6WnL0NMWj5xrWT4Rka1L4VlErhnpMFgK2G95XfPjZkpVXhid44WxWcZmKozNlhmdKTM8scDRc9OUKhEvz5SJ6q/eRCow6C5klsJ0X3s893pxNLu3PUt3IU1ve5b2rP6JFRG51uhfdhHZcjpyaW4d7OLWwa6mx1SjOqMzZSbmK0zMVTk/X+H8bJnxuQrjcxXOz1Y4P1fh2XPTjM1WmFqovuo9tnVk2VnMUcikaMum6GlLs6OYpzOXoj0bt/W2Z9jWkaW3LUshG5IJA00hERHZxBSeRUSWkQ6DpV0UWzE5X+GnY3NMzFc4P1dlZKbE8yNzjM2Wma/UOD25wKHhSUZmyiu+Tz4dsqevjUImpJAJGewp0JVP05ZNUciEDPUU2NPXRlR3+tuzdLdl1uPriohIixSeRUTWQVchw76hSwfZWlRnrhwxW6kxW6oxPltmdLbM2GyFUjVifLbCi+NzlGsR0wtVvvv0OaYWqstOIQHozKWoRk5bNqS/I0c+HdDbnmV3d55cOqQrn2ZHMUcYGJkwoKctQyoMKObTDPUUNH9bRGSVFJ5FRK6gVBhQLAQUC+mkpeOSr3F3KlGd2VKNn47NcfL8PKkwWLowMhMGzFVqjM6UWahGvDQ+x49PjFGu1ak1Cd0QXyiZTQUYEATxaic7ijnePFCktz1LOjQyqYDuQobetgyRO7l0SEcuRWcuTUcuRVsmRaAALiJbiMKziMgmZ2ZkUyHZ9pDe9iz79/S0/NrpUpWXp0o4UKpGTMxXiep1xmYrPD86S7kar51dd2d6ocpL5+f5s7996VVraq+kPZuiM5eiWMjQlU9TzMfBuhLVyaVCBrrzhIGRT8fTTgqZkDAwUmFARy5FMXmN1uQWkauBwrOIyDWsM5emM5e+9IENorpTqdWpRHUqtTrn5ypMzFcIA6NUjZgp1ZheqDJbrjFTim/TpSqT81WmFuJQPluukUkFzJUjxmZXnue9qJAJ6W3P0NOWxQAz2FnMkUuF8Trg+Thod+Xj7eEDM6K6s7OYp7c9Qy4VkksHZNMh+XRIOjRdfCki607hWURELhAGRj4TkiceCe7vyK7p/cq1CMOYKcXbs5eqURzQozqz5RpTC3HwnkhWMhmbLWNmRPU6z56boVKr4w7TC1VmyrWWPzcwaMuk6Myn6cynSYfGXLlGey5Nf3u8zGB7NkU+E5JLAnc+E//MNdzPp0MK2ZC2THxsIROSDoM11URErl4KzyIiclllU3EI723P0tu+tiBei+pMl+LA7e6YGacnFphcqFCq1ilVo4ZbHM6nS1WmF2rU6nUGuwtMJyH+4Kkp5is1FqrRBTtTtiITBhSyIYV0SDoVkA4DUkE8RzyfDunMp3GHdGj0tcfLEGaTkfH8RUG9kITyYj5FLh0yX4noaYvnmc9XItJhQCalsC6yWSg8i4jIVSOVrBjS07BE33V9bWt6T3enXIuD90I1YqES/yxVIxYq9aWAPVeOmK/UmK9EyS2+X43q1KJ4JL0a1ZkvR5w6P4+ZUalFPPLCOPOVaFXzyCEO6JWojhl05dPMlmvkUiE7u3LkMymyqYBcOiSbXPiZTYVk0w33U0HyuOGYdEgu+dn0demAtkxKK7GINKHwLCIiW5qZkUumajTfNmft6slUlYVKxPxiSF8M4clc8oVKjUImxehMmXPTJboLmXgJw7ky7dk0pWrEmckFSrU65Wq8nGG5VqdciyhXG3/Gc9bXoiObolhIEwYWz4Gv1QkCoyPZ4CeTCgiMpV9mMmGwNEq+dAvjUP7K4/CC5xZXfFl8XTo00sn7NN5fDPKLv+jo4lLZSArPIiIiV0AQGLkgDundV+DzFsP6UqhOQnYpCddLbdXFn688vzgXfXqhSuS+FHSjujNbrjFXrlGJ6kR15/nROZ44ObkUsBfb11Ng8cZF1ahO3aGvPUMxGY1PBUE8HaZhvvriHPZcOh6dd4/fY1tnjmpUZ65co7stQyEdEgTxhaXZxak4mRAjvhh1W2eWQia1VCt3lt4zHQSYwbbO7NLUpHrdcdCo/TVO4VlEROQa1BjWYXUrrqzV0oottTrlKKIavfI4DthxGG0M3I1TX2qRU41eaV+8n0qWPHxpfJ7Zco2OXIpa3ZfmuC9UImbL8Zrn5Vr8uFSLCMyoRnVmSvEFp7l0QKm6tpH5RWbxqjal5JeQMDAGuvK0ZVOEAYRBQGiQCgKCIP4lYGmqTDKyXo3qcYBPps+4xxfabu/M0ZFLU65FZMKg4ZeCkDCASuR0ZFO051LY0vlYMqWnccpO/HmZMLhgXXZ3p1Stk0sHWplmFRSeRUREZF0trdiSufLBfSXzldrSVJD5So1ytU7dncjjcD9fiZhLVnQxM0amSyxUI3LpcOmizcUR+mrk1OvO6ckFJucr5DIhuVRINaozPLHAQrKqzMW3mWqNsVqFSjKaXYucVGhLgblUrRMYZFIh43PlVV/MeimN02XmKjVK1TrpMJ66ZMn3LmRC+tqzmEEtcuruBGakw3h99lQQT6lJLU2tMVJBQK0e/xKwuztPKrCl6wS62zIMdhcoVeNfbkrVOsV8mt62DB25FJMLVYx4CtDiSju1pF65dMi7b9mxvkVYI4VnERER2RIKmdQF9wuZFQ7eBBZXjsmlQypRnVIlGWGvRtTqdTJhwGy5xmzDEo6Lo/6LF8G+MnWncapOfL+QCekqZJgp1ShVIyAejZ4tx/PsjWTkPIjftxo5tXqdauTMV2rUFtuSvxikw4Ba3fn+kXPUPV67PZ8OmZivUI1e+S0gHdoFj1cy1FNQeBYRERGRS1ucv714f7UbHm2Uet0xY2kqSC2KdzVdXC89MJivRJyfqzBdqtJdyODA+GwZwwiDeJQ7TJZ/3GwUnkVERERk3QQXXTCZCgN2FHMXtLUlq7Y0GujKX/ZzWw8txXkzu8vMnjOzE2b2iWWe/ydmdii5/djMbk3aB83sR2Z21MyOmNlvN7zm02Z22swOJrdfXr+vJSIiIiKy/i458mxmIfDHwLuAYeAxM3vA3Z9pOOynwJ3uPmFm7wHuB34GqAH/xt2fMLMO4HEz+0HDa/+bu//X9fxCIiIiIiKXSysjz28DTrj7C+5eAb4C3N14gLv/2N0nkod/C+xO2s+6+xPJ/RngKDCwXicvIiIiInIltRKeB4BTDY+HWTkA/wbwfy5uNLM9wD7g0Ybm+5KpHl8ws2XXjDeze83sgJkdGB0dbeF0RUREREQuj1bC83KrZi+7voiZ/SJxeP53F7W3A18HPubu00nznwA3ALcBZ4E/WO493f1+d9/v7vv7+/tbOF0RERERkcujlfA8DAw2PN4NnLn4IDN7M/A54G53H29oTxMH5y+5+zcW2939ZXeP3L0O/Cnx9BARERERkU2rlfD8GLDXzK4zswzwIeCBxgPMbAj4BvBhdz/W0G7A54Gj7v6HF71mZ8PD9wNPv7avICIiIiJyZVxytQ13r5nZfcD3gBD4grsfMbOPJM9/FvgU0Av892RB7Jq77wfeAXwYOGxmB5O3/KS7fwf4fTO7jXgKyIvAb67rNxMRERERWWfm671p+mW0f/9+P3DgwEafhoiIiIhc48zs8WQw+AKbb89DEREREZFNSuFZRERERKRFCs8iIiIiIi1SeBYRERERaZHCs4iIiIhIixSeRURERERapPAsIiIiItIihWcRERERkRYpPIuIiIiItOiqCM9m9j4zu39qamqjT0VEREREtrCrIjy7+7fc/d5isbjRpyIiIiIiW9hVEZ5FRERERDYDhWcRERERkRYpPIuIiIiItEjhWURERESkRQrPIiIiIiItUngWEREREWmRwrOIiIiISIsUnkVEREREWqTwLCIiIiLSIoVnEREREZEWKTyLiIiIiLRI4VlEREREpEUKzyIiIiIiLVJ4FhERERFpkcKziIiIiEiLFJ5FRERERFqk8CwiIiIi0iKFZxERERGRFik8i4iIiIi0SOFZRERERKRFCs8iIiIiIi1aU3g2s7vM7DkzO2Fmn1jm+TeY2SNmVjazj1/0XJeZ/YWZPWtmR83sZ9dyLiIiIiIil1vqtb7QzELgj4F3AcPAY2b2gLs/03DYeeBfAfcs8xafAb7r7v/IzDJA4bWei4iIiIjIlbCWkee3ASfc/QV3rwBfAe5uPMDdR9z9MaDa2G5mncAdwOeT4yruPrmGcxERERERuezWEp4HgFMNj4eTtlZcD4wC/8PMnjSzz5lZ2xrORURERETksnvN0zYAW6bNV/G5twMfdfdHzewzwCeAf/+qDzG7F7g3eThrZs+9lpNdoz5gbAM+92qmmq2O6rU6qtfqqWaro3qtjuq1OqrX6m1EzV63XONawvMwMNjweDdwZhWvHXb3R5PHf0Ecnl/F3e8H7n+tJ7kezOyAu+/fyHO42qhmq6N6rY7qtXqq2eqoXqujeq2O6rV6m6lma5m28Riw18yuSy74+xDwQCsvdPdzwCkze33S9EvAMyu8RERERERkw73mkWd3r5nZfcD3gBD4grsfMbOPJM9/1sx2AAeATqBuZh8Dbnb3aeCjwJeS4P0C8M/X+F1ERERERC6rtUzbwN2/A3znorbPNtw/RzydY7nXHgQ2xfB7CzZ02shVSjVbHdVrdVSv1VPNVkf1Wh3Va3VUr9XbNDUz91av8RMRERER2dq0PbeIiIiISIsUni/hUluQC5jZi2Z22MwOmtmBpK3HzH5gZseTn90bfZ4bycy+YGYjZvZ0Q1vTGpnZ7yR97jkze/fGnPXGaVKvT5vZ6aSfHTSzX254bqvXa9DMfmRmR83siJn9dtKuPraMFeqlPrYMM8uZ2U/M7KmkXv8xaVf/amKFmqmPrcDMwmT/jweTx5uzj7m7bk1uxBdCPk+8qUsGeIr4gscNP7fNdANeBPouavt94BPJ/U8A/2Wjz3ODa3QH8drmT1+qRsDNSV/LAtclfTDc6O+wCer1aeDjyxyresFO4PbkfgdwLKmL+tjq6qU+tny9DGhP7qeBR4G3q3+9ppqpj61ct38NfBl4MHm8KfuYRp5XdsktyKWpu4EvJve/CNyzgeey4dz9YeD8Rc3NanQ38BV3L7v7T4ETxH1xy2hSr2ZUL/ez7v5Ecn8GOEq846v62DJWqFczW71e7u6zycN0cnPUv5paoWbNbPmamdlu4L3A5xqaN2UfU3he2Vq2IN9KHPi+mT2e7AgJsN3dz0L8HxWwbcPObvNqViP1u+buM7NDybSOxT/fqV4NzGwPsI94pEt97BIuqheojy0r+XP6QWAE+IHHm5ypf62gSc1AfayZPwL+LVBvaNuUfUzheWVr2YJ8K3mHu98OvAf4LTO7Y6NP6Cqnfre8PwFuAG4DzgJ/kLSrXgkzawe+DnzM4/X0mx66TNuWq9ky9VIfa8LdI3e/jXj52beZ2ZtWOHzL1wua1kx9bBlm9g+AEXd/vNWXLNN2xeql8LyytWxBvmW4+5nk5wjwl8R/OnnZzHYCJD9HNu4MN61mNVK/W4a7v5z8Z1QH/pRX/kSnegFmliYOgl9y928kzepjTSxXL/WxS3P3SeD/Aneh/tWSxpqpjzX1DuBXzOxF4imyf8/M/hebtI8pPK/sNW9BvlWYWZuZdSzeB/4+8DRxnX49OezXgW9uzBluas1q9ADwITPLmtl1wF7gJxtwfpvK4j+gifcT9zNQvTAzAz4PHHX3P2x4Sn1sGc3qpT62PDPrN7Ou5H4eeCfwLOpfTTWrmfrY8tz9d9x9t7vvIc5aP3T3f8om7WNr2mHwWudNtiDf4NPabLYDfxn/X0QK+LK7f9fMHgO+Zma/AZwEPrCB57jhzOzPgV8A+sxsGPgPwO+xTI083ub+a8AzQA34LXePNuTEN0iTev2Cmd1G/Ke5F4HfBNUr8Q7gw8DhZI4lwCdRH2umWb1+TX1sWTuBL5pZSDzo9jV3f9DMHkH9q5lmNfuf6mOrsin/DdMOgyIiIiIiLdK0DRERERGRFik8i4iIiIi0SOFZRERERKRFCs8iIiIiIi1SeBYRERERaZHCs4iIiIhIixSeRURERERapPAsIiIiItKi/w+4d7m9Qa7SJQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsIAAAHSCAYAAADmLK3fAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3hc5Z328e+ZrlEZddmSbMsV925TjMFA6KGFFgJkIQGSsCHvZjdsSCFLCrtcQHaTbCokhEACgZCEhF6ydghg4wLGvUu2JVt1VEYzmn7ePx5JbjI2WPYY5v5cly80M2fOPOfMCN3zO0+xbNtGRERERCTbODLdABERERGRTFAQFhEREZGspCAsIiIiIllJQVhEREREspKCsIiIiIhkJQVhEREREclKrky9cGlpqV1TU5OplxcRERGRLLFixYpW27bL9r8/Y0G4pqaG5cuXZ+rlRURERCRLWJa1faD71TVCRERERLKSgrCIiIiIZCUFYRERERHJShnrIzyQRCJBfX090Wg0002RQ/D5fFRXV+N2uzPdFBEREZEP5LgKwvX19eTn51NTU4NlWZlujhyEbdu0tbVRX1/PyJEjM90cERERkQ/kuOoaEY1GKSkpUQg+zlmWRUlJiSr3IiIi8qF2XAVhQCH4Q0Lvk4iIiHzYHXdBOJM6Ojr46U9/+oGee8EFF9DR0THILRIRERGRo0VBeC/vFYRTqdR7Pvf555+nsLDwaDTriNi2TTqdznQzRERERI47CsJ7ueOOO9i6dSvTp0/n9ttvZ9GiRZxxxhl86lOfYsqUKQBceumlzJo1i0mTJvHAAw/0P7empobW1lbq6uqYMGECN998M5MmTeKcc86hp6fngNd65plnOPHEE5kxYwYf+9jHaGpqAqC7u5sbb7yRKVOmMHXqVP74xz8C8OKLLzJz5kymTZvGWWedBcBdd93F/fff37/PyZMnU1dX19+GW2+9lZkzZ7Jz506+8IUvMHv2bCZNmsR//Md/9D9n2bJlnHLKKUybNo25c+cSCoWYP38+K1eu7N9m3rx5rFq1ahDPtIiIiEjmHVezRuzt28+sZd2urkHd58TKAv7jokkHffyee+5hzZo1/SFw0aJFLF26lDVr1vTPjvDQQw9RXFxMT08Pc+bM4fLLL6ekpGSf/WzevJnHH3+cBx98kKuuuoo//vGPXHfddftsc+qpp7JkyRIsy+KXv/wl9957L9///vf57ne/SyAQYPXq1QC0t7fT0tLCzTffzGuvvcbIkSMJBoOHPNaNGzfy61//ur/Cfffdd1NcXEwqleKss85i1apVjB8/nquvvponnniCOXPm0NXVRU5ODjfddBMPP/wwP/jBD9i0aROxWIypU6ce/okWERER+RA4boPw8WLu3Ln7TBH2ox/9iD//+c8A7Ny5k82bNx8QhEeOHMn06dMBmDVrFnV1dQfst76+nquvvprdu3cTj8f7X+PVV1/l97//ff92RUVFPPPMM5x22mn92xQXFx+y3SNGjOCkk07qv/3kk0/ywAMPkEwm2b17N+vWrcOyLIYOHcqcOXMAKCgoAODKK6/ku9/9Lvfddx8PPfQQN9xwwyFfT0REROTD5rgNwu9VuT2WcnNz+39etGgRr776KosXL8bv97NgwYIBpxDzer39PzudzgG7Rtx2223867/+KxdffDGLFi3irrvuAkyf3v1nZBjoPgCXy7VP/9+927J3u2tra7n//vtZtmwZRUVF3HDDDUSj0YPu1+/3c/bZZ/OXv/yFJ598kuXLlw90akREREQ+1NRHeC/5+fmEQqGDPt7Z2UlRURF+v58NGzawZMmSD/xanZ2dVFVVAfCb3/ym//5zzjmHH//4x/2329vbOfnkk/n73/9ObW0tQH/XiJqaGt5++20A3n777f7H99fV1UVubi6BQICmpiZeeOEFAMaPH8+uXbtYtmwZAKFQiGQyCcBNN93El770JebMmXNYFWgRERGRDxsF4b2UlJQwb948Jk+ezO23337A4+eddx7JZJKpU6dy55137tP14P266667uPLKK5k/fz6lpaX993/zm9+kvb2dyZMnM23aNBYuXEhZWRkPPPAAn/jEJ5g2bRpXX301AJdffjnBYJDp06fzs5/9jHHjxg34WtOmTWPGjBlMmjSJz3zmM8ybNw8Aj8fDE088wW233ca0adM4++yz+6vKs2bNoqCggBtvvPEDH6OIiIjI8cyybTsjLzx79mx7/0vu69evZ8KECRlpj+xr165dLFiwgA0bNuBwDPx9Se+XiIiIfBhYlrXCtu3Z+9+virAc4JFHHuHEE0/k7rvvPmgIFhERERlIpoqsH8RxO1hOMufTn/40n/70pzPdDBERkSMSiibY0Bgi3+diTFkeLufAxZ3mUJSf/N8WRpfncf1JI0jb0NWToDuWpDuWJM/roqowB4fDoqkryqr6TmYML6TY72FrSzfbWsPsaItQ3x4hnrIZUuDjupOG43RYtIRijCzNZVVDJ5ubQswaUUyu10k8mWZ4sR/LsrBtmx3BCMFwnEg8xdLaILs7e3BYFrNripk+rBCf28G6XV3EU2mmDytkaCCH1za18LNFW6kI+JhWHaDA56Y7lqQnkcLvcVLbGmZ7WwS302JUWR4ep4PfvbWd4cV+/uXscTgti1gyjdflYGRpLnVtYZZsa2NoIAe302JnsIdQNEE4niKaSFHodzOsyM/I0lyeXtnAtpYws0YU4XY68LocnDiqmEcXb+eNrW1MqQqQ43YSiiXxuRz4PU5yPE6+c8lkSvO8A74PmaAgLCIicgxFEym8LseAs/bsLZZM0dDew8hSMwtQeyRBkd/Na5tbeeyt7ZTle5lUGeDEkcWMLM3Fsiw6IvH+8NPYFWVOTTFjyvNY09BJQ3sPWDClKkBDew/NoRhjyvM4YUg+XpeDFdvbWdPQSUsoxqgyc39loY+61ghbmkM0dEQpy/dSVegjlYaH36zF53Zy18WTGF2Wx85ghJfWNva2Nc6ahi48LgfFfg9FuR5Kcs1/AzlumrqiBMNxyvO9BCNx2sNxJgwtIBJP0RyKMaemCJfDwcbGLjY2dRMMx3BYFjOHF5GybZbXtZNKp8n3uRkS8NEZSeBxOZg1oohEKk13LEl7JMEji+voiCQAGFacwzkTh1DbGsa2bdxOB41dUWKJNPXtEcJxs4Lsn99poLY13P+8Pp7esBeKmUHlTodFrsdJVzTZv02+z4XP7aS1O8ZPFm4hkU5j2+ByWCTTB1ZJKwq8lOZ5aQ7FaAnF+u93WFCW7yWWTPP7ZTsH/Hw4LEjb5rjq2sI88+6uA7bJ9TgZWZZLMmXz2qZW4qk0Z5xQxrrdXdz462Xv+fnrO8Y8r4s8rwuvy0F7JE5773nJ97mYWh3gLyt3YVkQS6T5xWvbyPU4uXxmFZuauonEkxT4XMSSaVq740TiyUO84rGnPsLygen9EpEjEUumWF3fybgh+RT43AB09iSIJlKU53sPCIqJVJpEKk2O29n/2O7OHtbt6iLH7STf5ybf5yLf56I7lmTdri5eWd9EOm1z9sQhRBMpVtV3sLK+k7HlecwaUUSO28k/NrfSHIrysQkVTK4K4Pc42d3ZQ0NHlO5okmHFOUwYWkBpnpdFG5tpCcVo6Y7xytombGDGsELKCrwkkjZt4Ri2bQJEMm2zblcnXdEkVYU5TK0O0NmT4LlVuwnkuBle4qcjkqAgx83IEj+zRhTx2uZW1jR0MmtEEcvr2mnsilJdlEMsmaYlFKM0z0trd4yyfC/RRIpQbwgrz/fidTvYGTxwus7DsXdQ87gcxJPpA7ZxOixSe4W56qIcQlFTMS3McdMWju+z7biKfGzbJhiO0x6Jk0gdPG94nA7iqQNfE6DQ76a8NxRub4sAMLosF7/HRUdPnKbOGAU5bnriyf4w22f+2FKuP2kEoWiSR5dsZ3VDJ2PK8nC7LOLJNBUFPnI9LgI5bj53+iheWtvE797azuwRRUytLiTfZ0JgZ0+C2rYwiaRNeYGXqVUB/rGllfZwnNk1xYyryGNEcS4Bv/kcb2nu5rdLtlOS66GyMIeNTSHGlOUxY3ghK7a303cal2xrIxxLEvC7mTm8iKqiHFwOi6nVhQRy3KTTNut2d7G1pZtwLMW4ijy8Lifv1nfQ1GW+mHxyznDcTovO3gp2nteE8e5YkiK/B6fD/K5EEym6ognK832EY0mWbGujIMdNjttJJJ5iW0s3RbkeTh9XRls4TjptU1mY0//8Pq3dMTY1hphSHSC/9/cWTPV9aW2QyVUBKgp8B32vM+VgfYQVhOUD0/slcuxFEyncTscBf5z2F44leXV9E6ePK6PQ7wHor8Z4nA7e2NqGhQkKqbRNNJnG73aytC7I2zva6YgkyPW4cLssGjujjCnP45TRpTgseHFtI+/u7GDm8CJsYGNjiE1NIYYV+blkeiUdPQnaI3F64il2BCOEoklcDovSfC9NnVGWbGuj0O8hFE3QFU3iczs4YUgB9cFIf5gqzfPicVrEU2l8biddPYn+ylu+10VVUQ7BcJzmvapoAwnkuLEs+qt7OW4nk6sK2NgY2rM/n4vyfC9bW8KH/T44HRanjC7B63KypqGTYDiOy2lRmufFYUHKtrGwGFeRR0mulx3BCO/Wd+CwLD4xs4ruWJLGzihFuR66eszl+5ZQjCK/m7kji1le187o8jzOmzSE17e04vc4mVQZYENjF+Mq8rlp/kg8TgfbWsO8tS3IW7VtJFM2U6oDjCnLo6bUT5Hfw982mOA+tTpATUku8VSaVfUdVAZyqCzMYUtzNxsaQ3T2JDhxZDEzhhdS4HPT0NHDhsYQjZ091JTmMrY8n4oCLx2RBA0dPXRFE8ypKaY9HOeRxdtpj8QZGvBxyfQqAn43HqcDn9vZf75s26Y7luwNxQnK8r2U5nlo7opR6DdhbFtrmFyvi8IcN0vrgjgsi/FD8vf5UtTYGcWyGDBoJVNptraEe78UuXA6rf4vWH1SafuQvzvy0aQgLINO75dkG9u22RnsIc/nosjvJhJPkeN24tjrD2s0kWLtri66ogm8LgfDivy0heOs29VFKp2mNM/L5KoA+T4XuV4X8WSa3y7Zzjs7Okim0xT43ITjSdbt7qKmJJczTijnqjnDWLeri0cW1/HyuiYAhhXlUJDjxgISKZtEb2CsLPSx4IRyHnq9lg2NIXLcTmpKc9nd2dMfBr0uB7Heit+Y8jyau6J0RZP73L/3z3leU2HdW1VhDg0dpvo4pMDH2Apz+b19r8vJDgsqC3Mo9LtJJG1aumPkep2cOqaMnngSt9PB/HFlvLWtjW0tYWpK/dSU5OJxOVi7qwvbBq/bQTSeIs/noiTXi8flMNXa9h6Kcj2MH5LPjOGFJFI2XT0JQtEkoWgCv8fF6PI8plYHsG1Y3dBJkd/NsGI/bqeDZCpNY1eU7liSkaW5eF2mP2VdW5hILMWQgI/qohz8Hic7gz2sbuhgV0eU08aVMbYib8Cgd6iuDslUmrRtKq4Dfba2t0WoKPCR43EO8GwRORIKwoeho6ODxx57jFtvvfUDPf8HP/gBt9xyC36/f5BbdnzK9Psl0iedtvsvEed6nf0DYna0Rfjd0u30xFOMKc9jTFkeQwI+/B4X3TET2AI5ps9iIpVm/e4unli2k+3BCMOK/DgsiCbTxBIpYsk0ta1hdgTNpdm+y8cFPhdVRX7qgxGiyRSptM0AXQEPqm8/o8pMGOvqSeB2WkysLGBLczebmrr7Q2mh383F0yrxuZ3Ut0f6j9ntdOByWPQkUmxu6qaxK0q+z8WdF05kxfZ2mkNRqopMBdC2zaXNk0eV0NGT4PdLdzC6LI9RZXk0h6LMGF7EGSeUke9zE0+argi5XhdbmkO8u7OTlG0zc3ghY8rzaQ5F8Tgd/RXnaCLF6oZOhgZ8vRVdxz5fEkREMkVB+DDU1dXx8Y9/nDVr1nyg59fU1LB8+fJ9Fsg41pLJJC7XsRkDmen3SzLrcCpg+0uk0liAy+mgPRzn9S2trN3VxaUzKqltCfPjhVsYWZrLjOFFlOR62BGMmEv0XidbW7rpiCSoKsyhORQjGI6T53XR1BVlRzDS37/R73FyxaxqdndGeXV9E07L6u8vdzj8Hifjh+SzuzOKBXjdTrwuB163k7I8D6eNKyOWSNPSbS7p7gz2sLuzhxHFfnK9LtxOB5MqCyjN99ITT7EzGCHf52basABel5OGjh7W7+4iEk/R1ZOgsyfBRdMqmTWiaMD2vLOjnSeW7WT8kHyunjP8kNXCdNpmZe+l7yGB46+fnohIJhwsCGvWiL3ccccdbN26lenTp3P22Wdz3333cd999/Hkk08Si8W47LLL+Pa3v004HOaqq66ivr6eVCrFnXfeSVNTE7t27eKMM86gtLSUhQsX7rPv73znOzzzzDP09PRwyimn8Itf/ALLstiyZQuf//znaWlpwel08oc//IHRo0dz77338uijj+JwODj//PO55557WLBgAffffz+zZ8+mtbWV2bNnU1dXx8MPP8xzzz1HNBolHA7z17/+lUsuuYT29nYSiQTf+973uOSSSwAzR/D999+PZVlMnTqVn/70p0ydOpVNmzbhdrvp6upi6tSpbN68GbfbPdBpko+odNqmoaOHLc3d+NxOZteYKXEaOnp4ZW0jw0v8eJxO/rahib+tb6axK8q43kvEeT43H5tQzsbGEGt2dVFdmENLd4z6YIQcj5M8r4u0bfqSpmybQI6b4F4Da37x2lZs2wyAeas2yLOrdgNgWeB3OwnHUwwv9lOc62HRphbK8ryU5XvpjiUZV5HPuZOHUJLrwbIs1jZ08thbOyjIcfPFM8Zw3UkjKM/30tQVY0tzNy3dUSLxVP8gj87eUdAup8WI4lxOG1e6zwCQwVaW72X6sMLD3n7G8CJmDB84JA/E4TAj60VE5NCO3yD8wh3QuHpw9zlkCpx/z0Efvueee1izZg0rV64E4OWXX2bz5s0sXboU27a5+OKLee2112hpaaGyspLnnnsOgM7OTgKBAP/93//NwoULB6wIf/GLX+Rb3/oWANdffz3PPvssF110Eddeey133HEHl112GdFolHQ6zQsvvMDTTz/NW2+9hd/vJxgMHvLQFi9ezKpVqyguLiaZTPLnP/+ZgoICWltbOemkk7j44otZt24dd999N2+88QalpaUEg0Hy8/NZsGABzz33HJdeeim///3vufzyyxWCP+Rau2N0R5P43E6cDotFG5tZVhekKNdDeb6PeDLN0+80kLZtJlYWsCMYYVNjaJ8R17keJ4V+D7s7e/a51O9xOZg3uoSPTahgS0s36bTNzmCEb/1lLT63g2nVhazZ1UlxroeTRpcQS6YJx5Kk0jY3zKvB5bAIhuOMLM1l5ogiRpXm8tAbtXicTm49YzQuh0VHJEFbOMbQQA65XhfptP2+LrHfdckkU8V17ameDgn4VCEVEZF9HL9B+Djw8ssv8/LLLzNjxgwAuru72bx5M/Pnz+crX/kKX/3qV/n4xz/O/PnzD7mvhQsXcu+99xKJRAgGg0yaNIkFCxbQ0NDAZZddBoDPZ/5Iv/rqq9x44439fY2Li4sPuf+zzz67fzvbtvn617/Oa6+9hsPhoKGhgaamJv7v//6PK664oj+o921/0003ce+993LppZfy61//mgcffPB9nikZDLZt09QVI8fjpMBnfjXr2iI0tPcQTaSIJs1sAfleF3/f1ML6xhCpdJpkyiaZtknbNsOL/cQSaV5e13hAP9Uiv5twLNU/RVHf6PCltUGGF/u5YlY144bkM64in7buOG9ubSUcS1FZ6OPSGVU0dUaJJlOcNKoEv8d1QNu3toSpKPB+oGrq7eeO37etvfON9nm//Uz3HykuIiIykOM3CL9H5fZYsW2br33ta3zuc5874LEVK1bw/PPP87WvfY1zzjmnv9o7kGg0yq233sry5csZNmwYd911F9Fo9KBLEB6s76XL5SKdTvfvc2+5ubn9P//ud7+jpaWFFStW4Ha7qamp6X+9gfY7b9486urq+Pvf/04qlWLy5MkHPRZ5/5q7omwPRogn0wwN+Fi/O8S63Z2MLsujJM9LLJFiZ3sPf1nZwKr6TsDM6el1OQ6YE7OPy2EGU3l6p9Hyuc3gsLe2BYkmU9xy2mjGVeQRTaSJJVNMrgowu7cPakckQSyZPmR19LzJQ/a5Pbos76DbWpbFmPKDPy4iInI8On6DcAbk5+cTCoX6b5977rnceeedXHvtteTl5dHQ0IDb7SaZTFJcXMx1111HXl4eDz/88D7P379rRF9oLS0tpbu7m6eeeoorrriCgoICqqurefrpp7n00kuJxWKkUinOOeccvvOd7/CpT32qv2tEcXExNTU1rFixgrlz5/LUU08d9Dg6OzspLy/H7XazcOFCtm/fDsBZZ53FZZddxpe//GVKSkr69wtmWeVrrrmGO++8czBP6UdaNJHitU0txJLp/lH74XiSjkiiN2ymWL+7i0WbWjicMamjynL52vnjcTos2iNxwrEUJwzJZ1RpLjkeJz63WZKzLRxnWnWgf6T+/g41iG3vSquIiEg2UxDeS0lJCfPmzWPy5Mmcf/753Hfffaxfv56TTz4ZgLy8PH7729+yZcsWbr/9dhwOB263m5/97GcA3HLLLZx//vkMHTp0n8FyhYWF3HzzzUyZMoWamhrmzJnT/9ijjz7K5z73Ob71rW/hdrv5wx/+wHnnncfKlSuZPXs2Ho+HCy64gP/8z//kK1/5CldddRWPPvooZ5555kGP49prr+Wiiy5i9uzZTJ8+nfHjzWXnSZMm8Y1vfIPTTz8dp9PJjBkz+kP8tddeyze/+U2uueaawT6tH3rRRIra1jDdsSRLa4O8ur4Jl8NiS3P3PnOm7s/ttCjL83LbGWOYXVOM2+mgvj3CsGI/M4YXsqMtQlfvQgNVRTn9g72O1GDsQ0REJBto+jQB4KmnnuIvf/kLjz766GE/58P+fm1t6SYUTRKJJ3nm3V3Ut/eQStv4PWaJVsuCDbvNill7rxE/fVghPreDkjwvn5wzjKEBH/Gk3T/fapHfTUGOG7fzwEnzRURE5NjT9GlyULfddhsvvPACzz//fKabclR0x5I8tXwnb9UGcTsd+NwOdgQjLNm2ZzaOXI+TcUPycVoW7ZEewrEk8WSasRV53HzCKCYOLaDQ76amJJdhxdmxYIqIiMhHnYKw8L//+7+ZbsIRiSZSPPxmHb9+o5ax5flMH1bI7s4o9e0R6tt7aOyKkkqbGRUsy2zv97j46nnjGVeRRyptc+rY0gNmQhAREZGPNv3ll+NaPJlmV0cPDR09+NwOnA4HDe09/GZxHZ2RBLNqinhlXRMtIbNk7I5ghDe2tjKkwEd1UQ5zRxZTXZTDmePL39eiBCIiIvLRd9wF4Q+ybKsce0ezb/nq+k5+/WYtL65pJHKQ6cOqCnMYVpzDE8t2Mm9MKT/85ChOGV2KbZs5ddU/V0RERA7luArCPp+PtrY2SkpKFIaPY7Zt09bW1r8AyAcViSdp6orxzo52Hn6zjnjvNGSrGzrJ9Ti5eFol1UU5lBf4GFbkJ5ZMkbZt8n1uZgwrxOV0HLDimGVZuJ367IiIiMihHVdBuLq6mvr6elpaWjLdFDkEn89HdXX1+37ezmCEHcEIr21q4ZHF2+lJmIrvuIo8qgpzCIbj3PnxiVw5u/qwVgd7vyuOiYiIiPQ5roKw2+1m5MiRmW6GDIJdHT388NXNLKltw+ty4HM7aY/E2RnsAcBhwUXTKjl9XBlVhTnMqSlWqBUREZFj6rgKwvLhlU7btEfiNHXFeHplAw+/WQc2nDm+HIBoMkVVYQ43zx/FuIp8RpbmUlFwZF0rREREBlW4FboaYMhUGIwumqEmCG6DEScf+b72Z9vmn+MIxsQ0rIBoJ1TNAssBybjZX06R+XnzyzD6TPAMMG1osBbatkD5RCio3HO+ElHYsRg6dsCIU6B07L5thsE5t4NEQVjel1TaZnldEIfDoqkrytLaIEtrg2xp7u5fdMKy4BMzqvny2WOpLtKcuyIixwXbBjsNDue+93fWm7BWM9/8D9y2ofY1KB4JhcMhEoSO7eD0mNDTvB6WPgCn3AaeXFj+a5hzE+SV7dlnPAxr/gi+QhOELAe8+3uItMKCr0PB0D1tsixo2wr/+D44XHDC+TDydHD5INoB3nxw9naVa1wNsW6onm3uS6ehfqk5htbN0LoJZv2TeX7bFvPctq2w+CdQdgJMugw6d0KiB3raYfe7MPxkGHMW/PVLJvhhw7x/gY/dBbEQdDfBuqdh/TOQPxRGn2VeY82fYMsrJjwPmwuVM6CnAxIRSMXN85b9CuLdMPly8AWgeYNpR8kYczzbFppzNPI0cHkhnQSn17RnzR9h4X+a8x6oNu9FcCt0t4CdMu1PJcBXAOPON6/hdJnH00koG2/20bgKLrjPvCebXjTHkFcOO5bA4h8P/FmZcR20boGdS6B0HCy4A9y5JvB2N8E7v4X1fzWfJwB/CVTPNaF56QPQttnc73CbdsVC0F4L7XXw5bXgLz7yz/MgOa5WlpPjU1t3jLd3dBCKJnjojVrWNHT1P+b3OJk1oohJlQGGFHgpL/AxcWgBNaW5GWyxiByX+kLPkYp1Q6gRSsfse5/TAy7PwZ8XCcKud0w7RpxyYJUrnT6wupaIwtJfmNBRUGVCRHArNK2FwhHmD//wkw5+XO110LLJBIhwiwlgOUUmYHTshOJRENptqpCePMgfYgJP4XBTpfPmm+06d5rQkVNsglVPuwmuqRhs/T8TsopHmcD5ly/CrpVQMhpyS0247Gk3IcxOw8RLzD66m0yA2fyyCW4jToVRp8PWhbDjTdPOk26FN/8XYr3/36+YYo4/ETEBzumBcDMMnQ7X/9mEzp1vwVs/N23em+UwwciTC2PPhmgX1L1uQl0qbgKg5YB4yIRgywmJsHluYBjkFJogDODJN+GxYzs0rel7AXMeop0mvLVu3PPaOcXmHLBf5vHkmXPR99qn3AZdu2Dlb83x97Tv2XbYieZ26yYTChNhKKiG3BLTrr5QuLcTLjBfHt74gTlXFZPM86Od5vGSsSbsh/cbG8pNGHoAACAASURBVGU5zXmpmQ9FNeY427ebLyeBatPWnCLT7q5dsPbPe87V/vvx5pnPXSp+4OOzPwvjLzDttxzmvAe3wbJfmvbO+xdY/hB0N+77PF8hzLoBxnwMWjbA7pWwdRF01UNgOJz7PSg9Ad78kfkCUVBl2l40Ek798r5fmo6Rg60spyAsA2ruivLEsp38bUMz79Z39F/NKM/3cvu5J1BR4COQ42ZSZQEuTVUmkjm2bf44H0mFJRYyf5gLqgYOdLZtLnOmEvuGz77HWjeZMBVuMX+cp1wFgap9t4uH4cl/Mn+0L/ohDJuz57H2ut7LqPMOrFbatqlqNa0xlbfuJhOeEhGYfi1MuAg2vwIrf2dC3SU/gYrJpkrm8u45vtV/gFfv2hNA8irMH/mJl8C7j8HqP5oq1pizYdrVpnppp3sv/242YTKd3NOu/KGmLXbaVN6inYBlAmznThOwqmbBmqcGDiAA3gITMF0+c+7jYRMq+wKV22+qh7veGfj5voB5TxKRA/c7+RPmnPZ0mHb7AqYym+iBdX8Bf6kJVKFdUHOq6Qqw8O49n4MTPw/vPGre22EnwSlfNMf71i9MWD/jG/DslyGdgjmfhRe/ZoJbnyFT4Jy7zXlorzWvO3K+udz+wr+bsOV0m9f25Jnze9IXTGDd/jps6q3MFg43gblti6n6TrzYtHvrQvMFwJMLp3zJVGMDVSbMvXynCXaTL+8N3w7zWenaZboCFI82nw93DuRXwvJfwYbn4Ny7TVC1bXjjh+Z9LxlrjrdyhnkvwLTtnUdgwiUw5QrzOxNuNWHVX2JCsstjQqq7twtguM28nsdv9h/tNOekYKj5AtZRZ7aznKZqvuZP5tjn3Hx4XR96OqB5nfkM5paa+xpXmXa7c+Glr5sq9Mn/bD5zkTbTvvLxA++vbas5d8UjzZfMti3mve7caT6vo8/Y8/vVJ52GlvUmuHuOv2KYgrAcUlc0wYurG3lnZzt/fqeBaCLNtGGFnHlCOaeOLSGQ46a6yI/P7Tz0zkRkYLZt/qD7Swbud7f2aXNpsbvZhIQTLjBBKlBt/pBuXWiCTaAahk6FF75qQmjZBBMsuptNhWfceaZCmE7Ciodh9VOmejNqgQlObr+p3mxdaCp46aQJJKXjTLArGW2qrk1rTPWzryI47VNm27p/mD+I4RZTUQNTQUrFTdUvUG3akoiYYOXNM6Eqt9wEqqpZ5l8yCisfg3TCbOcLmFDR024qcE63uXzscJnwlltqLot78mDJT3sv9bthypWmyhrcZtpiOUwV0bLM+U4nTcXztK+Y8PjGD2D7G3vOe818c+zvPm7anF9pKn02cPa3YdQZ5li7GkwILhhqguvKx8wl4oLe4N+xwxx7Z4Opqk65EmZ+2pyL3DLznodbTVgvqDTH6S0wl7TBtK2rwRzH2qdNmBn/caiaaY4pEjQhw+GEVU+aMDLjehNSWjeZoDLjeiga8d6fwYG+8CR7A3tfVb2nw7zP487f0769pVOAZYLappfN8VbONO9PQeXBX18kAxSE5QB1rWH+vqmF7W0R8n0uHlu6g5ZQjDyvi3MmVvCls8aqi4McvsG67P1+pRImZPRVIBM98PajJnCMO89UgTy5prrSshFWP2nC3/CTTFDs6TDhMN5tKo9DpprQtPIxEyryKuCsO2HnUtOnrqDShJ5Ade+/YaZqklsKyZgJL6//j6mgYJvgl1/R2y+vAuqXQ1Pv5d3i0SbsOpxmn+UT4cnrzeXDktGwbZEJiu/F4TJVo+a1pprkK4BNL+37PMth+jVuf/PAy6dDp5nL+4Fq08eyZYM5T6HdJmxWTDKhbchkU7ld/BMTosedZ47PX2LaPeYss4/27aZq2N1ojtnlM/tsXgdnfcu0462fmza2bjLtnHKlacPaP5s2+UvMe7bhOROaz/meuYS7f2WseYOprFVMMkE71m32kYyaz0RwW28grjb7rzl138/o7lWmz2TNqaarBJgg27zeBN/36mZxONKpAyvcIpIRRxSELcs6D/gh4AR+adv2Pfs9XgQ8BIwGosBnbNtec8CO9qIgnDm2bfOr12u554UNJNM2XpeDWDLN1OoAd108ienVhZrKTAZm2yaY5Jbt+we+4W147GpTJaueYwLXtE+aS5Zr/mhCVfkEGHWmGU28+CemmplbZqp1iZ49YRXb9Knc+Za53D90Gky42Gyz6UVzqTlviLk0N/osePkbJlzOuM6EyXcfN5coD2CZfVtOU4lsXHVgyPTkm/6JYAbQDJ0OW/9mApvlNGEp0mb6bPZt16d4lAlf8W4THMd8zISu7hYTCkON5jwUVJlzk4iYdu9YYs5lpM3sp3Qc3PSqqYxGgua1XT5zeTnaZY7bW2CC9vY3TXeC6ln7tiUWMlXc9jrT7soZpktDLGTa6PZDsge8AVP1HEis22y3f/gMt/Ze4j0GX5LTafOl5EgDqYhkvQ8chC3LcgKbgLOBemAZcI1t2+v22uY+oNu27W9bljUe+Ilt22e9134VhI+tjkicZ1btZmltkGW1QRq7opw7qYJvXjiRYcV+wrEkfo9TK/p9WKXT5jJ3QdW+AdW2TVhy55gwZFnQtA42v2T68I2YZx4bSKgRnr7VBDV/iQlMfaPHc4rNKGl3julntuE5M5DFV2gGqBQON0HthAth43N79lkxxTzu9vf2qbRNRdPtN5eZ+/oYOr0mrMZDpr199/dVNqOd0LDcXBZ355p+nqufNBW4yhlmtLfLa0J34QgTTtu3m5A5cr6p6sbDJly7vLDxRXNuJlxsqsCpxJ6+c8kYbHgWhkzbt39stNOE+M56U/HcudSMxB5/oQn873dKo8Y1JsTPuclUmEVEZNAcSRA+GbjLtu1ze29/DcC27f/aa5vngP+ybfv13ttbgVNs22462H4VhI+dho4ern1wCXVtESoKvMwdWcKZ48u4dHqVgu+xtn/3gbatpv/mqV/ed7CTbff2Raw0gcq2zSjeDc+ZUDfrRjNw49W7TIU21ASx3rkgL/4xFA4zQfSNH+653BwYZiqwu96hf+S0v8Rcch52oql8vvu4qbZ680xVNp2AadeY0BhpNZXJYXNNaGteZ0JiKmb2ffkvTcBMp00/0d9cZKY1mnWjuSS+6SX427fNgI2rHzXbJaPm8rnDYcJnqNGEXX/JnkEm3c2w5VUTviunm4ErYLo8bHqxt2/nWDMYxeUxo+xFRET2ciRB+ArgPNu2b+q9fT1wom3bX9xrm/8EfLZt/6tlWXOBN3u3WbHfvm4BbgEYPnz4rO3bB7p8KUfqH5tb+P7Lm+jqSZC2bdrCZgDEg5+ezYkjixV+D1e4zYTTvc9X01r4w41mLsqSMSacnvR5cxtMVbBzJ4y/6MDLubWvwdP/bAaSXPpTc4n5ofNMJbdsAnz8f8xzgrVm7skdb5qQOOIUEwbr/mH6YmKZ/qBggvKwuaadgWoTfPtGxYMZRLTgq6af6rZFJsxWzjRVx5YNpv2bXjTbWg5TEU3Gegc4VZpR9QcbVXwokaCpRo+/cM85TKfM6+gzKCIix9CRBOErgXP3C8Jzbdu+ba9tCjB9iGcAq4HxwE22bb97sP2qIjy4kqk0r65v4qkV9by6vpmaEj+TqwI4LAu308FnTq1hUmUg083MnIFWs+lpN4OBnG5453emf+voM02l8qWvmQE9gWHmUruvwEyV89LXTaW2b7qivjkoJ33C9OHsm+aobw5QX6G5HL/jTVj/rAmXXbtMxTYSNP03z7rTVHb7Rt6DqcrO/ozpx9q01lRO5/0LnPg5cywrfm36f552u2lbn65dsPF50xe0cITp59o3cf3BhFtNKO6b51FEROQj5qh2jdhvewuoBabatt010DagIDxYbNvmT2838D+vbqK+vYfyfC/XzB3OFxaMzr5pzvaeDN+2zbRPO5aYwLj5ZVMpvfRnJuy++3t47l9NhXXaJ+H5r5hK5el3mH6l2xaaLgE9HWbkeXeTmfTccsCn/2rCc7jVDFx69stm/yVjYPIVZg7FxT82l/nDLeZ5/lIzmOv0fzeX+Zc+aALyrBvMoLGOnaargZ02ld3ScQfO0SgiIiIfyJEEYRdmsNxZQANmsNynbNteu9c2hUDEtu24ZVk3A/Nt2/70e+1XQfiDs22bV9Y1saExxPrdXbywppFpwwr55wWjOWtCBc6PwowP4VYzaX5gmJkaqa+/aP0KM/k5lukLmlNkgmzLRnjqM2aS/qpZZsL41k3mOQ63Wee9u8UM1PIVQk/QVHqb1vauqDTPzIG6baHpn3r6V031tU8iagZjeQtg0qWHfxy2bQKxv0Qj30VERDLkYEF4gBmy92XbdtKyrC8CL2GmT3vItu21lmV9vvfxnwMTgEcsy0oB64DPDmrrpV9bd4x/eWIl/9jcCoDbafFvZ4/j1jPGfHgDcDxsQqjTbaaHeuVOM4dr32pMvkIzoXx7nVlxyBvonTM0ZP794/tmVoHiUWYQ19aFJtie+Hkz12nfYKxYyKzbHg+bfrrTrzX9Zt/+DVz43ybk7lhs5pfdvxrr9plJ8d8vyzp01wQRERHJCC2o8SESTaS49pdvsaahk69fMIFr5g7HssB9PC1xnEqa/qzevIEf71sVqS9ovvljePmb9C+lmU6ZuVZnf8Ys09rdBGv/ZFYtKh1rVtk6+dY9MwN01sPfvmtWvbrs52ZWg3j4yJabFRERkY+UD1wRlsxq7ory7KrdvFXbRl1rhI1NIX567UwumHIcVhm7dsNjV5puDbcsMtNchVth4wtmGq5dK+Gd35p5Y2tONbMJvPIt08922IlmNadwK1zxkKnK9pl48cFfM1ANn/jFvvepb62IiIgcBgXh49imphBX/nwxnT0JRpbmUpbv5Z5PTMlcCA7WmmqvrxCe/rwJrkOnw2lfMbMfPPlpMzDMTpuf84fC+mf2WiTBA7NvNFXb1X8w89YWDocrHzaraImIiIgcQwrCx6GdwQiLt7bx369swuNy8ML/m8+EoQWHfuJgC9aauWujXWY1sWW/MqHW4Tah9oTzYfsb8KtzzGpm+ZVww3NmkNqfbjZ9bk++FaZebWZNcPvM4DaAM74Bq54wc+QqBIuIiEgGKAgfZ5bVBbn+V28RTaQpy/fyyGfmHp0QHAnC0gfM3LFTrjQh9e1HYPFPzOpjHduhY8deT7Bg1j/B0Gmwe5UZiFY+3oTkhXebqcnO+y8TdCunQ8loKBm77xy3e/P4TXVYREREJEM0WO448eaWVpbWBfnV67WU5Xv5xXWzGF2Wh2OwZ4LobjaV2Dd+aOa4BdPVoWombP0/E3SdXsgrh1ELYORppq+vnd5TzRURERH5ENFguePYA69t5T+f34BlwcShBTzw6dlUFeYc+Y6b1poldLe/aW4nInuqvMNPgev+ZFZXe/dx2PZ3mPlPcOH3zTRmIiIiIh9xCsIZZNs2//PqZn70t81cOHUo918xjRzPIKwGZ9tmINqT/2R+rpkH7hwzW8OsG03f3vIJe7YfdfqRv6aIiIjIh4yCcIbYts33nlvPr16v5arZ1fzXJ6Z+8AUx6t4wC0NYlvnv7lWQ7IHySXDdH7Wgg4iIiMgAFIQzwLZt7nlxA796vZYbTqnhWx+fePh9gdMp2PCsmcHBX2Lm2P3TLXtWYRs6zSxGUTTCLD2sGRlEREREBqQgfIyl0zb3vrSRX/x9G9eeOJz/uGgilnWYIXjbInjxa9C8DgLDYccSs+pa2Xi48QVw+83sDyIiIiJySArCx1BnJMG///FdXlrbxDVzh/PdSyYffghuWAG/uxICw8zKaxMvhZYNsPRBmP9vWlJYRERE5H1SED5Glmxr40uPv0MwHOebF07gs6eOfO8Q3NMBnfVQMcnM6fvE9WYas5te3RN6KybBRT84NgcgIiIi8hGjIHwMrNvVxU2/WU55gZeHbpjD5KoB+u2mEtDdBA1vw4qHofbvkE6aAW/ttWYJ4xufU+VXREREZJAoCB9lm5tCfObhZeR5XTx200kMCQzQhzcShAfPgPY6c7ugGk7+IgSqzWpvY86C8+4xt0VERERkUCgIH0Vvbmnlc4+uwOt28uhn5w4cggH+9m3o2GnCbtkJUHMaOHvfmrk3H7sGi4iIiGQRBeGjpD0c57bH32FIwMdvPjOXyr1XikunTL/fvCGw7mnTFeLkL8JJX8hYe0VERESyjYLwUfLdZ9fR2ZPgtzeduG8Ijkfgd1fA9jf23Fc+ERZ87dg3UkRERCSLKQgPsh1tEb7z7FpeXd/MbWeOYcLQgj0PdjbA05+HHYthwdfBTkHlTBh7NjgGYWllERERETlsCsKDKJFKc8PDS2nuivHV88Zz8/yRex58+xF4/naw03DJT2D6pzLXUBERERFREB5Mjy7ezraWMA/dMJszx1fseWDb3+GZf4GaU+Hi/zXLH4uIiIhIRikID5Kmrig/eHUT88eWcsYJ5Xse2Pii6Q5ROhau/i34Cg6+ExERERE5ZhyZbsBHQWNnlE8+sIRU2uZbH5+4Z8W4v30XHr8a8ivhmscVgkVERESOI6oIHyHbtvnC71bQEorxyGfnMrYi3zxQ9zr8436Ydg1c9CNweTLbUBERERHZhyrCR+i1za28s6ODb1w4gVkjepc/bngbnr4Vimrgwu8rBIuIiIgch1QRPgK2bfO/f9vM0ICPy2f2Ln/83L/Bsl+CNwDX/gE8uZltpIiIiIgMSBXhI/DS2iaWb2/n86ePxuNywOqnTAie/Vn48moYfmKmmygiIiIiB6GK8Ae0pbmbr/zhXaZUBbh6zjBo3w7PfhmGnQjn3wtOnVoRERGR45kqwh9AMpXmi4+9jdfl4BfXz8LnsOFPt5gHP/GAQrCIiIjIh4AS2wfw+NIdbGgM8bNrZ1KZ74JX7oSdS+ATD5oBciIiIiJy3FMQfp86Iwn++5VNnDSqmPOGp+GBM6BpNcz+DEy9KtPNExEREZHDpCD8Pj34j220RxLceeF4rL/cAMFtZsW4CRdlumkiIiIi8j4oCL8PnT0JfvNmHedPHsKknb+HbYvg4/+jECwiIiLyIaQg/D488mYdoViCbxc+By9+H8aeC7NuzHSzREREROQD0KwRh6kzkuCXr9fy7apllC//Pkz9JFz1CFhWppsmIiIiIh+AKsKH6SeLtpCIhri25zEYfjJc9nOFYBEREZEPMQXhw1DfHuHhN+r44bA3cTU3w8d+qxAsIiIi8iGnrhGH4fGlO6iyGzm3/TE44QItnSwiIiLyEaCK8CHYts3z79bz87wHcVhuuOC+TDdJRERERAaBgvAhrN3VxWmdf+UE91q47AEIVGe6SSIiIiIyCNQ14hBeeXs9X3Y9RWLE6Vo5TkREROQjRBXh92DbNtUrf0SB1YPjgv/SADkRERGRjxBVhN/D6m07uTj5MtuHXwYVkzLdHBEREREZRArC76HuH0/gtRKUnX5LppsiIiIiIoNMQfgg0mmbiu3P0OwaSt4oTZcmIiIi8lGjIHwQqzdtYnZ6Fe2jLlbfYBEREZGPIAXhg9i9+Emclk31/Osz3RQREREROQoUhA+iqGERja5KcodNyXRTREREROQoUBAeQH1zG1MTq2gbenqmmyIiIiIiR4mC8AA2LHmRHCtOyfQLM90UERERETlKFIQHkN78MjE8DJn6sUw3RURERESOEgXh/cSTacZ2LWFHwSxw52S6OSIiIiJylCgI72fLxtWMtBqJjzwz000RERERkaNIQXg/naueB6B85kUZbomIiIiIHE0KwvvJr1/ITmsoZSMmZLopIiIiInIUKQjvLdHDmPA7bAmcnOmWiIiIiMhRpiC8l/Z1C/ERJ1ZzVqabIiIiIiJHmYLwXtrX/o2Y7WLIVAVhERERkY86BeG9OBtXstEezsThFZluioiIiIgcZQrCfdJpyrrXs907Do9Lp0VERETko06Jr097Lf50mPbCSZluiYiIiIgcAwrCvZL1K8x/h0zPcEtERERE5FhQEO7VXbucqO0mMHxKppsiIiIiIseAgnCvdMM7rLdHMKqiMNNNEREREZFjQEEYIJ0mL7iWVemRjCrLy3RrREREROQYUBAG6KrHkwrT4BlFIMed6daIiIiIyDGgIAzQuhmAeOHoDDdERERERI4VBWHA7g3CniEnZLglIiIiInKsuDLdgONBonkTUTuH0orqTDdFRERERI4RVYQxQXibPZShhf5MN0VEREREjhEFYcAZ3Mw2u5KhAV+mmyIiIiIix4iCcDyML9LItvRQhigIi4iIiGQNBeG2LQBso5LyfAVhERERkWyhINw7Y0QwZwQel06HiIiISLZQ8mvbQhqLZKAm0y0RERERkWNIQbiznnariOJAINMtEREREZFjSEE41EijXUhlYU6mWyIiIiIix1DWB+FU1252pQo1Y4SIiIhIlsn6IGyHdtNsF2kOYREREZEsk91BOBnH1dNGs13IkAIFYREREZFsclhB2LKs8yzL2mhZ1hbLsu4Y4PGAZVnPWJb1rmVZay3LunHwm3oUhJsBaKKIoQH1ERYRERHJJocMwpZlOYGfAOcDE4FrLMuauN9m/wyss217GrAA+L5lWZ5BbuvgCzUC0GQXUV7gzXBjRERERORYOpyK8Fxgi23b22zbjgO/By7ZbxsbyLcsywLygCCQHNSWHg2h3QCEPaX43M4MN0ZEREREjqXDCcJVwM69btf33re3HwMTgF3AauD/2bad3n9HlmXdYlnWcsuylre0tHzAJg+i3opw3F+R4YaIiIiIyLF2OEHYGuA+e7/b5wIrgUpgOvBjy7IKDniSbT9g2/Zs27Znl5WVve/GDrrQblI4wF+a6ZaIiIiIyDF2OEG4Hhi21+1qTOV3bzcCf7KNLUAtMH5wmngUhRppdxQT8Kt/sIiIiEi2OZwgvAwYa1nWyN4BcJ8E/rrfNjuAswAsy6oATgC2DWZDj4rQbpopotDvznRLREREROQYcx1qA9u2k5ZlfRF4CXACD9m2vdayrM/3Pv5z4LvAw5ZlrcZ0pfiqbdutR7HdgyPUSGM6QGGOgrCIiIhItjlkEAawbft54Pn97vv5Xj/vAs4Z3KYdfXaokYbkTAL+43+mNxEREREZXNm7slwyhtUTpNkupEhdI0RERESyTvYG4Z4OAIIUqI+wiIiISBbK3iAcCwEQsnMozFHXCBEREZFsk8VBuAuAMD4CqgiLiIiIZJ0sDsKmItxt+ynSYDkRERGRrKMgTI6mTxMRERHJQlkfhEPkUKAgLCIiIpJ1sjcIx7sBsLz5OB1WhhsjIiIiIsda9gbh3sFyrpxAhhsiIiIiIpmQxUE4RAI3ebn+TLdERERERDIgq4NwxMrR8soiIiIiWSqrg3BYM0aIiIiIZK2sDsJdtk/LK4uIiIhkqawNwnYsRFc6h0J1jRARERHJSlkbhFM9XYTsHAp8rkw3RUREREQyIGuDMLEQ3eSQ43FmuiUiIiIikgFZG4SteIhuOwefS0FYREREJBtldRAOkYPPrSAsIiIiko2yMwinEjiSUcK2D587O0+BiIiISLbLzhQYCwHQTQ5edY0QERERyUrZGYTj3YAJwqoIi4iIiGSn7EyBvRXhkO1XH2ERERGRLJXVQVgVYREREZHslZ0psC8I2+ojLCIiIpKtsjQIdwFo+jQRERGRLJalQXivirC6RoiIiIhkpexMgTEza0QYn1aWExEREclSWRqETUW4x/LhdloZboyIiIiIZELWBuGYw4/X7cayFIRFREREslF2BuFEmLjDp4FyIiIiIlnMlekGZEQqSdJy43Vm5/cAEREREcnWinAqThKnKsIiIiIiWSw7g3A6QQIXXld2Hr6IiIiIZGsQTiVI4lJFWERERCSLZWkQjhO3nfi0mIaIiIhI1srOJJhKkFAfYREREZGslrVBOG6rj7CIiIhINsvOJJhO9HaNUEVYREREJFtlZxDu6yPsUhAWERERyVZZGoSTxDRYTkRERCSrZWcSTMWJpdU1QkRERCSbZWUQttMJYrYDr4KwiIiISNbKyiBMMk5Cs0aIiIiIZLWsTIK25hEWERERyXpZGYTNghouDZYTERERyWLZmQTTCZK4NH2aiIiISBbLziCcivdWhBWERURERLJVVgZhK50kgVOD5URERESyWPYlwXQKy06RsFURFhEREclm2ReEUwkADZYTERERyXLZlwTTfUFY06eJiIiIZLPsC8KqCIuIiIgIWRyEkzjxavo0ERERkayVhUE4DkAcF15VhEVERESyVvYlwd4+wklbfYRFREREsln2BeG9+wira4SIiIhI1sraIJyynLidVoYbIyIiIiKZkoVB2PQRxunBshSERURERLJV9gXhdBIAy+nOcENEREREJJOyLwj3VoTTDgVhERERkWyWhUHY9BFOW64MN0REREREMilrg3ASVYRFREREsln2BeF0X0VYU6eJiIiIZLPsC8K9fYSTlirCIiIiItksC4Nwb0XYoYqwiIiISDbL2iCcUkVYREREJKtlYRDunT5NQVhEREQkq2VfEO5dUCOp6dNEREREslr2BeHeinBKs0aIiIiIZLUsDMJ906d5MtwQEREREcmkLA7C6hohIiIiks2yLwhrQQ0RERERIRuDcCpOAjeWI/sOXURERET2yL40mEqQspw4rEw3REREREQyKSuDcNJy47CUhEVERESyWfYF4XSCFKoIi4iIiGS77AvCqThJy42lirCIiIhIVsvCIKyKsIiIiIhkaRBOWi5VhEVERESy3GEFYcuyzrMsa6NlWVssy7pjgMdvtyxrZe+/NZZlpSzLKh785g6CVJwkLlWE/397dxtj6VnWAfx/dfsib7FoFyRtA8WUD9WEik3VNEI1IkXQSqJJMSofTCqGJhpjTGui4jeVaEwEbSo2EBUaEt4aUikkChhjpAULtJRirZWuJXSRKEKM3e25/HDObCfDzO5MO8uZev1+yeQ555lnd++9Zv/N/QAAD9hJREFUcmf3n3uuc98AAMOdMghX1aEkb0nyyiSXJHltVV2y+ZnuflN3X9rdlya5IclHu/srp2PAT9riuBVhAAB2tSJ8eZL7u/uB7n40yS1Jrj7J869N8s79GNxpYUUYAIDsLgifn+ShTe+PrO59g6p6epKrkrx7h+9fW1V3VtWdR48e3etY98djx1ZBWBIGAJhsN0F4u8TYOzz740n+fqe2iO6+qbsv6+7LDh8+vNsx7q/HjuV4HRKEAQCG200QPpLkwk3vL0jy8A7PXpOD3BaRJIvlirAcDAAw226C8B1JLq6qi6rq7CzD7q1bH6qqb03ysiTv398h7rNVj7APywEAzHbmqR7o7uNVdV2S25McSnJzd99TVa9fff/G1aOvSfKh7v76aRvtfnjsuA/LAQBw6iCcJN19W5Lbtty7ccv7tyV5234N7LR57NEcix5hAIDp5p0stziWY1aEAQDGmxeEHzumRxgAgJlB+FgObbsnHAAAc4wMwg7UAABgYBB+dNkjPO9vDgDAJvPi4OJYjueQHmEAgOFmBeHuZHE8x1prBADAdLOC8GPHkiSP+rAcAMB4w4Lwo0liH2EAAIYF4cVyRdjJcgAAzArCq9aIY+1ADQCA6WYG4RzSGgEAMNysIHz2M5KXXZ/P5YVaIwAAhpsVhJ92bvJDN+TeemHkYACA2WYF4ZVFR48wAMBwI4Nw0nqEAQCGGxmEFx09wgAAww0NwlaEAQCmmxmEF61HGABguJFBuDt2jQAAGG5mEI4eYQCA6UYGYT3CAAAMDsKSMADAZEODsAM1AACmGxmEW2sEAMB4I4Pwwq4RAADjjQzCrUcYAGC8kUFYjzAAAOOCcHcniR5hAIDhxgXhxTIHa40AABhuYBC2IgwAwOAgrEcYAGC2cUF4lYNtnwYAMNzYIKxHGABgtnFBWI8wAADJ6CAsCQMATDYwCK97BAAAHATjgnBbEQYAICOD8PKqRxgAYLZxQfhEj7AkDAAw2sAgvLw6UAMAYLZxQbhtnwYAQAYG4RMrwpGEAQAmGxeEO1aEAQAYGIQXjlgGACATg/AqCcvBAACzjQvCbUUYAIAMDMKP7yO85oEAALBW4+LgRhC2awQAwGzjgvCqM0KPMADAcPOC8IkDNSRhAIDJxgVh26cBAJCMDMIO1AAAYGIQXiyvZUUYAGC0eUG4HagBAMDAILxBjzAAwGzjgrAeYQAAkpFBeHm1IgwAMNvAIKxHGACAgUG4TwRhSRgAYLJxQfjx1oj1jgMAgPUaF4RbjzAAABkYhPUIAwCQDA7CVoQBAGYbF4S1RgAAkAwMwlojAABIRgbh5dWuEQAAs40LwvYRBgAgGRmEl1c9wgAAs40Lwo/vGrHmgQAAsFYDg/DyakUYAGC2gUG41z0EAAAOgHFBWI8wAADJyCC86hEe9zcHAGCzcXFQjzAAAMnIIGzXCAAABgdhB2oAAMw2LghvfFhODAYAmG1eEM5Ga4QoDAAw2a6CcFVdVVX3VdX9VXX9Ds9cWVV3VdU9VfXR/R3m/lkslldBGABgtjNP9UBVHUryliQvT3IkyR1VdWt3f3bTM+cm+ZMkV3X3F6rqOadrwE/W4z3Cax4IAABrtZsV4cuT3N/dD3T3o0luSXL1lmd+Jsl7uvsLSdLdj+zvMPfPiQM1bBsBADDaboLw+Uke2vT+yOreZi9K8uyq+khVfaKqfn6736iqrq2qO6vqzqNHjz6xET9Jtk8DACDZXRDeLjL2lvdnJvneJK9K8ookv1lVL/qGX9R9U3df1t2XHT58eM+D3Q+LE7tGSMIAAJOdskc4yxXgCze9vyDJw9s88+Xu/nqSr1fVx5K8OMnn92WU++jxXSPWPBAAANZqNyvCdyS5uKouqqqzk1yT5NYtz7w/yQ9W1ZlV9fQk35fk3v0d6v44sSLs03IAAKOdckW4u49X1XVJbk9yKMnN3X1PVb1+9f0bu/veqvpgkk8nWSR5a3fffToH/kS1HmEAALK71oh0921Jbtty78Yt79+U5E37N7TTY7FwoAYAAANPlnu8NWK94wAAYL0GBuGNAzUkYQCAycYF4Q16hAEAZhsXhB8/UEMSBgCYbGAQXl4FYQCA2QYG4Y0e4TUPBACAtRoXhNuuEQAAZGQQ1iMMAMDAIKxHGACAZGQQdsQyAAAjg/Dy6kANAIDZxgXh7rYaDADAvCC86LYaDADAvCDcrT8YAICBQXjR+oMBABgYhPUIAwCQDAzCi257CAMAMDEIO0wDAICRQbgjBgMAMC4IdycWhAEAGBiEO2f4tBwAwHjjgrAeYQAAkpFB2PZpAACMDMIO1AAAYGAQbrtGAACQkUFYjzAAAAODsB5hAACSkUFYjzAAAAOD8HIf4XWPAgCAdRsXCZdHLFsRBgCYbmAQjh5hAADmBeGOXSMAABgYhBfdkYMBABgXhLvbijAAAPOC8GKhNQIAgIlBWGsEAAAZGIQ7DtQAAGBiEHbEMgAAGRiEl/sIS8IAANMNDMJWhAEAGBmE9QgDADAwCLddIwAAyMggrEcYAICBQViPMAAAydAgrEcYAICBQThWhAEAmBeElwdqSMIAANONC8LL7dPWPQoAANZtXBC2IgwAQDIwCDtQAwCAZGAQbtunAQCQgUF44UANAAAyMgh3xGAAAAYGYT3CAAAMDMJ6hAEASEYGYT3CAAAMDMKL7pwx7m8NAMBW4yLholuPMAAA84Jwd+waAQDAwCAcPcIAAAwMwgu7RgAAkLFBWBIGAJhuXhBeOFADAICBQdiBGgAAJAOD8PKI5XWPAgCAdRsXhDt6hAEAGBiElyvCgjAAwHTjgrAeYQAAkoFBeNEO1AAAYGQQtiIMAMDEILxoPcIAAMwLwh3bpwEAMDEI6xEGACADg7AeYQAAkrFBWBIGAJhuYBB2oAYAAAODcHf7sBwAABODcPQIAwAwLwjrEQYAINllEK6qq6rqvqq6v6qu3+b7V1bVf1XVXauv39r/oe4PPcIAACTJmad6oKoOJXlLkpcnOZLkjqq6tbs/u+XRv+vuV5+GMe6b7k6iNQIAgN2tCF+e5P7ufqC7H01yS5KrT++wTo/FMgenIgkDAEy3myB8fpKHNr0/srq31Q9U1aeq6q+r6ru2+42q6tqqurOq7jx69OgTGO6Ts7AiDADAym6C8Haxsbe8/2SS53f3i5P8cZL3bfcbdfdN3X1Zd192+PDhvY10H6xycM6QhAEAxttNED6S5MJN7y9I8vDmB7r7q939tdXr25KcVVXn7dso98nGirDPygEAsJsgfEeSi6vqoqo6O8k1SW7d/EBVfUettmKoqstXv+9/7Pdgn6wTK8KSMADAeKfcNaK7j1fVdUluT3Ioyc3dfU9VvX71/RuT/FSSX6qq40n+J8k1vbFFwwGiRxgAgA2nDMLJiXaH27bcu3HT6zcnefP+Dm3/nWiNsGsEAMB4o06W21ii1hkBAMCsILxYXvUIAwAwKgjrEQYAYMPMICwJAwCMNywIL6+lNQIAYLxRQbhP7BoBAMB0s4Lw6urDcgAAjArCPiwHAMCGUUH4zDPOyKUXnptvf+Y56x4KAABrtquT5f6/OPysc/K+N1yx7mEAAHAAjFoRBgCADYIwAAAjCcIAAIwkCAMAMJIgDADASIIwAAAjCcIAAIwkCAMAMJIgDADASIIwAAAjCcIAAIwkCAMAMJIgDADASIIwAAAjCcIAAIwkCAMAMJIgDADASIIwAAAjVXev5w+uOprk39byhyfnJfnymv7spyL12hv12hv12js12xv12hv12js125t11Ov53X146821BeF1qqo7u/uydY/jqUK99ka99ka99k7N9ka99ka99k7N9uYg1UtrBAAAIwnCAACMNDUI37TuATzFqNfeqNfeqNfeqdneqNfeqNfeqdneHJh6jewRBgCAqSvCAAAMNyoIV9VVVXVfVd1fVdevezwHUVU9WFWfqaq7qurO1b1vq6oPV9U/r67PXvc416mqbq6qR6rq7k33dqxRVd2wmnP3VdUr1jPq9dmhXm+sqn9fzbO7qurHNn1ver0urKq/rap7q+qeqvrl1X1zbBsnqZc5to2q+paq+nhVfWpVr99Z3Te/dnCSmpljJ1FVh6rqn6rqA6v3B3OOdfeIrySHkvxLkhcmOTvJp5Jcsu5xHbSvJA8mOW/Lvd9Pcv3q9fVJfm/d41xzjV6a5CVJ7j5VjZJcsppr5yS5aDUHD63773AA6vXGJL+2zbPqlTwvyUtWr5+V5POruphje6uXObZ9vSrJM1evz0ryj0m+3/x6QjUzx05et19N8o4kH1i9P5BzbNKK8OVJ7u/uB7r70SS3JLl6zWN6qrg6ydtXr9+e5CfXOJa16+6PJfnKlts71ejqJLd09/92978muT/LuTjGDvXaiXp1f7G7P7l6/d9J7k1yfsyxbZ2kXjuZXq/u7q+t3p61+uqYXzs6Sc12Mr5mVXVBklcleeum2wdyjk0KwucneWjT+yM5+T+WU3WSD1XVJ6rq2tW953b3F5PlfzpJnrO20R1cO9XIvNvZdVX16VXrxMaPyNRrk6p6QZLvyXIFyhw7hS31Ssyxba1+ZH1XkkeSfLi7za9T2KFmiTm2kz9K8utJFpvuHcg5NikI1zb3bJnxja7o7pckeWWSN1TVS9c9oKc48257f5rkO5NcmuSLSf5gdV+9VqrqmUneneRXuvurJ3t0m3vjarZNvcyxHXT3Y919aZILklxeVd99ksfH1yvZsWbm2Daq6tVJHunuT+z2l2xz75tWr0lB+EiSCze9vyDJw2say4HV3Q+vro8keW+WP574UlU9L0lW10fWN8IDa6camXfb6O4vrf5jWST5szz+YzD1SlJVZ2UZ6v6qu9+zum2O7WC7epljp9bd/5nkI0muivm1K5trZo7t6IokP1FVD2bZhvrDVfWXOaBzbFIQviPJxVV1UVWdneSaJLeueUwHSlU9o6qetfE6yY8muTvLOr1u9djrkrx/PSM80Haq0a1Jrqmqc6rqoiQXJ/n4GsZ3oGz8Y7jymiznWaJeqapK8udJ7u3uP9z0LXNsGzvVyxzbXlUdrqpzV6+fluRHknwu5teOdqqZOba97r6huy/o7hdkmbX+prt/Ngd0jp35zfqD1q27j1fVdUluz3IHiZu7+541D+ugeW6S9y7/X8mZSd7R3R+sqjuSvKuqfiHJF5L89BrHuHZV9c4kVyY5r6qOJPntJL+bbWrU3fdU1buSfDbJ8SRv6O7H1jLwNdmhXldW1aVZ/vjrwSS/mKjXyhVJfi7JZ1Y9iUnyGzHHdrJTvV5rjm3reUneXlWHslwMe1d3f6Cq/iHm1052qtlfmGN7ciD/DXOyHAAAI01qjQAAgBMEYQAARhKEAQAYSRAGAGAkQRgAgJEEYQAARhKEAQAYSRAGAGCk/wMBaJ9SRi3NwAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(12,8))\n",
    "plt.plot(result['train_loss'], label=\"train loss\")\n",
    "plt.plot(result['test_loss'], label=\"test loss\")\n",
    "plt.yscale('log')\n",
    "ticks = [0.16, 0.225, 0.32, 0.45, 0.64, 0.9, 1.28, 1.8, 2.56, 3.6, 5.12]\n",
    "plt.yticks(ticks=ticks, labels=ticks)\n",
    "plt.legend()\n",
    "plt.show()\n",
    "\n",
    "plt.figure(figsize=(12,8))\n",
    "plt.plot(result['train_accuracy'], label=\"train accuracy\")\n",
    "plt.plot(result['test_accuracy'], label=\"test accuracy\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we may see, the accuracies are close to the one-vs-rest approach we used in the last notebook. This is correct, as we just refactor the implementation from the last notebook - under the hood, it is still the same model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Before we move on\n",
    "\n",
    "We are finally going to see real neural networks in the following notebook. But before you open it, I have one request.\n",
    "\n",
    "> **Please make sure you understand everything in this notebook.**\n",
    "\n",
    "Even read this notebook once again. Make sure you are fully aware of activation functions, softmax, dense layer, losses, optimizers and you really know, how the gradients are computed along the way. Makes also sure to understand how weights, layers, models, optimizers, losses assembly together to create the resulting trained model.\n",
    "\n",
    "This is probably the hardest and technical notebook you will see and I know it was challenging. Please, if you had problems with the formulations or you were missing something, don't hesitate to contact me. The concepts discussed here will be the fundamentals of neural networks that we are going to use over and over again and you should understand them before moving forward."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bibliography\n",
    "\n",
    "- \\[1\\] The Softmax Function Derivative, Stephen Oman, 17th June 2019, [online](https://aimatters.wordpress.com/2019/06/17/the-softmax-function-derivative/)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}