Neural Network from scratch using NumPy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Quiz on Neural Networks Concepts\n",
    "\n",
    "## 1. What is the primary purpose of weights in a neural network?\n",
    "- A) To store input data\n",
    "- B) To adjust the influence of inputs\n",
    "- C) To provide bias to the output\n",
    "- D) To calculate activation functions\n",
    "\n",
    "**Answer:** B) To adjust the influence of inputs\n",
    "\n",
    "## 2. What does the bias term in a neural network do?\n",
    "- A) Shifts the activation function\n",
    "- B) Multiplies the weights\n",
    "- C) Reduces the error rate\n",
    "- D) Increases the number of neurons\n",
    "\n",
    "**Answer:** A) Shifts the activation function\n",
    "\n",
    "## 3. What is forward propagation in neural networks?\n",
    "- A) The process of training the network\n",
    "- B) The process of calculating the output of the network\n",
    "- C) The process of initializing weights\n",
    "- D) The process of optimizing hyperparameters\n",
    "\n",
    "**Answer:** B) The process of calculating the output of the network\n",
    "\n",
    "## 4. What is backward propagation (backpropagation)?\n",
    "- A) Calculating the loss of the model\n",
    "- B) Updating the weights based on errors\n",
    "- C) Initializing weights to random values\n",
    "- D) Forwarding inputs through the network\n",
    "\n",
    "**Answer:** B) Updating the weights based on errors\n",
    "\n",
    "## 5. Which activation function is defined as \\(f(x) = \\frac{1}{1 + e^{-x}}\\)?\n",
    "- A) ReLU\n",
    "- B) Sigmoid\n",
    "- C) Tanh\n",
    "- D) Softmax\n",
    "\n",
    "**Answer:** B) Sigmoid\n",
    "\n",
    "## 6. What is the purpose of an activation function in a neural network?\n",
    "- A) To normalize input data\n",
    "- B) To introduce non-linearity to the model\n",
    "- C) To update weights during training\n",
    "- D) To calculate the gradient of the loss function\n",
    "\n",
    "**Answer:** B) To introduce non-linearity to the model\n",
    "\n",
    "## 7. Which of the following is a common loss function for binary classification?\n",
    "- A) Mean Squared Error (MSE)\n",
    "- B) Cross-Entropy Loss\n",
    "- C) Mean Absolute Error (MAE)\n",
    "- D) Hinge Loss\n",
    "\n",
    "**Answer:** B) Cross-Entropy Loss\n",
    "\n",
    "## 8. What is the primary goal of gradient descent in neural networks?\n",
    "- A) To maximize the loss function\n",
    "- B) To minimize the loss function\n",
    "- C) To increase the learning rate\n",
    "- D) To initialize the weights\n",
    "\n",
    "**Answer:** B) To minimize the loss function\n",
    "\n",
    "## 9. Which of these is NOT an activation function?\n",
    "- A) ReLU\n",
    "- B) Tanh\n",
    "- C) Leaky ReLU\n",
    "- D) Cross-Entropy\n",
    "\n",
    "**Answer:** D) Cross-Entropy\n",
    "\n",
    "## 10. What does the learning rate control in gradient descent?\n",
    "- A) The number of hidden layers\n",
    "- B) The size of the steps taken during optimization\n",
    "- C) The type of activation function\n",
    "- D) The number of neurons in each layer\n",
    "\n",
    "**Answer:** B) The size of the steps taken during optimization\n",
    "\n",
    "## 11. In which scenario would you use the Softmax activation function?\n",
    "- A) Regression problems\n",
    "- B) Multi-class classification\n",
    "- C) Binary classification\n",
    "- D) Clustering\n",
    "\n",
    "**Answer:** B) Multi-class classification\n",
    "\n",
    "## 12. Which optimization technique helps to avoid overfitting in neural networks?\n",
    "- A) Stochastic Gradient Descent (SGD)\n",
    "- B) Dropout\n",
    "- C) ReLU Activation\n",
    "- D) Forward Propagation\n",
    "\n",
    "**Answer:** B) Dropout\n",
    "\n",
    "## 13. What does the term \"epoch\" mean in neural network training?\n",
    "- A) One pass through all of the training data\n",
    "- B) One update of a single weight\n",
    "- C) One forward pass of a single input\n",
    "- D) One backward pass of a single input\n",
    "\n",
    "**Answer:** A) One pass through all of the training data\n",
    "\n",
    "## 14. Which activation function is most likely to cause the vanishing gradient problem?\n",
    "- A) ReLU\n",
    "- B) Sigmoid\n",
    "- C) Softmax\n",
    "- D) Leaky ReLU\n",
    "\n",
    "**Answer:** B) Sigmoid\n",
    "\n",
    "## 15. What is the primary advantage of using ReLU over Sigmoid or Tanh?\n",
    "- A) ReLU can output only binary values\n",
    "- B) ReLU is less computationally expensive\n",
    "- C) ReLU has no derivative\n",
    "- D) ReLU does not have a bias term\n",
    "\n",
    "**Answer:** B) ReLU is less computationally expensive\n",
    "\n",
    "## 16. What is the purpose of the bias in a neural network?\n",
    "- A) To prevent the model from overfitting\n",
    "- B) To allow the model to fit better by shifting the activation function\n",
    "- C) To make the model train faster\n",
    "- D) To control the learning rate\n",
    "\n",
    "**Answer:** B) To allow the model to fit better by shifting the activation function\n",
    "\n",
    "## 17. What does \"gradient\" refer to in the context of gradient descent?\n",
    "- A) The total error of the model\n",
    "- B) The slope of the loss function with respect to the weights\n",
    "- C) The number of neurons in a layer\n",
    "- D) The difference between predicted and actual values\n",
    "\n",
    "**Answer:** B) The slope of the loss function with respect to the weights\n",
    "\n",
    "## 18. Which technique is used to prevent the exploding gradient problem?\n",
    "- A) Learning Rate Decay\n",
    "- B) Gradient Clipping\n",
    "- C) Dropout\n",
    "- D) Cross-Validation\n",
    "\n",
    "**Answer:** B) Gradient Clipping\n",
    "\n",
    "## 19. What does \"stochastic\" mean in Stochastic Gradient Descent?\n",
    "- A) Deterministic\n",
    "- B) Based on a small, random sample\n",
    "- C) Always uses all data\n",
    "- D) Optimizes without randomness\n",
    "\n",
    "**Answer:** B) Based on a small, random sample\n",
    "\n",
    "## 20. Which type of loss function is typically used for regression tasks?\n",
    "- A) Hinge Loss\n",
    "- B) Mean Squared Error (MSE)\n",
    "- C) Cross-Entropy Loss\n",
    "- D) Categorical Cross-Entropy\n",
    "\n",
    "**Answer:** B) Mean Squared Error (MSE)\n",
    "\n",
    "## 21. What is the main drawback of using a high learning rate in gradient descent?\n",
    "- A) Slow convergence\n",
    "- B) Oscillation around the minimum\n",
    "- C) Overfitting the model\n",
    "- D) Increased computation time\n",
    "\n",
    "**Answer:** B) Oscillation around the minimum\n",
    "\n",
    "## 22. What is the function of batch normalization in a neural network?\n",
    "- A) To normalize input data\n",
    "- B) To speed up training and reduce internal covariate shift\n",
    "- C) To increase model complexity\n",
    "- D) To regularize the model\n",
    "\n",
    "**Answer:** B) To speed up training and reduce internal covariate shift\n",
    "\n",
    "## 23. Which of the following is NOT a type of gradient descent?\n",
    "- A) Mini-batch Gradient Descent\n",
    "- B) Stochastic Gradient Descent\n",
    "- C) Batch Gradient Descent\n",
    "- D) Dynamic Gradient Descent\n",
    "\n",
    "**Answer:** D) Dynamic Gradient Descent\n",
    "\n",
    "## 24. What is \"momentum\" in the context of neural network optimization?\n",
    "- A) A method to accelerate gradient vectors\n",
    "- B) A technique to reduce the number of epochs\n",
    "- C) An activation function\n",
    "- D) A regularization technique\n",
    "\n",
    "**Answer:** A) A method to accelerate gradient vectors\n",
    "\n",
    "## 25. Which function represents a linear activation function?\n",
    "- A) $f(x) = max(0, x)$\n",
    "- B) $f(x) = x$\n",
    "- C) $f(x) = \\frac{1}{1 + e^{-x}}$\n",
    "- D) $f(x) = tanh(x)$\n",
    "\n",
    "**Answer:** B) $f(x) = x$\n",
    "\n",
    "## 26. What happens during the backpropagation process?\n",
    "- A) Weights are initialized randomly\n",
    "- B) The gradient of the loss is calculated with respect to each weight\n",
    "- C) The model's architecture is defined\n",
    "- D) The input is passed through the network\n",
    "\n",
    "**Answer:** B) The gradient of the loss is calculated with respect to each weight\n",
    "\n",
    "## 27. Which of the following is true for the Rectified Linear Unit (ReLU) function?\n",
    "- A) It has a range of (-1, 1)\n",
    "- B) It introduces non-linearity in the model\n",
    "- C) It always outputs values greater than zero\n",
    "- D) It is computationally expensive\n",
    "\n",
    "**Answer:** B) It introduces non-linearity in the model\n",
    "\n",
    "## 28. Why is Cross-Entropy Loss often preferred over Mean Squared Error for classification tasks?\n",
    "- A) Cross-Entropy Loss is faster\n",
    "- B) Mean Squared Error does not work with classification\n",
    "- C) Cross-Entropy Loss provides probabilistic interpretation\n",
    "- D) Mean Squared Error is computationally expensive\n",
    "\n",
    "**Answer:** C) Cross-Entropy Loss provides probabilistic interpretation\n",
    "\n",
    "## 29. What is the \"learning rate decay\" in neural network training?\n",
    "- A) Increasing the learning rate during training\n",
    "- B) Decreasing the learning rate as training progresses\n",
    "- C) Keeping the learning rate constant\n",
    "- D) Adjusting the weights directly\n",
    "\n",
    "**Answer:** B) Decreasing the learning rate as training progresses\n",
    "\n",
    "## 30. Which of the following methods is used to update the weights in a neural network?\n",
    "- A) Cross-Validation\n",
    "- B) Forward Propagation\n",
    "- C) Backpropagation\n",
    "- D) Activation Function\n",
    "\n",
    "**Answer:** C) Backpropagation\n",
    "\n",
    "## 31. Which function has outputs only in the range of 0 to 1?\n",
    "- A) Tanh\n",
    "- B) ReLU\n",
    "- C) Sigmoid\n",
    "- D) Linear\n",
    "\n",
    "**Answer:** C) Sigmoid\n",
    "\n",
    "## 32. What is the result of using a very small learning rate in gradient descent?\n",
    "- A) Fast convergence\n",
    "- B) Slow convergence\n",
    "- C) Skipping over the minimum\n",
    "- D) No effect on convergence\n",
    "\n",
    "**Answer:** B) Slow convergence\n",
    "\n",
    "## 33. Which type of gradient descent uses the entire dataset for every step?\n",
    "- A) Stochastic Gradient Descent\n",
    "- B) Mini-Batch Gradient Descent\n",
    "- C) Batch Gradient Descent\n",
    "- D) Momentum-based Gradient Descent\n",
    "\n",
    "**Answer:** C) Batch Gradient Descent\n",
    "\n",
    "## 34. What is the main advantage of the Leaky ReLU activation function over ReLU?\n",
    "- A) It has a fixed output range\n",
    "- B) It does not suffer from dying neurons\n",
    "- C) It is more computationally efficient\n",
    "- D) It has a probabilistic interpretation\n",
    "\n",
    "**Answer:** B) It does not suffer from dying neurons\n",
    "\n",
    "## 35. What is the typical shape of a loss function for neural network training?\n",
    "- A) Linear\n",
    "- B) Quadratic\n",
    "- C) Convex\n",
    "- D) Non-convex\n",
    "\n",
    "**Answer:** D) Non-convex\n",
    "\n",
    "## 36. Which optimization technique adapts the learning rate during training?\n",
    "- A) Adam\n",
    "- B) SGD\n",
    "- C) Momentum\n",
    "- D) Adagrad\n",
    "\n",
    "**Answer:** A) Adam\n",
    "\n",
    "## 37. Which activation function is zero-centered?\n",
    "- A) Sigmoid\n",
    "- B) ReLU\n",
    "- C) Tanh\n",
    "- D) Softmax\n",
    "\n",
    "**Answer:** C) Tanh\n",
    "\n",
    "## 38. What does the term \"epoch\" in neural network training indicate?\n",
    "- A) One complete cycle of forward and backward propagation for all training samples\n",
    "- B) One iteration through a mini-batch of data\n",
    "- C) The number of hidden layers in the network\n",
    "- D) The total number of neurons in the network\n",
    "\n",
    "**Answer:** A) One complete cycle of forward and backward propagation for all training samples\n",
    "\n",
    "## 39. What is the main purpose of using the Softmax function?\n",
    "- A) To reduce the number of features\n",
    "- B) To scale values to the range of 0 to 1\n",
    "- C) To convert values into probabilities\n",
    "- D) To minimize loss\n",
    "\n",
    "**Answer:** C) To convert values into probabilities\n",
    "\n",
    "## 40. Which loss function is suitable for multi-class classification?\n",
    "- A) Mean Squared Error\n",
    "- B) Hinge Loss\n",
    "- C) Categorical Cross-Entropy\n",
    "- D) Mean Absolute Error\n",
    "\n",
    "**Answer:** C) Categorical Cross-Entropy\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Quiz on Neural Network Numerical Concepts\n",
    "\n",
    "## 1. If a neural network has 3 input neurons, 2 hidden layers with 5 neurons each, and 1 output neuron, how many total neurons are there in the network?\n",
    "- A) 11\n",
    "- B) 14\n",
    "- C) 10\n",
    "- D) 13\n",
    "\n",
    "**Answer:** D) 13\n",
    "\n",
    "## 2. For a neural network with 1000 training examples and a batch size of 50, how many iterations are required per epoch?\n",
    "- A) 10\n",
    "- B) 20\n",
    "- C) 50\n",
    "- D) 200\n",
    "\n",
    "**Answer:** B) 20\n",
    "\n",
    "## 3. How many parameters (weights) are there between the input layer (3 neurons) and the hidden layer (4 neurons) of a neural network?\n",
    "- A) 7\n",
    "- B) 10\n",
    "- C) 12\n",
    "- D) 15\n",
    "\n",
    "**Answer:** C) 12\n",
    "\n",
    "## 4. If a neural network has 2 hidden layers with 8 neurons each and uses the ReLU activation function, how many ReLU operations are performed per forward pass?\n",
    "- A) 8\n",
    "- B) 16\n",
    "- C) 2\n",
    "- D) 10\n",
    "\n",
    "**Answer:** B) 16\n",
    "\n",
    "## 5. If you train a neural network for 100 epochs, with a batch size of 32 and 1024 training samples, how many iterations are performed in total?\n",
    "- A) 3200\n",
    "- B) 100\n",
    "- C) 320\n",
    "- D) 1024\n",
    "\n",
    "**Answer:** A) 3200\n",
    "\n",
    "## 6. A neural network has 2 hidden layers: the first with 10 neurons and the second with 5 neurons. If the input layer has 4 neurons, what is the total number of weight parameters excluding biases?\n",
    "- A) 50\n",
    "- B) 70\n",
    "- C) 65\n",
    "- D) 60\n",
    "\n",
    "**Answer:** B) 70\n",
    "\n",
    "## 7. If the input dimension is 5 and the output dimension is 3, how many weights are in the output layer?\n",
    "- A) 5\n",
    "- B) 8\n",
    "- C) 15\n",
    "- D) 3\n",
    "\n",
    "**Answer:** C) 15\n",
    "\n",
    "## 8. How many different weights are required in a neural network with 6 input neurons, 3 hidden neurons, and 2 output neurons?\n",
    "- A) 18\n",
    "- B) 24\n",
    "- C) 26\n",
    "- D) 20\n",
    "\n",
    "**Answer:** D) 20\n",
    "\n",
    "## 9. How many epochs are needed to complete 100 iterations with a batch size of 10 for a dataset with 1000 samples?\n",
    "- A) 1\n",
    "- B) 5\n",
    "- C) 10\n",
    "- D) 50\n",
    "\n",
    "**Answer:** C) 10\n",
    "\n",
    "## 10. In a neural network, if there are 128 hidden neurons in the first hidden layer, how many bias terms are associated with this layer?\n",
    "- A) 128\n",
    "- B) 64\n",
    "- C) 256\n",
    "- D) 1\n",
    "\n",
    "**Answer:** A) 128\n",
    "\n",
    "## 11. For a network with 3 layers and each layer having 5 neurons, how many total neurons are present in the network?\n",
    "- A) 5\n",
    "- B) 10\n",
    "- C) 15\n",
    "- D) 20\n",
    "\n",
    "**Answer:** C) 15\n",
    "\n",
    "## 12. A neural network uses the Sigmoid activation function for its output layer. If the input to this layer is 0.5, what is the output?\n",
    "- A) 0.73\n",
    "- B) 0.62\n",
    "- C) 0.47\n",
    "- D) 0.59\n",
    "\n",
    "**Answer:** B) 0.62\n",
    "\n",
    "## 13. If a neural network has 100,000 parameters and takes 1 second per epoch, how long will it take to train for 500 epochs?\n",
    "- A) 500 seconds\n",
    "- B) 50 minutes\n",
    "- C) 10 minutes\n",
    "- D) 1 hour\n",
    "\n",
    "**Answer:** A) 500 seconds\n",
    "\n",
    "## 14. If a neural network has an input layer of 4 neurons and a hidden layer of 6 neurons, what is the dimensionality of the weight matrix connecting these two layers?\n",
    "- A) (6, 4)\n",
    "- B) (4, 6)\n",
    "- C) (6, 5)\n",
    "- D) (5, 6)\n",
    "\n",
    "**Answer:** B) (4, 6)\n",
    "\n",
    "## 15. For a neural network with 2 hidden layers, each containing 16 neurons, and 1 output layer containing 1 neuron, how many total neurons are in the hidden layers?\n",
    "- A) 16\n",
    "- B) 32\n",
    "- C) 17\n",
    "- D) 33\n",
    "\n",
    "**Answer:** B) 32\n",
    "\n",
    "## 16. If a dataset has 5000 samples and the neural network is trained using 5 epochs, how many samples are processed in total?\n",
    "- A) 1000\n",
    "- B) 5000\n",
    "- C) 25000\n",
    "- D) 10000\n",
    "\n",
    "**Answer:** C) 25000\n",
    "\n",
    "## 17. How many epochs does it take for a neural network to have seen 10,000 samples with a batch size of 250?\n",
    "- A) 4\n",
    "- B) 20\n",
    "- C) 40\n",
    "- D) 50\n",
    "\n",
    "**Answer:** B) 40\n",
    "\n",
    "## 18. If the learning rate is set to 0.001, and the initial loss is 2.5, what will be the new loss after one iteration if the loss gradient is -0.4?\n",
    "- A) 2.1\n",
    "- B) 2.4\n",
    "- C) 2.6\n",
    "- D) 2.9\n",
    "\n",
    "**Answer:** B) 2.4\n",
    "\n",
    "## 19. How many iterations are required to complete 1 epoch for a dataset of 1200 samples and a batch size of 60?\n",
    "- A) 10\n",
    "- B) 20\n",
    "- C) 30\n",
    "- D) 60\n",
    "\n",
    "**Answer:** A) 20\n",
    "\n",
    "## 20. A neural network has 3 layers with the following number of neurons: 10, 20, 5. How many total weights are there between the layers (excluding biases)?\n",
    "- A) 50\n",
    "- B) 200\n",
    "- C) 250\n",
    "- D) 400\n",
    "\n",
    "**Answer:** B) 200\n",
    "\n",
    "## 21. For a neural network using the Tanh activation function, if the input is 1, what is the approximate output value?\n",
    "- A) 0.46\n",
    "- B) 0.76\n",
    "- C) 0.95\n",
    "- D) 0.99\n",
    "\n",
    "**Answer:** C) 0.95\n",
    "\n",
    "## 22. If you train a neural network with 500 epochs and reduce the learning rate by half every 100 epochs, how many times will the learning rate be reduced?\n",
    "- A) 2\n",
    "- B) 3\n",
    "- C) 4\n",
    "- D) 5\n",
    "\n",
    "**Answer:** D\n",
    "\n",
    "**Answer:** D) 5\n",
    "\n",
    "## 23. How many neurons are in the output layer of a neural network designed to classify digits from 0 to 9?\n",
    "- A) 1\n",
    "- B) 9\n",
    "- C) 10\n",
    "- D) 11\n",
    "\n",
    "**Answer:** C) 10\n",
    "\n",
    "## 24. A neural network has an input layer with 8 neurons, a hidden layer with 16 neurons, and an output layer with 4 neurons. How many total bias terms are in the network?\n",
    "- A) 4\n",
    "- B) 16\n",
    "- C) 20\n",
    "- D) 24\n",
    "\n",
    "**Answer:** C) 20\n",
    "\n",
    "## 25. If a neural network has 5000 weights and takes 2 milliseconds to update all weights per iteration, how much time does it take to complete 100 iterations?\n",
    "- A) 0.5 seconds\n",
    "- B) 1 second\n",
    "- C) 2 seconds\n",
    "- D) 10 seconds\n",
    "\n",
    "**Answer:** B) 1 second\n",
    "\n",
    "## 26. For a neural network trained with a batch size of 64, and there are 1280 samples, how many batches are processed per epoch?\n",
    "- A) 10\n",
    "- B) 15\n",
    "- C) 20\n",
    "- D) 25\n",
    "\n",
    "**Answer:** C) 20\n",
    "\n",
    "## 27. In a neural network, if the learning rate is 0.01 and the gradient of a weight is 0.5, what is the weight update value?\n",
    "- A) 0.005\n",
    "- B) 0.015\n",
    "- C) 0.05\n",
    "- D) 0.10\n",
    "\n",
    "**Answer:** A) 0.005\n",
    "\n",
    "## 28. How many hidden neurons are recommended in a hidden layer if the input layer has 20 neurons and the output layer has 10 neurons, using the \"rule of thumb\" method?\n",
    "- A) 5\n",
    "- B) 10\n",
    "- C) 15\n",
    "- D) 20\n",
    "\n",
    "**Answer:** C) 15\n",
    "\n",
    "## 29. If a neural network has 3 hidden layers with 50 neurons each and an output layer with 2 neurons, how many total neurons are there in the hidden and output layers?\n",
    "- A) 102\n",
    "- B) 150\n",
    "- C) 152\n",
    "- D) 200\n",
    "\n",
    "**Answer:** C) 152\n",
    "\n",
    "## 30. A neural network's activation function is ReLU. If the input is -3, what is the output?\n",
    "- A) -3\n",
    "- B) 0\n",
    "- C) 1\n",
    "- D) 3\n",
    "\n",
    "**Answer:** B) 0\n",
    "\n",
    "## 31. How many iterations are needed to complete 5 epochs for a dataset with 800 samples and a batch size of 40?\n",
    "- A) 10\n",
    "- B) 50\n",
    "- C) 100\n",
    "- D) 200\n",
    "\n",
    "**Answer:** B) 100\n",
    "\n",
    "## 32. If a neural network has 2 hidden layers, the first with 30 neurons and the second with 15 neurons, what is the total number of connections (weights) between these two layers?\n",
    "- A) 450\n",
    "- B) 600\n",
    "- C) 300\n",
    "- D) 45\n",
    "\n",
    "**Answer:** A) 450\n",
    "\n",
    "## 33. A neural network uses a learning rate of 0.1. If the initial loss is 0.9 and the gradient is 0.5, what is the updated loss after one iteration?\n",
    "- A) 0.85\n",
    "- B) 0.80\n",
    "- C) 0.75\n",
    "- D) 0.45\n",
    "\n",
    "**Answer:** B) 0.80\n",
    "\n",
    "## 34. For a neural network with 4 hidden layers, each containing 25 neurons, how many neurons are there in total in the hidden layers?\n",
    "- A) 25\n",
    "- B) 50\n",
    "- C) 100\n",
    "- D) 125\n",
    "\n",
    "**Answer:** C) 100\n",
    "\n",
    "## 35. How many total weights are needed between an input layer with 6 neurons and a hidden layer with 8 neurons?\n",
    "- A) 14\n",
    "- B) 48\n",
    "- C) 64\n",
    "- D) 72\n",
    "\n",
    "**Answer:** B) 48\n",
    "\n",
    "## 36. A neural network has an input layer of 10 neurons and an output layer of 5 neurons. How many weights are in the output layer?\n",
    "- A) 5\n",
    "- B) 15\n",
    "- C) 50\n",
    "- D) 55\n",
    "\n",
    "**Answer:** C) 50\n",
    "\n",
    "## 37. How many bias terms are needed for a neural network with 3 hidden layers, each with 10 neurons?\n",
    "- A) 10\n",
    "- B) 20\n",
    "- C) 30\n",
    "- D) 40\n",
    "\n",
    "**Answer:** C) 30\n",
    "\n",
    "## 38. If a dataset has 10,000 samples and is trained over 100 epochs with a batch size of 100, how many total samples are processed during training?\n",
    "- A) 10,000\n",
    "- B) 100,000\n",
    "- C) 1,000,000\n",
    "- D) 10,000,000\n",
    "\n",
    "**Answer:** B) 1,000,000\n",
    "\n",
    "## 39. If the ReLU activation function is used in a hidden layer with 32 neurons, how many activation outputs are computed per forward pass?\n",
    "- A) 16\n",
    "- B) 32\n",
    "- C) 64\n",
    "- D) 128\n",
    "\n",
    "**Answer:** B) 32\n",
    "\n",
    "## 40. How many weight parameters exist in a neural network layer that has 15 input neurons and 10 output neurons?\n",
    "- A) 15\n",
    "- B) 25\n",
    "- C) 150\n",
    "- D) 160\n",
    "\n",
    "**Answer:** C) 150\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cEG1dhnuNFRk"
   },
   "source": [
    "# Steps to build a Neural Network in NumPy\n",
    "\n",
    "<ol>1. Loading the dataset (Input and Output)</ol>\n",
    "<ol>2. Architecture of the model (# input, hidden and output neurons)</ol>\n",
    "<ol>3. Initializing the weights for all the layers</ol>\n",
    "<ol>4. Implementing forward propagation</ol>\n",
    "<ol>5. Implementing backward propagation</ol>\n",
    "<ol>6. Train the model for n epochs </ol>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install --upgrade numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip uninstall numpy\n",
    "#!pip install numpy==1.23.5\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "sbgj7HfHNFRr"
   },
   "source": [
    "## 1. Loading the dataset (Input and Output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "E5S9HgBzNFRw"
   },
   "outputs": [],
   "source": [
    "# importing required libraries\n",
    "import numpy as np\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1657,
     "status": "ok",
     "timestamp": 1585485745721,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "ogs6CaXu2zeZ",
    "outputId": "f5055dbe-331b-461c-a1a1-13d28149d528"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Version of numpy: 1.23.5\n"
     ]
    }
   ],
   "source": [
    "# version of numpy library\n",
    "print('Version of numpy:',np.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1788,
     "status": "ok",
     "timestamp": 1585485751574,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "vNmvxGv723N6",
    "outputId": "340278d2-64d4-43b2-fd9e-02585dc66d21"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Version of matplotlib: 3.7.2\n"
     ]
    }
   ],
   "source": [
    "# version of matplotlib library\n",
    "print('Version of matplotlib:',matplotlib.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 139
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1409,
     "status": "ok",
     "timestamp": 1585485752142,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "H_h7HoPONFR_",
    "outputId": "552026b7-f129-41e9-c8f4-22a4c9d80e6e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Input:\n",
      "[[1 0 1 0]\n",
      " [1 0 1 1]\n",
      " [0 1 0 1]]\n",
      "\n",
      " Shape of Input: (3, 4)\n"
     ]
    }
   ],
   "source": [
    "# creating the input array\n",
    "X=np.array([[1,0,1,0], [1,0,1,1], [0,1,0,1]])\n",
    "print ('\\n Input:')\n",
    "print(X)\n",
    "# shape of input array\n",
    "print('\\n Shape of Input:', X.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 156
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 3240,
     "status": "ok",
     "timestamp": 1585485756254,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "LVvQz5g39wo3",
    "outputId": "e0f22a43-1caa-4ca0-bfdd-0168c8853d6b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Input in matrix form:\n",
      "[[1 1 0]\n",
      " [0 0 1]\n",
      " [1 1 0]\n",
      " [0 1 1]]\n",
      "\n",
      " Shape of Input Matrix: (4, 3)\n"
     ]
    }
   ],
   "source": [
    "# converting the input in matrix form\n",
    "X = X.T\n",
    "print('\\n Input in matrix form:')\n",
    "print(X)\n",
    "# shape of input matrix\n",
    "print('\\n Shape of Input Matrix:', X.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 191
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 3079,
     "status": "ok",
     "timestamp": 1585485756256,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "IRe8JE0xNFSL",
    "outputId": "bd0f8cc4-140c-4788-fe12-0967af4b0f0f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Actual Output:\n",
      "[[1]\n",
      " [1]\n",
      " [0]]\n",
      "\n",
      " Output in matrix form:\n",
      "[[1 1 0]]\n",
      "\n",
      " Shape of Output: (1, 3)\n"
     ]
    }
   ],
   "source": [
    "# creating the output array\n",
    "y=np.array([[1],[1],[0]])\n",
    "print ('\\n Actual Output:')\n",
    "print(y)\n",
    "\n",
    "# output in matrix form\n",
    "y = y.T\n",
    "\n",
    "print ('\\n Output in matrix form:')\n",
    "print(y)\n",
    "\n",
    "# shape of input array\n",
    "print('\\n Shape of Output:', y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tKf4Ji1-NFSV"
   },
   "source": [
    "## 2. Architecture of the model (# input, hidden and output neurons)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vlhBW0NNNFSg"
   },
   "outputs": [],
   "source": [
    "inputlayer_neurons = X.shape[0] # number of features in data set\n",
    "hiddenlayer_neurons = 3 # number of hidden layers neurons\n",
    "output_neurons = 1 # number of neurons at output layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OoOsLucmNFSo"
   },
   "source": [
    "![alt text](https://drive.google.com/uc?id=1zrEFVsc6bMQZ7fRxbK4DRceaG78k26Pc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_Nmwj8RfNFSr"
   },
   "source": [
    "## 3. Initializing the weights for all the layers\n",
    "\n",
    "NOTE: For simplicity, we are assuming that the bias for all the layers is 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "1T1IG-W8NFSu"
   },
   "outputs": [],
   "source": [
    "# initializing weight\n",
    "# Shape of w_ih should number of neurons at input layer X number of neurons at hidden layer\n",
    "w_ih=np.random.uniform(size=(inputlayer_neurons,hiddenlayer_neurons))\n",
    "\n",
    "\n",
    "# Shape of w_ho should number of neurons at hidden layer X number of neurons at output layer\n",
    "w_ho=np.random.uniform(size=(hiddenlayer_neurons,output_neurons))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1009,
     "status": "ok",
     "timestamp": 1585485756260,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "Fpa1--9KNFS1",
    "outputId": "407ba6ee-ddc9-4830-d771-15704f6e39e6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((4, 3), (3, 1))"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# shape of weight matrix\n",
    "w_ih.shape, w_ho.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "srrDW1MNNFS-"
   },
   "source": [
    "## 4. Implementing forward propagation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-g-SocwQNFTC"
   },
   "source": [
    "![alt text](https://drive.google.com/uc?id=1YwD7vY9k84vZmjmE5CXgQ69fYyadPsox)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sOcBji4iNFTE"
   },
   "outputs": [],
   "source": [
    "# We are using sigmoid as an activation function so defining the sigmoid function here\n",
    "\n",
    "# defining the Sigmoid Function\n",
    "def sigmoid (x):\n",
    "    return 1/(1 + np.exp(-x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DO6AYHtGNFTM"
   },
   "outputs": [],
   "source": [
    "# hidden layer activations\n",
    "\n",
    "hidden_layer_input=np.dot(w_ih.T,X)\n",
    "hiddenlayer_activations = sigmoid(hidden_layer_input)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "o8zzYX6pNFTT"
   },
   "source": [
    "![alt text](https://drive.google.com/uc?id=1ETMoLD1fwi5u1HHLqtAdVUs-P8HNOU_p)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "CuqKwiToNFTW"
   },
   "outputs": [],
   "source": [
    "# calculating the output\n",
    "output_layer_input=np.dot(w_ho.T,hiddenlayer_activations)\n",
    "output = sigmoid(output_layer_input)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 2132,
     "status": "ok",
     "timestamp": 1585485759226,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "BjPlMkVMNFTd",
    "outputId": "98153d29-d232-4e9f-da3f-f834a5ad8ad9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.71977029, 0.73322579, 0.75059794]])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# output\n",
    "output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mdFKMYyzNFTm"
   },
   "source": [
    "## 5. Implementing backward propagation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "c2m3XBgZNFTn"
   },
   "source": [
    "![alt text](https://drive.google.com/uc?id=1uYdg4mQL-B9o7BTOLnfoYUhh_LxTnpcW)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "IvUAAhlcNFTp"
   },
   "outputs": [],
   "source": [
    "# calculating error\n",
    "error = np.square(y-output)/2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3H0vjBdNNFTw"
   },
   "source": [
    "### Rate of change of error w.r.t weight between hidden and output layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4cncCd1WNFTz"
   },
   "source": [
    "![alt text](https://drive.google.com/uc?id=1_KexjgVJGRptZ6t1eobTter3mfIGo9rs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DqrhlDeDNFT1"
   },
   "source": [
    "**a. Rate of change of error w.r.t output**\n",
    "\n",
    "**b. Rate of change of output w.r.t Z2**\n",
    "\n",
    "**c. Rate of change of Z2 w.r.t weights between hidden and output layer**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bKdk5m4FNFT3"
   },
   "outputs": [],
   "source": [
    "# rate of change of error w.r.t. output\n",
    "error_wrt_output = -(y-output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Bl1PDwrBNFT9"
   },
   "outputs": [],
   "source": [
    "# rate of change of output w.r.t. Z2\n",
    "output_wrt_Z2 = np.multiply(output,(1-output))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "3vLk1nxLNFUD"
   },
   "outputs": [],
   "source": [
    "# rate of change of Z2 w.r.t. weights between hidden and output layer\n",
    "Z2_wrt_who = hiddenlayer_activations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1953,
     "status": "ok",
     "timestamp": 1585485762993,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "UXXifY9QNFUI",
    "outputId": "869382cd-4440-47f1-c249-db25f9073338"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1, 3), (1, 3), (3, 3))"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# checking the shapes of partial derivatives\n",
    "error_wrt_output.shape, output_wrt_Z2.shape, Z2_wrt_who.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1725,
     "status": "ok",
     "timestamp": 1585485762995,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "ZvtS7wCRNFUN",
    "outputId": "377481a7-6b8f-4d0c-abe2-82060ce48ab0"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 1)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# shape of weights of output layer\n",
    "w_ho.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gC9zQEH6HON5"
   },
   "source": [
    "![alt text](https://drive.google.com/uc?id=1VesmZOVpfgLFESvOFd7dE-YHNtSMMkvM)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "l3HNVYGONFUr"
   },
   "outputs": [],
   "source": [
    "# rate of change of error w.r.t weight between hidden and output layer\n",
    "error_wrt_who = np.dot(Z2_wrt_who,(error_wrt_output*output_wrt_Z2).T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1805,
     "status": "ok",
     "timestamp": 1585485763842,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "cwyI1EGZNFUw",
    "outputId": "93df5cbf-3413-4acb-cdaa-394a9ce50a37"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 1)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "error_wrt_who.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "sDFPg2SHNFU2"
   },
   "source": [
    "### Rate of change of error w.r.t weight between input and hidden layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_757MrjBNFU2"
   },
   "source": [
    "![alt text](https://drive.google.com/uc?id=1X4-iInwlv7ber3fwgtqHuHTuFRci-tMV)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_nPYGXkeNFU4"
   },
   "source": [
    "**a. Rate of change of error w.r.t output**\n",
    "\n",
    "**b. Rate of change of output w.r.t Z2**\n",
    "\n",
    "**c. Rate of change of Z2 w.r.t hidden layer activations**\n",
    "\n",
    "**d. Rate of change of hidden layer activations w.r.t Z1**\n",
    "\n",
    "**e. Rate of change of Z1 w.r.t weights between input and hidden layer**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Sb7Ezxw9NFU6"
   },
   "outputs": [],
   "source": [
    "# rate of change of error w.r.t. output\n",
    "error_wrt_output = -(y-output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "3-SGbNaoNFVA"
   },
   "outputs": [],
   "source": [
    "# rate of change of output w.r.t. Z2\n",
    "output_wrt_Z2 = np.multiply(output,(1-output))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "amuoR7h6NFVF"
   },
   "outputs": [],
   "source": [
    "# rate of change of Z2 w.r.t. hidden layer activations\n",
    "Z2_wrt_h1 = w_ho"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "YDUZEdWKNFVJ"
   },
   "outputs": [],
   "source": [
    "# rate of change of hidden layer activations w.r.t. Z1\n",
    "h1_wrt_Z1 = np.multiply(hiddenlayer_activations,(1-hiddenlayer_activations))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Ft4U6Td6NFVO"
   },
   "outputs": [],
   "source": [
    "# rate of change of Z1 w.r.t. weights between input and hidden layer\n",
    "Z1_wrt_wih = X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1069,
     "status": "ok",
     "timestamp": 1585485765410,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "A-hsfsi4NFVR",
    "outputId": "2d5a0542-4563-4989-90be-382eba30a03a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1, 3), (1, 3), (3, 1), (3, 3), (4, 3))"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# checking the shapes of partial derivatives\n",
    "error_wrt_output.shape, output_wrt_Z2.shape, Z2_wrt_h1.shape, h1_wrt_Z1.shape, Z1_wrt_wih.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1503,
     "status": "ok",
     "timestamp": 1585485766077,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "1uka_yPrNFVV",
    "outputId": "238e6afd-f960-4eb3-c02c-b95a31932b5a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4, 3)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# shape of weights of hidden layer\n",
    "w_ih.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gCeSm7vrHbHj"
   },
   "source": [
    "![alt text](https://drive.google.com/uc?id=1RkG5x1NEFWlF3tj0OlswOWvBcV5XNV1C)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "XTPNf3E5NFVs"
   },
   "outputs": [],
   "source": [
    "# rate of change of error w.r.t weights between input and hidden layer\n",
    "error_wrt_wih = np.dot(Z1_wrt_wih,(h1_wrt_Z1*np.dot(Z2_wrt_h1,(output_wrt_Z2*error_wrt_output))).T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 2480,
     "status": "ok",
     "timestamp": 1585485768146,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "_WN0I-mpNFVw",
    "outputId": "a7ab9b3d-3a2a-4480-f15e-11d1f5ae8e13"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4, 3)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "error_wrt_wih.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "W2bu4H5-NFVz"
   },
   "source": [
    "### Update the parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-nmJnY_PNFV1"
   },
   "source": [
    "![alt text](https://drive.google.com/uc?id=1A5jaB3WjZx9yrJkk9imVEvP3PZodjapE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_r59xEpINFV2"
   },
   "outputs": [],
   "source": [
    "# defining the learning rate\n",
    "lr = 0.01"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 69
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 2341,
     "status": "ok",
     "timestamp": 1585485769472,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "aiBFNXd3NFV7",
    "outputId": "a7362697-b6e0-41c1-8a5a-bb525ed22c3e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.80942365],\n",
       "       [0.53803984],\n",
       "       [0.18169794]])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# initial w_ho and w_ih\n",
    "w_ho"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 86
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1928,
     "status": "ok",
     "timestamp": 1585485769474,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "CuosFKUENFWB",
    "outputId": "fa8986c3-9960-4985-b6ec-e7d9a51f16e1",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.21961466, 0.2381788 , 0.78111823],\n",
       "       [0.87692472, 0.65980643, 0.56509714],\n",
       "       [0.02992463, 0.36017459, 0.46746584],\n",
       "       [0.247335  , 0.13266896, 0.11249356]])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w_ih"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "D_Va2xywNFWF"
   },
   "outputs": [],
   "source": [
    "# updating the weights of output layer\n",
    "w_ho = w_ho - lr * error_wrt_who"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ruFlc96BNFWL"
   },
   "outputs": [],
   "source": [
    "# updating the weights of hidden layer\n",
    "w_ih = w_ih - lr * error_wrt_wih"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 69
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1799,
     "status": "ok",
     "timestamp": 1585485770584,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "NTf4nS1xNFWP",
    "outputId": "feb1e8da-cdff-4374-cf88-a66e1449ea05"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.80900522],\n",
       "       [0.53778958],\n",
       "       [0.18162061]])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# updated w_ho and w_ih\n",
    "w_ho"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 86
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 2866,
     "status": "ok",
     "timestamp": 1585485772036,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "7VYNPPNlNFWU",
    "outputId": "ae17551f-c966-4124-9ad1-95f3a9e59fad"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.21982661, 0.23831   , 0.78115142],\n",
       "       [0.87671421, 0.65964425, 0.56504011],\n",
       "       [0.03013658, 0.36030579, 0.46749903],\n",
       "       [0.24722383, 0.13256837, 0.11245193]])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w_ih"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "SxLy6DZlNFWY"
   },
   "source": [
    "## 6. Training the model for n epochs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8HKS9vIyNFWZ"
   },
   "outputs": [],
   "source": [
    "# defining the model architecture\n",
    "inputlayer_neurons = X.shape[0] # number of features in data set\n",
    "hiddenlayer_neurons = 3 # number of hidden layers neurons\n",
    "output_neurons = 1 # number of neurons at output layer\n",
    "\n",
    "# initializing weight\n",
    "w_ih=np.random.uniform(size=(inputlayer_neurons,hiddenlayer_neurons))\n",
    "w_ho=np.random.uniform(size=(hiddenlayer_neurons,output_neurons))\n",
    "\n",
    "# defining the parameters\n",
    "lr = 0.01\n",
    "epochs = 10000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 86
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 2151,
     "status": "ok",
     "timestamp": 1585485772038,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "ojybM51LNFWc",
    "outputId": "b368f6a2-87e3-4777-fea0-1efc795e8b83"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.21259566, 0.35643491, 0.66756214],\n",
       "       [0.85431284, 0.73519421, 0.57051133],\n",
       "       [0.81998676, 0.51091734, 0.63665445],\n",
       "       [0.80413488, 0.93411459, 0.89563323]])"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# initial w_ih and w_ho\n",
    "w_ih"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 69
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1811,
     "status": "ok",
     "timestamp": 1585485772039,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "1RS_d3kdNFWg",
    "outputId": "00d4aa30-5bb8-4b2f-8c97-21e2da705d10"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.35471643],\n",
       "       [0.87338054],\n",
       "       [0.49066467]])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w_ho"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_yVAcyW_NFWk"
   },
   "outputs": [],
   "source": [
    "error_epoch = []\n",
    "for i in range(epochs):\n",
    "    # Forward Propogation\n",
    "    \n",
    "    # hidden layer activations\n",
    "    hidden_layer_input=np.dot(w_ih.T,X)\n",
    "    hiddenlayer_activations = sigmoid(hidden_layer_input)    \n",
    "    # calculating the output\n",
    "    output_layer_input=np.dot(w_ho.T,hiddenlayer_activations)\n",
    "    output = sigmoid(output_layer_input)\n",
    "    \n",
    "    \n",
    "    # Backward Propagation\n",
    "    \n",
    "    # calculating error\n",
    "    error = np.square(y-output)/2\n",
    "    error_wrt_output = -(y-output)\n",
    "    output_wrt_Z2 = np.multiply(output,(1-output))\n",
    "    Z2_wrt_who = hiddenlayer_activations\n",
    "    # rate of change of error w.r.t weight between hidden and output layer\n",
    "    error_wrt_who = np.dot(Z2_wrt_who,(error_wrt_output*output_wrt_Z2).T)\n",
    "    Z2_wrt_h1 = w_ho\n",
    "    h1_wrt_Z1 = np.multiply(hiddenlayer_activations,(1-hiddenlayer_activations))\n",
    "    Z1_wrt_wih = X\n",
    "    # rate of change of error w.r.t weights between input and hidden layer\n",
    "    error_wrt_wih = np.dot(Z1_wrt_wih,(h1_wrt_Z1*np.dot(Z2_wrt_h1,(error_wrt_output*output_wrt_Z2))).T)\n",
    "\n",
    "    # updating the weights between hidden and output layer\n",
    "    w_ho = w_ho - lr * error_wrt_who\n",
    "    # updating the weights between input and hidden layer\n",
    "    w_ih = w_ih - lr * error_wrt_wih\n",
    "    \n",
    "    # appending the error of each epoch\n",
    "    error_epoch.append(np.average(error))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 86
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 2409,
     "status": "ok",
     "timestamp": 1585485773423,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "Ra5mTgwUNFWo",
    "outputId": "20de9dc0-62b7-4f07-fd3d-cf290542ac8f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.68619684,  1.29524789,  1.18166918],\n",
       "       [ 1.46376597, -1.32255698, -0.82707917],\n",
       "       [-0.07880574,  1.44973033,  1.1507615 ],\n",
       "       [ 0.90818591, -0.652678  , -0.26035733]])"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# updated w_ih and w_ho\n",
    "w_ih"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 283
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 2186,
     "status": "ok",
     "timestamp": 1585485784562,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "WeN2dcc0NFW8",
    "outputId": "59ab0369-88ea-4f52-87b4-99da23925bb9",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2083832bd90>]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGdCAYAAAAxCSikAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABEeElEQVR4nO3deViU5cIG8HsWZgYRhn1HBFcUcRlKUVErw9Q0y8qstPUUnUyFY0cNy7KjlHayz6/Uk2l9nUo9R62sqMByS1ASwVBxZVUZ2WQGBFlm3u8PamoClUHgHWbu33XN1eU7z8zc86hx+y7PKxEEQQARERGRFZOKHYCIiIjoRlhYiIiIyOqxsBAREZHVY2EhIiIiq8fCQkRERFaPhYWIiIisHgsLERERWT0WFiIiIrJ6crEDtBej0YiLFy/C2dkZEolE7DhERETUCoIgoKqqCv7+/pBKr70fxWYKy8WLFxEUFCR2DCIiImqDoqIiBAYGXvN5mykszs7OAJq+sIuLi8hpiIiIqDX0ej2CgoJMP8evxWYKy2+HgVxcXFhYiIiIupgbnc7Bk26JiIjI6rGwEBERkdVjYSEiIiKrx8JCREREVo+FhYiIiKweCwsRERFZPRYWIiIisnosLERERGT1WFiIiIjI6rGwEBERkdVjYSEiIiKrx8JCREREVo+F5ToEQcCuE5cwe1M6ausNYschIiKyWyws19FgELB053HsO12Kj9PyxY5DRERkt1hYrkMhl2L++D4AgHV7z0F/tUHkRERERPaJheUG7h0agF5eTqisacAH+/PEjkNERGSX2lRY1q5di5CQEKhUKmg0Guzfv/+aY4uLi/Hwww+jX79+kEqlmD9/frMxGzZsQHR0NNzc3ODm5obx48cjPT29LdHanVwmxd9i+gEANu7PRXl1nciJiIiI7I/FhWXr1q2YP38+EhISkJmZiejoaEycOBGFhYUtjq+rq4OXlxcSEhIwePDgFsfs2bMHM2fOxO7du5GWloYePXogJiYGFy5csDReh5gY7otBAWpcqTdg7Z5zYschIiKyOxJBEARLXjB8+HAMGzYM69atM20LCwvDtGnTkJiYeN3Xjhs3DkOGDME777xz3XEGgwFubm549913MXv27Fbl0uv1UKvV0Ol0cHFxadVrLLH3dCke25QOhVyK3QvGIcDVsd0/g4iIyN609ue3RXtY6uvrkZGRgZiYGLPtMTExSE1NbVvSFtTU1KChoQHu7u7XHFNXVwe9Xm/26Ehj+nhieIg76huNWPndyQ79LCIiIjJnUWEpKyuDwWCAj4+P2XYfHx9otdp2C7Vo0SIEBARg/Pjx1xyTmJgItVptegQFBbXb57dEIpFgyeQBkEiAL7Mu4nB+RYd+HhEREf2uTSfdSiQSs18LgtBsW1utXLkSmzdvxo4dO6BSqa45bvHixdDpdKZHUVFRu3z+9QwKVONBTVMxeu2rEzAaLTqaRkRERG1kUWHx9PSETCZrtjelpKSk2V6XtnjrrbewYsUKJCcnIyIi4rpjlUolXFxczB6dYcGEfnBWypF9QYdtR853ymcSERHZO4sKi0KhgEajQUpKitn2lJQUjBw58qaCrFq1Cq+//jq+++47REZG3tR7dSQvZyXm3tG0mNzK706isqZe5ERERES2z+JDQvHx8fjggw+wadMm5OTkIC4uDoWFhYiNjQXQdKjmz1f2ZGVlISsrC9XV1SgtLUVWVhZOnDhhen7lypVYsmQJNm3ahJ49e0Kr1UKr1aK6uvomv17HeGxkT/TyckJZdT2Wf5MjdhwiIiKbZ/FlzUDTwnErV65EcXExwsPDsXr1aowZMwYA8PjjjyM/Px979uz5/UNaOL8lODgY+fn5AICePXuioKCg2ZilS5fi1VdfbVWmjr6s+c8O51fggX+lQRCAT54ajtF9PDv8M4mIiGxNa39+t6mwWKPOLiwA8MqXx/BxWgEC3RyRHDcG3RTyTvlcIiIiW9Eh67CQub/f1R/+ahXOX67FP3hoiIiIqMOwsNyE7ko53ry/6Wqmzw4V4rtjxSInIiIisk0sLDcpuo8Xnh0bCgBYuD0bFytrRU5ERERke1hY2sHf7uyHwYFq6Gob8PxnR1DXaBA7EhERkU1hYWkHCrkUa2YOhdrRAZmFlXhpxzHYyLnMREREVoGFpZ0EezjhvYeHQSaVYPuR89iwP1fsSERERDaDhaUdje7jiVfuHgAAWJF0EtszuHQ/ERFRe2BhaWezo4Lx5KgQAMCL247i22xeOURERHSzWFjamUQiwZLJYXgwMhBGAZi7JRNf/3JR7FhERERdGgtLB5BKJUi8LwJTB/ujwSDghc2Z+HdavtixiIiIuiwWlg4ik0qwesYQPDqiBwQBePnL43jtq+NoMBjFjkZERNTlsLB0IJlUgtfvCUfc+L4AgA8P5OPhDQdxSX9V5GRERERdCwtLB5NIJJg3vg/en6WBs1KOn/MvI2b1PnyeeZ5rtRAREbUSC0sniRnoi50vjEZ4gAt0tQ2I23oUT370M86VVosdjYiIyOqxsHSiEE8nfP7XUXhxQj8oZFLsPlWKCav3YemXx1BSxcNERERE1yIRbOS4hF6vh1qthk6ng4uLi9hxbuhsSTXe+DYHu3JKADQt7z99WCCeGROKEE8nkdMRERF1jtb+/GZhEVnq2TKsSj6FzMJKAIBEAozq5YkHIgMxYaAvVA4ycQMSERF1IBaWLkQQBPycfxnr957DjydLTNudVXKMD/NBzAAfjOnrBSelXMSURERE7Y+FpYsqqqjBtozz2JZxHhcqa03bFXIpokI9MKq3B0b28kSYnwtkUomISYmIiG4eC0sXZzQKyCi8jOTjWnx//BIKK2rMnlc7OmB4iDuG9nDDkCBXRASquQeGiIi6HBYWGyIIAk5fqsb+M6VIO1eOQ3kVqK5rNBsjlQB9fZx/LS+uGODvgn4+znBU8BwYIiKyXiwsNqzRYET2BR1+zq9AVlElsgorcVHX/LJoqaTpUuoB/moM8HPBAH8XhPk5w9tZJUJqIiKi5lhY7Mwl/VVkFlYiq6gSxy/qcOKiHuVX6lsc69ldiTA/Zwzwc0F/P2eE+bmgl1d3OMi4LA8REXUuFhY7JwgCSqvqcKJY3/S42PTfvLIraOl33EEmQW9vZ1ORCfNzQX9fZ3h0V3Z+eCIishssLNSimvpGnNRW4WRxFXKK9cgp1uOktqrZOTG/8XZWNpWXPxSZEE8n7o0hIqJ2wcJCrSYIAs5frsWJYv3vRUarR0F5TYvjFTIp+vh0N+2F+a3IuDkpOjk5ERF1dSwsdNOq6xpxSmu+J+ZksR5X6g0tjvd1UZnOiQnzc0GYrzNCPJ0g594YIiK6BhYW6hBGo4CiyzXI+cMhpRytHkUVtS2OV8ql6OvjjPAAF0QENq0X09fHmYeUiIgIAAuL2HHsTtXVht/3xvz631PaKtS0sDdGKZdioH9TgRkcpMbgQFf09HCClCv3EhHZHRYWEp3RKKCwogYnivX45bwOv5yvRPZ5HapaOMHXWSVHRKAaEYGuGBLkCk2wGzx5hRIRkc1jYSGrZDQKyCu/gl/OV+JoUVOJOX5Rj7pGY7OxPT26QRPsjsiebogMdkMvr+7cC0NEZGNYWKjLaDAYcfpSlWkvzJGCSpwuqWq2Xoza0QHDergisqc7NMFN91BSOfDWA0REXRkLC3VputoGHCm8jIz8yzhc0HQLgqsN5nthFDIphvZwRVQvD0SFemBID1co5SwwRERdCQsL2ZQGgxE5xXoczr+MjILL+Dm/AiVVdWZjVA5SRAa7NxWYXh6ICFDzkmoiIivHwkI2TRAE5JfXIPVcGdLOleNgbjnKqs3vneSkkCGqlwfG9vPGuL5eCHLvJlJaIiK6FhYWsiuCIOBMSTXSzpUj9VwZDuZWQFfbYDYm1MsJY/t6YVw/bwwPcef5L0REVoCFheya0SjgRLEee0+XYu+pUmQUXobB+PsfdaVcihGhHrgjzBsxA3zhq1aJmJaIyH6xsBD9ga62Aalny7DnVCn2ni6FVn/V7PnBgWrEDPTFhIE+6O3tLFJKIiL7w8JCdA2CIODUpSrsPlmKlBNaZBZVml1CHerlhJgBvrgr3BeDA9WQSLj2CxFRR2FhIWqlEv1V7MopwffHtUg9V4YGw+9/JYLcHTElwh9Th/ijvy//XBERtTcWFqI2qLragD2nSvH9cS1+PFlidi+kvj7dMXWwP6YM9kewh5OIKYmIbAcLC9FNqq034IeTl7Az6yL2nCpFveH3heuGBLnigchATBnsDxeVg4gpiYi6NhYWonakq23A98e1+OroRRw4W4bfLjhSyqWYGO6LByODMCLUg/c6IiKyEAsLUQcprarDl1kX8J/DRTh9qdq0PdDNEQ9ogvDgLYHwUzuKmJCIqOtgYSHqYIIg4JfzOvzncBF2Zl1EVV0jAEAmlSBmgA9mR/XEiFB3XmVERHQdLCxEnai23oDvj2uxOb0Qh/IqTNv7+nTH7KieuHdoAJyUchETEhFZJxYWIpGc0lbh47R87DhyAbUNTVcZOSvlmHFLEJ4cHQJ/Vx4uIiL6DQsLkch0tQ3YnnEe/z5YgLyyKwAAuVSCqUP88eyYXujnyxV1iYha+/Nb2pY3X7t2LUJCQqBSqaDRaLB///5rji0uLsbDDz+Mfv36QSqVYv78+S2O2759OwYMGAClUokBAwbg888/b0s0IquhdnTAk6ND8EP8WHz4xC0YEeqORqOAHUcuYMI7+/DkRz/jUG45bOTfDEREHcriwrJ161bMnz8fCQkJyMzMRHR0NCZOnIjCwsIWx9fV1cHLywsJCQkYPHhwi2PS0tIwY8YMzJo1C0ePHsWsWbPw4IMP4tChQ5bGI7I6UqkEt/XzxpZnovDF86MwMdwXEgnw48kSzHj/IB78VxpSz5WJHZOIyKpZfEho+PDhGDZsGNatW2faFhYWhmnTpiExMfG6rx03bhyGDBmCd955x2z7jBkzoNfr8e2335q23XXXXXBzc8PmzZtblYuHhKgrySu7gg37c7Ht8HnTgnQjQt0RN74vhod6iJyOiKjzdMghofr6emRkZCAmJsZse0xMDFJTU9uWFE17WP78nhMmTLjue9bV1UGv15s9iLqKEE8nrLh3EPb9/TbMjgqGQibFwdwKzHj/IB754CAyCipu/CZERHbEosJSVlYGg8EAHx8fs+0+Pj7QarVtDqHVai1+z8TERKjVatMjKCiozZ9PJBZftQrL7gnH7hfH4eHhPeAgk+DA2XJMX5eGZz4+jHOl1Td+EyIiO9Cmk27/vBCWIAg3vTiWpe+5ePFi6HQ606OoqOimPp9ITAGujlhx7yDsXjAOD90SBKkESD5xCTGr9+GVL4+hvLpO7IhERKKyqLB4enpCJpM12/NRUlLSbA+JJXx9fS1+T6VSCRcXF7MHUVcX6NYNb0yPwPfzx+CO/t4wGAV8nFaAsav24L3dZ3G1wXDjNyEiskEWFRaFQgGNRoOUlBSz7SkpKRg5cmSbQ0RFRTV7z+Tk5Jt6T6KurI+PMzY+fgs2/2UEBgWoUV3XiFXfn8Kdq/fih5xLYscjIup0Fq8VHh8fj1mzZiEyMhJRUVF4//33UVhYiNjYWABNh2ouXLiAjz/+2PSarKwsAEB1dTVKS0uRlZUFhUKBAQMGAADmzZuHMWPG4M0338Q999yDL7/8Ert27cJPP/3UDl+RqOuK6uWBL58fhZ1HL+KNb0+iqKIWT/3fYYwP88bSKQMR5N5N7IhERJ2iTSvdrl27FitXrkRxcTHCw8OxevVqjBkzBgDw+OOPIz8/H3v27Pn9Q1o4FyU4OBj5+fmmX2/btg1LlixBbm4uevXqheXLl+O+++5rdSZe1ky27kpdI9b8eAYb9+eh0ShAKZfir+N649mxoVA5yMSOR0TUJlyan8hGnS2pwitfHkfquXIAQKiXE1ZOj0BkT3eRkxERWa5Dl+YnIvH09nbGp08Px7sPD4WXsxK5pVfwwL/S8OrO47hS1yh2PCKiDsHCQtQFSSQS3B3hj11xY/GAJhCCAHyUmo8J7+zDT2e4zD8R2R4WFqIuTN3NAaseGIyPn7wVAa6OOH+5Fo9uPISEz7NRW89LoInIdrCwENmAMX298H3cGMyOCgYAfHqoEJP/dz+OXdCJnIyIqH2wsBDZiO5KOZbdE45PnhoOH5emc1vuXXsA6/acg8FoE+fWE5EdY2EhsjGj+3jiu3ljcNdAXzQYBLz53Uk8vOEgtLqrYkcjImozFhYiG+TmpMC6R4dh5fQIdFPIcCivApPX7OcJuUTUZbGwENkoiUSCB28JwjdzozHAzwXlV+oxa9MhrPnhDIw8REREXQwLC5GNC/F0wo6/jsRDtwRBEIC3U07jiY9+xuUr9WJHIyJqNRYWIjugcpDhjekRWHV/BJRyKfaeLsXd//sTryIioi6DhYXIjjwQGYQvnh+FEE8nXKisxQPr0/BtdrHYsYiIboiFhcjOhPm54IvnR2FMXy/UNhjw3KdHsOaHM7CR24oRkY1iYSGyQ2pHB2x6LBJPjgoB0HRey5zNmVwdl4isFgsLkZ2Sy6R4ZcoAvDl9EBxkEnzzSzEeej8NZdV1YkcjImqGhYXIzs24pQc+fXoE3J0UOHpeh+nrUpFfdkXsWEREZlhYiAi3hrhjW2wUgtwdUVBeg+nrUpFVVCl2LCIiExYWIgIAhHp1x47nRiE8oGmRuZnvH8SPJy+JHYuICAALCxH9gZezElueiTJdQfSXjzOw48h5sWMREbGwEJG57ko5Nj4WifuGBcBgFPC3/x7FZ4cKxY5FRHaOhYWImnGQSfHW/YPxWFQwBAF46fNsfLA/V+xYRGTHWFiIqEVSqQSvTh2IZ8eGAgD+8U0O3v3xjMipiMhesbAQ0TVJJBIsuqs/4u/sCwB4K/k0Vn1/kqviElGnY2EhouuSSCSYe0cfJEwKAwC8t/scVu/inhYi6lwsLETUKn8ZE4pX7h4AAFjzwxkeHiKiTsXCQkSt9uToECye2B9A0+Ghf+09J3IiIrIXLCxEZJFnx/bCgpimc1oSvz2JTT/liZyIiOwBCwsRWWzO7X0w9/beAIBlX5/AlnSu00JEHYuFhYjaJO7OvqZLnl/6PBvfH9eKnIiIbBkLCxG1yW+XPM+IDIJRAF7YnImDueVixyIiG8XCQkRtJpFIsPzecMQM8EF9oxF/+b/DOHFRL3YsIrJBLCxEdFPkMinWzByKW3u6o6quEbM3paOwvEbsWERkY1hYiOimqRxk2PBYJPr7OqOsug6zNx1CxZV6sWMRkQ1hYSGidqF2dMDHT96KAFdH5JfXIPbfGahrNIgdi4hsBAsLEbUbbxcVPnziFjgr5UjPr8Ci7dm87xARtQsWFiJqV319nLH20WGQSSX4PPMC1vxwVuxIRGQDWFiIqN1F9/HC6/eEAwBW7zqNL7MuiJyIiLo6FhYi6hAPD++BZ8Y0LSz34n9/QUZBhciJiKgrY2Ehog6z6K7+mDDQB/UGI2I/OYJL+qtiRyKiLoqFhYg6jFQqwdsPDkE/H2eUVtUh9hNeOUREbcPCQkQdykkpx/uzNVA7OiCzsBKvfHGcVw4RkcVYWIiowwV7OGHNzKGQSoCth4vwySHe3ZmILMPCQkSdYmxfL/z9rv4AgNd2Hkd6Hk/CJaLWY2Ehok7z7JhQ3B3hh0ajgL9+egQlPAmXiFqJhYWIOo1EIsHK+yNM9xx6YXMmGg1GsWMRURfAwkJEnaqbQo61jwyDk0KGQ3kVWL3rtNiRiKgLYGEhok4X6tUdb0yPAAC8t/scdp8qETkREVk7FhYiEsWUwf6YNSIYABC/NQsXK2tFTkRE1oyFhYhEs+TuMAwKUONyTQPmfHYEDTyfhYiuoU2FZe3atQgJCYFKpYJGo8H+/fuvO37v3r3QaDRQqVQIDQ3F+vXrm41555130K9fPzg6OiIoKAhxcXG4epVXEBDZMqVchvceHgZnlRxHCiux6vtTYkciIitlcWHZunUr5s+fj4SEBGRmZiI6OhoTJ05EYWHLC0Hl5eVh0qRJiI6ORmZmJl566SXMnTsX27dvN4359NNPsWjRIixduhQ5OTnYuHEjtm7disWLF7f9mxFRl9DDoxtW3T8YAPD+vlzsO10qciIiskYSwcI1socPH45hw4Zh3bp1pm1hYWGYNm0aEhMTm41fuHAhdu7ciZycHNO22NhYHD16FGlpaQCAOXPmICcnBz/88INpzN/+9jekp6ffcO/Nb/R6PdRqNXQ6HVxcXCz5SkRkBZZ8kY1PDhbCy1mJ7+ZFw6O7UuxIRNQJWvvz26I9LPX19cjIyEBMTIzZ9piYGKSmprb4mrS0tGbjJ0yYgMOHD6OhoQEAMHr0aGRkZCA9PR0AkJubi6SkJEyePPmaWerq6qDX680eRNR1JUwagN7e3VFaVYeF23/h/YaIyIxFhaWsrAwGgwE+Pj5m2318fKDValt8jVarbXF8Y2MjysrKAAAPPfQQXn/9dYwePRoODg7o1asXbrvtNixatOiaWRITE6FWq02PoKAgS74KEVkZR4UMax4aCoVMil05JbzfEBGZadNJtxKJxOzXgiA023aj8X/cvmfPHixfvhxr167FkSNHsGPHDnz99dd4/fXXr/meixcvhk6nMz2Kiora8lWIyIoM8HfB3+/qBwD4x9cncPpSlciJiMhayC0Z7OnpCZlM1mxvSklJSbO9KL/x9fVtcbxcLoeHhwcA4OWXX8asWbPw9NNPAwAGDRqEK1eu4JlnnkFCQgKk0ua9SqlUQqnkMW4iW/PkqBDsO1OGfadLMXdzJr54fhRUDjKxYxGRyCzaw6JQKKDRaJCSkmK2PSUlBSNHjmzxNVFRUc3GJycnIzIyEg4ODgCAmpqaZqVEJpNBEAQexyayM1KpBG89EAEPJwVOaqvwFi91JiK04ZBQfHw8PvjgA2zatAk5OTmIi4tDYWEhYmNjATQdqpk9e7ZpfGxsLAoKChAfH4+cnBxs2rQJGzduxIIFC0xjpkyZgnXr1mHLli3Iy8tDSkoKXn75ZUydOhUyGf9lRWRvvJ1VWHl/09L9Gw/k4VBuuciJiEhsFh0SAoAZM2agvLwcy5YtQ3FxMcLDw5GUlITg4KYltouLi83WZAkJCUFSUhLi4uLw3nvvwd/fH2vWrMH06dNNY5YsWQKJRIIlS5bgwoUL8PLywpQpU7B8+fJ2+IpE1BXdEeaDGZFB2Hq4CAu2HcV388bASWnx/7KIyEZYvA6LteI6LES2p+pqA+56Zz8uVNbikeE9sPzeQWJHIqJ21iHrsBARdSZnlQNW/Xpo6NNDhVwFl8iOsbAQkVUb2dsTj0U1HXJeuP0X6GobRE5ERGJgYSEiq7dwYn/09OiGYt1VLPvqhNhxiEgELCxEZPW6KeT454ODIZUA24+cR/LxllfWJiLbxcJCRF2CJtgdfxkTCgBI+OIYdDU8NERkT1hYiKjLiBvfF728nFBaVYflSTw0RGRPWFiIqMtQOcjw5vQISCTAfw6fx4GzZWJHIqJOwsJCRF1KZE93zB7RdNXQoh2/oKa+UeRERNQZWFiIqMt58a7+8FerUFRRi7eTT4sdh4g6AQsLEXU53ZVyLL+vadXbTQfykFVUKW4gIupwLCxE1CXd1s8b9w4NgFEAFm77BfWNRrEjEVEHYmEhoi7r5bsHwMNJgVOXqrBuzzmx4xBRB2JhIaIuy91JgaVTBwIA3t19BmcuVYmciIg6CgsLEXVpUyL8MD7MGw0GAQmfH4ON3ICeiP6EhYWIujSJRILX7gmHo4MM6fkV+G/GebEjEVEHYGEhoi4vwNURcXf2AQAkJuWg4kq9yImIqL2xsBCRTXhiVAj6+zrjck0DEpNyxI5DRO2MhYWIbIKDTIrl9w6CRAL8N+M8DuWWix2JiNoRCwsR2QxNsBtm3toDQNMdnbk2C5HtYGEhIpuycEJ/eHZX4GxJNTbszxU7DhG1ExYWIrIp6m4OWDJ5AABgzQ9nUFB+ReRERNQeWFiIyObcM8Qfo3p7oK7RiFe+PM61WYhsAAsLEdkciUSC1+8Jh0Imxd7Tpfgmu1jsSER0k1hYiMgmhXp1x3PjegEA/vF1Dq7UNYqciIhuBgsLEdms58b1QpC7I7T6q/jfH8+KHYeIbgILCxHZLJWDDEvvbro54safcnGutFrkRETUViwsRGTTxg/wwe39m26O+OpOnoBL1FWxsBCRzXvl7gFQyKTYf6YM3x/Xih2HiNqAhYWIbF5PTyc8OzYUAPD61zmorTeInIiILMXCQkR24a/jeiPA1REXKmvx3m6egEvU1bCwEJFdcFTI8PLdTSvgvr8vF3llXAGXqCthYSEiuzFhoA/G9PVCvcGI177iCbhEXQkLCxHZDYlEglenDICDTII9p0qxK6dE7EhE1EosLERkV0K9uuPp6KYTcF/76jiuNvAEXKKugIWFiOzOC7f3hp9ahfOXa7Fuzzmx4xBRK7CwEJHd6aaQY8nkphNw1+09h6KKGpETEdGNsLAQkV2aNMgXI3t5oL7RiBVJOWLHIaIbYGEhIrskkUjwypQBkEkl+PaYFqlny8SORETXwcJCRHarv68LHh3eAwDw2lcn0GgwipyIiK6FhYWI7FrcnX3h1s0Bpy5V4dNDhWLHIaJrYGEhIrvm2k2B+Jh+AIC3U07j8pV6kRMRUUtYWIjI7j18aw/093WGrrYB/0w5JXYcImoBCwsR2T2ZVIJXpw4EAHx2qBAnLupFTkREf8bCQkQEYESoByZH+MEoAK/yPkNEVoeFhYjoVy9NCoPKQYr0vAp8k10sdhwi+gMWFiKiXwW4OiJ2bC8AwIpvclBbz/sMEVkLFhYioj94dkwvBLg64qLuKtbv5X2GiKwFCwsR0R84KmRImBwGAFi/9xzOX+Z9hoisQZsKy9q1axESEgKVSgWNRoP9+/dfd/zevXuh0WigUqkQGhqK9evXNxtTWVmJ559/Hn5+flCpVAgLC0NSUlJb4hER3ZSJ4b4YEeqOOt5niMhqWFxYtm7divnz5yMhIQGZmZmIjo7GxIkTUVjY8gqReXl5mDRpEqKjo5GZmYmXXnoJc+fOxfbt201j6uvrceeddyI/Px/btm3DqVOnsGHDBgQEBLT9mxERtZFEIsHSKQMhlQBJ2VqknuN9hojEJhEsvHZv+PDhGDZsGNatW2faFhYWhmnTpiExMbHZ+IULF2Lnzp3Iyfn9XymxsbE4evQo0tLSAADr16/HqlWrcPLkSTg4OLTpi+j1eqjVauh0Ori4uLTpPYiI/ujlL47h3wcL0N/XGV+/MBpyGY+iE7W31v78tuhvX319PTIyMhATE2O2PSYmBqmpqS2+Ji0trdn4CRMm4PDhw2hoaAAA7Ny5E1FRUXj++efh4+OD8PBwrFixAgbDtc/Qr6urg16vN3sQEbWn+Dv7Qu3ogJPaKmxO532GiMRkUWEpKyuDwWCAj4+P2XYfHx9otdoWX6PValsc39jYiLKypt2subm52LZtGwwGA5KSkrBkyRL885//xPLly6+ZJTExEWq12vQICgqy5KsQEd2Qm5MCC2L6AgDeSuZ9hojE1Kb9mxKJxOzXgiA023aj8X/cbjQa4e3tjffffx8ajQYPPfQQEhISzA47/dnixYuh0+lMj6KiorZ8FSKi65r5h/sMvZ1yWuw4RHbLosLi6ekJmUzWbG9KSUlJs70ov/H19W1xvFwuh4eHBwDAz88Pffv2hUwmM40JCwuDVqtFfX3L/6JRKpVwcXExexARtTe5TIqlU5ruM/TpoQLkFPPwM5EYLCosCoUCGo0GKSkpZttTUlIwcuTIFl8TFRXVbHxycjIiIyNNJ9iOGjUKZ8+ehdFoNI05ffo0/Pz8oFAoLIlIRNTuonp5YNIgXxgF4DXeZ4hIFBYfEoqPj8cHH3yATZs2IScnB3FxcSgsLERsbCyApkM1s2fPNo2PjY1FQUEB4uPjkZOTg02bNmHjxo1YsGCBacxzzz2H8vJyzJs3D6dPn8Y333yDFStW4Pnnn2+Hr0hEdPNemhQGpVyKg7kV+PZYy+fsEVHHkVv6ghkzZqC8vBzLli1DcXExwsPDkZSUhODgYABAcXGx2ZosISEhSEpKQlxcHN577z34+/tjzZo1mD59umlMUFAQkpOTERcXh4iICAQEBGDevHlYuHBhO3xFIqKbF+jWDc+O7YU1P5zB8m9ycFs/bzgqZDd+IRG1C4vXYbFWXIeFiDpabb0B49/eiwuVtZh7Rx/E39lX7EhEXV6HrMNCRGTP/nyfoaIK3meIqLOwsBARWWBiuC9G9vJAfaMRr399Quw4RHaDhYWIyAISiQSvTh0ImVSC5BOXsO90qdiRiOwCCwsRkYX6+jjjsaieAIBXvzqO+kbj9V9ARDeNhYWIqA3m39kHnt0VyC29go9S88SOQ2TzWFiIiNrAReWAv9/VHwDwP7vOoER/VeRERLaNhYWIqI3uHxaIIUGuuFJvwBvfnhQ7DpFNY2EhImojqVSC16YOhEQC7Mi8gIyCCrEjEdksFhYiopswOMgVMyKDAACvfHkcBqNNrMVJZHVYWIiIbtKLE/rBRSXH8Yt6bE4vvPELiMhiLCxERDfJo7vStEz/W8mncPlKvciJiGwPCwsRUTt4dEQw+vs6o7KmAf9MOSV2HCKbw8JCRNQO5DIpXp06EADw2aFCHL+oEzkRkW1hYSEiaicjQj1wd4QfjALw6s7jEASegEvUXlhYiIjaUcLkMDg6yPBz/mV8mXVR7DhENoOFhYioHfmpHTHn9t4AgBVJOaiuaxQ5EZFtYGEhImpnT0eHoKdHN5RU1WF1ymmx4xDZBBYWIqJ2ppTL8No94QCAj1LzceKiXuRERF0fCwsRUQcY29cLkwf5wWAUsOSLbBi5Ai7RTWFhISLqIC/fPQBOChmOFFbiP4eLxI5D1KWxsBARdRBftQpxv66A+8Z3J1HBFXCJ2oyFhYioAz0+sqdpBdw3vs0ROw5Rl8XCQkTUgeQyKZbf23QC7n8On8fh/AqRExF1TSwsREQdTBPsjoduCQIAJHx+DA0Go8iJiLoeFhYiok6w8K7+cHdS4NSlKnx4IE/sOERdDgsLEVEncHNSYNHE/gCAd3adwcXKWpETEXUtLCxERJ3k/mGBiAx2Q029Acu+OiF2HKIuhYWFiKiTSKUS/OPecMikEnx3XIsfci6JHYmoy2BhISLqRP19XfD06BAAwJIvjvHmiEStxMJCRNTJ5o/vix7u3VCsu4pV350UOw5Rl8DCQkTUyRwVMqy4dxAA4OODBcgo4NosRDfCwkJEJILRfTxxvyYQggAs2p6NukaD2JGIrBoLCxGRSBImhcGzuwJnSqqxfk+u2HGIrBoLCxGRSNycFHhlykAAwLu7z+DMpSqRExFZLxYWIiIRTYnww+39vdFgELBoRzaMRkHsSERWiYWFiEhEEokEr08Lh5NChoyCy/g0vVDsSERWiYWFiEhkAa6OeHFCPwDAm9+eRLGOy/YT/RkLCxGRFZgV1RNDe7iiuq4Ri3dkQxB4aIjoj1hYiIisgEwqwcrpEVDIpdhzqhT/PXxe7EhEVoWFhYjISvTxcUb8nX0BAK9/fYJ3dCb6AxYWIiIr8pfoUAzt4YqqukYs4qEhIhMWFiIiKyKTSrDq/sFQyKXYd7oUW38uEjsSkVVgYSEisjK9vbvjxZimq4b+8U0OLvDQEBELCxGRNXpydAg0wW6ormvEou2/8NAQ2T0WFiIiK9R0aCgCSrkU+8+UYXM6Dw2RfWNhISKyUqFe3U0Lyi3/5gQKy2tETkQkHhYWIiIr9sSoENza0x1X6g2I+08WGg1GsSMRiYKFhYjIismkErw9YzCclXJkFFzG2j3nxI5EJIo2FZa1a9ciJCQEKpUKGo0G+/fvv+74vXv3QqPRQKVSITQ0FOvXr7/m2C1btkAikWDatGltiUZEZHMC3brh9WnhAID/+eEMsooqxQ1EJAKLC8vWrVsxf/58JCQkIDMzE9HR0Zg4cSIKC1u+w2heXh4mTZqE6OhoZGZm4qWXXsLcuXOxffv2ZmMLCgqwYMECREdHW/5NiIhs2D1D/DFlsD8MRgHzt2TiSl2j2JGIOpVEsPBaueHDh2PYsGFYt26daVtYWBimTZuGxMTEZuMXLlyInTt3Iicnx7QtNjYWR48eRVpammmbwWDA2LFj8cQTT2D//v2orKzEF1980epcer0earUaOp0OLi4ulnwlIqIuQVfTgIn/sw8XdVcx89YgJN4XIXYkopvW2p/fFu1hqa+vR0ZGBmJiYsy2x8TEIDU1tcXXpKWlNRs/YcIEHD58GA0NDaZty5Ytg5eXF5566ilLIhER2Q11Nwe89eBgSCTA5vQiJB/Xih2JqNNYVFjKyspgMBjg4+Njtt3Hxwdabct/cbRabYvjGxsbUVZWBgA4cOAANm7ciA0bNrQ6S11dHfR6vdmDiMjWjezliWeiQwEAi3Zko0R/VeRERJ2jTSfdSiQSs18LgtBs243G/7a9qqoKjz76KDZs2ABPT89WZ0hMTIRarTY9goKCLPgGRERdV3xMXwzwc0HFlXrM35oFg5Gr4JLts6iweHp6QiaTNdubUlJS0mwvym98fX1bHC+Xy+Hh4YFz584hPz8fU6ZMgVwuh1wux8cff4ydO3dCLpfj3LmWL+FbvHgxdDqd6VFUxFUgicg+KOUyrJk5FI4OMqSeK8d7u8+KHYmow1lUWBQKBTQaDVJSUsy2p6SkYOTIkS2+Jioqqtn45ORkREZGwsHBAf3790d2djaysrJMj6lTp+K2225DVlbWNfecKJVKuLi4mD2IiOxFb+/u+Mevlzq/s+s0DuaWi5yIqGNZfEgoPj4eH3zwATZt2oScnBzExcWhsLAQsbGxAJr2fMyePds0PjY2FgUFBYiPj0dOTg42bdqEjRs3YsGCBQAAlUqF8PBws4erqyucnZ0RHh4OhULRTl+ViMi2TNcE4n5NIIwCMG9LJsqr68SORNRh5Ja+YMaMGSgvL8eyZctQXFyM8PBwJCUlITg4GABQXFxstiZLSEgIkpKSEBcXh/feew/+/v5Ys2YNpk+f3n7fgojITi27ZyAyCy/jXOkVxP/nKD58/BZIpdc+p5Coq7J4HRZrxXVYiMhendTqcc+7B1DXaMSiif0RO7aX2JGIWq1D1mEhIiLr09/XBa9NHQgAWPX9KRzOrxA5EVH7Y2EhIrIBM24JwtRfl+7/66dHUFLF9VnItrCwEBHZAIlEghX3DUIf7+4oqarDnE8z0WAwih2LqN2wsBAR2YjuSjnWz9Kgu1KO9PwKJCadFDsSUbthYSEisiG9vLrjnw8OBgBsOpCHL7MuiJyIqH2wsBAR2ZgJA33x/G1NVwot2p6Nk1rea426PhYWIiIbFH9nP0T38URtgwHP/jsDutoGsSMR3RQWFiIiGySTSrDmoaEIcHVEQXkNXticiUaehEtdGAsLEZGNcnNS4F+zNHB0kGHf6VIsT8oROxJRm7GwEBHZsPAANVbPaDoJ98MD+fj0UIHIiYjahoWFiMjG3RXuh7/d2RcAsPTL40g9VyZyIiLLsbAQEdmBObf3xtTB/mg0CnjukyPIK7sidiQii7CwEBHZAYlEgpX3R2BwkCt0tQ146v9+5pVD1KWwsBAR2QmVgwwbZmngp1Yht/QKnv33YdQ1GsSORdQqLCxERHbE20WFjY/dgu5KOQ7mVmDBf3+B0SiIHYvohlhYiIjszAB/F6x7dBjkUgm+OnoRb37Hew6R9WNhISKyQ9F9vPDm9AgAwL/25eKjA3kiJyK6PhYWIiI7NV0TiBcn9AMAvPb1CXybXSxyIqJrY2EhIrJjfx3XC48M7wFBAOZtzcLB3HKxIxG1iIWFiMiOSSQSvDZ1IMaH+aC+0Yin/+8wjhZVih2LqBkWFiIiOyeXSfHuw0MRFeqB6rpGPPZhOk5pq8SORWSGhYWIiJrWaHksEkOCXFFZ04BHNx5CPlfDJSvCwkJERACA7ko5PnriFvT3dUZpVR0e+eAQinW1YsciAsDCQkREf+DaTYGPn7oVIZ5OuFBZi0c+OISSqqtixyJiYSEiInPezip88vRw+P+6hP/M9w+iRM/SQuJiYSEiomYCXB2x+ZkR8FercK70Ch7awNJC4mJhISKiFgV7OGHLM1EIcHVEbukVPPT+QVxiaSGRsLAQEdE19fDohi3PjGgqLWVNpUWrY2mhzsfCQkRE1xXk/ntpySu7gofeT0NRRY3YscjOsLAQEdEN/VZaAt0ckV9egwfWp+HMJS4uR52HhYWIiFolyL0btsWORG/v7tDqr+LBf6VxGX/qNCwsRETUar5qFf7zbBQGB6pxuaYBD284iNRzZWLHIjvAwkJERBZxd1Lg07+MQFSoB67UG/D4hz8j+bhW7Fhk41hYiIjIYt2Vcnz4xC24c0DTXZ5jP8nAv9PyxY5FNoyFhYiI2kTlIMO6R4bhwchAGAXg5S+PY0VSDoxGQexoZINYWIiIqM3kMinenB6BBTF9AQDv78vFnM1HcLXBIHIysjUsLEREdFMkEgnm3N4H78wYAoVMiqRsLR7ecBDl1XViRyMbwsJCRETtYtrQAHz81K1wUclxpLAS965NxSkt12qh9sHCQkRE7WZEqAd2/HUkgtwdUVhRg3vXHsB3x4rFjkU2gIWFiIjaVW9vZ+x8fjRG9vJATb0BsZ8cweqU0zwZl24KCwsREbU7NycFPn7yVjw5KgQA8D8/nEHsJxmormsUORl1VSwsRETUIeQyKV6ZMgCr7o+AQiZF8olLmPbeAZzmPYioDVhYiIioQz0QGYStz46Aj4sSZ0uqcc+7B7At47zYsaiLYWEhIqION7SHG76ZG43oPp6obTBgwX+P4u/bjqK2nuu1UOuwsBARUafw7K7ER0/civg7+0IiAf5z+DymvXcAZ0uqxY5GXQALCxERdRqZVIK5d/TBp08Nh2d3JU5dqsKU//0Jnx0qhCDwKiK6NhYWIiLqdCN7eyJp3miM6u2B2gYDXvo8G3/5+DDKuDouXQMLCxERicLbWYV/PzkcSyaHQSGTYldOCe56Zx9+PHlJ7GhkhdpUWNauXYuQkBCoVCpoNBrs37//uuP37t0LjUYDlUqF0NBQrF+/3uz5DRs2IDo6Gm5ubnBzc8P48eORnp7elmhERNSFSKUSPB0dii/njEI/H2eUVdfjyY8OI+HzbK7ZQmYsLixbt27F/PnzkZCQgMzMTERHR2PixIkoLCxscXxeXh4mTZqE6OhoZGZm4qWXXsLcuXOxfft205g9e/Zg5syZ2L17N9LS0tCjRw/ExMTgwoULbf9mRETUZYT5ueDLOaPw1OimheY+PVSICav3Yd/pUpGTkbWQCBae5TR8+HAMGzYM69atM20LCwvDtGnTkJiY2Gz8woULsXPnTuTk5Ji2xcbG4ujRo0hLS2vxMwwGA9zc3PDuu+9i9uzZrcql1+uhVquh0+ng4uJiyVciIiIrcuBsGRZu/wXnL9cCAO7XBGLJ5DC4dlOInIw6Qmt/flu0h6W+vh4ZGRmIiYkx2x4TE4PU1NQWX5OWltZs/IQJE3D48GE0NDS0+Jqamho0NDTA3d3dknhERGQDRvX2xPfzx+CJUT0hkQDbMs5j/Nv7eBNFO2dRYSkrK4PBYICPj4/Zdh8fH2i12hZfo9VqWxzf2NiIsrKyFl+zaNEiBAQEYPz48dfMUldXB71eb/YgIiLb4KSUY+mUgdgWG4VeXk4oq65D7CdH8PT/HUZRRY3Y8UgEbTrpViKRmP1aEIRm2240vqXtALBy5Ups3rwZO3bsgEqluuZ7JiYmQq1Wmx5BQUGWfAUiIuoCNMHu+GZuNObc1htyqQS7ci7hztV78e6PZ1DXyFVy7YlFhcXT0xMymazZ3pSSkpJme1F+4+vr2+J4uVwODw8Ps+1vvfUWVqxYgeTkZERERFw3y+LFi6HT6UyPoqIiS74KERF1ESoHGRZM6IekedEYEeqOqw1GvJV8GhNW78NenpRrNywqLAqFAhqNBikpKWbbU1JSMHLkyBZfExUV1Wx8cnIyIiMj4eDgYNq2atUqvP766/juu+8QGRl5wyxKpRIuLi5mDyIisl19fZyx+S8j8D8PDYG3sxL55TV4bFM6nv33YRSUXxE7HnUwiw8JxcfH44MPPsCmTZuQk5ODuLg4FBYWIjY2FkDTno8/XtkTGxuLgoICxMfHIycnB5s2bcLGjRuxYMEC05iVK1diyZIl2LRpE3r27AmtVgutVovqat5fgoiIfieRSHDPkAD88LexeHp0CGRSCb4/fgnj396LZV+dQGVNvdgRqYNYfFkz0LRw3MqVK1FcXIzw8HCsXr0aY8aMAQA8/vjjyM/Px549e0zj9+7di7i4OBw/fhz+/v5YuHChqeAAQM+ePVFQUNDsc5YuXYpXX321VZl4WTMRkf05fakKy7/JMR0aclHJMfeOPpgVFQylXCZyOmqN1v78blNhsUYsLERE9mvf6VKsSMrBSW0VAKCHezcsmNAPdw/yg1R67YtCSHwsLEREZFcMRgHbM87jreRTKKlquolif19nzB/fFxMG+lz3alYSDwsLERHZpSt1jdj4Ux427M9F1dWm+xGFB7jgb3f2w7h+XiwuVoaFhYiI7JqupgEb9ufiwwN5uFLftGbL0B6umHdHH4zty+JiLVhYiIiIAJRX1+Ff+3LxcVo+rjYYAQAD/V3w3LhemBjuBxnPcREVCwsREdEflOiv4l/7cvHZoULUNjTtcenp0Q3Pju2F+4YF8KoikbCwEBERteDylXp8lJqPj1Lzoattugmvj4sSj48Mwcxbg3hX6E7GwkJERHQdV+oasTm9EB/sz4NWfxUAoHKQ4t6hgXhiVE/09XEWOaF9YGEhIiJqhbpGA77MuogPD+Qjp1hv2j66tyeeGNUTt/Xz5louHYiFhYiIyAKCIOBQXgU+PJCHlBOXYPz1p2OwRzfMuCUI92sC4e2sEjekDWJhISIiaqOiihr8+2ABtqQXQv/rWi5yqQTjw3wwc3gPRPf25F6XdsLCQkREdJNq6hvx9S/F2JxeiMzCStP2AFdHPHRLEKZrAuHv6iheQBvAwkJERNSOTmr12JJehB1Hzpv2ukgkwIgQD9w7LAATw33hrHIQOWXXw8JCRETUAa42GJCUXYwtPxchPa/CtF0pl+LOAT64d2gAxvT1goNMKmLKroOFhYiIqIMVVdRg59GL2HHkPM6VXjFt93BS4K5wX0we5IdbQ9whZ3m5JhYWIiKiTiIIAo5d0GNH5nl8dfQiyqrrTc+5OykwYaAPJg3yw4hQD+55+RMWFiIiIhE0Goz46WwZvs3W4vsTWlTWNJiec+3mgJgBPrgr3Bcje3lC5cDbAbCwEBERiazBYMSh3Ap8k12M5ONalF/5fc+Lo4MMo3p74o4wb9zR3xveLva5xgsLCxERkRVpNBiRnl+BpOxi/JBTgmLdVbPnBwWofy0vPhjo72I367ywsBAREVkpQRBwoliPH3NKsOtkCY4WVZo979ldgVG9PTG6tyei+3jBV227e19YWIiIiLqIkqqr2HOyFD+cvIT9Z8pQU28we763d/dfy4snRoR6wEkpFylp+2NhISIi6oLqG404UngZP50pw/6zZcg+X2m6rxHQdIuAIUGuuDXEHbeGuEMT7NalF6xjYSEiIrIBupoGpJ4rw74zZfjpbCmKKmrNnpdKgPAANW7t2VRgbunpDjcnhUhpLcfCQkREZIMKy2twMK8c6XkVOJRX3qzAAEA/H2cMC3bD0B6uGNbDFaGe3a32JF4WFiIiIjtwsbIWP+dX4FBeBdLzKnC2pLrZGGeVHEOCXDE0yBVDe7hhSJCr1eyFYWEhIiKyQ2XVdTicfxmZRZeRWViJX85X4mqDsdm4EE8nRASqEe6vxsAAFwz0U0PdrfPPhWFhISIiIjQajDiprUJmUSWyCiuRWXQZuX+479EfBbk7ItxfjfAANQb6uyA8QA3P7soOzcfCQkRERC2qrKlHVlEljl/U49gFHY5d1LV4LgwA+LqoEObnjDA/FzwQGYQQT6d2zdLan9+2cyE3ERERtYprNwXG9fPGuH7epm26mgYcv9hUXo5d0OPYRR3yyq5Aq78Krf4qdp8qxe39vdu9sLQWCwsRERFB3c0BI3t7YmRvT9O2K3WNyCnWI0dbhZxiPfr5OouWj4WFiIiIWuSklCOypzsie7qLHQVSsQMQERER3QgLCxEREVk9FhYiIiKyeiwsREREZPVYWIiIiMjqsbAQERGR1WNhISIiIqvHwkJERERWj4WFiIiIrB4LCxEREVk9FhYiIiKyeiwsREREZPVYWIiIiMjq2czdmgVBAADo9XqRkxAREVFr/fZz+7ef49diM4WlqqoKABAUFCRyEiIiIrJUVVUV1Gr1NZ+XCDeqNF2E0WjExYsX4ezsDIlE0m7vq9frERQUhKKiIri4uLTb+5I5znPn4Vx3Ds5z5+A8d46OnGdBEFBVVQV/f39Ipdc+U8Vm9rBIpVIEBgZ22Pu7uLjwL0Mn4Dx3Hs515+A8dw7Oc+foqHm+3p6V3/CkWyIiIrJ6LCxERERk9VhYbkCpVGLp0qVQKpViR7FpnOfOw7nuHJznzsF57hzWMM82c9ItERER2S7uYSEiIiKrx8JCREREVo+FhYiIiKweCwsRERFZPRaWG1i7di1CQkKgUqmg0Wiwf/9+sSNZrcTERNxyyy1wdnaGt7c3pk2bhlOnTpmNEQQBr776Kvz9/eHo6Ihx48bh+PHjZmPq6urwwgsvwNPTE05OTpg6dSrOnz9vNuby5cuYNWsW1Go11Go1Zs2ahcrKyo7+ilYpMTEREokE8+fPN23jPLePCxcu4NFHH4WHhwe6deuGIUOGICMjw/Q85/nmNTY2YsmSJQgJCYGjoyNCQ0OxbNkyGI1G0xjOc9vs27cPU6ZMgb+/PyQSCb744guz5ztzXgsLCzFlyhQ4OTnB09MTc+fORX19vWVfSKBr2rJli+Dg4CBs2LBBOHHihDBv3jzByclJKCgoEDuaVZowYYLw4YcfCseOHROysrKEyZMnCz169BCqq6tNY9544w3B2dlZ2L59u5CdnS3MmDFD8PPzE/R6vWlMbGysEBAQIKSkpAhHjhwRbrvtNmHw4MFCY2Ojacxdd90lhIeHC6mpqUJqaqoQHh4u3H333Z36fa1Benq60LNnTyEiIkKYN2+eaTvn+eZVVFQIwcHBwuOPPy4cOnRIyMvLE3bt2iWcPXvWNIbzfPP+8Y9/CB4eHsLXX38t5OXlCf/973+F7t27C++8845pDOe5bZKSkoSEhARh+/btAgDh888/N3u+s+a1sbFRCA8PF2677TbhyJEjQkpKiuDv7y/MmTPHou/DwnIdt956qxAbG2u2rX///sKiRYtEStS1lJSUCACEvXv3CoIgCEajUfD19RXeeOMN05irV68KarVaWL9+vSAIglBZWSk4ODgIW7ZsMY25cOGCIJVKhe+++04QBEE4ceKEAEA4ePCgaUxaWpoAQDh58mRnfDWrUFVVJfTp00dISUkRxo4dayosnOf2sXDhQmH06NHXfJ7z3D4mT54sPPnkk2bb7rvvPuHRRx8VBIHz3F7+XFg6c16TkpIEqVQqXLhwwTRm8+bNglKpFHQ6Xau/Aw8JXUN9fT0yMjIQExNjtj0mJgapqakipepadDodAMDd3R0AkJeXB61WazanSqUSY8eONc1pRkYGGhoazMb4+/sjPDzcNCYtLQ1qtRrDhw83jRkxYgTUarVd/d48//zzmDx5MsaPH2+2nfPcPnbu3InIyEg88MAD8Pb2xtChQ7FhwwbT85zn9jF69Gj88MMPOH36NADg6NGj+OmnnzBp0iQAnOeO0pnzmpaWhvDwcPj7+5vGTJgwAXV1dWaHWG/EZm5+2N7KyspgMBjg4+Njtt3HxwdarVakVF2HIAiIj4/H6NGjER4eDgCmeWtpTgsKCkxjFAoF3Nzcmo357fVarRbe3t7NPtPb29tufm+2bNmCI0eO4Oeff272HOe5feTm5mLdunWIj4/HSy+9hPT0dMydOxdKpRKzZ8/mPLeThQsXQqfToX///pDJZDAYDFi+fDlmzpwJgH+eO0pnzqtWq232OW5ublAoFBbNPQvLDUgkErNfC4LQbBs1N2fOHPzyyy/46aefmj3Xljn985iWxtvL701RURHmzZuH5ORkqFSqa47jPN8co9GIyMhIrFixAgAwdOhQHD9+HOvWrcPs2bNN4zjPN2fr1q345JNP8Nlnn2HgwIHIysrC/Pnz4e/vj8cee8w0jvPcMTprXttj7nlI6Bo8PT0hk8matb+SkpJmTZHMvfDCC9i5cyd2796NwMBA03ZfX18AuO6c+vr6or6+HpcvX77umEuXLjX73NLSUrv4vcnIyEBJSQk0Gg3kcjnkcjn27t2LNWvWQC6Xm+aA83xz/Pz8MGDAALNtYWFhKCwsBMA/z+3lxRdfxKJFi/DQQw9h0KBBmDVrFuLi4pCYmAiA89xROnNefX19m33O5cuX0dDQYNHcs7Bcg0KhgEajQUpKitn2lJQUjBw5UqRU1k0QBMyZMwc7duzAjz/+iJCQELPnQ0JC4Ovrazan9fX12Lt3r2lONRoNHBwczMYUFxfj2LFjpjFRUVHQ6XRIT083jTl06BB0Op1d/N7ccccdyM7ORlZWlukRGRmJRx55BFlZWQgNDeU8t4NRo0Y1uyz/9OnTCA4OBsA/z+2lpqYGUqn5jyKZTGa6rJnz3DE6c16joqJw7NgxFBcXm8YkJydDqVRCo9G0PnSrT8+1Q79d1rxx40bhxIkTwvz58wUnJychPz9f7GhW6bnnnhPUarWwZ88eobi42PSoqakxjXnjjTcEtVot7NixQ8jOzhZmzpzZ4mV0gYGBwq5du4QjR44It99+e4uX0UVERAhpaWlCWlqaMGjQIJu+PPFG/niVkCBwnttDenq6IJfLheXLlwtnzpwRPv30U6Fbt27CJ598YhrDeb55jz32mBAQEGC6rHnHjh2Cp6en8Pe//900hvPcNlVVVUJmZqaQmZkpABDefvttITMz07Q0R2fN62+XNd9xxx3CkSNHhF27dgmBgYG8rLm9vffee0JwcLCgUCiEYcOGmS7RpeYAtPj48MMPTWOMRqOwdOlSwdfXV1AqlcKYMWOE7Oxss/epra0V5syZI7i7uwuOjo7C3XffLRQWFpqNKS8vFx555BHB2dlZcHZ2Fh555BHh8uXLnfAtrdOfCwvnuX189dVXQnh4uKBUKoX+/fsL77//vtnznOebp9frhXnz5gk9evQQVCqVEBoaKiQkJAh1dXWmMZznttm9e3eL/09+7LHHBEHo3HktKCgQJk+eLDg6Ogru7u7CnDlzhKtXr1r0fSSCIAit3x9DRERE1Pl4DgsRERFZPRYWIiIisnosLERERGT1WFiIiIjI6rGwEBERkdVjYSEiIiKrx8JCREREVo+FhYiIiKweCwsRERFZPRYWIiIisnosLERERGT1WFiIiIjI6v0/MChNK9sjToIAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# visualizing the error after each epoch\n",
    "plt.plot(np.arange(1,epochs+1), np.array(error_epoch))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1949,
     "status": "ok",
     "timestamp": 1585485784571,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "bJlcGoeUNFXA",
    "outputId": "052f7ac8-c10e-49e4-df8e-69bff60d4381"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.93598588, 0.8839978 , 0.18340013]])"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# final output from the model\n",
    "output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1904,
     "status": "ok",
     "timestamp": 1585485785455,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "ARNn3MiKNFXF",
    "outputId": "eb1606ed-53da-48f8-c5a0-f4c459e71fdc"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 1, 0]])"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# actual target\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1734,
     "status": "ok",
     "timestamp": 1585485785462,
     "user": {
      "displayName": "Pulkit Sharma",
      "photoUrl": "",
      "userId": "07234574884764057306"
     },
     "user_tz": -330
    },
    "id": "MtwXl0bjNFXJ",
    "outputId": "d5037572-8214-4175-fd97-92eaea9c3540"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.0020489 , 0.00672826, 0.0168178 ]])"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# error at last epoch\n",
    "error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "HkfQF2CC2ir6"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "Neural Network from scratch using NumPy.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}