Exercise_03_3_solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 3.3 - Solution\n",
    "## Checkerboard\n",
    "\n",
    "Open the Tensorflow Playground (www.playground.tensorflow.org) and select on the left the checkerboard pattern as the data basis.\n",
    "\n",
    "The data is taken from a two-dimensional probability distribution and is represented by the value pairs $x_1$ and $x_2$. The regions $x1$, $x_2 > 0$ and $x_1$, $x_2 < 0$ are shown by one color. For value pairs with $x_1 > 0$, $x_2 < 0$ and $x_1 < 0$, $x_2 > 0$, the regions are indicated by a different color.  \n",
    "\n",
    "In features, select the two independent variables $x_1$ and $x_2$ and start the network training. The network learns that $x_1$ and $x_2$ are for these data not independent variables, but are taken from the probability distribution of the checkerboard pattern.\n",
    "\n",
    "[![Checkerboard](./images/checkerboard_tf_playground.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=0&networkShape=4,2&seed=0.20784&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false)\n",
    "\n",
    "## Tasks\n",
    "1.  Try various settings for the number of layers and neurons using `ReLU` as activation function. What is the smallest network that gives a good fit result?\n",
    "2.  What do you observe when training networks with the same settings multiple times? Explain your observations.\n",
    "3.  Try additional input features: Which one is most helpful?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solutions\n",
    "Hint: click on the images to open the correct playground settings needed to solve the task, respectively."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 1\n",
    "Try various settings for the number of layers and neurons using ReLU as activation function. What is the smallest network that gives a good fit result?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Checkerboard](./images/checkerboard_tf_playground.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=0&networkShape=3&seed=0.10528&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false)\n",
    "\n",
    "<em> A network with a single layer holding 3 nodes. However, this configuration is not stable.\n",
    "\n",
    "A network with a single layer, holding 4 nodes, is way more stable. </em>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 2\n",
    "What do you observe when training networks with the same settings multiple times? Explain your observations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<em> Due to the random initialization of weights, the network training always develops a little bit differently, leading to different results. </em>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 3\n",
    "Try additional input features: Which one is most helpful?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Checkerboard](./images/checkerboard_3_2_task_3.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=0&networkShape=&seed=0.10528&showTestData=false&discretize=false&percTrainData=50&x=false&y=false&xTimesY=true&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false)\n",
    "\n",
    "<em> Obviously, the $x_1\\cdot x_2$ feature is most helpful. </em> "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}