DA0101EN_Review_Data_Wrangling.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "anaconda-cloud": {},
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.7"
    },
    "colab": {
      "name": "DA0101EN-Review-Data-Wrangling.ipynb",
      "provenance": []
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AVATdyj0qRCW"
      },
      "source": [
        "<h1>Data Wrangling</h1>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vHBY0PDlqRCX"
      },
      "source": [
        "<h3>Welcome!</h3>\n",
        "\n",
        "By the end of this notebook, you will have learned the basics of Data Wrangling! "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "52oO8J_LqRCY"
      },
      "source": [
        "<h2>Table of content</h2>\n",
        "\n",
        "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
        "<ul>\n",
        "    <li><a href=\"#identify_handle_missing_values\">Identify and handle missing values</a>\n",
        "        <ul>\n",
        "            <li><a href=\"#identify_missing_values\">Identify missing values</a></li>\n",
        "            <li><a href=\"#deal_missing_values\">Deal with missing values</a></li>\n",
        "            <li><a href=\"#correct_data_format\">Correct data format</a></li>\n",
        "        </ul>\n",
        "    </li>\n",
        "    <li><a href=\"#data_standardization\">Data standardization</a></li>\n",
        "    <li><a href=\"#data_normalization\">Data Normalization (centering/scaling)</a></li>\n",
        "    <li><a href=\"#binning\">Binning</a></li>\n",
        "    <li><a href=\"#indicator\">Indicator variable</a></li>\n",
        "</ul>\n",
        "    \n",
        "Estimated Time Needed: <strong>30 min</strong>\n",
        "</div>\n",
        " \n",
        "<hr>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JGmjAin1qRCZ"
      },
      "source": [
        "<h2>What is the purpose of Data Wrangling?</h2>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a8Llh2RsqRCa"
      },
      "source": [
        "Data Wrangling is the process of converting data from the initial format to a format that may be better for analysis."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_8l8DfYrqRCb"
      },
      "source": [
        "<h3>What is the fuel consumption (L/100k) rate for the diesel car?</h3>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QWAA18ZJqRCd"
      },
      "source": [
        "<h3>Import data</h3>\n",
        "<p>\n",
        "You can find the \"Automobile Data Set\" from the following link: <a href=\"https://archive.ics.uci.edu/ml/machine-learning-databases/autos/imports-85.data\">https://archive.ics.uci.edu/ml/machine-learning-databases/autos/imports-85.data</a>. \n",
        "We will be using this data set throughout this course.\n",
        "</p>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8JcBmv7MqRCe"
      },
      "source": [
        "<h4>Import pandas</h4> "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "id": "xpmpc88mqRCf"
      },
      "source": [
        "import pandas as pd\n",
        "import matplotlib.pylab as plt"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kx743DOwqRCj"
      },
      "source": [
        "<h2>Reading the data set from the URL and adding the related headers.</h2>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QzpWxYw3qRCk"
      },
      "source": [
        "URL of the dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k5encMYBqRCl"
      },
      "source": [
        "This dataset was hosted on IBM Cloud object click <a href=\"https://cocl.us/da0101en_object_storage_skills_network\">HERE</a> for free storage "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "id": "xBSw1-uhqRCo"
      },
      "source": [
        "filename = \"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DA0101EN/auto.csv\""
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B7EQ0hlsqRCq"
      },
      "source": [
        " Python list <b>headers</b> containing name of headers "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "id": "92vYZjP9qRCr"
      },
      "source": [
        "headers = [\"symboling\",\"normalized-losses\",\"make\",\"fuel-type\",\"aspiration\", \"num-of-doors\",\"body-style\",\n",
        "         \"drive-wheels\",\"engine-location\",\"wheel-base\", \"length\",\"width\",\"height\",\"curb-weight\",\"engine-type\",\n",
        "         \"num-of-cylinders\", \"engine-size\",\"fuel-system\",\"bore\",\"stroke\",\"compression-ratio\",\"horsepower\",\n",
        "         \"peak-rpm\",\"city-mpg\",\"highway-mpg\",\"price\"]"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vfbCMY5-qRCv"
      },
      "source": [
        "Use the Pandas method <b>read_csv()</b> to load the data from the web address. Set the parameter  \"names\" equal to the Python list \"headers\"."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "v9or-1A-qRCx"
      },
      "source": [
        "df = pd.read_csv(filename, names = headers)"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Se3FRniMqRCy"
      },
      "source": [
        " Use the method <b>head()</b> to display the first five rows of the dataframe. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 309
        },
        "id": "t0aZlJkqqRCz",
        "outputId": "6f199abe-e579-4cf9-d8f8-6dd38b76dfe3"
      },
      "source": [
        "# To see what the data set looks like, we'll use the head() method.\n",
        "df.head()"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>symboling</th>\n",
              "      <th>normalized-losses</th>\n",
              "      <th>make</th>\n",
              "      <th>fuel-type</th>\n",
              "      <th>aspiration</th>\n",
              "      <th>num-of-doors</th>\n",
              "      <th>body-style</th>\n",
              "      <th>drive-wheels</th>\n",
              "      <th>engine-location</th>\n",
              "      <th>wheel-base</th>\n",
              "      <th>length</th>\n",
              "      <th>width</th>\n",
              "      <th>height</th>\n",
              "      <th>curb-weight</th>\n",
              "      <th>engine-type</th>\n",
              "      <th>num-of-cylinders</th>\n",
              "      <th>engine-size</th>\n",
              "      <th>fuel-system</th>\n",
              "      <th>bore</th>\n",
              "      <th>stroke</th>\n",
              "      <th>compression-ratio</th>\n",
              "      <th>horsepower</th>\n",
              "      <th>peak-rpm</th>\n",
              "      <th>city-mpg</th>\n",
              "      <th>highway-mpg</th>\n",
              "      <th>price</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>3</td>\n",
              "      <td>?</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>168.8</td>\n",
              "      <td>64.1</td>\n",
              "      <td>48.8</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000</td>\n",
              "      <td>21</td>\n",
              "      <td>27</td>\n",
              "      <td>13495</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3</td>\n",
              "      <td>?</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>168.8</td>\n",
              "      <td>64.1</td>\n",
              "      <td>48.8</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000</td>\n",
              "      <td>21</td>\n",
              "      <td>27</td>\n",
              "      <td>16500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>?</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>hatchback</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>94.5</td>\n",
              "      <td>171.2</td>\n",
              "      <td>65.5</td>\n",
              "      <td>52.4</td>\n",
              "      <td>2823</td>\n",
              "      <td>ohcv</td>\n",
              "      <td>six</td>\n",
              "      <td>152</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>2.68</td>\n",
              "      <td>3.47</td>\n",
              "      <td>9.0</td>\n",
              "      <td>154</td>\n",
              "      <td>5000</td>\n",
              "      <td>19</td>\n",
              "      <td>26</td>\n",
              "      <td>16500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>fwd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.8</td>\n",
              "      <td>176.6</td>\n",
              "      <td>66.2</td>\n",
              "      <td>54.3</td>\n",
              "      <td>2337</td>\n",
              "      <td>ohc</td>\n",
              "      <td>four</td>\n",
              "      <td>109</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>10.0</td>\n",
              "      <td>102</td>\n",
              "      <td>5500</td>\n",
              "      <td>24</td>\n",
              "      <td>30</td>\n",
              "      <td>13950</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>4wd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.4</td>\n",
              "      <td>176.6</td>\n",
              "      <td>66.4</td>\n",
              "      <td>54.3</td>\n",
              "      <td>2824</td>\n",
              "      <td>ohc</td>\n",
              "      <td>five</td>\n",
              "      <td>136</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>8.0</td>\n",
              "      <td>115</td>\n",
              "      <td>5500</td>\n",
              "      <td>18</td>\n",
              "      <td>22</td>\n",
              "      <td>17450</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   symboling normalized-losses         make  ... city-mpg highway-mpg  price\n",
              "0          3                 ?  alfa-romero  ...       21          27  13495\n",
              "1          3                 ?  alfa-romero  ...       21          27  16500\n",
              "2          1                 ?  alfa-romero  ...       19          26  16500\n",
              "3          2               164         audi  ...       24          30  13950\n",
              "4          2               164         audi  ...       18          22  17450\n",
              "\n",
              "[5 rows x 26 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "caEZw8q5qRC1"
      },
      "source": [
        "As we can see, several question marks appeared in the dataframe; those are missing values which may hinder our further analysis. \n",
        "<div>So, how do we identify all those missing values and deal with them?</div> \n",
        "\n",
        "\n",
        "<b>How to work with missing data?</b>\n",
        "\n",
        "Steps for working with missing data:\n",
        "<ol>\n",
        "    <li>Identify missing data</li>\n",
        "    <li>deal with missing data</li>\n",
        "    <li>correct data format</li>\n",
        "</ol>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s7Wnx1JUqRC2"
      },
      "source": [
        "<h2 id=\"identify_handle_missing_values\">Identify and handle missing values</h2>\n",
        "\n",
        "\n",
        "<h3 id=\"identify_missing_values\">Identify missing values</h3>\n",
        "<h4>Convert \"?\" to NaN</h4>\n",
        "In the car dataset, missing data comes with the question mark \"?\".\n",
        "We replace \"?\" with NaN (Not a Number), which is Python's default missing value marker, for reasons of computational speed and convenience. Here we use the function: \n",
        " <pre>.replace(A, B, inplace = True) </pre>\n",
        "to replace A by B"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 309
        },
        "id": "wAcsfmMxqRC3",
        "outputId": "44cd51ec-9f58-4d83-aa4f-84a1afe20cf5"
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "# replace \"?\" to NaN\n",
        "df.replace(\"?\", np.nan, inplace = True)\n",
        "df.head(5)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>symboling</th>\n",
              "      <th>normalized-losses</th>\n",
              "      <th>make</th>\n",
              "      <th>fuel-type</th>\n",
              "      <th>aspiration</th>\n",
              "      <th>num-of-doors</th>\n",
              "      <th>body-style</th>\n",
              "      <th>drive-wheels</th>\n",
              "      <th>engine-location</th>\n",
              "      <th>wheel-base</th>\n",
              "      <th>length</th>\n",
              "      <th>width</th>\n",
              "      <th>height</th>\n",
              "      <th>curb-weight</th>\n",
              "      <th>engine-type</th>\n",
              "      <th>num-of-cylinders</th>\n",
              "      <th>engine-size</th>\n",
              "      <th>fuel-system</th>\n",
              "      <th>bore</th>\n",
              "      <th>stroke</th>\n",
              "      <th>compression-ratio</th>\n",
              "      <th>horsepower</th>\n",
              "      <th>peak-rpm</th>\n",
              "      <th>city-mpg</th>\n",
              "      <th>highway-mpg</th>\n",
              "      <th>price</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>3</td>\n",
              "      <td>NaN</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>168.8</td>\n",
              "      <td>64.1</td>\n",
              "      <td>48.8</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000</td>\n",
              "      <td>21</td>\n",
              "      <td>27</td>\n",
              "      <td>13495</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3</td>\n",
              "      <td>NaN</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>168.8</td>\n",
              "      <td>64.1</td>\n",
              "      <td>48.8</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000</td>\n",
              "      <td>21</td>\n",
              "      <td>27</td>\n",
              "      <td>16500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>NaN</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>hatchback</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>94.5</td>\n",
              "      <td>171.2</td>\n",
              "      <td>65.5</td>\n",
              "      <td>52.4</td>\n",
              "      <td>2823</td>\n",
              "      <td>ohcv</td>\n",
              "      <td>six</td>\n",
              "      <td>152</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>2.68</td>\n",
              "      <td>3.47</td>\n",
              "      <td>9.0</td>\n",
              "      <td>154</td>\n",
              "      <td>5000</td>\n",
              "      <td>19</td>\n",
              "      <td>26</td>\n",
              "      <td>16500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>fwd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.8</td>\n",
              "      <td>176.6</td>\n",
              "      <td>66.2</td>\n",
              "      <td>54.3</td>\n",
              "      <td>2337</td>\n",
              "      <td>ohc</td>\n",
              "      <td>four</td>\n",
              "      <td>109</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>10.0</td>\n",
              "      <td>102</td>\n",
              "      <td>5500</td>\n",
              "      <td>24</td>\n",
              "      <td>30</td>\n",
              "      <td>13950</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>4wd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.4</td>\n",
              "      <td>176.6</td>\n",
              "      <td>66.4</td>\n",
              "      <td>54.3</td>\n",
              "      <td>2824</td>\n",
              "      <td>ohc</td>\n",
              "      <td>five</td>\n",
              "      <td>136</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>8.0</td>\n",
              "      <td>115</td>\n",
              "      <td>5500</td>\n",
              "      <td>18</td>\n",
              "      <td>22</td>\n",
              "      <td>17450</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   symboling normalized-losses         make  ... city-mpg highway-mpg  price\n",
              "0          3               NaN  alfa-romero  ...       21          27  13495\n",
              "1          3               NaN  alfa-romero  ...       21          27  16500\n",
              "2          1               NaN  alfa-romero  ...       19          26  16500\n",
              "3          2               164         audi  ...       24          30  13950\n",
              "4          2               164         audi  ...       18          22  17450\n",
              "\n",
              "[5 rows x 26 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oAHK1L5CqRC4"
      },
      "source": [
        "Identify_missing_values\n",
        "\n",
        "<h4>Evaluating for Missing Data</h4>\n",
        "\n",
        "The missing values are converted to Python's default. We use Python's built-in functions to identify these missing values. There are two methods to detect missing data:\n",
        "<ol>\n",
        "    <li><b>.isnull()</b></li>\n",
        "    <li><b>.notnull()</b></li>\n",
        "</ol>\n",
        "The output is a boolean value indicating whether the value that is passed into the argument is in fact missing data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 258
        },
        "id": "7kg6IwIpqRC5",
        "outputId": "514e3b03-99d1-4389-fc54-def4413dc799"
      },
      "source": [
        "missing_data = df.isnull()\n",
        "missing_data.head(5)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>symboling</th>\n",
              "      <th>normalized-losses</th>\n",
              "      <th>make</th>\n",
              "      <th>fuel-type</th>\n",
              "      <th>aspiration</th>\n",
              "      <th>num-of-doors</th>\n",
              "      <th>body-style</th>\n",
              "      <th>drive-wheels</th>\n",
              "      <th>engine-location</th>\n",
              "      <th>wheel-base</th>\n",
              "      <th>length</th>\n",
              "      <th>width</th>\n",
              "      <th>height</th>\n",
              "      <th>curb-weight</th>\n",
              "      <th>engine-type</th>\n",
              "      <th>num-of-cylinders</th>\n",
              "      <th>engine-size</th>\n",
              "      <th>fuel-system</th>\n",
              "      <th>bore</th>\n",
              "      <th>stroke</th>\n",
              "      <th>compression-ratio</th>\n",
              "      <th>horsepower</th>\n",
              "      <th>peak-rpm</th>\n",
              "      <th>city-mpg</th>\n",
              "      <th>highway-mpg</th>\n",
              "      <th>price</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>False</td>\n",
              "      <td>True</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>False</td>\n",
              "      <td>True</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>False</td>\n",
              "      <td>True</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   symboling  normalized-losses   make  ...  city-mpg  highway-mpg  price\n",
              "0      False               True  False  ...     False        False  False\n",
              "1      False               True  False  ...     False        False  False\n",
              "2      False               True  False  ...     False        False  False\n",
              "3      False              False  False  ...     False        False  False\n",
              "4      False              False  False  ...     False        False  False\n",
              "\n",
              "[5 rows x 26 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n5zv-I9KqRC6"
      },
      "source": [
        "\"True\" stands for missing value, while \"False\" stands for not missing value."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iqqfkv4kqRC6"
      },
      "source": [
        "<h4>Count missing values in each column</h4>\n",
        "<p>\n",
        "Using a for loop in Python, we can quickly figure out the number of missing values in each column. As mentioned above, \"True\" represents a missing value, \"False\"  means the value is present in the dataset.  In the body of the for loop the method  \".value_counts()\"  counts the number of \"True\" values. \n",
        "</p>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aj6y_9hoqRC7",
        "outputId": "8b5af0ad-7b22-49f4-a4ab-ecba985b8013"
      },
      "source": [
        "for column in missing_data.columns.values.tolist():\n",
        "    print(column)\n",
        "    print (missing_data[column].value_counts())\n",
        "    print(\"\")    "
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "symboling\n",
            "False    205\n",
            "Name: symboling, dtype: int64\n",
            "\n",
            "normalized-losses\n",
            "False    164\n",
            "True      41\n",
            "Name: normalized-losses, dtype: int64\n",
            "\n",
            "make\n",
            "False    205\n",
            "Name: make, dtype: int64\n",
            "\n",
            "fuel-type\n",
            "False    205\n",
            "Name: fuel-type, dtype: int64\n",
            "\n",
            "aspiration\n",
            "False    205\n",
            "Name: aspiration, dtype: int64\n",
            "\n",
            "num-of-doors\n",
            "False    203\n",
            "True       2\n",
            "Name: num-of-doors, dtype: int64\n",
            "\n",
            "body-style\n",
            "False    205\n",
            "Name: body-style, dtype: int64\n",
            "\n",
            "drive-wheels\n",
            "False    205\n",
            "Name: drive-wheels, dtype: int64\n",
            "\n",
            "engine-location\n",
            "False    205\n",
            "Name: engine-location, dtype: int64\n",
            "\n",
            "wheel-base\n",
            "False    205\n",
            "Name: wheel-base, dtype: int64\n",
            "\n",
            "length\n",
            "False    205\n",
            "Name: length, dtype: int64\n",
            "\n",
            "width\n",
            "False    205\n",
            "Name: width, dtype: int64\n",
            "\n",
            "height\n",
            "False    205\n",
            "Name: height, dtype: int64\n",
            "\n",
            "curb-weight\n",
            "False    205\n",
            "Name: curb-weight, dtype: int64\n",
            "\n",
            "engine-type\n",
            "False    205\n",
            "Name: engine-type, dtype: int64\n",
            "\n",
            "num-of-cylinders\n",
            "False    205\n",
            "Name: num-of-cylinders, dtype: int64\n",
            "\n",
            "engine-size\n",
            "False    205\n",
            "Name: engine-size, dtype: int64\n",
            "\n",
            "fuel-system\n",
            "False    205\n",
            "Name: fuel-system, dtype: int64\n",
            "\n",
            "bore\n",
            "False    201\n",
            "True       4\n",
            "Name: bore, dtype: int64\n",
            "\n",
            "stroke\n",
            "False    201\n",
            "True       4\n",
            "Name: stroke, dtype: int64\n",
            "\n",
            "compression-ratio\n",
            "False    205\n",
            "Name: compression-ratio, dtype: int64\n",
            "\n",
            "horsepower\n",
            "False    203\n",
            "True       2\n",
            "Name: horsepower, dtype: int64\n",
            "\n",
            "peak-rpm\n",
            "False    203\n",
            "True       2\n",
            "Name: peak-rpm, dtype: int64\n",
            "\n",
            "city-mpg\n",
            "False    205\n",
            "Name: city-mpg, dtype: int64\n",
            "\n",
            "highway-mpg\n",
            "False    205\n",
            "Name: highway-mpg, dtype: int64\n",
            "\n",
            "price\n",
            "False    201\n",
            "True       4\n",
            "Name: price, dtype: int64\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s0PMbFJMqRC8"
      },
      "source": [
        "Based on the summary above, each column has 205 rows of data, seven columns containing missing data:\n",
        "<ol>\n",
        "    <li>\"normalized-losses\": 41 missing data</li>\n",
        "    <li>\"num-of-doors\": 2 missing data</li>\n",
        "    <li>\"bore\": 4 missing data</li>\n",
        "    <li>\"stroke\" : 4 missing data</li>\n",
        "    <li>\"horsepower\": 2 missing data</li>\n",
        "    <li>\"peak-rpm\": 2 missing data</li>\n",
        "    <li>\"price\": 4 missing data</li>\n",
        "</ol>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BaMGGcubqRC9"
      },
      "source": [
        "<h3 id=\"deal_missing_values\">Deal with missing data</h3>\n",
        "<b>How to deal with missing data?</b>\n",
        "\n",
        "<ol>\n",
        "    <li>drop data<br>\n",
        "        a. drop the whole row<br>\n",
        "        b. drop the whole column\n",
        "    </li>\n",
        "    <li>replace data<br>\n",
        "        a. replace it by mean<br>\n",
        "        b. replace it by frequency<br>\n",
        "        c. replace it based on other functions\n",
        "    </li>\n",
        "</ol>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FlsNPAgGqRC-"
      },
      "source": [
        "Whole columns should be dropped only if most entries in the column are empty. In our dataset, none of the columns are empty enough to drop entirely.\n",
        "We have some freedom in choosing which method to replace data; however, some methods may seem more reasonable than others. We will apply each method to many different columns:\n",
        "\n",
        "<b>Replace by mean:</b>\n",
        "<ul>\n",
        "    <li>\"normalized-losses\": 41 missing data, replace them with mean</li>\n",
        "    <li>\"stroke\": 4 missing data, replace them with mean</li>\n",
        "    <li>\"bore\": 4 missing data, replace them with mean</li>\n",
        "    <li>\"horsepower\": 2 missing data, replace them with mean</li>\n",
        "    <li>\"peak-rpm\": 2 missing data, replace them with mean</li>\n",
        "</ul>\n",
        "\n",
        "<b>Replace by frequency:</b>\n",
        "<ul>\n",
        "    <li>\"num-of-doors\": 2 missing data, replace them with \"four\". \n",
        "        <ul>\n",
        "            <li>Reason: 84% sedans is four doors. Since four doors is most frequent, it is most likely to occur</li>\n",
        "        </ul>\n",
        "    </li>\n",
        "</ul>\n",
        "\n",
        "<b>Drop the whole row:</b>\n",
        "<ul>\n",
        "    <li>\"price\": 4 missing data, simply delete the whole row\n",
        "        <ul>\n",
        "            <li>Reason: price is what we want to predict. Any data entry without price data cannot be used for prediction; therefore any row now without price data is not useful to us</li>\n",
        "        </ul>\n",
        "    </li>\n",
        "</ul>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JHoWT9ZvqRDA"
      },
      "source": [
        "<h4>Calculate the average of the column </h4>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ot4C5H4nqRDC",
        "outputId": "2cddaa8c-8af3-4ac6-8152-2d2a5a540818"
      },
      "source": [
        "avg_norm_loss = df[\"normalized-losses\"].astype(\"float\").mean(axis=0)\n",
        "print(\"Average of normalized-losses:\", avg_norm_loss)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Average of normalized-losses: 122.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PAoFFquVqRDD"
      },
      "source": [
        "<h4>Replace \"NaN\" by mean value in \"normalized-losses\" column</h4>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "id": "P_fPRNGwqRDE"
      },
      "source": [
        "df[\"normalized-losses\"].replace(np.nan, avg_norm_loss, inplace=True)"
      ],
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iGlaJFPEqRDF"
      },
      "source": [
        "<h4>Calculate the mean value for 'bore' column</h4>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dVZUGpNBqRDG",
        "outputId": "7eb49990-309b-45c1-bf8c-b29336cf73d6"
      },
      "source": [
        "avg_bore=df['bore'].astype('float').mean(axis=0)\n",
        "print(\"Average of bore:\", avg_bore)"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Average of bore: 3.3297512437810957\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_KuZfwcwqRDG"
      },
      "source": [
        "<h4>Replace NaN by mean value</h4>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "id": "wx88OcK_qRDH"
      },
      "source": [
        "df[\"bore\"].replace(np.nan, avg_bore, inplace=True)"
      ],
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yS_2r65pqRDH"
      },
      "source": [
        "<div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
        "<h1> Question  #1: </h1>\n",
        "\n",
        "<b>According to the example above, replace NaN in \"stroke\" column by mean.</b>\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RkNpb25MqRDI",
        "outputId": "95f5603a-689a-465d-a926-46f6c15903c0"
      },
      "source": [
        "# Write your code below and press Shift+Enter to execute \n",
        "avg_stroke = df['stroke'].astype('float').mean(axis=0)\n",
        "print('Average of Stroke:', avg_stroke)\n",
        "\n",
        "df['stroke'].replace(np.nan, avg_stroke, inplace=True)"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Average of Stroke: 3.2554228855721337\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9YUHZm4LqRDP"
      },
      "source": [
        "Double-click <b>here</b> for the solution.\n",
        "\n",
        "<!-- The answer is below:\n",
        "\n",
        "# calculate the mean vaule for \"stroke\" column\n",
        "avg_stroke = df[\"stroke\"].astype(\"float\").mean(axis = 0)\n",
        "print(\"Average of stroke:\", avg_stroke)\n",
        "\n",
        "# replace NaN by mean value in \"stroke\" column\n",
        "df[\"stroke\"].replace(np.nan, avg_stroke, inplace = True)\n",
        "\n",
        "-->\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YKbNmt32qRDQ"
      },
      "source": [
        "<h4>Calculate the mean value for the  'horsepower' column:</h4>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hkHBhCmJqRDQ",
        "outputId": "15351066-0e14-4fb0-c26d-9720e91905de"
      },
      "source": [
        "avg_horsepower = df['horsepower'].astype('float').mean(axis=0)\n",
        "print(\"Average horsepower:\", avg_horsepower)"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Average horsepower: 104.25615763546799\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ekOvkWOMqRDR"
      },
      "source": [
        "<h4>Replace \"NaN\" by mean value:</h4>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "id": "uN70OkWrqRDR"
      },
      "source": [
        "df['horsepower'].replace(np.nan, avg_horsepower, inplace=True)"
      ],
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-qdA6UKFqRDS"
      },
      "source": [
        "<h4>Calculate the mean value for 'peak-rpm' column:</h4>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_3EkNJn1qRDS",
        "outputId": "5a5c747c-f22d-4ddb-9c31-4b0367536a42"
      },
      "source": [
        "avg_peakrpm=df['peak-rpm'].astype('float').mean(axis=0)\n",
        "print(\"Average peak rpm:\", avg_peakrpm)"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Average peak rpm: 5125.369458128079\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g8f9sbZlqRDT"
      },
      "source": [
        "<h4>Replace NaN by mean value:</h4>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "id": "4dWiydxoqRDU"
      },
      "source": [
        "df['peak-rpm'].replace(np.nan, avg_peakrpm, inplace=True)"
      ],
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y7uW0pbLqRDU"
      },
      "source": [
        "To see which values are present in a particular column, we can use the \".value_counts()\" method:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YomgT5dpqRDV",
        "outputId": "f4fd0e5d-ae94-47f5-ede4-b37828453d90"
      },
      "source": [
        "df['num-of-doors'].value_counts()"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "four    114\n",
              "two      89\n",
              "Name: num-of-doors, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nHE8VFMtqRDV"
      },
      "source": [
        "We can see that four doors are the most common type. We can also use the \".idxmax()\" method to calculate for us the most common type automatically:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "I2wkg0QuqRDW",
        "outputId": "1e242eae-2b69-48db-8da5-f73a07920c7d"
      },
      "source": [
        "df['num-of-doors'].value_counts().idxmax()"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'four'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KVmDLDTiqRDW"
      },
      "source": [
        "The replacement procedure is very similar to what we have seen previously"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4EW8aYHrqRDX"
      },
      "source": [
        "#replace the missing 'num-of-doors' values by the most frequent \n",
        "df[\"num-of-doors\"].replace(np.nan, \"four\", inplace=True)"
      ],
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y_f6FyoxqRDY"
      },
      "source": [
        "Finally, let's drop all rows that do not have price data:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "id": "L4DBcGTBqRDY"
      },
      "source": [
        "# simply drop whole row with NaN in \"price\" column\n",
        "df.dropna(subset=[\"price\"], axis=0, inplace=True)\n",
        "\n",
        "# reset index, because we droped two rows\n",
        "df.reset_index(drop=True, inplace=True)"
      ],
      "execution_count": 23,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 309
        },
        "id": "UKzpzxdNqRDZ",
        "outputId": "c57e40f6-a909-421d-9880-a413dd3c1402"
      },
      "source": [
        "df.head()"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>symboling</th>\n",
              "      <th>normalized-losses</th>\n",
              "      <th>make</th>\n",
              "      <th>fuel-type</th>\n",
              "      <th>aspiration</th>\n",
              "      <th>num-of-doors</th>\n",
              "      <th>body-style</th>\n",
              "      <th>drive-wheels</th>\n",
              "      <th>engine-location</th>\n",
              "      <th>wheel-base</th>\n",
              "      <th>length</th>\n",
              "      <th>width</th>\n",
              "      <th>height</th>\n",
              "      <th>curb-weight</th>\n",
              "      <th>engine-type</th>\n",
              "      <th>num-of-cylinders</th>\n",
              "      <th>engine-size</th>\n",
              "      <th>fuel-system</th>\n",
              "      <th>bore</th>\n",
              "      <th>stroke</th>\n",
              "      <th>compression-ratio</th>\n",
              "      <th>horsepower</th>\n",
              "      <th>peak-rpm</th>\n",
              "      <th>city-mpg</th>\n",
              "      <th>highway-mpg</th>\n",
              "      <th>price</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>3</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>168.8</td>\n",
              "      <td>64.1</td>\n",
              "      <td>48.8</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000</td>\n",
              "      <td>21</td>\n",
              "      <td>27</td>\n",
              "      <td>13495</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>168.8</td>\n",
              "      <td>64.1</td>\n",
              "      <td>48.8</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000</td>\n",
              "      <td>21</td>\n",
              "      <td>27</td>\n",
              "      <td>16500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>hatchback</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>94.5</td>\n",
              "      <td>171.2</td>\n",
              "      <td>65.5</td>\n",
              "      <td>52.4</td>\n",
              "      <td>2823</td>\n",
              "      <td>ohcv</td>\n",
              "      <td>six</td>\n",
              "      <td>152</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>2.68</td>\n",
              "      <td>3.47</td>\n",
              "      <td>9.0</td>\n",
              "      <td>154</td>\n",
              "      <td>5000</td>\n",
              "      <td>19</td>\n",
              "      <td>26</td>\n",
              "      <td>16500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>fwd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.8</td>\n",
              "      <td>176.6</td>\n",
              "      <td>66.2</td>\n",
              "      <td>54.3</td>\n",
              "      <td>2337</td>\n",
              "      <td>ohc</td>\n",
              "      <td>four</td>\n",
              "      <td>109</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>10.0</td>\n",
              "      <td>102</td>\n",
              "      <td>5500</td>\n",
              "      <td>24</td>\n",
              "      <td>30</td>\n",
              "      <td>13950</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>4wd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.4</td>\n",
              "      <td>176.6</td>\n",
              "      <td>66.4</td>\n",
              "      <td>54.3</td>\n",
              "      <td>2824</td>\n",
              "      <td>ohc</td>\n",
              "      <td>five</td>\n",
              "      <td>136</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>8.0</td>\n",
              "      <td>115</td>\n",
              "      <td>5500</td>\n",
              "      <td>18</td>\n",
              "      <td>22</td>\n",
              "      <td>17450</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   symboling normalized-losses         make  ... city-mpg highway-mpg  price\n",
              "0          3               122  alfa-romero  ...       21          27  13495\n",
              "1          3               122  alfa-romero  ...       21          27  16500\n",
              "2          1               122  alfa-romero  ...       19          26  16500\n",
              "3          2               164         audi  ...       24          30  13950\n",
              "4          2               164         audi  ...       18          22  17450\n",
              "\n",
              "[5 rows x 26 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2o5b25ziqRDZ"
      },
      "source": [
        "<b>Good!</b> Now, we obtain the dataset with no missing values."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XxQurAqRqRDa"
      },
      "source": [
        "<h3 id=\"correct_data_format\">Correct data format</h3>\n",
        "<b>We are almost there!</b>\n",
        "<p>The last step in data cleaning is checking and making sure that all data is in the correct format (int, float, text or other).</p>\n",
        "\n",
        "In Pandas, we use \n",
        "<p><b>.dtype()</b> to check the data type</p>\n",
        "<p><b>.astype()</b> to change the data type</p>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7N3ekFLMqRDa"
      },
      "source": [
        "<h4>Lets list the data types for each column</h4>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "j0-qKX8GqRDb",
        "outputId": "7d63bb89-2e61-49bf-bea4-69fc49ba09ab"
      },
      "source": [
        "df.dtypes"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "symboling              int64\n",
              "normalized-losses     object\n",
              "make                  object\n",
              "fuel-type             object\n",
              "aspiration            object\n",
              "num-of-doors          object\n",
              "body-style            object\n",
              "drive-wheels          object\n",
              "engine-location       object\n",
              "wheel-base           float64\n",
              "length               float64\n",
              "width                float64\n",
              "height               float64\n",
              "curb-weight            int64\n",
              "engine-type           object\n",
              "num-of-cylinders      object\n",
              "engine-size            int64\n",
              "fuel-system           object\n",
              "bore                  object\n",
              "stroke                object\n",
              "compression-ratio    float64\n",
              "horsepower            object\n",
              "peak-rpm              object\n",
              "city-mpg               int64\n",
              "highway-mpg            int64\n",
              "price                 object\n",
              "dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tIXQo3HBqRDb"
      },
      "source": [
        "<p>As we can see above, some columns are not of the correct data type. Numerical variables should have type 'float' or 'int', and variables with strings such as categories should have type 'object'. For example, 'bore' and 'stroke' variables are numerical values that describe the engines, so we should expect them to be of the type 'float' or 'int'; however, they are shown as type 'object'. We have to convert data types into a proper format for each column using the \"astype()\" method.</p> "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RXBf3qLsqRDc"
      },
      "source": [
        "<h4>Convert data types to proper format</h4>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6UpnvYt4qRDd"
      },
      "source": [
        "df[[\"bore\", \"stroke\"]] = df[[\"bore\", \"stroke\"]].astype(\"float\")\n",
        "df[[\"normalized-losses\"]] = df[[\"normalized-losses\"]].astype(\"int\")\n",
        "df[[\"price\"]] = df[[\"price\"]].astype(\"float\")\n",
        "df[[\"peak-rpm\"]] = df[[\"peak-rpm\"]].astype(\"float\")\n",
        "df[[\"horsepower\"]] = df[[\"horsepower\"]].astype(\"int\")"
      ],
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tEono_uyqRDd"
      },
      "source": [
        "<h4>Let us list the columns after the conversion</h4>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oF4ci9mXqRDe",
        "outputId": "36376b22-8aa1-42cd-bc0b-dc988f0c6c25"
      },
      "source": [
        "df.dtypes"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "symboling              int64\n",
              "normalized-losses      int64\n",
              "make                  object\n",
              "fuel-type             object\n",
              "aspiration            object\n",
              "num-of-doors          object\n",
              "body-style            object\n",
              "drive-wheels          object\n",
              "engine-location       object\n",
              "wheel-base           float64\n",
              "length               float64\n",
              "width                float64\n",
              "height               float64\n",
              "curb-weight            int64\n",
              "engine-type           object\n",
              "num-of-cylinders      object\n",
              "engine-size            int64\n",
              "fuel-system           object\n",
              "bore                 float64\n",
              "stroke               float64\n",
              "compression-ratio    float64\n",
              "horsepower             int64\n",
              "peak-rpm             float64\n",
              "city-mpg               int64\n",
              "highway-mpg            int64\n",
              "price                float64\n",
              "dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qww_h0jRqRDe"
      },
      "source": [
        "<b>Wonderful!</b>\n",
        "\n",
        "Now, we finally obtain the cleaned dataset with no missing values and all data in its proper format."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D0a3XwfGqRDf"
      },
      "source": [
        "<h2 id=\"data_standardization\">Data Standardization</h2>\n",
        "<p>\n",
        "Data is usually collected from different agencies with different formats.\n",
        "(Data Standardization is also a term for a particular type of data normalization, where we subtract the mean and divide by the standard deviation)\n",
        "</p>\n",
        "    \n",
        "<b>What is Standardization?</b>\n",
        "<p>Standardization is the process of transforming data into a common format which allows the researcher to make the meaningful comparison.\n",
        "</p>\n",
        "\n",
        "<b>Example</b>\n",
        "<p>Transform mpg to L/100km:</p>\n",
        "<p>In our dataset, the fuel consumption columns \"city-mpg\" and \"highway-mpg\" are represented by mpg (miles per gallon) unit. Assume we are developing an application in a country that accept the fuel consumption with L/100km standard</p>\n",
        "<p>We will need to apply <b>data transformation</b> to transform mpg into L/100km?</p>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cfFgLXS3qRDf"
      },
      "source": [
        "<p>The formula for unit conversion is<p>\n",
        "L/100km = 235 / mpg\n",
        "<p>We can do many mathematical operations directly in Pandas.</p>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 309
        },
        "id": "3THvjLNwqRDg",
        "outputId": "624f6763-8630-41f2-cc06-1b3e7c8be89d"
      },
      "source": [
        "df.head()"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>symboling</th>\n",
              "      <th>normalized-losses</th>\n",
              "      <th>make</th>\n",
              "      <th>fuel-type</th>\n",
              "      <th>aspiration</th>\n",
              "      <th>num-of-doors</th>\n",
              "      <th>body-style</th>\n",
              "      <th>drive-wheels</th>\n",
              "      <th>engine-location</th>\n",
              "      <th>wheel-base</th>\n",
              "      <th>length</th>\n",
              "      <th>width</th>\n",
              "      <th>height</th>\n",
              "      <th>curb-weight</th>\n",
              "      <th>engine-type</th>\n",
              "      <th>num-of-cylinders</th>\n",
              "      <th>engine-size</th>\n",
              "      <th>fuel-system</th>\n",
              "      <th>bore</th>\n",
              "      <th>stroke</th>\n",
              "      <th>compression-ratio</th>\n",
              "      <th>horsepower</th>\n",
              "      <th>peak-rpm</th>\n",
              "      <th>city-mpg</th>\n",
              "      <th>highway-mpg</th>\n",
              "      <th>price</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>3</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>168.8</td>\n",
              "      <td>64.1</td>\n",
              "      <td>48.8</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>21</td>\n",
              "      <td>27</td>\n",
              "      <td>13495.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>168.8</td>\n",
              "      <td>64.1</td>\n",
              "      <td>48.8</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>21</td>\n",
              "      <td>27</td>\n",
              "      <td>16500.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>hatchback</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>94.5</td>\n",
              "      <td>171.2</td>\n",
              "      <td>65.5</td>\n",
              "      <td>52.4</td>\n",
              "      <td>2823</td>\n",
              "      <td>ohcv</td>\n",
              "      <td>six</td>\n",
              "      <td>152</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>2.68</td>\n",
              "      <td>3.47</td>\n",
              "      <td>9.0</td>\n",
              "      <td>154</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>19</td>\n",
              "      <td>26</td>\n",
              "      <td>16500.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>fwd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.8</td>\n",
              "      <td>176.6</td>\n",
              "      <td>66.2</td>\n",
              "      <td>54.3</td>\n",
              "      <td>2337</td>\n",
              "      <td>ohc</td>\n",
              "      <td>four</td>\n",
              "      <td>109</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>10.0</td>\n",
              "      <td>102</td>\n",
              "      <td>5500.0</td>\n",
              "      <td>24</td>\n",
              "      <td>30</td>\n",
              "      <td>13950.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>4wd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.4</td>\n",
              "      <td>176.6</td>\n",
              "      <td>66.4</td>\n",
              "      <td>54.3</td>\n",
              "      <td>2824</td>\n",
              "      <td>ohc</td>\n",
              "      <td>five</td>\n",
              "      <td>136</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>8.0</td>\n",
              "      <td>115</td>\n",
              "      <td>5500.0</td>\n",
              "      <td>18</td>\n",
              "      <td>22</td>\n",
              "      <td>17450.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   symboling  normalized-losses         make  ... city-mpg highway-mpg    price\n",
              "0          3                122  alfa-romero  ...       21          27  13495.0\n",
              "1          3                122  alfa-romero  ...       21          27  16500.0\n",
              "2          1                122  alfa-romero  ...       19          26  16500.0\n",
              "3          2                164         audi  ...       24          30  13950.0\n",
              "4          2                164         audi  ...       18          22  17450.0\n",
              "\n",
              "[5 rows x 26 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 309
        },
        "id": "9R5aE1sKqRDg",
        "outputId": "1279146f-db3e-4b09-f0f7-2881767a076b"
      },
      "source": [
        "# Convert mpg to L/100km by mathematical operation (235 divided by mpg)\n",
        "df['city-L/100km'] = 235/df[\"city-mpg\"]\n",
        "\n",
        "# check your transformed data \n",
        "df.head()"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>symboling</th>\n",
              "      <th>normalized-losses</th>\n",
              "      <th>make</th>\n",
              "      <th>fuel-type</th>\n",
              "      <th>aspiration</th>\n",
              "      <th>num-of-doors</th>\n",
              "      <th>body-style</th>\n",
              "      <th>drive-wheels</th>\n",
              "      <th>engine-location</th>\n",
              "      <th>wheel-base</th>\n",
              "      <th>length</th>\n",
              "      <th>width</th>\n",
              "      <th>height</th>\n",
              "      <th>curb-weight</th>\n",
              "      <th>engine-type</th>\n",
              "      <th>num-of-cylinders</th>\n",
              "      <th>engine-size</th>\n",
              "      <th>fuel-system</th>\n",
              "      <th>bore</th>\n",
              "      <th>stroke</th>\n",
              "      <th>compression-ratio</th>\n",
              "      <th>horsepower</th>\n",
              "      <th>peak-rpm</th>\n",
              "      <th>city-mpg</th>\n",
              "      <th>highway-mpg</th>\n",
              "      <th>price</th>\n",
              "      <th>city-L/100km</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>3</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>168.8</td>\n",
              "      <td>64.1</td>\n",
              "      <td>48.8</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>21</td>\n",
              "      <td>27</td>\n",
              "      <td>13495.0</td>\n",
              "      <td>11.190476</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>168.8</td>\n",
              "      <td>64.1</td>\n",
              "      <td>48.8</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>21</td>\n",
              "      <td>27</td>\n",
              "      <td>16500.0</td>\n",
              "      <td>11.190476</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>hatchback</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>94.5</td>\n",
              "      <td>171.2</td>\n",
              "      <td>65.5</td>\n",
              "      <td>52.4</td>\n",
              "      <td>2823</td>\n",
              "      <td>ohcv</td>\n",
              "      <td>six</td>\n",
              "      <td>152</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>2.68</td>\n",
              "      <td>3.47</td>\n",
              "      <td>9.0</td>\n",
              "      <td>154</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>19</td>\n",
              "      <td>26</td>\n",
              "      <td>16500.0</td>\n",
              "      <td>12.368421</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>fwd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.8</td>\n",
              "      <td>176.6</td>\n",
              "      <td>66.2</td>\n",
              "      <td>54.3</td>\n",
              "      <td>2337</td>\n",
              "      <td>ohc</td>\n",
              "      <td>four</td>\n",
              "      <td>109</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>10.0</td>\n",
              "      <td>102</td>\n",
              "      <td>5500.0</td>\n",
              "      <td>24</td>\n",
              "      <td>30</td>\n",
              "      <td>13950.0</td>\n",
              "      <td>9.791667</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>4wd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.4</td>\n",
              "      <td>176.6</td>\n",
              "      <td>66.4</td>\n",
              "      <td>54.3</td>\n",
              "      <td>2824</td>\n",
              "      <td>ohc</td>\n",
              "      <td>five</td>\n",
              "      <td>136</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>8.0</td>\n",
              "      <td>115</td>\n",
              "      <td>5500.0</td>\n",
              "      <td>18</td>\n",
              "      <td>22</td>\n",
              "      <td>17450.0</td>\n",
              "      <td>13.055556</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   symboling  normalized-losses  ...    price city-L/100km\n",
              "0          3                122  ...  13495.0    11.190476\n",
              "1          3                122  ...  16500.0    11.190476\n",
              "2          1                122  ...  16500.0    12.368421\n",
              "3          2                164  ...  13950.0     9.791667\n",
              "4          2                164  ...  17450.0    13.055556\n",
              "\n",
              "[5 rows x 27 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B7XTh1GYqRDh"
      },
      "source": [
        "<div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
        "<h1> Question  #2: </h1>\n",
        "\n",
        "<b>According to the example above, transform mpg to L/100km in the column of \"highway-mpg\", and change the name of column to \"highway-L/100km\".</b>\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 309
        },
        "id": "cOKxr4KJqRDh",
        "outputId": "36f5ce51-9bef-4141-f9f3-9846fc29d185"
      },
      "source": [
        "# Write your code below and press Shift+Enter to execute \n",
        "df['highway-mpg'] = 235/df['highway-mpg']\n",
        "df.rename(columns={'highway-mpg':'highway-L/100km'}, inplace=True)\n",
        "\n",
        "df.head()"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>symboling</th>\n",
              "      <th>normalized-losses</th>\n",
              "      <th>make</th>\n",
              "      <th>fuel-type</th>\n",
              "      <th>aspiration</th>\n",
              "      <th>num-of-doors</th>\n",
              "      <th>body-style</th>\n",
              "      <th>drive-wheels</th>\n",
              "      <th>engine-location</th>\n",
              "      <th>wheel-base</th>\n",
              "      <th>length</th>\n",
              "      <th>width</th>\n",
              "      <th>height</th>\n",
              "      <th>curb-weight</th>\n",
              "      <th>engine-type</th>\n",
              "      <th>num-of-cylinders</th>\n",
              "      <th>engine-size</th>\n",
              "      <th>fuel-system</th>\n",
              "      <th>bore</th>\n",
              "      <th>stroke</th>\n",
              "      <th>compression-ratio</th>\n",
              "      <th>horsepower</th>\n",
              "      <th>peak-rpm</th>\n",
              "      <th>city-mpg</th>\n",
              "      <th>highway-L/100km</th>\n",
              "      <th>price</th>\n",
              "      <th>city-L/100km</th>\n",
              "      <th>highway-L/100km</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>3</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>168.8</td>\n",
              "      <td>64.1</td>\n",
              "      <td>48.8</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>21</td>\n",
              "      <td>8.703704</td>\n",
              "      <td>13495.0</td>\n",
              "      <td>11.190476</td>\n",
              "      <td>8.703704</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>168.8</td>\n",
              "      <td>64.1</td>\n",
              "      <td>48.8</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>21</td>\n",
              "      <td>8.703704</td>\n",
              "      <td>16500.0</td>\n",
              "      <td>11.190476</td>\n",
              "      <td>8.703704</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>hatchback</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>94.5</td>\n",
              "      <td>171.2</td>\n",
              "      <td>65.5</td>\n",
              "      <td>52.4</td>\n",
              "      <td>2823</td>\n",
              "      <td>ohcv</td>\n",
              "      <td>six</td>\n",
              "      <td>152</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>2.68</td>\n",
              "      <td>3.47</td>\n",
              "      <td>9.0</td>\n",
              "      <td>154</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>19</td>\n",
              "      <td>9.038462</td>\n",
              "      <td>16500.0</td>\n",
              "      <td>12.368421</td>\n",
              "      <td>9.038462</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>fwd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.8</td>\n",
              "      <td>176.6</td>\n",
              "      <td>66.2</td>\n",
              "      <td>54.3</td>\n",
              "      <td>2337</td>\n",
              "      <td>ohc</td>\n",
              "      <td>four</td>\n",
              "      <td>109</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>10.0</td>\n",
              "      <td>102</td>\n",
              "      <td>5500.0</td>\n",
              "      <td>24</td>\n",
              "      <td>7.833333</td>\n",
              "      <td>13950.0</td>\n",
              "      <td>9.791667</td>\n",
              "      <td>7.833333</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>gas</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>4wd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.4</td>\n",
              "      <td>176.6</td>\n",
              "      <td>66.4</td>\n",
              "      <td>54.3</td>\n",
              "      <td>2824</td>\n",
              "      <td>ohc</td>\n",
              "      <td>five</td>\n",
              "      <td>136</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>8.0</td>\n",
              "      <td>115</td>\n",
              "      <td>5500.0</td>\n",
              "      <td>18</td>\n",
              "      <td>10.681818</td>\n",
              "      <td>17450.0</td>\n",
              "      <td>13.055556</td>\n",
              "      <td>10.681818</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   symboling  normalized-losses  ... city-L/100km highway-L/100km\n",
              "0          3                122  ...    11.190476        8.703704\n",
              "1          3                122  ...    11.190476        8.703704\n",
              "2          1                122  ...    12.368421        9.038462\n",
              "3          2                164  ...     9.791667        7.833333\n",
              "4          2                164  ...    13.055556       10.681818\n",
              "\n",
              "[5 rows x 28 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O56tEDadqRDi"
      },
      "source": [
        "Double-click <b>here</b> for the solution.\n",
        "\n",
        "<!-- The answer is below:\n",
        "\n",
        "# transform mpg to L/100km by mathematical operation (235 divided by mpg)\n",
        "df[\"highway-mpg\"] = 235/df[\"highway-mpg\"]\n",
        "\n",
        "# rename column name from \"highway-mpg\" to \"highway-L/100km\"\n",
        "df.rename(columns={'\"highway-mpg\"':'highway-L/100km'}, inplace=True)\n",
        "\n",
        "# check your transformed data \n",
        "df.head()\n",
        "\n",
        "-->\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VcFPIJTsqRDi"
      },
      "source": [
        "<h2 id=\"data_normalization\">Data Normalization</h2>\n",
        "\n",
        "<b>Why normalization?</b>\n",
        "<p>Normalization is the process of transforming values of several variables into a similar range. Typical normalizations include scaling the variable so the variable average is 0, scaling the variable so the variance is 1, or scaling variable so the variable values range from 0 to 1\n",
        "</p>\n",
        "\n",
        "<b>Example</b>\n",
        "<p>To demonstrate normalization, let's say we want to scale the columns \"length\", \"width\" and \"height\" </p>\n",
        "<p><b>Target:</b>would like to Normalize those variables so their value ranges from 0 to 1.</p>\n",
        "<p><b>Approach:</b> replace original value by (original value)/(maximum value)</p>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yApD9pACqRDj"
      },
      "source": [
        "# replace (original value) by (original value)/(maximum value)\n",
        "df['length'] = df['length']/df['length'].max()\n",
        "df['width'] = df['width']/df['width'].max()"
      ],
      "execution_count": 32,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4TjsPJgTqRDk"
      },
      "source": [
        "<div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
        "<h1> Questiont #3: </h1>\n",
        "\n",
        "<b>According to the example above, normalize the column \"height\".</b>\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "Rk6tzvguqRDk",
        "outputId": "75cbe6b5-6f87-4bba-fda7-a2ad3ad29e55"
      },
      "source": [
        "# Write your code below and press Shift+Enter to execute \n",
        "df['height'] = df['height'] / df['height'].max()\n",
        "df[[\"length\",\"width\",\"height\"]].head()"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>length</th>\n",
              "      <th>width</th>\n",
              "      <th>height</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.811148</td>\n",
              "      <td>0.890278</td>\n",
              "      <td>0.816054</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.811148</td>\n",
              "      <td>0.890278</td>\n",
              "      <td>0.816054</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.822681</td>\n",
              "      <td>0.909722</td>\n",
              "      <td>0.876254</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.848630</td>\n",
              "      <td>0.919444</td>\n",
              "      <td>0.908027</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.848630</td>\n",
              "      <td>0.922222</td>\n",
              "      <td>0.908027</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     length     width    height\n",
              "0  0.811148  0.890278  0.816054\n",
              "1  0.811148  0.890278  0.816054\n",
              "2  0.822681  0.909722  0.876254\n",
              "3  0.848630  0.919444  0.908027\n",
              "4  0.848630  0.922222  0.908027"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_g5PNajPqRDk"
      },
      "source": [
        "Double-click <b>here</b> for the solution.\n",
        "\n",
        "<!-- The answer is below:\n",
        "\n",
        "df['height'] = df['height']/df['height'].max() \n",
        "# show the scaled columns\n",
        "df[[\"length\",\"width\",\"height\"]].head()\n",
        "\n",
        "-->"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i4vcn2WLqRDl"
      },
      "source": [
        "Here we can see, we've normalized \"length\", \"width\" and \"height\" in the range of [0,1]."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m7CPZuEwqRDl"
      },
      "source": [
        "<h2 id=\"binning\">Binning</h2>\n",
        "<b>Why binning?</b>\n",
        "<p>\n",
        "    Binning is a process of transforming continuous numerical variables into discrete categorical 'bins', for grouped analysis.\n",
        "</p>\n",
        "\n",
        "<b>Example: </b>\n",
        "<p>In our dataset, \"horsepower\" is a real valued variable ranging from 48 to 288, it has 57 unique values. What if we only care about the price difference between cars with high horsepower, medium horsepower, and little horsepower (3 types)? Can we rearrange them into three ‘bins' to simplify analysis? </p>\n",
        "\n",
        "<p>We will use the Pandas method 'cut' to segment the 'horsepower' column into 3 bins </p>\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dx3splqTqRDm"
      },
      "source": [
        "<h3>Example of Binning Data In Pandas</h3>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dr8Wqo6xqRDm"
      },
      "source": [
        " Convert data to correct format "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "89G3ycjGqRDn"
      },
      "source": [
        "df[\"horsepower\"]=df[\"horsepower\"].astype(int, copy=True)"
      ],
      "execution_count": 36,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FxUegaDaqRDn"
      },
      "source": [
        "Lets plot the histogram of horspower, to see what the distribution of horsepower looks like."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 312
        },
        "id": "OygwFObRqRDo",
        "outputId": "bd2047cd-b63c-41b4-c1fa-7550b0762162"
      },
      "source": [
        "%matplotlib inline\n",
        "import matplotlib as plt\n",
        "from matplotlib import pyplot\n",
        "plt.pyplot.hist(df[\"horsepower\"])\n",
        "\n",
        "# set x/y labels and plot title\n",
        "plt.pyplot.xlabel(\"horsepower\")\n",
        "plt.pyplot.ylabel(\"count\")\n",
        "plt.pyplot.title(\"horsepower bins\")"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'horsepower bins')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 37
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEWCAYAAABhffzLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAVWElEQVR4nO3de7BlZX3m8e9DA165SocgGBsVnUHLoHYQxhuDiRdwAmPUaKnpmJ6gU0ppRSd2kknGpHQGNPFujYWDsU1UwBiECTUqIjRqItIoIpcYEBuFAN0gCjoTFfjNH+vtsOk+p/uAvfY+fd7vp2rXWde9fuft1c9+z9prvztVhSSpH7vMugBJ0nQZ/JLUGYNfkjpj8EtSZwx+SeqMwS9JnTH4NTNJNiT51VnXsdglOSrJ9dtY/8EkfzzNmrRz23XWBUj6+VTVa2Zdg3Yu9vi100uyZDowS+l30eJl8GvWDktyWZIfJjk9yQM3r0jyu0muSfL9JGcnefjEukry2iRXA1dn8K4kG5PcnuSbSZ7Qtn1Akj9P8t0kN7dLIw9q645Kcn2SP0xyS7v89PKJ4+yV5KNJNiW5Lsl/TbJLW3ddkqe06Ze3mh7f5lcn+XSb3iXJmiTfTnJrkjOS7NvWrWj7rU7yXeAL8zXUNmr8SJK3bvH7vLG1xY1JXjWx7TFJrkxyR5Ibkrzp5/nH087J4NesvQR4HnAw8ETgtwGSHA38j7b+AOA64LQt9j0eeCpwKPAc4JnAY4G92n63tu1OassPAx4DHAj8ycTz/CKwX1u+CjglyePauve153sU8Czgt4DNQboOOKpNPwu4ttWweX5dmz6x1fos4OHAbcAHtvhdngX8W+C5c7TR9mqca9u92rargQ8k2aetOxV4dVXtATyBbbzQaAmrKh8+ZvIANgCvmJh/O/DBNn0q8PaJdQ8FfgasaPMFHD2x/mjgn4AjgF0mlgf4MfDoiWVHAt9p00cBdwIPmVh/BvDHwDLgp8ChE+teDVzQplcDZ7fpq4D/BJzW5q8Dnjyx7tkTz3FA+112BVa03+VR22ineWts0x8B3jqx7f8Ddp3YdiNwRJv+bvsd9pz1v7+P2T3s8WvWbpqY/r8MAQ9Dz/i6zSuq6kcMPfgDJ7b/3sT6LwDvZ+hJb0xySpI9geXAg4FLkvwgyQ+Az7Tlm91WVT+emL+uHX8/YLfJOtr05hrWAc9IcgDDi8QZwNOSrGDocV/atnskcObE8a8C7gL2n+t3mcd8Nc7l1qq6c2J+sl1/AzgGuC7JuiRHbue4WoIMfi1W/8wQmAAkeQjwMOCGiW3uNbRsVb23qp7CcOnnscB/AW5h6AE/vqr2bo+9quqhE7vu055/s19qx7+FoWf+yC3W3dCOdw1DqJ4IXFhVtzO8kJ0AfKmq7m77fA94/sTx966qB1bVvL/LHOar8T6pqour6jjgF4BPM7xYqTMGvxarTwCvSnJYkgcA/x24qKo2zLVxkl9J8tQkuzFc2vkX4O4Wvh8C3pXkF9q2BybZ8lr6nybZPckzgBcAn6yquxiC8W1J9kjySOD3gL+e2G8d8DruuZ5/wRbzAB9sz/HIdvzlSY67H22yVY33Zee278uT7FVVPwNuB+7e3n5aegx+LUpV9XmG6+yfAm4EHg28dBu77MkQ8LcxXAa5FXhHW/dm4BrgK0luBz4PTL4xelPb75+BjwGvqap/bOtOZHghuRb4EvBx4MMT+64D9gAunGce4D3A2cDnktwBfIXhTen7Yls13hevBDa0dngN8PLtbK8lKFV+EYv6leQo4K+r6qBZ1yJNiz1+SeqMwS9JnfFSjyR1ZtRxQZJsAO5guGf5zqpa2T6qfjrDB1c2AC+pqtvGrEOSdI9Re/wt+FdW1S0Ty94OfL+qTkqyBtinqt68refZb7/9asWKFaPVKUlL0SWXXHJLVS3fcvksRgI8jnvGN1nLcN/zNoN/xYoVrF+/ftyqJGmJSXLdXMvHfnO3GO5dviTJCW3Z/lV1Y5u+iXt/bP1fJTkhyfok6zdt2jRymZLUj7F7/E+vqhvaJybPTXKvD5xUVSWZ81pTVZ0CnAKwcuVK34GWpB1k1B7/5rFIqmojcCZwOHBzG9SK9nPjmDVIku5ttOBP8pAke2yeZhgv/XKGj66vaputAs4aqwZJ0tbGvNSzP8NQtJuP8/Gq+kySi4EzkqxmGFPlJSPWIEnawmjBX1XXAr88x/JbgWePdVxJ0rY5ZIMkdcbgl6TOGPyS1JlZfHJXI1ux5pyZHHfDScfO5LiS7ht7/JLUGYNfkjpj8EtSZwx+SeqMwS9JnTH4JakzBr8kdcbgl6TOGPyS1BmDX5I6Y/BLUmcMfknqjIO0jWRWA6VJ0vbY45ekzhj8ktQZg1+SOrPkr/F7rV2S7s0evyR1xuCXpM4Y/JLUGYNfkjpj8EtSZwx+SeqMwS9JnTH4JakzBr8kdcbgl6TOGPyS1BmDX5I6Y/BLUmdGD/4ky5J8PcnftfmDk1yU5JokpyfZfewaJEn3mEaP//XAVRPzJwPvqqrHALcBq6dQgySpGTX4kxwEHAv8rzYf4Gjgb9oma4Hjx6xBknRvY/f43w38PnB3m38Y8IOqurPNXw8cONeOSU5Isj7J+k2bNo1cpiT1Y7TgT/ICYGNVXXJ/9q+qU6pqZVWtXL58+Q6uTpL6NeZXLz4N+PUkxwAPBPYE3gPsnWTX1us/CLhhxBokSVsYrcdfVX9QVQdV1QrgpcAXqurlwPnAi9pmq4CzxqpBkrS1WdzH/2bg95Jcw3DN/9QZ1CBJ3RrzUs+/qqoLgAva9LXA4dM4riRpa35yV5I6Y/BLUmcMfknqjMEvSZ0x+CWpMwa/JHXG4Jekzhj8ktQZg1+SOmPwS1JnDH5J6ozBL0mdMfglqTMGvyR1xuCXpM4Y/JLUGYNfkjpj8EtSZwx+SeqMwS9JnTH4JakzBr8kdcbgl6TOGPyS1BmDX5I6Y/BLUmcMfknqjMEvSZ0x+CWpMwa/JHXG4Jekzhj8ktQZg1+SOmPwS1JnRgv+JA9M8tUk30hyRZI/bcsPTnJRkmuSnJ5k97FqkCRtbcwe/0+Ao6vql4HDgOclOQI4GXhXVT0GuA1YPWINkqQtjBb8NfhRm92tPQo4GvibtnwtcPxYNUiStjbqNf4ky5JcCmwEzgW+Dfygqu5sm1wPHDjPvickWZ9k/aZNm8YsU5K6MmrwV9VdVXUYcBBwOPBv7sO+p1TVyqpauXz58tFqlKTeTOWunqr6AXA+cCSwd5Jd26qDgBumUYMkaTDmXT3Lk+zdph8E/BpwFcMLwIvaZquAs8aqQZK0tV23v8n9dgCwNskyhheYM6rq75JcCZyW5K3A14FTR6xBkrSF0YK/qi4DnjTH8msZrvdLkmbAT+5KUmcMfknqjMEvSZ0x+CWpMwsK/iTnLWSZJGnx2+ZdPUkeCDwY2C/JPkDaqj2ZZ6gFSdLitr3bOV8NvAF4OHAJ9wT/7cD7R6xLkjSSbQZ/Vb0HeE+SE6vqfVOqSZI0ogV9gKuq3pfk3wErJvepqo+OVJckaSQLCv4kfwU8GrgUuKstLsDgl6SdzEKHbFgJHFpVNWYxkqTxLfQ+/suBXxyzEEnSdCy0x78fcGWSrzJ8ly4AVfXro1QlSRrNQoP/LWMWIUmanoXe1bNu7EIkSdOx0Lt67mC4iwdgd2A34MdVtedYhUmSxrHQHv8em6eTBDgOOGKsoiRJ47nPo3PW4NPAc0eoR5I0soVe6nnhxOwuDPf1/8soFUmSRrXQu3r+w8T0ncAGhss9kqSdzEKv8b9q7EK081ux5pyZHXvDScfO7NjSzmahX8RyUJIzk2xsj08lOWjs4iRJO95C39z9S+BshnH5Hw7877ZMkrSTWWjwL6+qv6yqO9vjI8DyEeuSJI1kocF/a5JXJFnWHq8Abh2zMEnSOBYa/L8DvAS4CbgReBHw2yPVJEka0UJv5/wzYFVV3QaQZF/gzxleECRJO5GF9vifuDn0Aarq+8CTxilJkjSmhQb/Lkn22TzTevwL/WtBkrSILDS8/wL4hySfbPMvBt42TkmSpDEt9JO7H02yHji6LXphVV05XlmSpLEs+HJNC3rDXpJ2cvd5WGZJ0s7N4JekzowW/EkekeT8JFcmuSLJ69vyfZOcm+Tq9nOf7T2XJGnHGbPHfyfwxqo6lOFrGl+b5FBgDXBeVR0CnNfmJUlTMlrwV9WNVfW1Nn0HcBVwIMMXuKxtm60Fjh+rBknS1qZyjT/JCoZP+l4E7F9VN7ZVNwH7T6MGSdJg9OBP8lDgU8Abqur2yXVVVUDNs98JSdYnWb9p06axy5Skbowa/El2Ywj9j1XV37bFNyc5oK0/ANg4175VdUpVrayqlcuXO/S/JO0oY97VE+BU4KqqeufEqrOBVW16FXDWWDVIkrY25kBrTwNeCXwzyaVt2R8CJwFnJFkNXMcwzr8kaUpGC/6q+hKQeVY/e6zjSpK2zU/uSlJnDH5J6ozBL0mdMfglqTMGvyR1xuCXpM4Y/JLUGYNfkjpj8EtSZwx+SeqMwS9JnTH4JakzY47OKS15K9acM7Njbzjp2JkdWzs3e/yS1BmDX5I6Y/BLUmcMfknqjMEvSZ0x+CWpMwa/JHXG+/i1JMzyfnppZ2OPX5I6Y/BLUmcMfknqjMEvSZ0x+CWpMwa/JHXG4Jekzhj8ktQZg1+SOmPwS1JnDH5J6ozBL0mdMfglqTOjBX+SDyfZmOTyiWX7Jjk3ydXt5z5jHV+SNLcxe/wfAZ63xbI1wHlVdQhwXpuXJE3RaMFfVRcC399i8XHA2ja9Fjh+rONLkuY27Wv8+1fVjW36JmD/KR9fkro3szd3q6qAmm99khOSrE+yftOmTVOsTJKWtmkH/81JDgBoPzfOt2FVnVJVK6tq5fLly6dWoCQtddMO/rOBVW16FXDWlI8vSd0b83bOTwD/ADwuyfVJVgMnAb+W5GrgV9u8JGmKdh3riavqZfOsevZYx5QkbZ+f3JWkzhj8ktQZg1+SOmPwS1JnDH5J6ozBL0mdMfglqTMGvyR1xuCXpM4Y/JLUGYNfkjpj8EtSZwx+SeqMwS9JnTH4JakzBr8kdWa0L2KRNK4Va86ZyXE3nHTsTI6rHccevyR1xuCXpM4Y/JLUGYNfkjrjm7uS7pNZvakMvrG8o9jjl6TOGPyS1BmDX5I6Y/BLUmcMfknqjMEvSZ0x+CWpMwa/JHXG4Jekzhj8ktQZg1+SOuNYPZK0HUvtS2/s8UtSZ2YS/Emel+RbSa5JsmYWNUhSr6Ye/EmWAR8Ang8cCrwsyaHTrkOSejWLHv/hwDVVdW1V/RQ4DThuBnVIUpdm8ebugcD3JuavB5665UZJTgBOaLM/SvKtHXDs/YBbdsDzLEW2zfxsm/lNtW1y8rSOtEP83G2zA37fR861cNHe1VNVpwCn7MjnTLK+qlbuyOdcKmyb+dk287Nt5reY22YWl3puAB4xMX9QWyZJmoJZBP/FwCFJDk6yO/BS4OwZ1CFJXZr6pZ6qujPJ64DPAsuAD1fVFVM6/A69dLTE2Dbzs23mZ9vMb9G2Tapq1jVIkqbIT+5KUmcMfknqzJIN/iQbknwzyaVJ1rdl+yY5N8nV7ec+s65zWpJ8OMnGJJdPLJuzPTJ4bxtS47IkT55d5eObp23ekuSGdv5cmuSYiXV/0NrmW0meO5uqpyPJI5Kcn+TKJFckeX1b3v25s422WfznTlUtyQewAdhvi2VvB9a06TXAybOuc4rt8UzgycDl22sP4Bjg/wABjgAumnX9M2ibtwBvmmPbQ4FvAA8ADga+DSyb9e8wYtscADy5Te8B/FNrg+7PnW20zaI/d5Zsj38exwFr2/Ra4PgZ1jJVVXUh8P0tFs/XHscBH63BV4C9kxwwnUqnb562mc9xwGlV9ZOq+g5wDcMwJEtSVd1YVV9r03cAVzF8+r77c2cbbTOfRXPuLOXgL+BzSS5pwz8A7F9VN7bpm4D9Z1PaojFfe8w1rMa2Tuil6nXtcsWHJy4Ldts2SVYATwIuwnPnXrZoG1jk585SDv6nV9WTGUYBfW2SZ06urOFvL+9lbWyPrfxP4NHAYcCNwF/MtpzZSvJQ4FPAG6rq9sl1vZ87c7TNoj93lmzwV9UN7edG4EyGP6lu3vxnZ/u5cXYVLgrztUf3w2pU1c1VdVdV3Q18iHv+JO+ubZLsxhBsH6uqv22LPXeYu212hnNnSQZ/kock2WPzNPAc4HKGoSFWtc1WAWfNpsJFY772OBv4rXaHxhHADyf+rO/CFtel/yPD+QND27w0yQOSHAwcAnx12vVNS5IApwJXVdU7J1Z1f+7M1zY7xbkz63fGx3gAj2J49/wbwBXAH7XlDwPOA64GPg/sO+tap9gmn2D4s/NnDNcWV8/XHgx3ZHyA4a6DbwIrZ13/DNrmr9rvfhnDf9gDJrb/o9Y23wKeP+v6R26bpzNcxrkMuLQ9jvHc2WbbLPpzxyEbJKkzS/JSjyRpfga/JHXG4Jekzhj8ktQZg1+SOmPwa0lJsmJylE1JWzP4pSbJ1L+K9P7YWerU4mXwaylaluRDbYz0zyV5UJLDknylDZx15sT48RckeXf7zobXJ3lxksuTfCPJhW2bZUnekeTitv+r2/KjklyY5Jw2vvoHk+zS1r0sw/dBXJ7k5LbsxUne2aZfn+TaNv2oJF9u009Jsq4NLvjZiWER7lXndJtTS409By1FhwAvq6rfTXIG8BvA7wMnVtW6JH8G/DfgDW373atqJUCSbwLPraobkuzd1q9mGHrgV5I8APhyks+1dYczjLN+HfAZ4IVJ/h44GXgKcBvDKLHHA19sdQA8A7g1yYFt+sI27sv7gOOqalOS3wTeBvzOlnVKPw+DX0vRd6rq0jZ9CcNIiXtX1bq2bC3wyYntT5+Y/jLwkfaCsXlAsucAT0zyoja/F8OLy0+Br1bV5p77Jxg+xv8z4IKq2tSWfwx4ZlV9OslD2zhSjwA+zvAlMM9ox3oc8ATg3GEYGJYxDCUxV53S/Wbwayn6ycT0XcDe823Y/HjzRFW9JslTgWOBS5I8hWH8mROr6rOTOyU5iq2HI97eGCh/D7yKYayWLzL05o8E3gj8EnBFVR25vTqln4fX+NWDHwK3JXlGm38lsG6uDZM8uqouqqo/ATYx9Mw/C/zndimGJI9to74CHJ7k4HZt/zeBLzGMuPisJPslWQa8bOJ4XwTeBFwIfB3498BPquqHDC8Gy5Mc2Y6zW5LH77hmkAb2+NWLVcAHkzwYuJah1z2XdyQ5hKGXfx7DCK+XASuAr7WheDdxz1cNXgy8H3gMcD5wZlXdnWRNmw9wTlVtHrb4iwwvJhdW1V1Jvgf8I0BV/bRdTnpvkr0Y/n++m2GEWWmHcXRO6X5ql3reVFUvmHUt0n3hpR5J6ow9fknqjD1+SeqMwS9JnTH4JakzBr8kdcbgl6TO/H/tQAhv+ZEEugAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "03WiIkP_qRDo"
      },
      "source": [
        "<p>We would like 3 bins of equal size bandwidth so we use numpy's <code>linspace(start_value, end_value, numbers_generated</code> function.</p>\n",
        "<p>Since we want to include the minimum value of horsepower we want to set start_value=min(df[\"horsepower\"]).</p>\n",
        "<p>Since we want to include the maximum value of horsepower we want to set end_value=max(df[\"horsepower\"]).</p>\n",
        "<p>Since we are building 3 bins of equal length, there should be 4 dividers, so numbers_generated=4.</p>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hkNX1NCyqRDp"
      },
      "source": [
        "We build a bin array, with a minimum value to a maximum value, with bandwidth calculated above. The bins will be values used to determine when one bin ends and another begins."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c3Kqdl2eqRDp",
        "outputId": "8b51cbda-1c06-46ec-a328-6554717c9ce9"
      },
      "source": [
        "bins = np.linspace(min(df[\"horsepower\"]), max(df[\"horsepower\"]), 4)\n",
        "bins"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 48.        , 119.33333333, 190.66666667, 262.        ])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pJJ4h9nzqRDq"
      },
      "source": [
        " We set group  names:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "id": "OTdtS4c8qRDq"
      },
      "source": [
        "group_names = ['Low', 'Medium', 'High']"
      ],
      "execution_count": 39,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7qBEJetiqRDr"
      },
      "source": [
        " We apply the function \"cut\" the determine what each value of \"df['horsepower']\" belongs to. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 669
        },
        "id": "Yb912245qRDr",
        "outputId": "6ec0b197-6d8b-40f4-de2a-23b2be3bd39e"
      },
      "source": [
        "df['horsepower-binned'] = pd.cut(df['horsepower'], bins, labels=group_names, include_lowest=True )\n",
        "df[['horsepower','horsepower-binned']].head(20)"
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>horsepower</th>\n",
              "      <th>horsepower-binned</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>111</td>\n",
              "      <td>Low</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>111</td>\n",
              "      <td>Low</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>154</td>\n",
              "      <td>Medium</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>102</td>\n",
              "      <td>Low</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>115</td>\n",
              "      <td>Low</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>110</td>\n",
              "      <td>Low</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>110</td>\n",
              "      <td>Low</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>110</td>\n",
              "      <td>Low</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>140</td>\n",
              "      <td>Medium</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>101</td>\n",
              "      <td>Low</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>101</td>\n",
              "      <td>Low</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>121</td>\n",
              "      <td>Medium</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>121</td>\n",
              "      <td>Medium</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>121</td>\n",
              "      <td>Medium</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>182</td>\n",
              "      <td>Medium</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>182</td>\n",
              "      <td>Medium</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>182</td>\n",
              "      <td>Medium</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>48</td>\n",
              "      <td>Low</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>70</td>\n",
              "      <td>Low</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>70</td>\n",
              "      <td>Low</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    horsepower horsepower-binned\n",
              "0          111               Low\n",
              "1          111               Low\n",
              "2          154            Medium\n",
              "3          102               Low\n",
              "4          115               Low\n",
              "5          110               Low\n",
              "6          110               Low\n",
              "7          110               Low\n",
              "8          140            Medium\n",
              "9          101               Low\n",
              "10         101               Low\n",
              "11         121            Medium\n",
              "12         121            Medium\n",
              "13         121            Medium\n",
              "14         182            Medium\n",
              "15         182            Medium\n",
              "16         182            Medium\n",
              "17          48               Low\n",
              "18          70               Low\n",
              "19          70               Low"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-Qnv-AgJqRDr"
      },
      "source": [
        "Lets see the number of vehicles in each bin."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vGBLVfTmqRDs",
        "outputId": "efbd6f74-3d06-46ee-9793-93c3f92cb133"
      },
      "source": [
        "df[\"horsepower-binned\"].value_counts()"
      ],
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Low       153\n",
              "Medium     43\n",
              "High        5\n",
              "Name: horsepower-binned, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2ztD7DWuqRDs"
      },
      "source": [
        "Lets plot the distribution of each bin."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 312
        },
        "id": "1Tz5WSjBqRDt",
        "outputId": "84843260-b27e-4d9b-e2e0-dd6169a4b54a"
      },
      "source": [
        "%matplotlib inline\n",
        "import matplotlib as plt\n",
        "from matplotlib import pyplot\n",
        "pyplot.bar(group_names, df[\"horsepower-binned\"].value_counts())\n",
        "\n",
        "# set x/y labels and plot title\n",
        "plt.pyplot.xlabel(\"horsepower\")\n",
        "plt.pyplot.ylabel(\"count\")\n",
        "plt.pyplot.title(\"horsepower bins\")"
      ],
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'horsepower bins')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 42
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAY3ElEQVR4nO3deZhddZ3n8fcHUHEFNCWNBAgqOo2Oa0QcB0XpUVzG8LhCu6DSHXVoRqe1Fe1psW3pwWXcHXlig8Co0LjCjI4bCnFDTBRBQJo0giQNpkQWVxD4zh/nl+OlrEoqIffeStX79Tz15Jzf75x7vlX3yf3cs/1OqgpJkgC2G3cBkqS5w1CQJPUMBUlSz1CQJPUMBUlSz1CQJPUMBc1JSa5I8mfjrmOuS3JgkrUb6T8+yd+NsiZt23YYdwGShqeqXjnuGrRtcU9B81qSefPFZz79Lpq7DAXNZY9IckGSG5L8c5IdN3Qk+cska5L8IsmZSe430FdJjkxyGXBZOu9Jsj7JjUkuTPLQtuxdkrwryU+T/Kwdbrlr6zswydokb0ry83ZI64UD29kpySlJJpNcmeS/J9mu9V2Z5NFt+oWtpoe0+SOSfK5Nb5fk6CT/muTaJKcnuXfrW9LWOyLJT4GvzfSH2kiNJyV525Tf57Xtb3F1kpcNLPv0JBcn+WWSdUled0fePG2bDAXNZc8HDgb2Bh4GvBQgyZOB/9H6dwOuBE6bsu4hwGOBfYGnAE8AHgTs1Na7ti13XGt/BPBAYHfgzQOv8yfAotZ+OLAiyYNb3wfa690feCLwEmDDh+w5wIFt+onA5a2GDfPntOmjWq1PBO4HXAd8aMrv8kTgT4GnTvM32lSN0y27U1v2COBDSXZpfScAr6iqewIPZSMhpHmsqvzxZ879AFcALxqYfwdwfJs+AXjHQN89gN8DS9p8AU8e6H8y8C/A/sB2A+0Bfg08YKDtccBP2vSBwC3A3Qf6Twf+DtgeuBnYd6DvFcDZbfoI4Mw2fQnwF8Bpbf5K4FEDfQcNvMZu7XfZAVjSfpf7b+TvNGONbfok4G0Dy/4W2GFg2fXA/m36p+13uNe4339/xvfjnoLmsmsGpn9D9+EP3TfqKzd0VNWv6L757z6w/FUD/V8DPkj3DXx9khVJ7gVMAHcDVie5Psn1wBdb+wbXVdWvB+avbNtfBNxpsI42vaGGc4ADkuxGFyCnA49PsoTum/r5bbm9gM8ObP8S4FZg1+l+lxnMVON0rq2qWwbmB/+uzwGeDlyZ5Jwkj9vEdjUPGQraFv0b3YcpAEnuDtwHWDewzO2G/62q91fVo+kOJz0I+Bvg53TfnB9SVTu3n52q6h4Dq+7SXn+DPdv2f073jX6vKX3r2vbW0H3gHgWsrKob6UJuOfDNqrqtrXMV8LSB7e9cVTtW1Yy/yzRmqnGzVNX3qmoZcF/gc3RBpgXGUNC26FTgZUkekeQuwD8C362qK6ZbOMljkjw2yZ3oDhf9DritfTB/BHhPkvu2ZXdPMvXY/d8nuXOSA4BnAp+sqlvpPjSPTXLPJHsBfw18bGC9c4C/4g/nD86eMg9wfHuNvdr2J5Is24K/yR/VuDkrt3VfmGSnqvo9cCNw26bW0/xjKGibU1VfpTuu/2ngauABwKEbWeVedB/+19EdWrkWeGfrewOwBjg3yY3AV4HBk7TXtPX+Dfg48Mqq+nHrO4ouZC4Hvgl8AjhxYN1zgHsCK2eYB3gfcCbw5SS/BM6lO0G+OTZW4+Z4MXBF+zu8EnjhJpbXPJQqH7IjTSfJgcDHqmrxuGuRRsU9BUlSz1CQJPWGFgpJTmx3Tf5oSvtRSX6c5KIk7xhof2O7Q/XSaU70SSNXVWd76EgLzTDHUjmJ7trwUzY0JHkSsAx4eFXdNHDFx750JwofQnd99VeTPKhd4SFJGpGhhUJVrWw36gx6FXBcVd3Ullnf2pfR3e15E/CTJGuA/YDvbGwbixYtqiVLpm5CkrQxq1ev/nlVTUzXN+pRFx9Ed5fnsXTXir+uqr5HdxfouQPLreX2d6f2kiynuwGIPffck1WrVg23YkmaZ5JcOVPfqE807wDcm24Mmr8BTk+SzXmBqlpRVUuraunExLRBJ0naQqMOhbXAZ6pzHt0dk4vohgbYY2C5xdx+yAJJ0giMOhQ+BzwJIMmDgDvTjSFzJnBoG9t+b2Af4LwR1yZJC97QzikkOZVuqN5F6Z4hewzdEAAntstUbwYOr+6W6ouSnA5cTDcM8JFeeSRJo7dND3OxdOnS8kSzJG2eJKuraul0fd7RLEnqGQqSpJ6hIEnqGQqSpN6o72ieM5Yc/flxlzBvXXHcM8ZdgqQt5J6CJKlnKEiSeoaCJKlnKEiSeoaCJKlnKEiSeoaCJKlnKEiSeoaCJKlnKEiSeoaCJKlnKEiSeoaCJKk3tFBIcmKS9e15zFP7Xpukkixq80ny/iRrklyQ5FHDqkuSNLNh7imcBBw8tTHJHsBTgJ8OND8N2Kf9LAc+PMS6JEkzGFooVNVK4BfTdL0HeD1QA23LgFOqcy6wc5LdhlWbJGl6Iz2nkGQZsK6qfjila3fgqoH5ta1tutdYnmRVklWTk5NDqlSSFqaRhUKSuwFvAt58R16nqlZU1dKqWjoxMbF1ipMkAaN9HOcDgL2BHyYBWAx8P8l+wDpgj4FlF7c2SdIIjWxPoaourKr7VtWSqlpCd4joUVV1DXAm8JJ2FdL+wA1VdfWoapMkdYZ5SeqpwHeABydZm+SIjSz+BeByYA3wEeC/DKsuSdLMhnb4qKoO20T/koHpAo4cVi2SpNnxjmZJUs9QkCT1DAVJUs9QkCT1DAVJUs9QkCT1DAVJUs9QkCT1DAVJUs9QkCT1DAVJUs9QkCT1DAVJUs9QkCT1DAVJUs9QkCT1DAVJUs9QkCT1hvmM5hOTrE/yo4G2dyb5cZILknw2yc4DfW9MsibJpUmeOqy6JEkzG+aewknAwVPavgI8tKoeBvwL8EaAJPsChwIPaev8ryTbD7E2SdI0hhYKVbUS+MWUti9X1S1t9lxgcZteBpxWVTdV1U+ANcB+w6pNkjS9cZ5TeDnw/9r07sBVA31rW9sfSbI8yaokqyYnJ4dcoiQtLGMJhSR/C9wCfHxz162qFVW1tKqWTkxMbP3iJGkB22HUG0zyUuCZwEFVVa15HbDHwGKLW5skaYRGuqeQ5GDg9cCzquo3A11nAocmuUuSvYF9gPNGWZskaYh7CklOBQ4EFiVZCxxDd7XRXYCvJAE4t6peWVUXJTkduJjusNKRVXXrsGqTJE1vaKFQVYdN03zCRpY/Fjh2WPVIkjbNO5olST1DQZLUMxQkST1DQZLUMxQkST1DQZLUMxQkST1DQZLUMxQkST1DQZLUMxQkST1DQZLUMxQkST1DQZLUMxQkST1DQZLUMxQkST1DQZLUG1ooJDkxyfokPxpou3eSryS5rP27S2tPkvcnWZPkgiSPGlZdkqSZDXNP4STg4CltRwNnVdU+wFltHuBpwD7tZznw4SHWJUmawdBCoapWAr+Y0rwMOLlNnwwcMtB+SnXOBXZOstuwapMkTW/U5xR2raqr2/Q1wK5tenfgqoHl1ra2P5JkeZJVSVZNTk4Or1JJWoDGdqK5qgqoLVhvRVUtraqlExMTQ6hMkhauUYfCzzYcFmr/rm/t64A9BpZb3NokSSM06lA4Ezi8TR8OnDHQ/pJ2FdL+wA0Dh5kkSSOyw7BeOMmpwIHAoiRrgWOA44DTkxwBXAk8vy3+BeDpwBrgN8DLhlWXJGlmQwuFqjpshq6Dplm2gCOHVYskaXa8o1mS1DMUJEk9Q0GS1DMUJEk9Q0GS1DMUJEk9Q0GS1DMUJEk9Q0GS1DMUJEm9WYVCkrNm0yZJ2rZtdOyjJDsCd6Mb1G4XIK3rXszwEBxJ0rZrUwPivQJ4DXA/YDV/CIUbgQ8OsS5J0hhsNBSq6n3A+5IcVVUfGFFNkqQxmdXQ2VX1gST/AVgyuE5VnTKkuiRJYzCrUEjyv4EHAOcDt7bmAgwFSZpHZvuQnaXAvu1hOJKkeWq29yn8CPiTYRYiSRq/2e4pLAIuTnIecNOGxqp61pZsNMl/A/6C7hDUhXTPZN4NOA24D92VTi+uqpu35PUlSVtmtqHwlq21wSS7A/+V7nDUb5OcDhwKPB14T1WdluR44Ajgw1tru5KkTZvt1UfnDGG7d03ye7qb464Gngz8ees/mS6IDAVJGqHZDnPxyyQ3tp/fJbk1yY1bssGqWge8C/gpXRjcQHe46PqquqUtthbvmJakkZvtnsI9N0wnCbAM2H9LNtiGy1gG7A1cD3wSOHgz1l8OLAfYc889t6QESdIMNnuU1Op8DnjqFm7zz4CfVNVkVf0e+AzweGDnJBtCajGwbobtr6iqpVW1dGJiYgtLkCRNZ7Y3rz17YHY7uvsWfreF2/wpsH+SuwG/BQ4CVgFfB55LdwXS4cAZW/j6kqQtNNurj/7zwPQtwBV0h4A2W1V9N8mngO+31/oBsAL4PHBakre1thO25PUlSVtutucUXrY1N1pVxwDHTGm+HNhva25HkrR5Znv10eIkn02yvv18OsniYRcnSRqt2Z5o/ihwJt1zFe4H/J/WJkmaR2YbChNV9dGquqX9nAR46Y8kzTOzDYVrk7woyfbt50XAtcMsTJI0erMNhZcDzweuobsL+bnAS4dUkyRpTGZ7SepbgcOr6jqAJPemG6ri5cMqTJI0erPdU3jYhkAAqKpfAI8cTkmSpHGZbShs18YsAvo9hdnuZUiSthGz/WD/n8B3knyyzT8POHY4JUmSxmW2dzSfkmQV3TMPAJ5dVRcPryxJ0jjM+hBQCwGDQJLmsc0eOluSNH8ZCpKknqEgSeoZCpKknqEgSeoZCpKknqEgSeqNJRSS7JzkU0l+nOSSJI9Lcu8kX0lyWft3l02/kiRpaxrXnsL7gC9W1b8DHg5cAhwNnFVV+wBntXlJ0giNPBSS7AQ8ATgBoKpurqrrgWXAyW2xk4FDRl2bJC1049hT2BuYBD6a5AdJ/inJ3YFdq+rqtsw1wK5jqE2SFrRxhMIOwKOAD1fVI4FfM+VQUVUVUNOtnGR5klVJVk1OTg69WElaSMYRCmuBtVX13Tb/KbqQ+FmS3QDav+unW7mqVlTV0qpaOjExMZKCJWmhGHkoVNU1wFVJHtyaDqIbffVM4PDWdjhwxqhrk6SFblxPTzsK+HiSOwOXAy+jC6jTkxwBXAk8f0y1SdKCNZZQqKrzgaXTdB006lokSX/gHc2SpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqjesZzSTZHlgFrKuqZybZGzgNuA+wGnhxVd08rvo09yw5+vPjLmHeuuK4Z4y7BM0R49xTeDVwycD824H3VNUDgeuAI8ZSlSQtYGMJhSSLgWcA/9TmAzwZ+FRb5GTgkHHUJkkL2bj2FN4LvB64rc3fB7i+qm5p82uB3cdRmCQtZCMPhSTPBNZX1eotXH95klVJVk1OTm7l6iRpYRvHnsLjgWcluYLuxPKTgfcBOyfZcOJ7MbBuupWrakVVLa2qpRMTE6OoV5IWjJGHQlW9saoWV9US4FDga1X1QuDrwHPbYocDZ4y6Nkla6ObSfQpvAP46yRq6cwwnjLkeSVpwxnafAkBVnQ2c3aYvB/YbZz2StNDNpT0FSdKYGQqSpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqGQqSpJ6hIEnqjTwUkuyR5OtJLk5yUZJXt/Z7J/lKksvav7uMujZJWujGsadwC/DaqtoX2B84Msm+wNHAWVW1D3BWm5ckjdDIQ6Gqrq6q77fpXwKXALsDy4CT22InA4eMujZJWujGek4hyRLgkcB3gV2r6urWdQ2w6wzrLE+yKsmqycnJkdQpSQvF2EIhyT2ATwOvqaobB/uqqoCabr2qWlFVS6tq6cTExAgqlaSFYyyhkOROdIHw8ar6TGv+WZLdWv9uwPpx1CZJC9k4rj4KcAJwSVW9e6DrTODwNn04cMaoa5OkhW6HMWzz8cCLgQuTnN/a3gQcB5ye5AjgSuD5Y6hNkha0kYdCVX0TyAzdB42yFknS7XlHsySpZyhIknqGgiSpZyhIknqGgiSpZyhIknqGgiSpZyhIknqGgiSpZyhIknqGgiSpZyhIknqGgiSpZyhIknrjeJ6CpAViydGfH3cJ89YVxz1jKK/rnoIkqWcoSJJ6hoIkqTfnQiHJwUkuTbImydHjrkeSFpI5FQpJtgc+BDwN2Bc4LMm+461KkhaOORUKwH7Amqq6vKpuBk4Dlo25JklaMObaJam7A1cNzK8FHju4QJLlwPI2+6skl46otnFbBPx83EXMRt4+7grmDN+zbcs2837BHX7P9pqpY66FwiZV1QpgxbjrGLUkq6pq6bjr0Oz5nm1bfL86c+3w0Tpgj4H5xa1NkjQCcy0Uvgfsk2TvJHcGDgXOHHNNkrRgzKnDR1V1S5K/Ar4EbA+cWFUXjbmsuWLBHTKbB3zPti2+X0Cqatw1SJLmiLl2+EiSNEaGgiSpZyjMAUl+Ne4a1ElSST42ML9Dkskk/3czX+fsJEvb9BeS7Ly1a9XsTP3/leSlST7Ypl+Z5CWbWL9ffiGYUyeapTng18BDk9y1qn4L/Cfu4GXRVfX0rVKZtrqqOn7cNcw17inMUUkekeTcJBck+WySXZLcN8nq1v/w9q12zzb/r0nuNt6q540vABueYHIYcOqGjiR3T3JikvOS/CDJstZ+1ySnJbkkyWeBuw6sc0WSRUmWJPnRQPvrkrylTZ+d5D1JVrXXeEySzyS5LMnbRvA7L0hJ3pLkdW36Me3/2/lJ3jn4XgH3S/LF9n68Y0zljoShMHedAryhqh4GXAgcU1XrgR2T3As4AFgFHJBkL2B9Vf1mfOXOK6cBhybZEXgY8N2Bvr8FvlZV+wFPAt6Z5O7Aq4DfVNWfAscAj96C7d7c7qg9HjgDOBJ4KPDSJPfZ4t9Gd20f9OcnOR946wzLfRR4RVU9Arh1St8jgBcA/x54QZI9pq48X3j4aA5KshOwc1Wd05pOBj7Zpr8NPB54AvCPwMFAgG+Mus75qqouSLKEbi/hC1O6nwI8a8O3S2BHYE+69+P9A+tfsAWb3nCj5oXARVV1NUCSy+nu9L92C15T8Nv2QQ905wiA2w1n0c753LOqvtOaPgE8c2CRs6rqhrbsxXRjBw2O0zZvGArbnpV0ewl70X2bfANQgA/D3brOBN4FHAgMfksP8Jyqut1AjElm85q3cPu98x2n9N/U/r1tYHrDvP9Xx2vw/biVefx+ePhoDmrfSK5LckBrejGwYa/hG8CLgMuq6jbgF8DTgW+OvND57UTg76vqwintXwKOSkuBJI9s7SuBP29tD6U77DTVz4D7JrlPkrtw+2+iGqOquh74ZZINozIfOs56xmnept025m5J1g7Mvxs4HDi+nTy+HHgZQFVd0T6QVrZlvwksrqrrRlnwfFdVa2mHg6b4B+C9wAVJtgN+Qvfh/mHgo0kuAS4BVk/zmr9P8lbgPLormn48pPK1ZY4APpLkNrovYTeMuZ6xcJgLSQKS3KOqftWmjwZ2q6pXj7mskXNPQZI6z0jyRrrPxSuBl463nPFwT0GS1PNEsySpZyhIknqGgiSpZyhoQZg67pCk6RkK0iYk2Sau0ttW6tTcZihoIdk+yUeSXJTky21k0z8ajRb6UUvfm2QV8Ookz0vyoyQ/TLKyLbN9G03ze239V7T2A5OsTPL5JJcmOb7d6EaSw5Jc2F7r7a3teUne3aZf3cY6Isn9k3yrTT86yTlJVif5UpLdpqtztH9OzUd+s9BCsg9wWFX9ZZLTgecArweOqqpz2t3GxwCvacvfuY1aSpILgadW1br84YE5RwA3VNVj2rAV30ry5da3H7Av3fXuXwSeneTbwNvpRlC9DvhykkPohi55fVvvAODaJLu36ZVJ7gR8AFhWVZNJXgAcC7x8ap3SHWUoaCH5SVWd36ZXAw9g5tFoAf55YPpbwEktTD7T2p4CPCzJc9v8TnTBczNwXlVt+MZ/KvAfgd8DZ1fVZGv/OPCEqvpcknskuSfdaKifoBt19YC2rQfTDaH9lTbk0vbA1TPUKd0hhoIWkqkjXW7qEZm/3jBRVa9sg6U9A1id5NF0I6YeVVVfGlwpyYF0I9cO2tRdot+mG9/qUro9h5cDjwNeSzc090VV9bhN1SndUZ5T0EK2sdFobyfJA6rqu1X1ZmCS7hv9l4BXtcM7JHlQe+AOwH5J9m7nEl5AN3DhecAT0z2FbXu65zUMjn77OrqBDn9A9wCfm9qIuZcCE0ke17ZzpyQP2Xp/BukP3FPQQjftaLTTeGeSfej2Ds4CfghcACwBvt9Grp0EDmnLfw/4IPBA4OvAZ6vqtjbQ2tfb63y+qs5oy3+DLmhWVtWtSa6ijaJaVTe3Q1Tvbw9g2oFupNaLttLfQOo59pG0lbXDR6+rKp+XoG2Oh48kST33FCRJPfcUJEk9Q0GS1DMUJEk9Q0GS1DMUJEm9/w9eonJivNqE7QAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cGdh1GgAqRDt"
      },
      "source": [
        "<p>\n",
        "    Check the dataframe above carefully, you will find the last column provides the bins for \"horsepower\" with 3 categories (\"Low\",\"Medium\" and \"High\"). \n",
        "</p>\n",
        "<p>\n",
        "    We successfully narrow the intervals from 57 to 3!\n",
        "</p>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KuOBNxdvqRDu"
      },
      "source": [
        "<h3>Bins visualization</h3>\n",
        "Normally, a histogram is used to visualize the distribution of bins we created above. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 312
        },
        "id": "5p1MSGFfqRDu",
        "outputId": "5a9f3a2a-8a1e-4d81-cb6e-273c88d3a8cf"
      },
      "source": [
        "%matplotlib inline\n",
        "import matplotlib as plt\n",
        "from matplotlib import pyplot\n",
        "\n",
        "a = (0,1,2)\n",
        "\n",
        "# draw historgram of attribute \"horsepower\" with bins = 3\n",
        "plt.pyplot.hist(df[\"horsepower\"], bins = 3)\n",
        "\n",
        "# set x/y labels and plot title\n",
        "plt.pyplot.xlabel(\"horsepower\")\n",
        "plt.pyplot.ylabel(\"count\")\n",
        "plt.pyplot.title(\"horsepower bins\")"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'horsepower bins')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAYjUlEQVR4nO3dfbRddX3n8fcHUHxCAuZKIwGCCM6gyyreIoxVEboU0TGMVQvjQ8S00Q4yOtVa1Kk4XTKD2vrsyIoFgZZC8QHJLFoVUYhaAYMijyIpEEgK5Mqj9YHH7/yxdzaH673JTcg55ybn/VrrrLv3b+999vf+cnI/Z++z9++kqpAkCWCbYRcgSZo9DAVJUsdQkCR1DAVJUsdQkCR1DAVJUsdQ0KyU5MYkfzDsOma7JAclWb2e5Scm+ctB1qQt23bDLkBS/1TV24ddg7YsHiloq5Zkq3njszX9Lpq9DAXNZs9NcnmSu5P8Y5LHrVuQ5E+SrExyR5JlSZ7Ws6ySHJ3kOuC6ND6RZG2Se5JckeTZ7brbJ/nrJDclua093fL4dtlBSVYneX+Sn7entN7Qs58dk5yWZCLJqiT/M8k27bJVSZ7fTr+hrelZ7fziJF9rp7dJcmySf01ye5KzkuzcLlvQbrc4yU3At6frqPXUeEqSD0/6fd7d9sUtSY7qWfewJFcn+UWSNUne82j+8bRlMhQ0m70eOBTYE3gO8BaAJAcD/6ddPg9YBZw5advDgRcA+wIvA14M7APs2G53e7veCW37c4FnALsCH+x5nt8B5rbti4ClSZ7ZLvtM+3xPB14CvBlY90f2QuCgdvolwPVtDevmL2ynj2lrfQnwNOBO4HOTfpeXAP8RePkUfbShGqdad8d23cXA55Ls1C47CXhbVe0APJv1hJC2YlXlw8esewA3Am/smf8ocGI7fRLw0Z5lTwLuBxa08wUc3LP8YOBnwAHANj3tAX4J7NXTdiBwQzt9EPAA8MSe5WcBfwlsC9wH7Nuz7G3ABe30YmBZO30N8MfAme38KmC/nmWH9DzHvPZ32Q5Y0P4uT19PP01bYzt9CvDhnnV/DWzXs+5a4IB2+qb2d3jysP/9fQzv4ZGCZrNbe6Z/RfPHH5p31KvWLaiqf6d5579rz/o39yz/NvBZmnfga5MsTfJkYAx4AnBpkruS3AV8vW1f586q+mXP/Kp2/3OBx/TW0U6vq+FC4EVJ5tEEyFnAC5MsoHmnflm73h7A2T37vwZ4ENhlqt9lGtPVOJXbq+qBnvnefv1D4DBgVZILkxy4gf1qK2QoaEv0bzR/TAFI8kTgKcCannUeMfxvVX26qp5PczppH+DPgZ/TvHN+VlXNaR87VtWTejbdqX3+dXZv9/9zmnf0e0xatqbd30qaP7jHAMur6h6akFsCfK+qHmq3uRl4Rc/+51TV46pq2t9lCtPVuFGq6odVtRB4KvA1miDTiDEUtCU6AzgqyXOTbA/8b+DiqrpxqpWT/F6SFyR5DM3pot8AD7V/mL8AfCLJU9t1d00y+dz9/0ry2CQvAl4FfKmqHqT5o3l8kh2S7AH8GfD3PdtdCLyDhz8/uGDSPMCJ7XPs0e5/LMnCTeiT36pxYzZut31Dkh2r6n7gHuChDW2nrY+hoC1OVX2L5rz+V4BbgL2AI9azyZNp/vjfSXNq5XbgY+2yvwBWAhcluQf4FtD7Ie2t7Xb/BpwOvL2qftouO4YmZK4Hvgf8A3Byz7YXAjsAy6eZB/gUsAz4ZpJfABfRfEC+MdZX48Z4E3Bj2w9vB96wgfW1FUqVX7IjTSXJQcDfV9X8YdciDYpHCpKkjqEgSer0LRSSnNzeNXnlpPZjkvw0yVVJPtrT/r72DtVrp/igTxq4qrrAU0caNf0cS+UUmmvDT1vXkOSlwELgd6vq3p4rPval+aDwWTTXV38ryT7tFR6SpAHpWyhU1fL2Rp1efwqcUFX3tuusbdsX0tzteS9wQ5KVwP7AD9a3j7lz59aCBZN3IUlan0svvfTnVTU21bJBj7q4D81dnsfTXCv+nqr6Ic1doBf1rLeaR96d2kmyhOYGIHbffXdWrFjR34olaSuTZNV0ywb9QfN2wM40Y9D8OXBWkmzME1TV0qoar6rxsbEpg06StIkGHQqrga9W4xKaOybn0gwNsFvPevN55JAFkqQBGHQofA14KUCSfYDH0owhsww4oh3bfk9gb+CSAdcmSSOvb58pJDmDZqjeuWm+Q/Y4miEATm4vU70PWFTNLdVXJTkLuJpmGOCjvfJIkgZvix7mYnx8vPygWZI2TpJLq2p8qmXe0SxJ6hgKkqSOoSBJ6hgKkqTOoO9onjUWHHvusEvQFG484ZXDLkEaaR4pSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6fQuFJCcnWdt+H/PkZe9OUknmtvNJ8ukkK5NcnmS/ftUlSZpeP48UTgEOndyYZDfgZcBNPc2vAPZuH0uAz/exLknSNPoWClW1HLhjikWfAN4LVE/bQuC0alwEzEkyr1+1SZKmNtDPFJIsBNZU1U8mLdoVuLlnfnXbNtVzLEmyIsmKiYmJPlUqSaNpYKGQ5AnA+4EPPprnqaqlVTVeVeNjY2ObpzhJEjDYr+PcC9gT+EkSgPnAj5LsD6wBdutZd37bJkkaoIEdKVTVFVX11KpaUFULaE4R7VdVtwLLgDe3VyEdANxdVbcMqjZJUqOfl6SeAfwAeGaS1UkWr2f1fwKuB1YCXwD+W7/qkiRNr2+nj6rqyA0sX9AzXcDR/apFkjQz3tEsSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkTj+/o/nkJGuTXNnT9rEkP01yeZKzk8zpWfa+JCuTXJvk5f2qS5I0vX4eKZwCHDqp7Tzg2VX1HOBnwPsAkuwLHAE8q93m/ybZto+1SZKm0LdQqKrlwB2T2r5ZVQ+0sxcB89vphcCZVXVvVd0ArAT271dtkqSpDfMzhbcC/9xO7wrc3LNsddv2W5IsSbIiyYqJiYk+lyhJo2UooZDkA8ADwOkbu21VLa2q8aoaHxsb2/zFSdII227QO0zyFuBVwCFVVW3zGmC3ntXmt22SpAEa6JFCkkOB9wKvrqpf9SxaBhyRZPskewJ7A5cMsjZJUh+PFJKcARwEzE2yGjiO5mqj7YHzkgBcVFVvr6qrkpwFXE1zWunoqnqwX7VJkqbWt1CoqiOnaD5pPesfDxzfr3okSRvmHc2SpI6hIEnqGAqSpI6hIEnqGAqSpI6hIEnqGAqSpI6hIEnqGAqSpI6hIEnqGAqSpI6hIEnqGAqSpI6hIEnqGAqSpI6hIEnqGAqSpI6hIEnq9C0UkpycZG2SK3vadk5yXpLr2p87te1J8ukkK5NcnmS/ftUlSZpeP48UTgEOndR2LHB+Ve0NnN/OA7wC2Lt9LAE+38e6JEnT6FsoVNVy4I5JzQuBU9vpU4HDe9pPq8ZFwJwk8/pVmyRpaoP+TGGXqrqlnb4V2KWd3hW4uWe91W3bb0myJMmKJCsmJib6V6kkjaChfdBcVQXUJmy3tKrGq2p8bGysD5VJ0ugadCjctu60UPtzbdu+BtitZ735bZskaYAGHQrLgEXt9CLgnJ72N7dXIR0A3N1zmkmSNCDb9euJk5wBHATMTbIaOA44ATgryWJgFfD6dvV/Ag4DVgK/Ao7qV12SpOn1LRSq6shpFh0yxboFHN2vWiRJM+MdzZKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSerMKBSSnD+TNknSlm29Yx8leRzwBJpB7XYC0i56MtN8CY4kacu1oQHx3ga8C3gacCkPh8I9wGf7WJckaQjWGwpV9SngU0mOqarPDKgmSdKQzGjo7Kr6TJL/BCzo3aaqTutTXZKkIZhRKCT5O2Av4DLgwba5AENBkrYiM/2SnXFg3/bLcCRJW6mZ3qdwJfA7/SxEkjR8Mz1SmAtcneQS4N51jVX16k3ZaZL/AfwxzSmoK2i+k3kecCbwFJornd5UVfdtyvNLkjbNTEPhQ5trh0l2Bf47zemoXyc5CzgCOAz4RFWdmeREYDHw+c21X0nShs306qML+7Dfxye5n+bmuFuAg4H/2i4/lSaIDAVJGqCZDnPxiyT3tI/fJHkwyT2bssOqWgP8NXATTRjcTXO66K6qeqBdbTXeMS1JAzfTI4Ud1k0nCbAQOGBTdtgOl7EQ2BO4C/gScOhGbL8EWAKw++67b0oJkqRpbPQoqdX4GvDyTdznHwA3VNVEVd0PfBV4ITAnybqQmg+smWb/S6tqvKrGx8bGNrEESdJUZnrz2mt6ZrehuW/hN5u4z5uAA5I8Afg1cAiwAvgO8FqaK5AWAeds4vNLkjbRTK8++s890w8AN9KcAtpoVXVxki8DP2qf68fAUuBc4MwkH27bTtqU55ckbbqZfqZw1ObcaVUdBxw3qfl6YP/NuR9J0saZ6dVH85OcnWRt+/hKkvn9Lk6SNFgz/aD5i8Aymu9VeBrw/9o2SdJWZKahMFZVX6yqB9rHKYCX/kjSVmamoXB7kjcm2bZ9vBG4vZ+FSZIGb6ah8Fbg9cCtNHchvxZ4S59qkiQNyUwvSf0rYFFV3QmQZGeaoSre2q/CJEmDN9MjheesCwSAqroDeF5/SpIkDctMQ2GbdswioDtSmOlRhiRpCzHTP+x/A/wgyZfa+dcBx/enJEnSsMz0jubTkqyg+c4DgNdU1dX9K0uSNAwzPgXUhoBBIElbsY0eOluStPUyFCRJHUNBktQxFCRJHUNBktQxFCRJHUNBktQZSigkmZPky0l+muSaJAcm2TnJeUmua3/utOFnkiRtTsM6UvgU8PWq+g/A7wLXAMcC51fV3sD57bwkaYAGHgpJdgReDJwEUFX3VdVdwELg1Ha1U4HDB12bJI26YRwp7AlMAF9M8uMkf5vkicAuVXVLu86twC5DqE2SRtowQmE7YD/g81X1POCXTDpVVFUF1FQbJ1mSZEWSFRMTE30vVpJGyTBCYTWwuqoubue/TBMStyWZB9D+XDvVxlW1tKrGq2p8bGxsIAVL0qgYeChU1a3AzUme2TYdQjP66jJgUdu2CDhn0LVJ0qgb1renHQOcnuSxwPXAUTQBdVaSxcAq4PVDqk2SRtZQQqGqLgPGp1h0yKBrkSQ9zDuaJUkdQ0GS1DEUJEkdQ0GS1DEUJEkdQ0GS1DEUJEkdQ0GS1DEUJEkdQ0GS1DEUJEkdQ0GS1DEUJEkdQ0GS1DEUJEkdQ0GS1DEUJEkdQ0GS1BnWdzSTZFtgBbCmql6VZE/gTOApwKXAm6rqvmHVp+FYcOy5wy5Bk9x4wiuHXYIGaJhHCu8ErumZ/wjwiap6BnAnsHgoVUnSCBtKKCSZD7wS+Nt2PsDBwJfbVU4FDh9GbZI0yoZ1pPBJ4L3AQ+38U4C7quqBdn41sOswCpOkUTbwUEjyKmBtVV26idsvSbIiyYqJiYnNXJ0kjbZhHCm8EHh1khtpPlg+GPgUMCfJug++5wNrptq4qpZW1XhVjY+NjQ2iXkkaGQMPhap6X1XNr6oFwBHAt6vqDcB3gNe2qy0Czhl0bZI06mbTfQp/AfxZkpU0nzGcNOR6JGnkDO0+BYCqugC4oJ2+Hth/mPVI0qibTUcKkqQhMxQkSR1DQZLUMRQkSR1DQZLUMRQkSR1DQZLUMRQkSR1DQZLUMRQkSR1DQZLUMRQkSR1DQZLUMRQkSR1DQZLUMRQkSR1DQZLUMRQkSR1DQZLUGXgoJNktyXeSXJ3kqiTvbNt3TnJekuvanzsNujZJGnXDOFJ4AHh3Ve0LHAAcnWRf4Fjg/KraGzi/nZckDdDAQ6GqbqmqH7XTvwCuAXYFFgKntqudChw+6NokadQN9TOFJAuA5wEXA7tU1S3toluBXabZZkmSFUlWTExMDKROSRoVQwuFJE8CvgK8q6ru6V1WVQXUVNtV1dKqGq+q8bGxsQFUKkmjYyihkOQxNIFwelV9tW2+Lcm8dvk8YO0wapOkUTaMq48CnARcU1Uf71m0DFjUTi8Czhl0bZI06rYbwj5fCLwJuCLJZW3b+4ETgLOSLAZWAa8fQm2SNNIGHgpV9T0g0yw+ZJC1SJIeyTuaJUkdQ0GS1DEUJEkdQ0GS1DEUJEkdQ0GS1DEUJEkdQ0GS1DEUJEkdQ0GS1DEUJEkdQ0GS1DEUJEkdQ0GS1BnG9ylI2oIsOPbcYZegKdx4wiv78rweKUiSOoaCJKljKEiSOrMuFJIcmuTaJCuTHDvseiRplMyqUEiyLfA54BXAvsCRSfYdblWSNDpmVSgA+wMrq+r6qroPOBNYOOSaJGlkzLZLUncFbu6ZXw28oHeFJEuAJe3svye5djPsdy7w883wPFsj+2Z69s362T/Te9R9k488qv3vMd2C2RYKG1RVS4Glm/M5k6yoqvHN+ZxbC/tmevbN+tk/05vNfTPbTh+tAXbrmZ/ftkmSBmC2hcIPgb2T7JnkscARwLIh1yRJI2NWnT6qqgeSvAP4BrAtcHJVXTWAXW/W01FbGftmevbN+tk/05u1fZOqGnYNkqRZYradPpIkDZGhIEnqjGQoJLkxyRVJLkuyom3bOcl5Sa5rf+407DoHIcnJSdYmubKnbcq+SOPT7RAklyfZb3iV9980ffOhJGva185lSQ7rWfa+tm+uTfLy4VQ9GEl2S/KdJFcnuSrJO9v2kX/trKdvtozXTlWN3AO4EZg7qe2jwLHt9LHAR4Zd54D64sXAfsCVG+oL4DDgn4EABwAXD7v+IfTNh4D3TLHuvsBPgO2BPYF/BbYd9u/Qx76ZB+zXTu8A/Kztg5F/7aynb7aI185IHilMYyFwajt9KnD4EGsZmKpaDtwxqXm6vlgInFaNi4A5SeYNptLBm6ZvprMQOLOq7q2qG4CVNMO2bJWq6paq+lE7/QvgGpoRCUb+tbOevpnOrHrtjGooFPDNJJe2w2YA7FJVt7TTtwK7DKe0WWG6vphqGJL1vdi3Vu9oT4Gc3HOacWT7JskC4HnAxfjaeYRJfQNbwGtnVEPh96tqP5rRWI9O8uLehdUc03mtLvbFFD4P7AU8F7gF+JvhljNcSZ4EfAV4V1Xd07ts1F87U/TNFvHaGclQqKo17c+1wNk0h2q3rTucbX+uHV6FQzddX4z8MCRVdVtVPVhVDwFf4OHD/JHrmySPofmjd3pVfbVt9rXD1H2zpbx2Ri4UkjwxyQ7rpoGXAVfSDKexqF1tEXDOcCqcFabri2XAm9srSQ4A7u45VTASJp0H/y80rx1o+uaIJNsn2RPYG7hk0PUNSpIAJwHXVNXHexaN/Gtnur7ZYl47w/6kftAP4Ok0n/T/BLgK+EDb/hTgfOA64FvAzsOudUD9cQbNoez9NOcyF0/XFzRXjnyO5uqIK4DxYdc/hL75u/Z3v5zmP/O8nvU/0PbNtcArhl1/n/vm92lODV0OXNY+DvO1s96+2SJeOw5zIUnqjNzpI0nS9AwFSVLHUJAkdQwFSVLHUJAkdQwFjYQkC3pHO5U0NUNB2oAks+pra6ezpdSp2c1Q0CjZNskX2jHuv5nk8Umem+SidpCys3vG/78gySfb79t4Z5LXJbkyyU+SLG/X2TbJx5L8sN3+bW37QUmWJzm3HR//xCTbtMuOTPNdHlcm+Ujb9rokH2+n35nk+nb66Um+304/P8mF7SCO3+gZSuIRdQ62O7U18p2FRsnewJFV9SdJzgL+EHgvcExVXZjkr4DjgHe16z+2qsYBklwBvLyq1iSZ0y5fTDNcw+8l2R74fpJvtsv2pxknfxXwdeA1Sf4F+AjwfOBOmpF6Dwe+29YB8CLg9iS7ttPL23F0PgMsrKqJJH8EHA+8dXKd0qNlKGiU3FBVl7XTl9KMWDmnqi5s204FvtSz/j/2TH8fOKUNk3WDv70MeE6S17bzO9IEz33AJVW17h3/GTRDH9wPXFBVE2376cCLq+prSZ7Ujsm1G/APNF/w86J2X88Eng2c1wyrw7Y0w29MVaf0qBgKGiX39kw/CMyZbsXWL9dNVNXbk7wAeCVwaZLn04znc0xVfaN3oyQH8dtDRm9oPJl/AY6iGfvmuzRHAQcC7wZ2B66qqgM3VKf0aPmZgkbZ3cCdSV7Uzr8JuHCqFZPsVVUXV9UHgQmad/TfAP60Pb1Dkn3akXcB9k+yZ/tZwh8B36MZ+fIlSeYm2RY4smd/3wXeAywHfgy8FLi3qu6mCYqxJAe2+3lMkmdtvm6QHuaRgkbdIuDEJE8Arqd5tz6VjyXZm+bo4HyaUXYvBxYAP2qHS57g4a+f/CHwWeAZwHeAs6vqoSTHtvMBzq2qdUNLf5cmaJZX1YNJbgZ+ClBV97WnqD6dZEea/7efpBnlV9qsHCVV2sza00fvqapXDbsWaWN5+kiS1PFIQZLU8UhBktQxFCRJHUNBktQxFCRJHUNBktT5/0x6hsbYPod0AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S0Zjy1JnqRDv"
      },
      "source": [
        "The plot above shows the binning result for attribute \"horsepower\". "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J8GdlVFdqRDv"
      },
      "source": [
        "<h2 id=\"indicator\">Indicator variable (or dummy variable)</h2>\n",
        "<b>What is an indicator variable?</b>\n",
        "<p>\n",
        "    An indicator variable (or dummy variable) is a numerical variable used to label categories. They are called 'dummies' because the numbers themselves don't have inherent meaning. \n",
        "</p>\n",
        "\n",
        "<b>Why we use indicator variables?</b>\n",
        "<p>\n",
        "    So we can use categorical variables for regression analysis in the later modules.\n",
        "</p>\n",
        "<b>Example</b>\n",
        "<p>\n",
        "    We see the column \"fuel-type\" has two unique values, \"gas\" or \"diesel\". Regression doesn't understand words, only numbers. To use this attribute in regression analysis, we convert \"fuel-type\" into indicator variables.\n",
        "</p>\n",
        "\n",
        "<p>\n",
        "    We will use the panda's method 'get_dummies' to assign numerical values to different categories of fuel type. \n",
        "</p>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ifSUN4g4qRDw",
        "outputId": "9b8173cd-7877-4c2a-8b5a-33f9743a05e4"
      },
      "source": [
        "df.columns"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Index(['symboling', 'normalized-losses', 'make', 'fuel-type', 'aspiration',\n",
              "       'num-of-doors', 'body-style', 'drive-wheels', 'engine-location',\n",
              "       'wheel-base', 'length', 'width', 'height', 'curb-weight', 'engine-type',\n",
              "       'num-of-cylinders', 'engine-size', 'fuel-system', 'bore', 'stroke',\n",
              "       'compression-ratio', 'horsepower', 'peak-rpm', 'city-mpg',\n",
              "       'highway-L/100km', 'price', 'city-L/100km', 'highway-L/100km',\n",
              "       'horsepower-binned'],\n",
              "      dtype='object')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5CRmkhDqqRDy"
      },
      "source": [
        "get indicator variables and assign it to data frame \"dummy_variable_1\" "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "Me9uAXrSqRDz",
        "outputId": "23feb4be-917b-492c-fef6-9e8c5851e73f"
      },
      "source": [
        "dummy_variable_1 = pd.get_dummies(df[\"fuel-type\"])\n",
        "dummy_variable_1.head()"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>diesel</th>\n",
              "      <th>gas</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   diesel  gas\n",
              "0       0    1\n",
              "1       0    1\n",
              "2       0    1\n",
              "3       0    1\n",
              "4       0    1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dGp1zGl5qRD0"
      },
      "source": [
        "change column names for clarity "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "OoMPe783qRD1",
        "outputId": "687d119b-ad83-4546-de81-f51e573d7285"
      },
      "source": [
        "dummy_variable_1.rename(columns={'fuel-type-diesel':'gas', 'fuel-type-diesel':'diesel'}, inplace=True)\n",
        "dummy_variable_1.head()"
      ],
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>diesel</th>\n",
              "      <th>gas</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   diesel  gas\n",
              "0       0    1\n",
              "1       0    1\n",
              "2       0    1\n",
              "3       0    1\n",
              "4       0    1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lVdkwUViqRD1"
      },
      "source": [
        "We now have the value 0 to represent \"gas\" and 1 to represent \"diesel\" in the column \"fuel-type\". We will now insert this column back into our original dataset. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "id": "GlUJOqMEqRD2"
      },
      "source": [
        "# merge data frame \"df\" and \"dummy_variable_1\" \n",
        "df = pd.concat([df, dummy_variable_1], axis=1)\n",
        "\n",
        "# drop original column \"fuel-type\" from \"df\"\n",
        "df.drop(\"fuel-type\", axis = 1, inplace=True)"
      ],
      "execution_count": 47,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 309
        },
        "id": "o36mgIBXqRD2",
        "outputId": "0b75ab57-efee-4a60-f0f6-8f2119b60f51"
      },
      "source": [
        "df.head()"
      ],
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>symboling</th>\n",
              "      <th>normalized-losses</th>\n",
              "      <th>make</th>\n",
              "      <th>aspiration</th>\n",
              "      <th>num-of-doors</th>\n",
              "      <th>body-style</th>\n",
              "      <th>drive-wheels</th>\n",
              "      <th>engine-location</th>\n",
              "      <th>wheel-base</th>\n",
              "      <th>length</th>\n",
              "      <th>width</th>\n",
              "      <th>height</th>\n",
              "      <th>curb-weight</th>\n",
              "      <th>engine-type</th>\n",
              "      <th>num-of-cylinders</th>\n",
              "      <th>engine-size</th>\n",
              "      <th>fuel-system</th>\n",
              "      <th>bore</th>\n",
              "      <th>stroke</th>\n",
              "      <th>compression-ratio</th>\n",
              "      <th>horsepower</th>\n",
              "      <th>peak-rpm</th>\n",
              "      <th>city-mpg</th>\n",
              "      <th>highway-L/100km</th>\n",
              "      <th>price</th>\n",
              "      <th>city-L/100km</th>\n",
              "      <th>highway-L/100km</th>\n",
              "      <th>horsepower-binned</th>\n",
              "      <th>diesel</th>\n",
              "      <th>gas</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>3</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>0.811148</td>\n",
              "      <td>0.890278</td>\n",
              "      <td>0.816054</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>21</td>\n",
              "      <td>8.703704</td>\n",
              "      <td>13495.0</td>\n",
              "      <td>11.190476</td>\n",
              "      <td>8.703704</td>\n",
              "      <td>Low</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>0.811148</td>\n",
              "      <td>0.890278</td>\n",
              "      <td>0.816054</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>21</td>\n",
              "      <td>8.703704</td>\n",
              "      <td>16500.0</td>\n",
              "      <td>11.190476</td>\n",
              "      <td>8.703704</td>\n",
              "      <td>Low</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>std</td>\n",
              "      <td>two</td>\n",
              "      <td>hatchback</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>94.5</td>\n",
              "      <td>0.822681</td>\n",
              "      <td>0.909722</td>\n",
              "      <td>0.876254</td>\n",
              "      <td>2823</td>\n",
              "      <td>ohcv</td>\n",
              "      <td>six</td>\n",
              "      <td>152</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>2.68</td>\n",
              "      <td>3.47</td>\n",
              "      <td>9.0</td>\n",
              "      <td>154</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>19</td>\n",
              "      <td>9.038462</td>\n",
              "      <td>16500.0</td>\n",
              "      <td>12.368421</td>\n",
              "      <td>9.038462</td>\n",
              "      <td>Medium</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>fwd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.8</td>\n",
              "      <td>0.848630</td>\n",
              "      <td>0.919444</td>\n",
              "      <td>0.908027</td>\n",
              "      <td>2337</td>\n",
              "      <td>ohc</td>\n",
              "      <td>four</td>\n",
              "      <td>109</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>10.0</td>\n",
              "      <td>102</td>\n",
              "      <td>5500.0</td>\n",
              "      <td>24</td>\n",
              "      <td>7.833333</td>\n",
              "      <td>13950.0</td>\n",
              "      <td>9.791667</td>\n",
              "      <td>7.833333</td>\n",
              "      <td>Low</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>std</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>4wd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.4</td>\n",
              "      <td>0.848630</td>\n",
              "      <td>0.922222</td>\n",
              "      <td>0.908027</td>\n",
              "      <td>2824</td>\n",
              "      <td>ohc</td>\n",
              "      <td>five</td>\n",
              "      <td>136</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>8.0</td>\n",
              "      <td>115</td>\n",
              "      <td>5500.0</td>\n",
              "      <td>18</td>\n",
              "      <td>10.681818</td>\n",
              "      <td>17450.0</td>\n",
              "      <td>13.055556</td>\n",
              "      <td>10.681818</td>\n",
              "      <td>Low</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   symboling  normalized-losses         make  ... horsepower-binned diesel gas\n",
              "0          3                122  alfa-romero  ...               Low      0   1\n",
              "1          3                122  alfa-romero  ...               Low      0   1\n",
              "2          1                122  alfa-romero  ...            Medium      0   1\n",
              "3          2                164         audi  ...               Low      0   1\n",
              "4          2                164         audi  ...               Low      0   1\n",
              "\n",
              "[5 rows x 30 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1z3wmJtDqRD3"
      },
      "source": [
        "The last two columns are now the indicator variable representation of the fuel-type variable. It's all 0s and 1s now."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aajTFKZ3qRD3"
      },
      "source": [
        "<div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
        "<h1> Question  #4: </h1>\n",
        "\n",
        "<b>As above, create indicator variable to the column of \"aspiration\": \"std\" to 0, while \"turbo\" to 1.</b>\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "GU6zg5uAqRD4",
        "outputId": "ca03dd83-4a66-4bef-f5ec-2aea90ef5be5"
      },
      "source": [
        "# Write your code below and press Shift+Enter to execute \n",
        "dummy_variable_2 = pd.get_dummies(df['aspiration'])\n",
        "dummy_variable_2.head()"
      ],
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>std</th>\n",
              "      <th>turbo</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   std  turbo\n",
              "0    1      0\n",
              "1    1      0\n",
              "2    1      0\n",
              "3    1      0\n",
              "4    1      0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "Bo1S2zqFIl7G",
        "outputId": "5e954615-dd0e-4310-aaf4-da8690f6382e"
      },
      "source": [
        "dummy_variable_2.rename(columns={'aspiration':'std', 'aspiration':'turbo'}, inplace=True)\n",
        "dummy_variable_2.head()"
      ],
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>std</th>\n",
              "      <th>turbo</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   std  turbo\n",
              "0    1      0\n",
              "1    1      0\n",
              "2    1      0\n",
              "3    1      0\n",
              "4    1      0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 52
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "62DyTDapqRD4"
      },
      "source": [
        "Double-click <b>here</b> for the solution.\n",
        "\n",
        "<!-- The answer is below:\n",
        "\n",
        "# get indicator variables of aspiration and assign it to data frame \"dummy_variable_2\"\n",
        "dummy_variable_2 = pd.get_dummies(df['aspiration'])\n",
        "\n",
        "# change column names for clarity\n",
        "dummy_variable_2.rename(columns={'std':'aspiration-std', 'turbo': 'aspiration-turbo'}, inplace=True)\n",
        "\n",
        "# show first 5 instances of data frame \"dummy_variable_1\"\n",
        "dummy_variable_2.head()\n",
        "\n",
        "-->"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xFdJn3EeqRD5"
      },
      "source": [
        " <div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
        "<h1> Question  #5: </h1>\n",
        "\n",
        "<b>Merge the new dataframe to the original dataframe then drop the column 'aspiration'</b>\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ni6VlWsTqRD5"
      },
      "source": [
        "# Write your code below and press Shift+Enter to execute \n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hh7ycwT7GZhD"
      },
      "source": [
        "df = pd.concat([df, dummy_variable_2], axis=1)\n",
        "df.drop('aspiration', axis =1, inplace=True)"
      ],
      "execution_count": 50,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 309
        },
        "id": "vo_ov1HUH9fF",
        "outputId": "8bd7d8b4-28fc-493f-bd20-c04f6954b40c"
      },
      "source": [
        "df.head()"
      ],
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>symboling</th>\n",
              "      <th>normalized-losses</th>\n",
              "      <th>make</th>\n",
              "      <th>num-of-doors</th>\n",
              "      <th>body-style</th>\n",
              "      <th>drive-wheels</th>\n",
              "      <th>engine-location</th>\n",
              "      <th>wheel-base</th>\n",
              "      <th>length</th>\n",
              "      <th>width</th>\n",
              "      <th>height</th>\n",
              "      <th>curb-weight</th>\n",
              "      <th>engine-type</th>\n",
              "      <th>num-of-cylinders</th>\n",
              "      <th>engine-size</th>\n",
              "      <th>fuel-system</th>\n",
              "      <th>bore</th>\n",
              "      <th>stroke</th>\n",
              "      <th>compression-ratio</th>\n",
              "      <th>horsepower</th>\n",
              "      <th>peak-rpm</th>\n",
              "      <th>city-mpg</th>\n",
              "      <th>highway-L/100km</th>\n",
              "      <th>price</th>\n",
              "      <th>city-L/100km</th>\n",
              "      <th>highway-L/100km</th>\n",
              "      <th>horsepower-binned</th>\n",
              "      <th>diesel</th>\n",
              "      <th>gas</th>\n",
              "      <th>std</th>\n",
              "      <th>turbo</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>3</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>0.811148</td>\n",
              "      <td>0.890278</td>\n",
              "      <td>0.816054</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>21</td>\n",
              "      <td>8.703704</td>\n",
              "      <td>13495.0</td>\n",
              "      <td>11.190476</td>\n",
              "      <td>8.703704</td>\n",
              "      <td>Low</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>two</td>\n",
              "      <td>convertible</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>88.6</td>\n",
              "      <td>0.811148</td>\n",
              "      <td>0.890278</td>\n",
              "      <td>0.816054</td>\n",
              "      <td>2548</td>\n",
              "      <td>dohc</td>\n",
              "      <td>four</td>\n",
              "      <td>130</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.47</td>\n",
              "      <td>2.68</td>\n",
              "      <td>9.0</td>\n",
              "      <td>111</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>21</td>\n",
              "      <td>8.703704</td>\n",
              "      <td>16500.0</td>\n",
              "      <td>11.190476</td>\n",
              "      <td>8.703704</td>\n",
              "      <td>Low</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>122</td>\n",
              "      <td>alfa-romero</td>\n",
              "      <td>two</td>\n",
              "      <td>hatchback</td>\n",
              "      <td>rwd</td>\n",
              "      <td>front</td>\n",
              "      <td>94.5</td>\n",
              "      <td>0.822681</td>\n",
              "      <td>0.909722</td>\n",
              "      <td>0.876254</td>\n",
              "      <td>2823</td>\n",
              "      <td>ohcv</td>\n",
              "      <td>six</td>\n",
              "      <td>152</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>2.68</td>\n",
              "      <td>3.47</td>\n",
              "      <td>9.0</td>\n",
              "      <td>154</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>19</td>\n",
              "      <td>9.038462</td>\n",
              "      <td>16500.0</td>\n",
              "      <td>12.368421</td>\n",
              "      <td>9.038462</td>\n",
              "      <td>Medium</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>fwd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.8</td>\n",
              "      <td>0.848630</td>\n",
              "      <td>0.919444</td>\n",
              "      <td>0.908027</td>\n",
              "      <td>2337</td>\n",
              "      <td>ohc</td>\n",
              "      <td>four</td>\n",
              "      <td>109</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>10.0</td>\n",
              "      <td>102</td>\n",
              "      <td>5500.0</td>\n",
              "      <td>24</td>\n",
              "      <td>7.833333</td>\n",
              "      <td>13950.0</td>\n",
              "      <td>9.791667</td>\n",
              "      <td>7.833333</td>\n",
              "      <td>Low</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2</td>\n",
              "      <td>164</td>\n",
              "      <td>audi</td>\n",
              "      <td>four</td>\n",
              "      <td>sedan</td>\n",
              "      <td>4wd</td>\n",
              "      <td>front</td>\n",
              "      <td>99.4</td>\n",
              "      <td>0.848630</td>\n",
              "      <td>0.922222</td>\n",
              "      <td>0.908027</td>\n",
              "      <td>2824</td>\n",
              "      <td>ohc</td>\n",
              "      <td>five</td>\n",
              "      <td>136</td>\n",
              "      <td>mpfi</td>\n",
              "      <td>3.19</td>\n",
              "      <td>3.40</td>\n",
              "      <td>8.0</td>\n",
              "      <td>115</td>\n",
              "      <td>5500.0</td>\n",
              "      <td>18</td>\n",
              "      <td>10.681818</td>\n",
              "      <td>17450.0</td>\n",
              "      <td>13.055556</td>\n",
              "      <td>10.681818</td>\n",
              "      <td>Low</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   symboling  normalized-losses         make  ... gas std turbo\n",
              "0          3                122  alfa-romero  ...   1   1     0\n",
              "1          3                122  alfa-romero  ...   1   1     0\n",
              "2          1                122  alfa-romero  ...   1   1     0\n",
              "3          2                164         audi  ...   1   1     0\n",
              "4          2                164         audi  ...   1   1     0\n",
              "\n",
              "[5 rows x 31 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "475UMQiJqRD6"
      },
      "source": [
        "Double-click <b>here</b> for the solution.\n",
        "\n",
        "<!-- The answer is below:\n",
        "\n",
        "#merge the new dataframe to the original datafram\n",
        "df = pd.concat([df, dummy_variable_2], axis=1)\n",
        "\n",
        "# drop original column \"aspiration\" from \"df\"\n",
        "df.drop('aspiration', axis = 1, inplace=True)\n",
        "\n",
        "-->"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZZESdwA7qRD6"
      },
      "source": [
        "save the new csv "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": true,
        "id": "8wjK_6hPqRD7"
      },
      "source": [
        "df.to_csv('clean_df.csv')"
      ],
      "execution_count": 53,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U5YyElVGqRD7"
      },
      "source": [
        "<h1>Thank you for completing this notebook</h1>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mF6g-R4uqRD7"
      },
      "source": [
        "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
        "\n",
        "    <p><a href=\"https://cocl.us/skills_network_DA0101EN_bottom\"><img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DA0101EN/Images/BottomAd.png\" width=\"750\" align=\"center\"></a></p>\n",
        "</div>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bJhfVdpKqRD8"
      },
      "source": [
        "<h3>About the Authors:</h3>\n",
        "\n",
        "This notebook was written by <a href=\"https://www.linkedin.com/in/mahdi-noorian-58219234/\" target=\"_blank\">Mahdi Noorian PhD</a>, <a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\" target=\"_blank\">Joseph Santarcangelo</a>, Bahare Talayian, Eric Xiao, Steven Dong, Parizad, Hima Vsudevan and <a href=\"https://www.linkedin.com/in/fiorellawever/\" target=\"_blank\">Fiorella Wenver</a> and <a href=\" https://www.linkedin.com/in/yi-leng-yao-84451275/ \" target=\"_blank\" >Yi Yao</a>.\n",
        "\n",
        "<p><a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\" target=\"_blank\">Joseph Santarcangelo</a> is a Data Scientist at IBM, and holds a PhD in Electrical Engineering. His research focused on using Machine Learning, Signal Processing, and Computer Vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.</p>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lpqYQxjRqRD8"
      },
      "source": [
        "<hr>\n",
        "<p>Copyright &copy; 2018 IBM Developer Skills Network. This notebook and its source code are released under the terms of the <a href=\"https://cognitiveclass.ai/mit-license/\">MIT License</a>.</p>"
      ]
    }
  ]
}