Projeto_2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Projeto 2 - Ciência dos Dados\n",
    "\n",
    "## Integrantes:\n",
    "* Gabriela Kimi\n",
    "* Luiza Ehrenberger\n",
    "* Pedro Barão\n",
    "* Rafael Paolino"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introdução:\n",
    "\n",
    "## Objetivo: fazer uso de métodos de regressão para prever o preço de carros, usando suas características como base.\n",
    "\n",
    "Para isso, fizemos uso de uma base de dados que contém as características de um carro e o preço sugerido pelo fabricante. Alguns dos atributos utilizados para estimar o preço são, por exemplo: \n",
    "\n",
    "* Tipo de transmissão (Câmbio)\n",
    "* Tipo de combustível utilizado \n",
    "* Ano de lançamento do carro \n",
    "* Tipo de tração\n",
    "\n",
    "<a href= \" https://www.kaggle.com/CooperUnion/cardataset \" > Link para a base de dados \"Car Features and MSRP\"</a>\n",
    "https://www.kaggle.com/CooperUnion/cardataset "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Importando as bibiotecas necessárias: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib notebook\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from scipy.stats import norm, probplot\n",
    "import statsmodels.api as sm\n",
    "import matplotlib.pyplot as plt\n",
    "from statsmodels.formula.api import ols\n",
    "from sklearn.tree import DecisionTreeRegressor \n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from IPython.display import display\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DataFrame da base de dados"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.read_csv(\"data.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine Fuel Type</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Driven_Wheels</th>\n",
       "      <th>Number of Doors</th>\n",
       "      <th>Market Category</th>\n",
       "      <th>Vehicle Size</th>\n",
       "      <th>Vehicle Style</th>\n",
       "      <th>highway MPG</th>\n",
       "      <th>city mpg</th>\n",
       "      <th>Popularity</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BMW</td>\n",
       "      <td>1 Series M</td>\n",
       "      <td>2011</td>\n",
       "      <td>premium unleaded (required)</td>\n",
       "      <td>335.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>rear wheel drive</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Factory Tuner,Luxury,High-Performance</td>\n",
       "      <td>Compact</td>\n",
       "      <td>Coupe</td>\n",
       "      <td>26</td>\n",
       "      <td>19</td>\n",
       "      <td>3916</td>\n",
       "      <td>46135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BMW</td>\n",
       "      <td>1 Series</td>\n",
       "      <td>2011</td>\n",
       "      <td>premium unleaded (required)</td>\n",
       "      <td>300.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>rear wheel drive</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Luxury,Performance</td>\n",
       "      <td>Compact</td>\n",
       "      <td>Convertible</td>\n",
       "      <td>28</td>\n",
       "      <td>19</td>\n",
       "      <td>3916</td>\n",
       "      <td>40650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BMW</td>\n",
       "      <td>1 Series</td>\n",
       "      <td>2011</td>\n",
       "      <td>premium unleaded (required)</td>\n",
       "      <td>300.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>rear wheel drive</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Luxury,High-Performance</td>\n",
       "      <td>Compact</td>\n",
       "      <td>Coupe</td>\n",
       "      <td>28</td>\n",
       "      <td>20</td>\n",
       "      <td>3916</td>\n",
       "      <td>36350</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Make       Model  Year             Engine Fuel Type  Engine HP  \\\n",
       "0  BMW  1 Series M  2011  premium unleaded (required)      335.0   \n",
       "1  BMW    1 Series  2011  premium unleaded (required)      300.0   \n",
       "2  BMW    1 Series  2011  premium unleaded (required)      300.0   \n",
       "\n",
       "   Engine Cylinders Transmission Type     Driven_Wheels  Number of Doors  \\\n",
       "0               6.0            MANUAL  rear wheel drive              2.0   \n",
       "1               6.0            MANUAL  rear wheel drive              2.0   \n",
       "2               6.0            MANUAL  rear wheel drive              2.0   \n",
       "\n",
       "                         Market Category Vehicle Size Vehicle Style  \\\n",
       "0  Factory Tuner,Luxury,High-Performance      Compact         Coupe   \n",
       "1                     Luxury,Performance      Compact   Convertible   \n",
       "2                Luxury,High-Performance      Compact         Coupe   \n",
       "\n",
       "   highway MPG  city mpg  Popularity   MSRP  \n",
       "0           26        19        3916  46135  \n",
       "1           28        19        3916  40650  \n",
       "2           28        20        3916  36350  "
      ]
     },
     "execution_count": 167,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mineirando Dados e Características do Dataset\n",
    "\n",
    "### Colunas que serão utilizadas: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Make', 'Model', 'Year', 'Engine Fuel Type', 'Engine HP',\n",
       "       'Engine Cylinders', 'Transmission Type', 'Driven_Wheels',\n",
       "       'Number of Doors', 'Market Category', 'Vehicle Size', 'Vehicle Style',\n",
       "       'highway MPG', 'city mpg', 'MSRP'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data=data.drop('Popularity',axis=1) # Remoção de varíavel não explicada pela base de dados\n",
    "data.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Descrevendo as variáveis que serão utilizadas:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Targets (varíaveis analisadas):\n",
    "* Make - Marca do carro \n",
    "* Modelo - Modelo do carro \n",
    "* Year - Ano de lançamento do carro \n",
    "* Engine Fuel Type - Tipo de combustível utilizado \n",
    "* Engine HP - Potência do motor (HP)\n",
    "* Engine Cylinders - Número de cilindros do motor \n",
    "* Transmission Type - Tipo de transmissão (Câmbio)\n",
    "* Driven wheels - Tipo de tração\n",
    "* Number of doors - Número de portas \n",
    "* Market Category - Categorias usadas no mercado \n",
    "* Vehicle Size - Classificação do tamanho do veículo \n",
    "* Vehicle Style - Classificação de estilo do veículo\n",
    "* Highway MPG - Média de rendimento de gasolina na auto-estrada (milhas por galão)\n",
    "* City MPG - Média de rendimento de gasolina na cidade (milhas por galão)\n",
    "\n",
    "### Feature (varíavel buscada):\n",
    "* MSRP - preço de varejo recomendado pelo fabricante do carro (dólares)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Remoção de células sem valores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Make                 0\n",
       "Model                0\n",
       "Year                 0\n",
       "Engine Fuel Type     0\n",
       "Engine HP            0\n",
       "Engine Cylinders     0\n",
       "Transmission Type    0\n",
       "Driven_Wheels        0\n",
       "Number of Doors      0\n",
       "Market Category      0\n",
       "Vehicle Size         0\n",
       "Vehicle Style        0\n",
       "highway MPG          0\n",
       "city mpg             0\n",
       "MSRP                 0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 169,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data=data.dropna()  \n",
    "data.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Codificação de colunas categóricas para integer e hot encode:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Isso é feito para facilitar o uso de colunas com valores categóricos nos modelos de regressão"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [],
   "source": [
    "data=data.drop(['Make','Model','Vehicle Style','Market Category'],axis=1) # Remoção de colunas com quantidade excessiva de categorias individuais\n",
    "data_2=pd.get_dummies(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Criação de DataFrames de treinamento e de teste:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [],
   "source": [
    "train=data_2.sample(frac=0.9,random_state=0)\n",
    "\n",
    "test=data_2.drop(train.index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modelos de Predição"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Regressão Linear MMQ: tenta-se aproximar o comportamento da variável target a uma reta, em função dos valores das features. Esta reta pode ser usada para prever o valor target de interesse\n",
    "\n",
    "Para tal, utilizamos a função OLS da biblioteca statsmodels.formula.api\n",
    "\n",
    "Regressão em Árvore: usa-se uma árvore de decisões como modelo preditivo, a fim de chegar a uma estimativa do valor de interesse\n",
    "\n",
    "Para tal, utilizamos a função DecisionTreeRegressor da biblioteca sklearn\n",
    "\n",
    "Ambas são treinadas pela porção de treinamento dos dados e têm seus erros médios medidos com relação à porção de teste"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Regressão linear MMQ:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Função de regressão linear:\n",
    "\n",
    "#### Y: coluna do DataFrame utilizada como variável resposta. (TARGET)\n",
    "#### X: coluna(s) do DataFrame utilizada(s) como variável(is) explicativas. (FEATURES)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [],
   "source": [
    "def regress(Y,X):\n",
    "    X_cp = sm.add_constant(X)\n",
    "    model = sm.OLS(Y,X_cp)\n",
    "    results = model.fit()\n",
    "    \n",
    "    return results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda3\\lib\\site-packages\\statsmodels\\tsa\\tsatools.py:142: FutureWarning: In a future version of pandas all arguments of concat except for the argument 'objs' will be keyword-only\n",
      "  x = pd.concat(x[::order], 1)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>          <td>MSRP</td>       <th>  R-squared:         </th> <td>   0.540</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.538</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   386.5</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Tue, 30 Nov 2021</td> <th>  Prob (F-statistic):</th>  <td>  0.00</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>21:06:55</td>     <th>  Log-Likelihood:    </th> <td> -88380.</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>  7276</td>      <th>  AIC:               </th> <td>1.768e+05</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>  7253</td>      <th>  BIC:               </th> <td>1.770e+05</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>    22</td>      <th>                     </th>     <td> </td>    \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>    \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "                                <td></td>                                   <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>const</th>                                                         <td>-2.432e+05</td> <td> 1.23e+05</td> <td>   -1.979</td> <td> 0.048</td> <td>-4.84e+05</td> <td>-2271.421</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Year</th>                                                          <td>  163.0812</td> <td>  117.902</td> <td>    1.383</td> <td> 0.167</td> <td>  -68.040</td> <td>  394.202</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine HP</th>                                                     <td>  270.2741</td> <td>   11.449</td> <td>   23.607</td> <td> 0.000</td> <td>  247.831</td> <td>  292.717</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine Cylinders</th>                                              <td> 1.331e+04</td> <td>  632.669</td> <td>   21.046</td> <td> 0.000</td> <td> 1.21e+04</td> <td> 1.46e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Number of Doors</th>                                               <td>-1118.9402</td> <td>  767.863</td> <td>   -1.457</td> <td> 0.145</td> <td>-2624.174</td> <td>  386.294</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>highway MPG</th>                                                   <td>  -31.7046</td> <td>  126.324</td> <td>   -0.251</td> <td> 0.802</td> <td> -279.337</td> <td>  215.928</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>city mpg</th>                                                      <td> 1266.3160</td> <td>  169.276</td> <td>    7.481</td> <td> 0.000</td> <td>  934.487</td> <td> 1598.145</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine Fuel Type_diesel</th>                                       <td>-1.626e+04</td> <td>  1.7e+04</td> <td>   -0.955</td> <td> 0.340</td> <td>-4.96e+04</td> <td> 1.71e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine Fuel Type_electric</th>                                     <td>-6.866e+04</td> <td> 3.34e+04</td> <td>   -2.054</td> <td> 0.040</td> <td>-1.34e+05</td> <td>-3142.879</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine Fuel Type_flex-fuel (premium unleaded recommended/E85)</th> <td> -2.68e+04</td> <td> 1.87e+04</td> <td>   -1.431</td> <td> 0.152</td> <td>-6.35e+04</td> <td> 9898.482</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine Fuel Type_flex-fuel (premium unleaded required/E85)</th>    <td>-9654.5159</td> <td> 1.74e+04</td> <td>   -0.554</td> <td> 0.580</td> <td>-4.38e+04</td> <td> 2.45e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine Fuel Type_flex-fuel (unleaded/E85)</th>                     <td>-4.054e+04</td> <td> 1.71e+04</td> <td>   -2.370</td> <td> 0.018</td> <td>-7.41e+04</td> <td>-7011.859</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine Fuel Type_premium unleaded (recommended)</th>               <td>-3.365e+04</td> <td> 1.68e+04</td> <td>   -2.000</td> <td> 0.046</td> <td>-6.66e+04</td> <td> -659.918</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine Fuel Type_premium unleaded (required)</th>                  <td>-1.535e+04</td> <td> 1.66e+04</td> <td>   -0.924</td> <td> 0.356</td> <td>-4.79e+04</td> <td> 1.72e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine Fuel Type_regular unleaded</th>                             <td>-3.231e+04</td> <td> 1.65e+04</td> <td>   -1.953</td> <td> 0.051</td> <td>-6.47e+04</td> <td>  121.929</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Transmission Type_AUTOMATED_MANUAL</th>                            <td>-1.972e+04</td> <td> 2.62e+04</td> <td>   -0.752</td> <td> 0.452</td> <td>-7.11e+04</td> <td> 3.17e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Transmission Type_AUTOMATIC</th>                                   <td>-3.912e+04</td> <td> 2.64e+04</td> <td>   -1.482</td> <td> 0.138</td> <td>-9.09e+04</td> <td> 1.26e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Transmission Type_DIRECT_DRIVE</th>                                <td>-4.573e+04</td> <td> 3.63e+04</td> <td>   -1.261</td> <td> 0.207</td> <td>-1.17e+05</td> <td> 2.54e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Transmission Type_MANUAL</th>                                      <td>-4.704e+04</td> <td>  2.6e+04</td> <td>   -1.808</td> <td> 0.071</td> <td> -9.8e+04</td> <td> 3957.568</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Transmission Type_UNKNOWN</th>                                     <td>-9.161e+04</td> <td> 3.65e+04</td> <td>   -2.509</td> <td> 0.012</td> <td>-1.63e+05</td> <td>   -2e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Driven_Wheels_all wheel drive</th>                                 <td> -5.36e+04</td> <td> 3.08e+04</td> <td>   -1.740</td> <td> 0.082</td> <td>-1.14e+05</td> <td> 6773.548</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Driven_Wheels_four wheel drive</th>                                <td>-7.126e+04</td> <td> 3.09e+04</td> <td>   -2.307</td> <td> 0.021</td> <td>-1.32e+05</td> <td>-1.07e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Driven_Wheels_front wheel drive</th>                               <td>-5.027e+04</td> <td> 3.07e+04</td> <td>   -1.639</td> <td> 0.101</td> <td> -1.1e+05</td> <td> 9850.730</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Driven_Wheels_rear wheel drive</th>                                <td>-6.809e+04</td> <td> 3.07e+04</td> <td>   -2.220</td> <td> 0.026</td> <td>-1.28e+05</td> <td>-7970.166</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Vehicle Size_Compact</th>                                          <td>-6.764e+04</td> <td>  4.1e+04</td> <td>   -1.649</td> <td> 0.099</td> <td>-1.48e+05</td> <td> 1.28e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Vehicle Size_Large</th>                                            <td>-8.875e+04</td> <td>  4.1e+04</td> <td>   -2.165</td> <td> 0.030</td> <td>-1.69e+05</td> <td>-8407.970</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Vehicle Size_Midsize</th>                                          <td>-8.684e+04</td> <td>  4.1e+04</td> <td>   -2.121</td> <td> 0.034</td> <td>-1.67e+05</td> <td>-6562.896</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td>14013.228</td> <th>  Durbin-Watson:     </th>   <td>   2.012</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th>  <td> 0.000</td>   <th>  Jarque-Bera (JB):  </th> <td>55711502.785</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>           <td>14.712</td>   <th>  Prob(JB):          </th>   <td>    0.00</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>       <td>430.667</td>  <th>  Cond. No.          </th>   <td>7.57e+16</td>  \n",
       "</tr>\n",
       "</table><br/><br/>Notes:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The smallest eigenvalue is 5.24e-24. This might indicate that there are<br/>strong multicollinearity problems or that the design matrix is singular."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                   MSRP   R-squared:                       0.540\n",
       "Model:                            OLS   Adj. R-squared:                  0.538\n",
       "Method:                 Least Squares   F-statistic:                     386.5\n",
       "Date:                Tue, 30 Nov 2021   Prob (F-statistic):               0.00\n",
       "Time:                        21:06:55   Log-Likelihood:                -88380.\n",
       "No. Observations:                7276   AIC:                         1.768e+05\n",
       "Df Residuals:                    7253   BIC:                         1.770e+05\n",
       "Df Model:                          22                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "=================================================================================================================================\n",
       "                                                                    coef    std err          t      P>|t|      [0.025      0.975]\n",
       "---------------------------------------------------------------------------------------------------------------------------------\n",
       "const                                                         -2.432e+05   1.23e+05     -1.979      0.048   -4.84e+05   -2271.421\n",
       "Year                                                            163.0812    117.902      1.383      0.167     -68.040     394.202\n",
       "Engine HP                                                       270.2741     11.449     23.607      0.000     247.831     292.717\n",
       "Engine Cylinders                                               1.331e+04    632.669     21.046      0.000    1.21e+04    1.46e+04\n",
       "Number of Doors                                               -1118.9402    767.863     -1.457      0.145   -2624.174     386.294\n",
       "highway MPG                                                     -31.7046    126.324     -0.251      0.802    -279.337     215.928\n",
       "city mpg                                                       1266.3160    169.276      7.481      0.000     934.487    1598.145\n",
       "Engine Fuel Type_diesel                                       -1.626e+04    1.7e+04     -0.955      0.340   -4.96e+04    1.71e+04\n",
       "Engine Fuel Type_electric                                     -6.866e+04   3.34e+04     -2.054      0.040   -1.34e+05   -3142.879\n",
       "Engine Fuel Type_flex-fuel (premium unleaded recommended/E85)  -2.68e+04   1.87e+04     -1.431      0.152   -6.35e+04    9898.482\n",
       "Engine Fuel Type_flex-fuel (premium unleaded required/E85)    -9654.5159   1.74e+04     -0.554      0.580   -4.38e+04    2.45e+04\n",
       "Engine Fuel Type_flex-fuel (unleaded/E85)                     -4.054e+04   1.71e+04     -2.370      0.018   -7.41e+04   -7011.859\n",
       "Engine Fuel Type_premium unleaded (recommended)               -3.365e+04   1.68e+04     -2.000      0.046   -6.66e+04    -659.918\n",
       "Engine Fuel Type_premium unleaded (required)                  -1.535e+04   1.66e+04     -0.924      0.356   -4.79e+04    1.72e+04\n",
       "Engine Fuel Type_regular unleaded                             -3.231e+04   1.65e+04     -1.953      0.051   -6.47e+04     121.929\n",
       "Transmission Type_AUTOMATED_MANUAL                            -1.972e+04   2.62e+04     -0.752      0.452   -7.11e+04    3.17e+04\n",
       "Transmission Type_AUTOMATIC                                   -3.912e+04   2.64e+04     -1.482      0.138   -9.09e+04    1.26e+04\n",
       "Transmission Type_DIRECT_DRIVE                                -4.573e+04   3.63e+04     -1.261      0.207   -1.17e+05    2.54e+04\n",
       "Transmission Type_MANUAL                                      -4.704e+04    2.6e+04     -1.808      0.071    -9.8e+04    3957.568\n",
       "Transmission Type_UNKNOWN                                     -9.161e+04   3.65e+04     -2.509      0.012   -1.63e+05      -2e+04\n",
       "Driven_Wheels_all wheel drive                                  -5.36e+04   3.08e+04     -1.740      0.082   -1.14e+05    6773.548\n",
       "Driven_Wheels_four wheel drive                                -7.126e+04   3.09e+04     -2.307      0.021   -1.32e+05   -1.07e+04\n",
       "Driven_Wheels_front wheel drive                               -5.027e+04   3.07e+04     -1.639      0.101    -1.1e+05    9850.730\n",
       "Driven_Wheels_rear wheel drive                                -6.809e+04   3.07e+04     -2.220      0.026   -1.28e+05   -7970.166\n",
       "Vehicle Size_Compact                                          -6.764e+04    4.1e+04     -1.649      0.099   -1.48e+05    1.28e+04\n",
       "Vehicle Size_Large                                            -8.875e+04    4.1e+04     -2.165      0.030   -1.69e+05   -8407.970\n",
       "Vehicle Size_Midsize                                          -8.684e+04    4.1e+04     -2.121      0.034   -1.67e+05   -6562.896\n",
       "==============================================================================\n",
       "Omnibus:                    14013.228   Durbin-Watson:                   2.012\n",
       "Prob(Omnibus):                  0.000   Jarque-Bera (JB):         55711502.785\n",
       "Skew:                          14.712   Prob(JB):                         0.00\n",
       "Kurtosis:                     430.667   Cond. No.                     7.57e+16\n",
       "==============================================================================\n",
       "\n",
       "Notes:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The smallest eigenvalue is 5.24e-24. This might indicate that there are\n",
       "strong multicollinearity problems or that the design matrix is singular.\n",
       "\"\"\""
      ]
     },
     "execution_count": 173,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X=train.drop(['MSRP'],axis=1)\n",
    "Y=train[['MSRP']]\n",
    "\n",
    "results=regress(Y,X)\n",
    "results.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Remoção de colunas com valor p > 10%:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=X.drop(['Year','Number of Doors','highway MPG','Engine Fuel Type_diesel','Transmission Type_UNKNOWN','Engine Fuel Type_electric','Engine Fuel Type_electric','Engine Fuel Type_electric','Engine Fuel Type_flex-fuel (premium unleaded recommended/E85)','Transmission Type_AUTOMATIC','Engine Fuel Type_premium unleaded (required)','Engine Fuel Type_flex-fuel (premium unleaded required/E85)'],axis=1)\n",
    "Y=train[['MSRP']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda3\\lib\\site-packages\\statsmodels\\tsa\\tsatools.py:142: FutureWarning: In a future version of pandas all arguments of concat except for the argument 'objs' will be keyword-only\n",
      "  x = pd.concat(x[::order], 1)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>          <td>MSRP</td>       <th>  R-squared:         </th> <td>   0.539</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.538</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   606.3</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Tue, 30 Nov 2021</td> <th>  Prob (F-statistic):</th>  <td>  0.00</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>21:06:55</td>     <th>  Log-Likelihood:    </th> <td> -88385.</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>  7276</td>      <th>  AIC:               </th> <td>1.768e+05</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>  7261</td>      <th>  BIC:               </th> <td>1.769e+05</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>    14</td>      <th>                     </th>     <td> </td>    \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>    \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "                         <td></td>                            <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>const</th>                                           <td>-7.426e+04</td> <td> 3088.922</td> <td>  -24.039</td> <td> 0.000</td> <td>-8.03e+04</td> <td>-6.82e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine HP</th>                                       <td>  279.7097</td> <td>    9.557</td> <td>   29.266</td> <td> 0.000</td> <td>  260.974</td> <td>  298.445</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine Cylinders</th>                                <td> 1.309e+04</td> <td>  588.908</td> <td>   22.222</td> <td> 0.000</td> <td> 1.19e+04</td> <td> 1.42e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>city mpg</th>                                        <td> 1264.8061</td> <td>  127.511</td> <td>    9.919</td> <td> 0.000</td> <td> 1014.847</td> <td> 1514.765</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine Fuel Type_flex-fuel (unleaded/E85)</th>       <td> -2.44e+04</td> <td> 2267.887</td> <td>  -10.759</td> <td> 0.000</td> <td>-2.88e+04</td> <td>   -2e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine Fuel Type_premium unleaded (recommended)</th> <td>-1.783e+04</td> <td> 1749.535</td> <td>  -10.193</td> <td> 0.000</td> <td>-2.13e+04</td> <td>-1.44e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Engine Fuel Type_regular unleaded</th>               <td>-1.691e+04</td> <td> 1642.268</td> <td>  -10.298</td> <td> 0.000</td> <td>-2.01e+04</td> <td>-1.37e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Transmission Type_AUTOMATED_MANUAL</th>              <td> 1.943e+04</td> <td> 2277.781</td> <td>    8.530</td> <td> 0.000</td> <td>  1.5e+04</td> <td> 2.39e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Transmission Type_DIRECT_DRIVE</th>                  <td>-5.464e+04</td> <td>  1.6e+04</td> <td>   -3.425</td> <td> 0.001</td> <td>-8.59e+04</td> <td>-2.34e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Transmission Type_MANUAL</th>                        <td>-8016.7089</td> <td> 1573.681</td> <td>   -5.094</td> <td> 0.000</td> <td>-1.11e+04</td> <td>-4931.836</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Driven_Wheels_all wheel drive</th>                   <td>-1.157e+04</td> <td> 1202.265</td> <td>   -9.620</td> <td> 0.000</td> <td>-1.39e+04</td> <td>-9209.197</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Driven_Wheels_four wheel drive</th>                  <td>-2.878e+04</td> <td> 1857.529</td> <td>  -15.494</td> <td> 0.000</td> <td>-3.24e+04</td> <td>-2.51e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Driven_Wheels_front wheel drive</th>                 <td>-8187.8522</td> <td> 1394.814</td> <td>   -5.870</td> <td> 0.000</td> <td>-1.09e+04</td> <td>-5453.612</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Driven_Wheels_rear wheel drive</th>                  <td>-2.572e+04</td> <td> 1296.877</td> <td>  -19.833</td> <td> 0.000</td> <td>-2.83e+04</td> <td>-2.32e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Vehicle Size_Compact</th>                            <td>-1.075e+04</td> <td> 1357.650</td> <td>   -7.915</td> <td> 0.000</td> <td>-1.34e+04</td> <td>-8084.978</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Vehicle Size_Large</th>                              <td>-3.292e+04</td> <td> 1605.159</td> <td>  -20.506</td> <td> 0.000</td> <td>-3.61e+04</td> <td>-2.98e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Vehicle Size_Midsize</th>                            <td>-3.059e+04</td> <td> 1225.053</td> <td>  -24.973</td> <td> 0.000</td> <td> -3.3e+04</td> <td>-2.82e+04</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td>13955.601</td> <th>  Durbin-Watson:     </th>   <td>   2.012</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th>  <td> 0.000</td>   <th>  Jarque-Bera (JB):  </th> <td>54418923.501</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>           <td>14.579</td>   <th>  Prob(JB):          </th>   <td>    0.00</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>       <td>425.672</td>  <th>  Cond. No.          </th>   <td>3.51e+18</td>  \n",
       "</tr>\n",
       "</table><br/><br/>Notes:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The smallest eigenvalue is 5.24e-29. This might indicate that there are<br/>strong multicollinearity problems or that the design matrix is singular."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                   MSRP   R-squared:                       0.539\n",
       "Model:                            OLS   Adj. R-squared:                  0.538\n",
       "Method:                 Least Squares   F-statistic:                     606.3\n",
       "Date:                Tue, 30 Nov 2021   Prob (F-statistic):               0.00\n",
       "Time:                        21:06:55   Log-Likelihood:                -88385.\n",
       "No. Observations:                7276   AIC:                         1.768e+05\n",
       "Df Residuals:                    7261   BIC:                         1.769e+05\n",
       "Df Model:                          14                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "===================================================================================================================\n",
       "                                                      coef    std err          t      P>|t|      [0.025      0.975]\n",
       "-------------------------------------------------------------------------------------------------------------------\n",
       "const                                           -7.426e+04   3088.922    -24.039      0.000   -8.03e+04   -6.82e+04\n",
       "Engine HP                                         279.7097      9.557     29.266      0.000     260.974     298.445\n",
       "Engine Cylinders                                 1.309e+04    588.908     22.222      0.000    1.19e+04    1.42e+04\n",
       "city mpg                                         1264.8061    127.511      9.919      0.000    1014.847    1514.765\n",
       "Engine Fuel Type_flex-fuel (unleaded/E85)        -2.44e+04   2267.887    -10.759      0.000   -2.88e+04      -2e+04\n",
       "Engine Fuel Type_premium unleaded (recommended) -1.783e+04   1749.535    -10.193      0.000   -2.13e+04   -1.44e+04\n",
       "Engine Fuel Type_regular unleaded               -1.691e+04   1642.268    -10.298      0.000   -2.01e+04   -1.37e+04\n",
       "Transmission Type_AUTOMATED_MANUAL               1.943e+04   2277.781      8.530      0.000     1.5e+04    2.39e+04\n",
       "Transmission Type_DIRECT_DRIVE                  -5.464e+04    1.6e+04     -3.425      0.001   -8.59e+04   -2.34e+04\n",
       "Transmission Type_MANUAL                        -8016.7089   1573.681     -5.094      0.000   -1.11e+04   -4931.836\n",
       "Driven_Wheels_all wheel drive                   -1.157e+04   1202.265     -9.620      0.000   -1.39e+04   -9209.197\n",
       "Driven_Wheels_four wheel drive                  -2.878e+04   1857.529    -15.494      0.000   -3.24e+04   -2.51e+04\n",
       "Driven_Wheels_front wheel drive                 -8187.8522   1394.814     -5.870      0.000   -1.09e+04   -5453.612\n",
       "Driven_Wheels_rear wheel drive                  -2.572e+04   1296.877    -19.833      0.000   -2.83e+04   -2.32e+04\n",
       "Vehicle Size_Compact                            -1.075e+04   1357.650     -7.915      0.000   -1.34e+04   -8084.978\n",
       "Vehicle Size_Large                              -3.292e+04   1605.159    -20.506      0.000   -3.61e+04   -2.98e+04\n",
       "Vehicle Size_Midsize                            -3.059e+04   1225.053    -24.973      0.000    -3.3e+04   -2.82e+04\n",
       "==============================================================================\n",
       "Omnibus:                    13955.601   Durbin-Watson:                   2.012\n",
       "Prob(Omnibus):                  0.000   Jarque-Bera (JB):         54418923.501\n",
       "Skew:                          14.579   Prob(JB):                         0.00\n",
       "Kurtosis:                     425.672   Cond. No.                     3.51e+18\n",
       "==============================================================================\n",
       "\n",
       "Notes:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The smallest eigenvalue is 5.24e-29. This might indicate that there are\n",
       "strong multicollinearity problems or that the design matrix is singular.\n",
       "\"\"\""
      ]
     },
     "execution_count": 175,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results=regress(Y,X)\n",
    "results.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Regressão em árvore:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=test.drop(['MSRP','Year','Number of Doors','highway MPG','Engine Fuel Type_diesel','Engine Fuel Type_electric','Engine Fuel Type_electric','Engine Fuel Type_electric','Engine Fuel Type_flex-fuel (premium unleaded recommended/E85)','Transmission Type_AUTOMATIC','Engine Fuel Type_premium unleaded (required)','Engine Fuel Type_flex-fuel (premium unleaded required/E85)'],axis=1)\n",
    "Y=test['MSRP']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeRegressor(random_state=0)"
      ]
     },
     "execution_count": 177,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regressor = DecisionTreeRegressor(random_state = 0) \n",
    "\n",
    "regressor.fit(X, Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testes e cálculo de erro médio :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [],
   "source": [
    "tree_pred = regressor.predict(X)\n",
    "\n",
    "linear_y=results.predict(test.drop(['Year','Number of Doors','Transmission Type_UNKNOWN','highway MPG','Engine Fuel Type_diesel','Engine Fuel Type_electric','Engine Fuel Type_electric','Engine Fuel Type_electric','Engine Fuel Type_flex-fuel (premium unleaded recommended/E85)','Transmission Type_AUTOMATIC','Engine Fuel Type_premium unleaded (required)','Engine Fuel Type_flex-fuel (premium unleaded required/E85)'],axis=1))\n",
    "\n",
    "tree_pred=pd.Series(tree_pred)\n",
    "\n",
    "MSE_linear=mean_squared_error(test['MSRP'],linear_y,squared=False)\n",
    "MSE_tree=mean_squared_error(test['MSRP'],tree_pred,squared=False)\n",
    "\n",
    "df=pd.DataFrame({'MSRP(test)':test['MSRP'],'Preço(tree)':tree_pred,'Preço(linear)':linear_y})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Erro médio da Regressão em Árvore: 3032.18 dólares\n",
      "Erro médio da Regressão linear: 122771319.08 dólares\n",
      "Erro médio da Regressão em Árvore / média: 5.617%\n",
      "Erro médio da Regressão linear / média: 227412.190%\n"
     ]
    }
   ],
   "source": [
    "print('Erro médio da Regressão em Árvore: {:.2f} dólares'.format(MSE_tree))\n",
    "print('Erro médio da Regressão linear: {:.2f} dólares'.format(MSE_linear))\n",
    "print('Erro médio da Regressão em Árvore / média: {:.3f}%'.format(MSE_tree/test['MSRP'].mean()*100))\n",
    "print('Erro médio da Regressão linear / média: {:.3f}%'.format(MSE_linear/test['MSRP'].mean()*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusões:\n",
    "\n",
    "Considerando que o erro médio da Regressão em Árvore consiste em apenas 5.6% do valor médio dos carros, pode-se afirmar que esse modelo consegue usar os dados para prever o comportamento do preço de maneira satisfatória.\n",
    "\n",
    "Entretanto, como o erro da regressão linear consiste em 227412% do valor médio, pode-se perceber que esse modelo não consegue prever o preço dos carros com base em nas características consideradas.\n",
    "\n",
    "Isso nos indica que o preço dos veículo não varia conforme as features de forma linear "
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "5f64f01a6cdb813bd23f3c2980301e513b65108ef9650403b1a46775be8a9390"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}