mnist-cnn-em-portugu-s.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.021354,
     "end_time": "2020-10-23T18:56:04.965238",
     "exception": false,
     "start_time": "2020-10-23T18:56:04.943884",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Introdução"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.020649,
     "end_time": "2020-10-23T18:56:05.005781",
     "exception": false,
     "start_time": "2020-10-23T18:56:04.985132",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Esse código, comentado e explicado em português, foi feito para pessoas que tem interesse em aprender mais sobre o uso de Redes Neurais Covolucionais, mas que ainda não tenham tanta facilidade com inglês.\n",
    "\n",
    "Caso tenha alguma sugestão de melhoria coloque nos comentários."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.02025,
     "end_time": "2020-10-23T18:56:05.047127",
     "exception": false,
     "start_time": "2020-10-23T18:56:05.026877",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "MNIST é uma base de dados com números \"desenhados\" e com seus respectivos rotulos que indicam qual número esta desenhado. Nesse desafio, temos a base de dados com os rótulos que será usada para treinar o modelo(base de treino), e a sem rótulos, que devemos prever qual número será.\n",
    "\n",
    "Para isso, no código foi usado, principalmente, a API keras(Tensorflow backend) e foi obtido uma acurácia de  99,47%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.020284,
     "end_time": "2020-10-23T18:56:05.087983",
     "exception": false,
     "start_time": "2020-10-23T18:56:05.067699",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Importações Iniciais"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:05.142585Z",
     "iopub.status.busy": "2020-10-23T18:56:05.141750Z",
     "iopub.status.idle": "2020-10-23T18:56:12.190360Z",
     "shell.execute_reply": "2020-10-23T18:56:12.189130Z"
    },
    "papermill": {
     "duration": 7.08064,
     "end_time": "2020-10-23T18:56:12.190506",
     "exception": false,
     "start_time": "2020-10-23T18:56:05.109866",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy as np # Algebra linear\n",
    "import pandas as pd\n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "\n",
    "import itertools\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "from tensorflow.keras.utils import to_categorical #(one-hot-encoding)\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D, MaxPooling2D\n",
    "from tensorflow.keras.optimizers import RMSprop\n",
    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
    "from tensorflow.keras.callbacks import ReduceLROnPlateau"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:12.245887Z",
     "iopub.status.busy": "2020-10-23T18:56:12.245241Z",
     "iopub.status.idle": "2020-10-23T18:56:18.652359Z",
     "shell.execute_reply": "2020-10-23T18:56:18.651217Z"
    },
    "papermill": {
     "duration": 6.436172,
     "end_time": "2020-10-23T18:56:18.652484",
     "exception": false,
     "start_time": "2020-10-23T18:56:12.216312",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Importando a base de dados\n",
    "train = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\n",
    "test = pd.read_csv('/kaggle/input/digit-recognizer/test.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:18.702379Z",
     "iopub.status.busy": "2020-10-23T18:56:18.701361Z",
     "iopub.status.idle": "2020-10-23T18:56:18.706994Z",
     "shell.execute_reply": "2020-10-23T18:56:18.706450Z"
    },
    "papermill": {
     "duration": 0.032989,
     "end_time": "2020-10-23T18:56:18.707103",
     "exception": false,
     "start_time": "2020-10-23T18:56:18.674114",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((42000, 785), (28000, 784))"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.shape, test.shape # Modo como os dados estão estrutudados (42000 linhas, 785 colunas), (28000 linhas, 784 colunas) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:18.767388Z",
     "iopub.status.busy": "2020-10-23T18:56:18.764918Z",
     "iopub.status.idle": "2020-10-23T18:56:18.780756Z",
     "shell.execute_reply": "2020-10-23T18:56:18.781372Z"
    },
    "papermill": {
     "duration": 0.052723,
     "end_time": "2020-10-23T18:56:18.781524",
     "exception": false,
     "start_time": "2020-10-23T18:56:18.728801",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>pixel0</th>\n",
       "      <th>pixel1</th>\n",
       "      <th>pixel2</th>\n",
       "      <th>pixel3</th>\n",
       "      <th>pixel4</th>\n",
       "      <th>pixel5</th>\n",
       "      <th>pixel6</th>\n",
       "      <th>pixel7</th>\n",
       "      <th>pixel8</th>\n",
       "      <th>...</th>\n",
       "      <th>pixel774</th>\n",
       "      <th>pixel775</th>\n",
       "      <th>pixel776</th>\n",
       "      <th>pixel777</th>\n",
       "      <th>pixel778</th>\n",
       "      <th>pixel779</th>\n",
       "      <th>pixel780</th>\n",
       "      <th>pixel781</th>\n",
       "      <th>pixel782</th>\n",
       "      <th>pixel783</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 785 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   label  pixel0  pixel1  pixel2  pixel3  pixel4  pixel5  pixel6  pixel7  \\\n",
       "0      1       0       0       0       0       0       0       0       0   \n",
       "1      0       0       0       0       0       0       0       0       0   \n",
       "2      1       0       0       0       0       0       0       0       0   \n",
       "3      4       0       0       0       0       0       0       0       0   \n",
       "4      0       0       0       0       0       0       0       0       0   \n",
       "\n",
       "   pixel8  ...  pixel774  pixel775  pixel776  pixel777  pixel778  pixel779  \\\n",
       "0       0  ...         0         0         0         0         0         0   \n",
       "1       0  ...         0         0         0         0         0         0   \n",
       "2       0  ...         0         0         0         0         0         0   \n",
       "3       0  ...         0         0         0         0         0         0   \n",
       "4       0  ...         0         0         0         0         0         0   \n",
       "\n",
       "   pixel780  pixel781  pixel782  pixel783  \n",
       "0         0         0         0         0  \n",
       "1         0         0         0         0  \n",
       "2         0         0         0         0  \n",
       "3         0         0         0         0  \n",
       "4         0         0         0         0  \n",
       "\n",
       "[5 rows x 785 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.022074,
     "end_time": "2020-10-23T18:56:18.825993",
     "exception": false,
     "start_time": "2020-10-23T18:56:18.803919",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Como pode-se ver acima, cada coluna da linha representa um pixel da imagem, e o número no pixel representa a cor(RGB) dele.\n",
    "\n",
    "OBS: As imagens dos números estão em escala de cinza. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:18.876339Z",
     "iopub.status.busy": "2020-10-23T18:56:18.875260Z",
     "iopub.status.idle": "2020-10-23T18:56:18.982318Z",
     "shell.execute_reply": "2020-10-23T18:56:18.981716Z"
    },
    "papermill": {
     "duration": 0.135272,
     "end_time": "2020-10-23T18:56:18.982457",
     "exception": false,
     "start_time": "2020-10-23T18:56:18.847185",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "X_train = train.iloc[:,1:].values.astype('float32')\n",
    "y_train = train['label'].values.astype('int32')\n",
    "test = test.values.astype('float32')\n",
    "\n",
    "del train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:19.033129Z",
     "iopub.status.busy": "2020-10-23T18:56:19.031861Z",
     "iopub.status.idle": "2020-10-23T18:56:19.106288Z",
     "shell.execute_reply": "2020-10-23T18:56:19.105712Z"
    },
    "papermill": {
     "duration": 0.101695,
     "end_time": "2020-10-23T18:56:19.106414",
     "exception": false,
     "start_time": "2020-10-23T18:56:19.004719",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "X_train = X_train / 255.0 # Normalizando os dados de treino\n",
    "test = test / 255.0 # Normalizando os dados de teste"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:19.159259Z",
     "iopub.status.busy": "2020-10-23T18:56:19.157739Z",
     "iopub.status.idle": "2020-10-23T18:56:19.159997Z",
     "shell.execute_reply": "2020-10-23T18:56:19.160487Z"
    },
    "papermill": {
     "duration": 0.030674,
     "end_time": "2020-10-23T18:56:19.160634",
     "exception": false,
     "start_time": "2020-10-23T18:56:19.129960",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Converte o dataset de treino para o formato (num_imagens, img_linhas, img_colunas)\n",
    "X_train = X_train.reshape(X_train.shape[0], 28, 28, 1)\n",
    "test = test.reshape(test.shape[0], 28, 28, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:19.214561Z",
     "iopub.status.busy": "2020-10-23T18:56:19.213202Z",
     "iopub.status.idle": "2020-10-23T18:56:19.217275Z",
     "shell.execute_reply": "2020-10-23T18:56:19.217783Z"
    },
    "papermill": {
     "duration": 0.035202,
     "end_time": "2020-10-23T18:56:19.217914",
     "exception": false,
     "start_time": "2020-10-23T18:56:19.182712",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0, 1, ..., 7, 6, 9], dtype=int32)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.021778,
     "end_time": "2020-10-23T18:56:19.262507",
     "exception": false,
     "start_time": "2020-10-23T18:56:19.240729",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "É comum que bases de dados como essa, tenham os resultados das classificações de imagens como mostrado no ultimo código. Entretanto, o algoritmo utilizado lida melhor com outro tipo de dado, por isso, vamos categoriza-lo de uma forma diferente. Nesse caso, usaremos o one-hot encoding:\n",
    "\n",
    "Exemplo:\n",
    "\n",
    "    O que era apenas o número 2 vira um array de zeros no qual o indice 2 do array é igual a 1\n",
    "    2 -> [0,0,1,0,0,0,0,0,0,0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:19.312988Z",
     "iopub.status.busy": "2020-10-23T18:56:19.312245Z",
     "iopub.status.idle": "2020-10-23T18:56:19.322029Z",
     "shell.execute_reply": "2020-10-23T18:56:19.321354Z"
    },
    "papermill": {
     "duration": 0.036673,
     "end_time": "2020-10-23T18:56:19.322153",
     "exception": false,
     "start_time": "2020-10-23T18:56:19.285480",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 1., 0., ..., 0., 0., 0.],\n",
       "       [1., 0., 0., ..., 0., 0., 0.],\n",
       "       [0., 1., 0., ..., 0., 0., 0.],\n",
       "       ...,\n",
       "       [0., 0., 0., ..., 1., 0., 0.],\n",
       "       [0., 0., 0., ..., 0., 0., 0.],\n",
       "       [0., 0., 0., ..., 0., 0., 1.]], dtype=float32)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train = to_categorical(y_train)\n",
    "y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:19.378286Z",
     "iopub.status.busy": "2020-10-23T18:56:19.376413Z",
     "iopub.status.idle": "2020-10-23T18:56:19.379056Z",
     "shell.execute_reply": "2020-10-23T18:56:19.379553Z"
    },
    "papermill": {
     "duration": 0.031764,
     "end_time": "2020-10-23T18:56:19.379687",
     "exception": false,
     "start_time": "2020-10-23T18:56:19.347923",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "np.random.seed(2) #usado para que seja possivel reproduzir a aleatoriedade do experimento com os mesmos resultados"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.023417,
     "end_time": "2020-10-23T18:56:19.428045",
     "exception": false,
     "start_time": "2020-10-23T18:56:19.404628",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Agora, vamos separar o conjunto de dados entre a parte que sera usada para treinar o modelo e a parte de teste, que sera utilizada para descobrirmos o quão bom o modelo é, antes de submetelo na Competição."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:19.485408Z",
     "iopub.status.busy": "2020-10-23T18:56:19.483992Z",
     "iopub.status.idle": "2020-10-23T18:56:19.827548Z",
     "shell.execute_reply": "2020-10-23T18:56:19.826929Z"
    },
    "papermill": {
     "duration": 0.375827,
     "end_time": "2020-10-23T18:56:19.827675",
     "exception": false,
     "start_time": "2020-10-23T18:56:19.451848",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X_train, y_train, test_size = 0.0001, random_state = 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.023162,
     "end_time": "2020-10-23T18:56:19.874284",
     "exception": false,
     "start_time": "2020-10-23T18:56:19.851122",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Rede Neural Covolucional\n",
    "\n",
    "Quando se trata de imagens, as CNN(Covolutional Neural Network) ganham destaque. Isso ocorre pois, se utiliza de forma correta, esse modelo tende a ter um dos melhores indices de acerto na classificação de imagens.\n",
    "\n",
    "**Assim, no modelo abaixo:**\n",
    "\n",
    "1. Começamos com camadas de Convolução que recebem a imagem e aplicam filtros nela.\n",
    "2. Colocamos a camada de pooling, a qual reduz as dimenções dos dados .\n",
    "3. A camada de Dropout, simplesmente exclui parte do processamento na imagem até o momento. Ela é necessária para minimizar a possibilidade de superajustamento aos dados de treino.\n",
    "\n",
    "\n",
    "4. Depois todas as camadas anteriores são repetidas para uma melhor performance.\n",
    "\n",
    "\n",
    "5. A camada de Flattening, transforma os dados(que até então eram matrizes) em vetores, para que as camadas Densas consigam processar os dados.\n",
    "6. A primeira camada Densa possui 256 \"neurônios\", que processam o dado e manda para a ultima camada.\n",
    "7. Essa ultima camada possui 10 \"neûronios\" pois é a quantidade de saidas(os números de 0 á 9) que precisamos."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.02404,
     "end_time": "2020-10-23T18:56:19.922235",
     "exception": false,
     "start_time": "2020-10-23T18:56:19.898195",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    " Keras é uma biblioteca desenvolvida pela google que possui ótimas funções a serem ultilizadas \n",
    " em algoritimos de aprendizado de maquina "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:19.983799Z",
     "iopub.status.busy": "2020-10-23T18:56:19.982727Z",
     "iopub.status.idle": "2020-10-23T18:56:23.736859Z",
     "shell.execute_reply": "2020-10-23T18:56:23.736083Z"
    },
    "papermill": {
     "duration": 3.791147,
     "end_time": "2020-10-23T18:56:23.736977",
     "exception": false,
     "start_time": "2020-10-23T18:56:19.945830",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "model = Sequential()\n",
    "\n",
    "model.add(Conv2D(filters = 64, kernel_size = (5,5), padding = 'Same', activation = 'relu', input_shape = (28,28,1)))\n",
    "model.add(Conv2D(filters = 64, kernel_size = (5,5), padding = 'Same', activation = 'relu'))\n",
    "model.add(MaxPool2D(pool_size=(2,2)))\n",
    "model.add(Dropout(0.2))\n",
    "\n",
    "model.add(Conv2D(filters = 128, kernel_size = (3,3), padding = 'Same', activation = 'relu'))\n",
    "model.add(Conv2D(filters = 128, kernel_size = (3,3), padding = 'Same', activation = 'relu'))\n",
    "model.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))\n",
    "model.add(Dropout(0.2))\n",
    "\n",
    "\n",
    "model.add(Flatten())\n",
    "model.add(Dense(512, activation = 'relu'))\n",
    "model.add(Dropout(0.2))\n",
    "model.add(Dense(10, activation = 'softmax'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:23.799452Z",
     "iopub.status.busy": "2020-10-23T18:56:23.798388Z",
     "iopub.status.idle": "2020-10-23T18:56:23.806551Z",
     "shell.execute_reply": "2020-10-23T18:56:23.806016Z"
    },
    "papermill": {
     "duration": 0.04579,
     "end_time": "2020-10-23T18:56:23.806669",
     "exception": false,
     "start_time": "2020-10-23T18:56:23.760879",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Colocamos como metrica de acerto: acurácia\n",
    "# Função de perda(exibe o quanto o algoritimo esta errado): categorical_crossentropy\n",
    "# optimizer: adam (um dos melhores optimizadores atualmente)\n",
    "model.compile(optimizer=\"adam\", loss='categorical_crossentropy', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:23.860524Z",
     "iopub.status.busy": "2020-10-23T18:56:23.858473Z",
     "iopub.status.idle": "2020-10-23T18:56:23.861322Z",
     "shell.execute_reply": "2020-10-23T18:56:23.861884Z"
    },
    "papermill": {
     "duration": 0.031206,
     "end_time": "2020-10-23T18:56:23.862029",
     "exception": false,
     "start_time": "2020-10-23T18:56:23.830823",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "epochs = 38 # Número epocas do modelo\n",
    "batch_size = 64 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.024638,
     "end_time": "2020-10-23T18:56:23.911309",
     "exception": false,
     "start_time": "2020-10-23T18:56:23.886671",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Aumento de Dados\n",
    "   Essa tecnica pode ser utilizada para aumentar a quantidade de dados de forma artificial. No codigo abaixo, por exemplo, essa tecnica foi empregada pegando os dados das imagens originais do dataset, e gerando novas imagens modificadas.\n",
    "\n",
    "**Caracteristicas modificadas:**\n",
    "* Zoom da imagem\n",
    "* Inclinação/rotação da imagem(Nesse caso, temos que tomar cuidado com a rotação pois o número 6 pode ser confundido com o 9 ou o contrário)\n",
    "* Centro da imagem\n",
    "* Largura\n",
    "* Altura"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:23.968608Z",
     "iopub.status.busy": "2020-10-23T18:56:23.967258Z",
     "iopub.status.idle": "2020-10-23T18:56:24.020460Z",
     "shell.execute_reply": "2020-10-23T18:56:24.019880Z"
    },
    "papermill": {
     "duration": 0.084076,
     "end_time": "2020-10-23T18:56:24.020589",
     "exception": false,
     "start_time": "2020-10-23T18:56:23.936513",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "datagen = ImageDataGenerator(featurewise_center=False,\n",
    "                            samplewise_center=False,\n",
    "                            featurewise_std_normalization=False,\n",
    "                            samplewise_std_normalization=False,\n",
    "                            zca_whitening=False,\n",
    "                            rotation_range=10,\n",
    "                            zoom_range=0.1,\n",
    "                            width_shift_range=0.1,\n",
    "                            height_shift_range=0.1,\n",
    "                            horizontal_flip=False,\n",
    "                            vertical_flip=False)\n",
    "datagen.fit(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T18:56:24.078179Z",
     "iopub.status.busy": "2020-10-23T18:56:24.077287Z",
     "iopub.status.idle": "2020-10-23T19:07:39.276279Z",
     "shell.execute_reply": "2020-10-23T19:07:39.276890Z"
    },
    "papermill": {
     "duration": 675.231852,
     "end_time": "2020-10-23T19:07:39.277053",
     "exception": false,
     "start_time": "2020-10-23T18:56:24.045201",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/38\n",
      "656/656 - 18s - loss: 0.2255 - accuracy: 0.9289 - val_loss: 1.8357e-04 - val_accuracy: 1.0000\n",
      "Epoch 2/38\n",
      "656/656 - 16s - loss: 0.0783 - accuracy: 0.9756 - val_loss: 0.1881 - val_accuracy: 0.8000\n",
      "Epoch 3/38\n",
      "656/656 - 19s - loss: 0.0588 - accuracy: 0.9820 - val_loss: 0.0162 - val_accuracy: 1.0000\n",
      "Epoch 4/38\n",
      "656/656 - 16s - loss: 0.0539 - accuracy: 0.9837 - val_loss: 0.0624 - val_accuracy: 1.0000\n",
      "Epoch 5/38\n",
      "656/656 - 17s - loss: 0.0462 - accuracy: 0.9864 - val_loss: 0.3172 - val_accuracy: 0.8000\n",
      "Epoch 6/38\n",
      "656/656 - 17s - loss: 0.0426 - accuracy: 0.9870 - val_loss: 0.1666 - val_accuracy: 0.8000\n",
      "Epoch 7/38\n",
      "656/656 - 18s - loss: 0.0368 - accuracy: 0.9885 - val_loss: 0.0142 - val_accuracy: 1.0000\n",
      "Epoch 8/38\n",
      "656/656 - 17s - loss: 0.0373 - accuracy: 0.9886 - val_loss: 0.0550 - val_accuracy: 1.0000\n",
      "Epoch 9/38\n",
      "656/656 - 17s - loss: 0.0323 - accuracy: 0.9899 - val_loss: 0.0087 - val_accuracy: 1.0000\n",
      "Epoch 10/38\n",
      "656/656 - 18s - loss: 0.0307 - accuracy: 0.9908 - val_loss: 0.0675 - val_accuracy: 1.0000\n",
      "Epoch 11/38\n",
      "656/656 - 17s - loss: 0.0301 - accuracy: 0.9908 - val_loss: 0.7035 - val_accuracy: 0.8000\n",
      "Epoch 12/38\n",
      "656/656 - 17s - loss: 0.0294 - accuracy: 0.9908 - val_loss: 0.2639 - val_accuracy: 0.8000\n",
      "Epoch 13/38\n",
      "656/656 - 19s - loss: 0.0287 - accuracy: 0.9914 - val_loss: 0.0297 - val_accuracy: 1.0000\n",
      "Epoch 14/38\n",
      "656/656 - 17s - loss: 0.0272 - accuracy: 0.9918 - val_loss: 0.0936 - val_accuracy: 1.0000\n",
      "Epoch 15/38\n",
      "656/656 - 17s - loss: 0.0268 - accuracy: 0.9917 - val_loss: 0.0041 - val_accuracy: 1.0000\n",
      "Epoch 16/38\n",
      "656/656 - 18s - loss: 0.0246 - accuracy: 0.9930 - val_loss: 0.3055 - val_accuracy: 0.8000\n",
      "Epoch 17/38\n",
      "656/656 - 18s - loss: 0.0237 - accuracy: 0.9922 - val_loss: 8.4460e-04 - val_accuracy: 1.0000\n",
      "Epoch 18/38\n",
      "656/656 - 18s - loss: 0.0235 - accuracy: 0.9932 - val_loss: 0.0011 - val_accuracy: 1.0000\n",
      "Epoch 19/38\n",
      "656/656 - 17s - loss: 0.0217 - accuracy: 0.9935 - val_loss: 5.7488e-04 - val_accuracy: 1.0000\n",
      "Epoch 20/38\n",
      "656/656 - 19s - loss: 0.0219 - accuracy: 0.9929 - val_loss: 5.6211e-05 - val_accuracy: 1.0000\n",
      "Epoch 21/38\n",
      "656/656 - 18s - loss: 0.0235 - accuracy: 0.9930 - val_loss: 0.0076 - val_accuracy: 1.0000\n",
      "Epoch 22/38\n",
      "656/656 - 17s - loss: 0.0215 - accuracy: 0.9934 - val_loss: 5.0544e-06 - val_accuracy: 1.0000\n",
      "Epoch 23/38\n",
      "656/656 - 19s - loss: 0.0214 - accuracy: 0.9937 - val_loss: 0.0055 - val_accuracy: 1.0000\n",
      "Epoch 24/38\n",
      "656/656 - 18s - loss: 0.0202 - accuracy: 0.9940 - val_loss: 0.0072 - val_accuracy: 1.0000\n",
      "Epoch 25/38\n",
      "656/656 - 18s - loss: 0.0214 - accuracy: 0.9932 - val_loss: 4.0384e-05 - val_accuracy: 1.0000\n",
      "Epoch 26/38\n",
      "656/656 - 17s - loss: 0.0203 - accuracy: 0.9939 - val_loss: 0.0332 - val_accuracy: 1.0000\n",
      "Epoch 27/38\n",
      "656/656 - 19s - loss: 0.0180 - accuracy: 0.9944 - val_loss: 4.5194e-04 - val_accuracy: 1.0000\n",
      "Epoch 28/38\n",
      "656/656 - 17s - loss: 0.0193 - accuracy: 0.9943 - val_loss: 5.1877e-04 - val_accuracy: 1.0000\n",
      "Epoch 29/38\n",
      "656/656 - 18s - loss: 0.0193 - accuracy: 0.9941 - val_loss: 2.0181e-04 - val_accuracy: 1.0000\n",
      "Epoch 30/38\n",
      "656/656 - 18s - loss: 0.0211 - accuracy: 0.9936 - val_loss: 0.4554 - val_accuracy: 0.8000\n",
      "Epoch 31/38\n",
      "656/656 - 18s - loss: 0.0180 - accuracy: 0.9949 - val_loss: 4.1965e-04 - val_accuracy: 1.0000\n",
      "Epoch 32/38\n",
      "656/656 - 18s - loss: 0.0199 - accuracy: 0.9937 - val_loss: 1.0852e-04 - val_accuracy: 1.0000\n",
      "Epoch 33/38\n",
      "656/656 - 18s - loss: 0.0163 - accuracy: 0.9949 - val_loss: 0.2963 - val_accuracy: 0.8000\n",
      "Epoch 34/38\n",
      "656/656 - 18s - loss: 0.0182 - accuracy: 0.9946 - val_loss: 0.2427 - val_accuracy: 0.8000\n",
      "Epoch 35/38\n",
      "656/656 - 17s - loss: 0.0185 - accuracy: 0.9944 - val_loss: 0.0293 - val_accuracy: 1.0000\n",
      "Epoch 36/38\n",
      "656/656 - 18s - loss: 0.0184 - accuracy: 0.9938 - val_loss: 0.0355 - val_accuracy: 1.0000\n",
      "Epoch 37/38\n",
      "656/656 - 18s - loss: 0.0156 - accuracy: 0.9951 - val_loss: 0.0099 - val_accuracy: 1.0000\n",
      "Epoch 38/38\n",
      "656/656 - 17s - loss: 0.0167 - accuracy: 0.9951 - val_loss: 0.0013 - val_accuracy: 1.0000\n"
     ]
    }
   ],
   "source": [
    "# Ajusta e Treina o modelo\n",
    "history = model.fit_generator(datagen.flow(X_train,y_train, batch_size=batch_size),\n",
    "                              epochs = epochs, validation_data = (X_test,y_test),\n",
    "                              verbose = 2, steps_per_epoch=X_train.shape[0] // batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T19:07:39.390956Z",
     "iopub.status.busy": "2020-10-23T19:07:39.389770Z",
     "iopub.status.idle": "2020-10-23T19:07:40.930220Z",
     "shell.execute_reply": "2020-10-23T19:07:40.929096Z"
    },
    "papermill": {
     "duration": 1.598933,
     "end_time": "2020-10-23T19:07:40.930361",
     "exception": false,
     "start_time": "2020-10-23T19:07:39.331428",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Usa o modelo treinado para prever e classificar os dados sem classificação\n",
    "# nesse caso, os dados que serão submetidos na competição.\n",
    "results = model.predict(test) \n",
    "\n",
    "# O algoritimo retorna uma lista de probabilidades pra cada número"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T19:07:41.046156Z",
     "iopub.status.busy": "2020-10-23T19:07:41.045205Z",
     "iopub.status.idle": "2020-10-23T19:07:41.050411Z",
     "shell.execute_reply": "2020-10-23T19:07:41.049746Z"
    },
    "papermill": {
     "duration": 0.064984,
     "end_time": "2020-10-23T19:07:41.050552",
     "exception": false,
     "start_time": "2020-10-23T19:07:40.985568",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Seleciona os indices com maior probabilidade\n",
    "results = np.argmax(results,axis = 1)\n",
    "\n",
    "results = pd.Series(results,name=\"Label\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0",
    "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a",
    "execution": {
     "iopub.execute_input": "2020-10-23T19:07:41.174897Z",
     "iopub.status.busy": "2020-10-23T19:07:41.173934Z",
     "iopub.status.idle": "2020-10-23T19:07:41.178059Z",
     "shell.execute_reply": "2020-10-23T19:07:41.178649Z"
    },
    "papermill": {
     "duration": 0.073566,
     "end_time": "2020-10-23T19:07:41.178797",
     "exception": false,
     "start_time": "2020-10-23T19:07:41.105231",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ImageId</th>\n",
       "      <th>Label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27995</th>\n",
       "      <td>27996</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27996</th>\n",
       "      <td>27997</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27997</th>\n",
       "      <td>27998</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27998</th>\n",
       "      <td>27999</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27999</th>\n",
       "      <td>28000</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>28000 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       ImageId  Label\n",
       "0            1      2\n",
       "1            2      0\n",
       "2            3      9\n",
       "3            4      0\n",
       "4            5      3\n",
       "...        ...    ...\n",
       "27995    27996      9\n",
       "27996    27997      7\n",
       "27997    27998      3\n",
       "27998    27999      9\n",
       "27999    28000      2\n",
       "\n",
       "[28000 rows x 2 columns]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Gera um dataframe com o Id e a resposta de cada imagem\n",
    "output = pd.concat([pd.Series(range(1,28001), name = 'ImageId'), results], axis = 1)\n",
    "output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-10-23T19:07:41.295034Z",
     "iopub.status.busy": "2020-10-23T19:07:41.294045Z",
     "iopub.status.idle": "2020-10-23T19:07:41.412399Z",
     "shell.execute_reply": "2020-10-23T19:07:41.411870Z"
    },
    "papermill": {
     "duration": 0.178578,
     "end_time": "2020-10-23T19:07:41.412515",
     "exception": false,
     "start_time": "2020-10-23T19:07:41.233937",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Para finalizar, vamos gerar um aquivo csv a partir do dataframe.\n",
    "output.to_csv('submission.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.055148,
     "end_time": "2020-10-23T19:07:41.521507",
     "exception": false,
     "start_time": "2020-10-23T19:07:41.466359",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Fontes:\n",
    "\n",
    "* https://keras.io/\n",
    "* https://www.kaggle.com/yassineghouzam/introduction-to-cnn-keras-0-997-top-6\n",
    "* https://www.kaggle.com/elcaiseri/mnist-simple-cnn-keras-accuracy-0-99-top-1"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "papermill": {
   "duration": 703.0518,
   "end_time": "2020-10-23T19:07:43.633098",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2020-10-23T18:56:00.581298",
   "version": "2.1.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}