data/examples/use-pytorch-to-predict-handwritten-digits.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Use PyTorch to predict handwritten digits\n",
    "<table style=\"border: none\" align=\"left\">\n",
    "   <tr style=\"border: none\">\n",
    "       <td style=\"border: none\"><img src=\"https://github.com/IBM/pytorch-on-watson-studio/raw/master/doc/source/images/pytorch-pattern-header.jpg\" width=\"600\" alt=\"Icon\"></td>\n",
    "   </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook contains steps and code to demonstrate Deep Learning model\n",
    "training in the\n",
    "<a href=\"https://www.ibm.com/cloud/machine-learning\">Watson Machine Learning</a>\n",
    "service.\n",
    "\n",
    "<a href=\"https://pytorch.org/\" target=\"_blank\" rel=\"noopener no\n",
    "referrer\">PyTorch</a> is a relatively new deep learning framework. Yet, it has\n",
    "begun to gain adoption especially among researchers and data scientists. The\n",
    "strength of PyTorch is its support of dynamic computational graph while most deep\n",
    "learning frameworks are based on static computational graph. In addition, its strong\n",
    "NumPy like GPU accelerated tensor computation has allowed Python developers to easily\n",
    "learn and build deep learning networks for GPUs and CPUs alike.\n",
    "\n",
    "Some familiarity with Python is helpful. This notebook uses Python 3 and\n",
    "<a href=\"https://dataplatform.cloud.ibm.com/docs/content/analyze-data/environments-parent.html\" target=\"_blank\" rel=\"noopener no referrer\">Watson Studio</a> to configure and initiate training of a PyTorch base\n",
    "workload using Watson Machine Learning service.\n",
    "\n",
    "\n",
    "## Learning goals\n",
    "\n",
    "In this notebook, you will learn how to:\n",
    "\n",
    "-  Work with Watson Machine Learning to train Deep Learning models\n",
    "-  Use PyTorch features, tools and libraries  \n",
    "-  Save trained models in the Watson Machine Learning repository\n",
    "\n",
    "## Contents\n",
    "\n",
    "1.\t[Set up](#setup)\n",
    "2.\t[Create the training definitions](#model)\n",
    "3.  [Train the model](#train)\n",
    "4.\t[Work with the trained models](#work)\n",
    "5.\t[Summary and next steps](#summary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"setup\"></a>\n",
    "## 1. Set up\n",
    "\n",
    "Before you use the sample code in this notebook, you must perform the following setup tasks:\n",
    "\n",
    "-  Create a <a href=\"https://console.bluemix.net/catalog/services/machine-learning\" target=\"_blank\" rel=\"noopener no referrer\">Watson Machine Learning (WML) Service</a> instance (a free plan is offered and information about how to create the instance is <a href=\"https://dataplatform.ibm.com/docs/content/analyze-data/wml-setup.html\" target=\"_blank\" rel=\"noopener no referrer\">here</a>).\n",
    "-  Create a <a href=\"https://console.bluemix.net/catalog/services/cloud-object-storage\" target=\"_blank\" rel=\"noopener no referrer\">Cloud Object Storage (COS)</a> instance (a lite plan is offered and information about how to order storage is <a href=\"https://console.bluemix.net/docs/services/cloud-object-storage/basics/order-storage.html#order-storage\" target=\"_blank\" rel=\"noopener no referrer\">here</a>). <br/>**Note: When using Watson Studio, you already have a COS instance associated with the project you are running the notebook in.**\n",
    "- Create new credentials with HMAC: \n",
    "    - Go to your COS dashboard (see Tip).\n",
    "    - In the **Service credentials** tab, click **New Credential+**.\n",
    "    - In the **Add Inline Configuration Parameters(Optional):** box, add {\"HMAC\":true}\n",
    "    - Click **Add**. (For more information, see <a href=\"https://console.bluemix.net/docs/services/cloud-object-storage/hmac/credentials.html#using-hmac-credentials\" target=\"_blank\" rel=\"noopener no referrer\">HMAC</a>.)\n",
    "\n",
    "    This configuration parameter adds the following section to the instance credentials, (for use later in this notebook):\n",
    "    ```\n",
    "      \"cos_hmac_keys\": {\n",
    "            \"access_key_id\": \"-------\",\n",
    "            \"secret_access_key\": \"-------\"\n",
    "       }\n",
    "    ```\n",
    " \n",
    "**Tip:** follow the steps below to access your COS instance dashboard. From the Watson Studio dashboard:\n",
    "- Click the **Services** tab on the top of the page\n",
    "- Click the **Data Services** tab\n",
    "- Select and click your target object storage (COS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 Work with Cloud Object Storage  (COS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Install the boto library. This library allows Python developers to manage Cloud Object Storage (COS)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Tip:** If `ibm_boto3` is not preinstalled in your environment, run the following command to install it: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: ibm-cos-sdk in /opt/conda/envs/Python36/lib/python3.6/site-packages (2.4.3)\n",
      "Requirement already satisfied: ibm-cos-sdk-s3transfer==2.*,>=2.0.0 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from ibm-cos-sdk) (2.4.3)\n",
      "Requirement already satisfied: ibm-cos-sdk-core==2.*,>=2.0.0 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from ibm-cos-sdk) (2.4.3)\n",
      "Requirement already satisfied: urllib3<1.25,>=1.20 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk) (1.24.1)\n",
      "Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk) (2.7.5)\n",
      "Requirement already satisfied: docutils>=0.10 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk) (0.14)\n",
      "Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk) (0.9.3)\n",
      "Requirement already satisfied: six>=1.5 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from python-dateutil<3.0.0,>=2.1->ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk) (1.12.0)\n"
     ]
    }
   ],
   "source": [
    "# Run the command if ibm_boto3 is not installed.\n",
    "!pip install ibm-cos-sdk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install the boto library.\n",
    "import ibm_boto3\n",
    "from ibm_botocore.client import Config"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Replace** the information in the following cell with your COS credentials. \n",
    "\n",
    "You can find these credentials in your COS instance dashboard under the **Service credentials** tab.\n",
    "\n",
    "**Note:** the HMAC key, described in [set up the environment](#setup) is included in these credentials.\n",
    "\n",
    "`\n",
    "cos_credentials = {\n",
    "  \"apikey\": \"-------\",\n",
    "  \"cos_hmac_keys\": {\n",
    "    \"access_key_id\": \"------\",\n",
    "    \"secret_access_key\": \"------\"\n",
    "  },\n",
    "  \"endpoints\": \"https://cos-service.bluemix.net/endpoints\",\n",
    "  \"iam_apikey_description\": \"------\",\n",
    "  \"iam_apikey_name\": \"------\",\n",
    "  \"iam_role_crn\": \"------\",\n",
    "  \"iam_serviceid_crn\": \"------\",\n",
    "  \"resource_instance_id\": \"-------\"\n",
    "}\n",
    "`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# @hidden_cell\n",
    "cos_credentials = {\n",
    "     # Credentials deleted\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define the endpoint.\n",
    "\n",
    "To do this, go to the **Endpoint** tab in the COS instance's dashboard to get the endpoint information, then enter it in the cell below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define endpoint information.\n",
    "service_endpoint = 'https://s3-api.us-geo.objectstorage.softlayer.net'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You also need the IBM Cloud authorization endpoint to be able to create COS resource object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the authorization endpoint.\n",
    "auth_endpoint = 'https://iam.bluemix.net/oidc/token'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a Boto resource to be able to write data to COS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a COS resource.\n",
    "cos = ibm_boto3.resource('s3',\n",
    "                         ibm_api_key_id=cos_credentials['apikey'],\n",
    "                         ibm_service_instance_id=cos_credentials['resource_instance_id'],\n",
    "                         ibm_auth_endpoint=auth_endpoint,\n",
    "                         config=Config(signature_version='oauth'),\n",
    "                         endpoint_url=service_endpoint)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create two buckets, which you will use to store training data and training results.\n",
    "\n",
    "**Note:** The bucket names must be unique."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating bucket \"training-mnist-data-5c70e75e-660e-4847-ba75-9897680e59d1\"...\n",
      "Creating bucket \"training-mnist-results-5c70e75e-660e-4847-ba75-9897680e59d1\"...\n"
     ]
    }
   ],
   "source": [
    "from uuid import uuid4\n",
    "\n",
    "bucket_uid = str(uuid4())\n",
    "buckets = ['training-mnist-data-' + bucket_uid, 'training-mnist-results-' + bucket_uid]\n",
    "\n",
    "for bucket in buckets:\n",
    "    if not cos.Bucket(bucket) in cos.buckets.all():\n",
    "        print('Creating bucket \"{}\"...'.format(bucket))\n",
    "        try:\n",
    "            cos.create_bucket(Bucket=bucket)\n",
    "        except ibm_boto3.exceptions.ibm_botocore.client.ClientError as e:\n",
    "            print('Error: {}.'.format(e.response['Error']['Message']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you should have 2 buckets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[s3.Bucket(name='pytorchproject-donotdelete-pr-ijmqiswj6qxhcu'), s3.Bucket(name='test-donotdelete-pr-yogxn1hpefkkbl'), s3.Bucket(name='test20190426-donotdelete-pr-y2l6voewh8doky'), s3.Bucket(name='test20200615-donotdelete-pr-c8ul2bk0dgfauk'), s3.Bucket(name='training-mnist-data-1974b929-2e55-4bce-9990-fd2d794d64a5'), s3.Bucket(name='training-mnist-data-5c70e75e-660e-4847-ba75-9897680e59d1'), s3.Bucket(name='training-mnist-data-62ef940b-a1aa-4cbe-9a45-a835f60ccf66'), s3.Bucket(name='training-mnist-results-1974b929-2e55-4bce-9990-fd2d794d64a5'), s3.Bucket(name='training-mnist-results-5c70e75e-660e-4847-ba75-9897680e59d1'), s3.Bucket(name='training-mnist-results-62ef940b-a1aa-4cbe-9a45-a835f60ccf66')]\n"
     ]
    }
   ],
   "source": [
    "# Display a list of created buckets.\n",
    "print(list(cos.buckets.all()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Download the training data and upload it to the COS buckets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**PyTorch Tools & Libraries**\n",
    "\n",
    "An active community of researchers and developers have built a rich\n",
    "ecosystem of tools and libraries for extending PyTorch and supporting\n",
    "development in areas from computer vision to reinforcement learning.\n",
    "\n",
    "PyTorch's <a href=\"https://github.com/pytorch/vision\" target=\"_blank\"\n",
    "rel=\"noopener no referrer\">torchvision</a> is one of those packages.\n",
    "`torchvision` consists of popular datasets, model architectures, and common\n",
    "image transformations for computer vision.\n",
    "\n",
    "This tutorial will use `torchvision's MNIST dataset` package to download\n",
    "and process the training data. The processed data files will be uploaded to\n",
    "the `training-data-mnist` bucket.\n",
    "\n",
    "**Tip:** If PyTorch or `torchvision` is not preinstalled in your environment, run the\n",
    "following command to install it: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting torchvision\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/8e/dc/4a939cfbd38398f4765f712576df21425241020bfccc200af76d19088533/torchvision-0.7.0-cp36-cp36m-manylinux1_x86_64.whl (5.9MB)\n",
      "\u001b[K     |████████████████████████████████| 5.9MB 5.6MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: pillow>=4.1.1 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from torchvision) (5.4.1)\n",
      "Requirement already satisfied: numpy in /opt/conda/envs/Python36/lib/python3.6/site-packages (from torchvision) (1.15.4)\n",
      "Collecting torch==1.6.0 (from torchvision)\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/38/53/914885a93a44b96c0dd1c36f36ff10afe341f091230aad68f7228d61db1e/torch-1.6.0-cp36-cp36m-manylinux1_x86_64.whl (748.8MB)\n",
      "\u001b[K     |████████████████████\u001b[K     |████████████████████████████████| 748.8MB 11kB/s  eta 0:00:011████████▋  | 691.8MB 3.6MB/s eta 0:00:16��████████████████████▋  | 693.0MB 1.4MB/s eta 0:00:3993.3MB 1.4MB/s eta 0:00:39█████████▊  | 695.1MB 1.4MB/s eta 0:00:3895.2MB 1.4MB/s eta 0:00:38��█████████████████▊  | 695.5MB 1.4MB/s eta 0:00:384MB/s eta 0:00:37��██████████████▊  | 696.2MB 1.4MB/s eta 0:00:37��████████████████████▊  | 696.3MB 1.4MB/s eta 0:00:3796.6MB 1.4MB/s eta 0:00:37��█████████████████▉  | 697.0MB 1.4MB/s eta 0:00:374MB/s eta 0:00:36��██████████████▉  | 697.6MB 1.4MB/s eta 0:00:36███▉  | 697.7MB 1.4MB/s eta 0:00:36�███████████████████████▉  | 698.1MB 1.4MB/s eta 0:00:36▉  | 698.4MB 2.0MB/s eta 0:00:26�████████████████████▉  | 698.7MB 2.0MB/s eta 0:00:26█████████▉  | 699.4MB 2.0MB/s eta 0:00:25██████████  | 700.8MB 2.0MB/s eta 0:00:25|██████████████████████████████  | 701.1MB 2.0MB/s eta 0:00:24███████  | 701.4MB 2.0MB/s eta 0:00:24██████████  | 702.8MB 2.0MB/s eta 0:00:2402.9MB 2.0MB/s eta 0:00:24��██████████████████  | 703.3MB 2.0MB/s eta 0:00:230MB/s eta 0:00:23��███████████████  | 703.9MB 2.0MB/s eta 0:00:23████  | 704.0MB 2.0MB/s eta 0:00:23██████████  | 704.5MB 276kB/s eta 0:02:40��██▏ | 704.8MB 276kB/s eta 0:02:39� | 705.2MB 276kB/s eta 0:02:38    |██████████████████████████████▏ | 706.4MB 276kB/s eta 0:02:34�████████████████████████████▏ | 706.7MB 276kB/s eta 0:02:33█████████████████████████▏ | 707.0MB 276kB/s eta 0:02:31��█████████████████████▎ | 707.3MB 276kB/s eta 0:02:30�██████████████████▎ | 707.6MB 276kB/s eta 0:02:29██████████████████████████████▎ | 708.4MB 276kB/s eta 0:02:26��██████████████████████████▎ | 708.7MB 276kB/s eta 0:02:25�███████████████████████▎ | 709.0MB 276kB/s eta 0:02:24████████████████████▎ | 709.3MB 276kB/s eta 0:02:23��████████████████▎ | 709.6MB 276kB/s eta 0:02:22██████████████████████████████▍ | 710.3MB 356kB/s eta 0:01:48��██████████████████████████▍ | 710.6MB 356kB/s eta 0:01:48�███████████████████████▍ | 710.9MB 356kB/s eta 0:01:47████████████████████▍ | 711.2MB 356kB/s eta 0:01:46    |██████████████████████████████▌ | 712.7MB 356kB/s eta 0:01:42�████████████████████████████▌ | 713.0MB 356kB/s eta 0:01:41█████████████████████████▌ | 713.3MB 356kB/s eta 0:01:40��█████████████████████▌ | 713.6MB 356kB/s eta 0:01:39�██████████████████▌ | 713.9MB 356kB/s eta 0:01:38██████████████████████████████▌ | 714.8MB 356kB/s eta 0:01:36��██████████████████████████▋ | 715.1MB 356kB/s eta 0:01:35�███████████████████████▋ | 715.4MB 356kB/s eta 0:01:34████████████████████▋ | 715.7MB 356kB/s eta 0:01:33��████████████████▋ | 716.1MB 356kB/s eta 0:01:32�█████████████▋ | 716.4MB 356kB/s eta 0:01:31██████████▋ | 716.7MB 356kB/s eta 0:01:31    |██████████████████████████████▋ | 716.9MB 304kB/s eta 0:01:45�████████████████████████████▋ | 717.2MB 304kB/s eta 0:01:44��██████████████████████████▋ | 717.3MB 304kB/s eta 0:01:44██████████████████████████████▊ | 719.0MB 304kB/s eta 0:01:38��██████████████████████████▊ | 719.3MB 304kB/s eta 0:01:37��████████████████▉ | 720.2MB 304kB/s eta 0:01:34██████████████████████████████▉ | 721.9MB 304kB/s eta 0:01:29�████████████████████████████▉ | 722.0MB 304kB/s eta 0:01:29�███████████████████████▉ | 722.5MB 304kB/s eta 0:01:27████████████████████▉ | 722.8MB 304kB/s eta 0:01:26��█████████████████ | 723.1MB 304kB/s eta 0:01:25    |███████████████████████████████ | 724.9MB 304kB/s eta 0:01:19██████████████████████████████ | 725.2MB 20.0MB/s eta 0:00:0220.0MB/s eta 0:00:02███ | 727.2MB 20.0MB/s eta 0:00:02��███████████████▏| 730.6MB 20.0MB/s eta 0:00:019MB 20.0MB/s eta 0:00:01��███████████████▎| 732.6MB 20.0MB/s eta 0:00:019MB 20.0MB/s eta 0:00:01��███████████████████████▎| 733.2MB 20.0MB/s eta 0:00:01█████▍| 733.5MB 20.0MB/s eta 0:00:01 |███████████████████████████████▍| 733.8MB 20.0MB/s eta 0:00:01�██▍| 734.0MB 20.0MB/s eta 0:00:01█████████████████████████████▍| 734.3MB 20.0MB/s eta 0:00:01�██████████▍| 734.6MB 20.0MB/s eta 0:00:015.7MB 641kB/s eta 0:00:21�██████████████▌| 736.0MB 641kB/s eta 0:00:20███████████████████████████████▌| 736.3MB 641kB/s eta 0:00:20��██▌| 736.6MB 641kB/s eta 0:00:20�███████████████████▌| 736.9MB 641kB/s eta 0:00:197.1MB 641kB/s eta 0:00:19�██████████████▌| 737.4MB 641kB/s eta 0:00:18███████████████████████████████▌| 737.7MB 641kB/s eta 0:00:18��██▌| 738.0MB 641kB/s eta 0:00:17�███████████████████▌| 738.3MB 641kB/s eta 0:00:17██████▋| 738.8MB 641kB/s eta 0:00:16███████████████████████████████▋| 739.3MB 641kB/s eta 0:00:15��██▋| 739.6MB 641kB/s eta 0:00:15�██▋| 740.1MB 21.5MB/s eta 0:00:01█████████████████████████████▋| 740.4MB 21.5MB/s eta 0:00:015MB 21.5MB/s eta 0:00:01��███████████████████████▋| 740.9MB 21.5MB/s eta 0:00:01�██▊| 741.0MB 21.5MB/s eta 0:00:01█████████████████████████████▊| 741.3MB 21.5MB/s eta 0:00:01�██████████▊| 741.6MB 21.5MB/s eta 0:00:01 0:00:01��███████████████▊| 742.4MB 21.5MB/s eta 0:00:01��██████▉| 743.8MB 7.9MB/s eta 0:00:01�███████████████████████▉| 744.1MB 7.9MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: future in /opt/conda/envs/Python36/lib/python3.6/site-packages (from torch==1.6.0->torchvision) (0.17.1)\n",
      "Installing collected packages: torch, torchvision\n",
      "Successfully installed torch-1.6.0 torchvision-0.7.0\n"
     ]
    }
   ],
   "source": [
    "#Install torchvision\n",
    "!pip install torchvision"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following code will download and process the MNIST training and test data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to ./data/MNIST/raw/train-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "681d550b1f234658b73b55cf367c77d2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Extracting ./data/MNIST/raw/train-images-idx3-ubyte.gz to ./data/MNIST/raw\n",
      "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to ./data/MNIST/raw/train-labels-idx1-ubyte.gz\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "44db148543cf4c588742bc0205903c76",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Extracting ./data/MNIST/raw/train-labels-idx1-ubyte.gz to ./data/MNIST/raw\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to ./data/MNIST/raw/t10k-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9a7a43905806425189b7c430a3a5dc03",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Extracting ./data/MNIST/raw/t10k-images-idx3-ubyte.gz to ./data/MNIST/raw\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to ./data/MNIST/raw/t10k-labels-idx1-ubyte.gz\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "866f595a1ac74770906ae4ce3c9be09c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Extracting ./data/MNIST/raw/t10k-labels-idx1-ubyte.gz to ./data/MNIST/raw\n",
      "Processing...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/torchvision/datasets/mnist.py:469: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /pytorch/torch/csrc/utils/tensor_numpy.cpp:141.)\n",
      "  return torch.from_numpy(parsed.astype(m[2], copy=False)).view(*s)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Dataset MNIST\n",
       "    Number of datapoints: 60000\n",
       "    Root location: ./data\n",
       "    Split: Train\n",
       "    StandardTransform\n",
       "Transform: Compose(\n",
       "               ToTensor()\n",
       "               Normalize(mean=(0.1307,), std=(0.3081,))\n",
       "           )"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "from torchvision import datasets, transforms\n",
    "\n",
    "data_dir = './data'\n",
    "\n",
    "datasets.MNIST(data_dir, train=True, download=True,\n",
    "                       transform=transforms.Compose([\n",
    "                           transforms.ToTensor(),\n",
    "                           transforms.Normalize((0.1307,), (0.3081,))\n",
    "                       ]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code in the next cell uploads the processed files to your COS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Uploading data processed/test.pt...\n",
      "processed/test.pt is uploaded.\n",
      "Uploading data processed/training.pt...\n",
      "processed/training.pt is uploaded.\n",
      "Done\n"
     ]
    }
   ],
   "source": [
    "import glob\n",
    "import os\n",
    "\n",
    "files_search = os.path.join(data_dir,\"MNIST\", \"processed\", \"*\")\n",
    "files = glob.glob(files_search)\n",
    "\n",
    "bucket_obj = cos.Bucket(buckets[0])\n",
    "\n",
    "for file in files:\n",
    "    filename = file.split('/')[-1]\n",
    "    filename = os.path.join(\"processed\", filename)\n",
    "    print('Uploading data {}...'.format(filename))\n",
    "    bucket_obj.upload_file(file, filename )\n",
    "    print('{} is uploaded.'.format(filename))\n",
    "print(\"Done\")    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Have a look at the list of the created buckets and their contents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training-mnist-data-5c70e75e-660e-4847-ba75-9897680e59d1\n",
      "  File: processed/test.pt, 7735.44kB\n",
      "  File: processed/training.pt, 46407.31kB\n",
      "training-mnist-results-5c70e75e-660e-4847-ba75-9897680e59d1\n"
     ]
    }
   ],
   "source": [
    "for bucket_name in buckets:\n",
    "    print(bucket_name)\n",
    "    bucket_obj = cos.Bucket(bucket_name)\n",
    "    for obj in bucket_obj.objects.all():\n",
    "        print(\"  File: {}, {:4.2f}kB\".format(obj.key, obj.size/1024))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You are done with COS, and you are ready to train your model!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3. Work with the WML service instance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import the libraries you need to work with your WML instance.\n",
    "\n",
    "**Hint:** You may also need to install `wget` using the following command `!pip install wget`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting wget\n",
      "  Downloading https://files.pythonhosted.org/packages/47/6a/62e288da7bcda82b935ff0c6cfe542970f04e29c756b0e147251b2fb251f/wget-3.2.zip\n",
      "Building wheels for collected packages: wget\n",
      "  Building wheel for wget (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Stored in directory: /home/dsxuser/.cache/pip/wheels/40/15/30/7d8f7cea2902b4db79e3fea550d7d7b85ecb27ef992b618f3f\n",
      "Successfully built wget\n",
      "Installing collected packages: wget\n",
      "Successfully installed wget-3.2\n"
     ]
    }
   ],
   "source": [
    "!pip install wget"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib3, requests, json, base64, time, os, wget"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Authenticate to the Watson Machine Learning (WML) service on IBM Cloud.\n",
    "\n",
    "**Tip**: Authentication information (your credentials) can be found in the <a href=\"https://console.bluemix.net/docs/services/service_credentials.html#service_credentials\" target=\"_blank\" rel=\"noopener noreferrer\">Service credentials</a> tab of the service instance that you created on IBM Cloud. \n",
    "If there are no credentials listed for your instance in **Service credentials**, click **New credential (+)** and enter the information required to generate new authentication information. \n",
    "\n",
    "**Action**: Enter your WML service instance credentials here.\n",
    "\n",
    "`\n",
    "wml_credentials = {\n",
    "  \"apikey\": \"------\",\n",
    "  \"iam_apikey_description\": \"------:\",\n",
    "  \"iam_apikey_name\": \"------\",\n",
    "  \"iam_role_crn\": \"-------\",\n",
    "  \"iam_serviceid_crn\": \"-------\",\n",
    "  \"instance_id\": \"-------\",\n",
    "  \"url\": \"------\"\n",
    "}\n",
    "`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# @hidden_cell\n",
    "wml_credentials = {\n",
    " # Credentials deleted\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import the `watson-machine-learning-client` and authenticate to the service instance.\n",
    "\n",
    "**Tip:** If `watson-machine-learning-client` is not preinstalled in your environment, run the following command to install it: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: watson-machine-learning-client in /opt/conda/envs/Python36/lib/python3.6/site-packages (1.0.378)\n",
      "Requirement already satisfied: tabulate in /opt/conda/envs/Python36/lib/python3.6/site-packages (from watson-machine-learning-client) (0.8.2)\n",
      "Requirement already satisfied: tqdm in /opt/conda/envs/Python36/lib/python3.6/site-packages (from watson-machine-learning-client) (4.31.1)\n",
      "Requirement already satisfied: pandas in /opt/conda/envs/Python36/lib/python3.6/site-packages (from watson-machine-learning-client) (0.24.1)\n",
      "Requirement already satisfied: ibm-cos-sdk in /opt/conda/envs/Python36/lib/python3.6/site-packages (from watson-machine-learning-client) (2.4.3)\n",
      "Requirement already satisfied: requests in /opt/conda/envs/Python36/lib/python3.6/site-packages (from watson-machine-learning-client) (2.21.0)\n",
      "Requirement already satisfied: lomond in /opt/conda/envs/Python36/lib/python3.6/site-packages (from watson-machine-learning-client) (0.3.3)\n",
      "Requirement already satisfied: urllib3 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from watson-machine-learning-client) (1.24.1)\n",
      "Requirement already satisfied: certifi in /opt/conda/envs/Python36/lib/python3.6/site-packages (from watson-machine-learning-client) (2020.6.20)\n",
      "Requirement already satisfied: pytz>=2011k in /opt/conda/envs/Python36/lib/python3.6/site-packages (from pandas->watson-machine-learning-client) (2018.9)\n",
      "Requirement already satisfied: python-dateutil>=2.5.0 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from pandas->watson-machine-learning-client) (2.7.5)\n",
      "Requirement already satisfied: numpy>=1.12.0 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from pandas->watson-machine-learning-client) (1.15.4)\n",
      "Requirement already satisfied: ibm-cos-sdk-s3transfer==2.*,>=2.0.0 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from ibm-cos-sdk->watson-machine-learning-client) (2.4.3)\n",
      "Requirement already satisfied: ibm-cos-sdk-core==2.*,>=2.0.0 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from ibm-cos-sdk->watson-machine-learning-client) (2.4.3)\n",
      "Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from requests->watson-machine-learning-client) (2.8)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from requests->watson-machine-learning-client) (3.0.4)\n",
      "Requirement already satisfied: six>=1.10.0 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from lomond->watson-machine-learning-client) (1.12.0)\n",
      "Requirement already satisfied: docutils>=0.10 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk->watson-machine-learning-client) (0.14)\n",
      "Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk->watson-machine-learning-client) (0.9.3)\n"
     ]
    }
   ],
   "source": [
    "!pip install watson-machine-learning-client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from watson_machine_learning_client import WatsonMachineLearningAPIClient"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:** A deprecation warning is returned from scikit-learn package that does not impact watson machine learning client functionalities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "client = WatsonMachineLearningAPIClient(wml_credentials)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0.378\n"
     ]
    }
   ],
   "source": [
    "# Display the client version number.\n",
    "print(client.version)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:** `watson-machine-learning-client` documentation can be found <a href=\"http://wml-api-pyclient.mybluemix.net/\" target=\"_blank\" rel=\"noopener noreferrer\">here</a>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"model\"></a>\n",
    "## 2. Create the training definitions\n",
    "\n",
    "In this section you:\n",
    "\n",
    "- [2.1 Prepare the training definition metadata](#prep)\n",
    "- [2.2 Get the sample model definition content files from Git](#get)\n",
    "- [2.3 Store the training definition in the WML repository](#store)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Prepare the training definition metadata<a id=\"prep\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare the training definition metadata. The main program will be called with\n",
    "enviroment variables `$DATA_DIR` and `$RESULT_DIR` as the inputs for the\n",
    "`--data-dir` and `--result-dir` options.\n",
    "\n",
    "**Tip:** You may want to change the number of epoch to run with a larger epoch number."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_definition_metadata = {\n",
    "            client.repository.DefinitionMetaNames.NAME: \"My definition name\",\n",
    "            client.repository.DefinitionMetaNames.DESCRIPTION: \"My description\",\n",
    "            client.repository.DefinitionMetaNames.AUTHOR_NAME: \"John Smith\",\n",
    "            client.repository.DefinitionMetaNames.FRAMEWORK_NAME: \"pytorch\",\n",
    "            client.repository.DefinitionMetaNames.FRAMEWORK_VERSION: \"0.4\",\n",
    "            client.repository.DefinitionMetaNames.RUNTIME_NAME: \"python\",\n",
    "            client.repository.DefinitionMetaNames.RUNTIME_VERSION: \"3.6\",\n",
    "            client.repository.DefinitionMetaNames.EXECUTION_COMMAND: \"python3 main.py --epochs 1 --data-dir $DATA_DIR --result-dir $RESULT_DIR\"\n",
    "            }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Get the sample model definition content file from GitHub <a id=\"get\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The sample model used here is the <a href=\"https://github.com/pytorch/examples/tree/master/mnist\">MNIST model</a> from the official PyTorch examples repository. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pytorch-mnist.zip was downloaded\n"
     ]
    }
   ],
   "source": [
    "filename='pytorch-mnist.zip'\n",
    "\n",
    "if not os.path.isfile(filename):\n",
    "    filename = wget.download('https://github.com/IBM/pytorch-on-watson-studio/raw/master/data/code/pytorch-mnist.zip')\n",
    "    print(filename, \"was downloaded\")\n",
    "else:\n",
    "    print(filename, \"was downloaded previously.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can verify the size of the model definition file by running the following command.\n",
    "```\n",
    "ls -o\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Store the training definition in the WML repository<a id=\"store\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "definition_details = client.repository.store_definition(filename, model_definition_metadata)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "definition_uid = client.repository.get_definition_uid(definition_details)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f62fbb05-7f3c-4337-b270-d7f2a661c421\n"
     ]
    }
   ],
   "source": [
    "# Display the training definition uid.\n",
    "print(definition_uid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Train the model<a id=\"train\"></a>\n",
    "\n",
    "In this section, learn how to:\n",
    "- [3.1 Enter training configuration metadata](#meta)\n",
    "- [3.2 Train the model in the background](#backg)\n",
    "- [3.3 Monitor the training log](#log)\n",
    "- [3.4 Cancel the training run](#cancel)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 Enter training configuration metadata<a id=\"meta\"></a>\n",
    "\n",
    "- `TRAINING_DATA_REFERENCE` - references the uploaded training data.\n",
    "- `TRAINING_RESULTS_REFERENCE` - location where trained model will be saved.\n",
    "\n",
    "**Note** Your COS credentials are referenced in this code.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Configure the training metadata for the TRAINING_DATA_REFERENCE and TRAINING_RESULTS_REFERENCE.\n",
    "training_configuration_metadata = {\n",
    "            client.training.ConfigurationMetaNames.NAME: \"Hand-written Digit Recognition\", \n",
    "            client.training.ConfigurationMetaNames.AUTHOR_NAME: \"John Smith\",              \n",
    "            client.training.ConfigurationMetaNames.DESCRIPTION: \"Hand-written Digit Recognition training\",\n",
    "            client.training.ConfigurationMetaNames.COMPUTE_CONFIGURATION: {\"name\": \"k80\"},\n",
    "            client.training.ConfigurationMetaNames.TRAINING_DATA_REFERENCE: {\n",
    "                    \"connection\": {\n",
    "                        \"endpoint_url\": service_endpoint,\n",
    "                        \"access_key_id\": cos_credentials['cos_hmac_keys']['access_key_id'],\n",
    "                        \"secret_access_key\": cos_credentials['cos_hmac_keys']['secret_access_key']\n",
    "                    },\n",
    "                    \"source\": {\n",
    "                        \"bucket\": buckets[0],\n",
    "                    },\n",
    "                    \"type\": \"s3\"\n",
    "                },\n",
    "            client.training.ConfigurationMetaNames.TRAINING_RESULTS_REFERENCE: {\n",
    "                \"connection\": {\n",
    "                    \"endpoint_url\": service_endpoint,\n",
    "                    \"access_key_id\": cos_credentials['cos_hmac_keys']['access_key_id'],\n",
    "                    \"secret_access_key\": cos_credentials['cos_hmac_keys']['secret_access_key']\n",
    "                },\n",
    "                \"target\": {\n",
    "                    \"bucket\": buckets[1],\n",
    "                },\n",
    "                \"type\": \"s3\"\n",
    "            },\n",
    "        }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Train the model in the background<a id=\"backg\"></a>\n",
    "\n",
    "To run the training in the **background**, set the optional parameter `asynchronous=True` (or remove it). In this case the parameter has been removed. \n",
    "\n",
    "**Note:** To run the training in **active** mode, set `asynchronous=False`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_run_details = client.training.run(definition_uid, training_configuration_metadata)\n",
    "# print(json.dumps(training_run_details, indent=2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training_run_guid_async= model-shg4e7ha\n"
     ]
    }
   ],
   "source": [
    "training_run_guid_async = client.training.get_run_uid(training_run_details)\n",
    "print(\"training_run_guid_async=\",training_run_guid_async)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the status of the training run by calling the method the next cell:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"current_at\": \"2020-08-06T21:18:45.655Z\",\n",
      "  \"metrics\": [],\n",
      "  \"state\": \"pending\",\n",
      "  \"submitted_at\": \"2020-08-06T21:18:34.554Z\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "# Get training run status.\n",
    "status = client.training.get_status(training_run_guid_async)\n",
    "print(json.dumps(status, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3  Monitor the training log<a id=\"log\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the cell below to monitor the training log. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "####################################################\n",
      "\n",
      "Log monitor started for training run: model-shg4e7ha\n",
      "\n",
      "####################################################\n",
      "\n",
      "\n",
      "training-OArTxnSGg:   LOG_DIR: /job/logs\n",
      "\n",
      "training-OArTxnSGg: Thu Aug  6 21:18:53 UTC 2020: Running PyTorch job\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [0/60000 (0%)]\tLoss: 2.351818\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [640/60000 (1%)]\tLoss: 2.290547\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [1280/60000 (2%)]\tLoss: 2.287427\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [1920/60000 (3%)]\tLoss: 2.304919\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [2560/60000 (4%)]\tLoss: 2.278541\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [3200/60000 (5%)]\tLoss: 2.216028\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [3840/60000 (6%)]\tLoss: 2.230055\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [4480/60000 (7%)]\tLoss: 2.180564\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [5120/60000 (9%)]\tLoss: 2.127563\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [5760/60000 (10%)]\tLoss: 2.063140\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [6400/60000 (11%)]\tLoss: 1.945805\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [7040/60000 (12%)]\tLoss: 1.995426\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [7680/60000 (13%)]\tLoss: 1.831923\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [8320/60000 (14%)]\tLoss: 1.669878\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [8960/60000 (15%)]\tLoss: 1.792871\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [9600/60000 (16%)]\tLoss: 1.557927\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [10240/60000 (17%)]\tLoss: 1.505638\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [10880/60000 (18%)]\tLoss: 1.383776\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [11520/60000 (19%)]\tLoss: 1.315710\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [12160/60000 (20%)]\tLoss: 1.234057\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [12800/60000 (21%)]\tLoss: 1.181712\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [13440/60000 (22%)]\tLoss: 1.187185\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [14080/60000 (23%)]\tLoss: 0.941361\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [14720/60000 (25%)]\tLoss: 1.079440\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [15360/60000 (26%)]\tLoss: 0.864651\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [16000/60000 (27%)]\tLoss: 1.246875\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [16640/60000 (28%)]\tLoss: 0.783206\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [17280/60000 (29%)]\tLoss: 1.024021\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [17920/60000 (30%)]\tLoss: 0.792960\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [18560/60000 (31%)]\tLoss: 0.803963\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [19200/60000 (32%)]\tLoss: 0.859197\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [19840/60000 (33%)]\tLoss: 0.954909\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [20480/60000 (34%)]\tLoss: 0.949569\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [21120/60000 (35%)]\tLoss: 0.914748\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [21760/60000 (36%)]\tLoss: 0.712012\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [22400/60000 (37%)]\tLoss: 0.566637\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [23040/60000 (38%)]\tLoss: 0.607327\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [23680/60000 (39%)]\tLoss: 0.948613\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [24320/60000 (41%)]\tLoss: 0.685381\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [24960/60000 (42%)]\tLoss: 0.806403\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [25600/60000 (43%)]\tLoss: 0.676512\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [26240/60000 (44%)]\tLoss: 0.503738\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [26880/60000 (45%)]\tLoss: 0.726284\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [27520/60000 (46%)]\tLoss: 0.675021\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [28160/60000 (47%)]\tLoss: 0.595621\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [28800/60000 (48%)]\tLoss: 0.598795\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [29440/60000 (49%)]\tLoss: 0.766931\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [30080/60000 (50%)]\tLoss: 0.791907\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [30720/60000 (51%)]\tLoss: 0.508784\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [31360/60000 (52%)]\tLoss: 0.917543\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [32000/60000 (53%)]\tLoss: 0.555115\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [32640/60000 (54%)]\tLoss: 0.543236\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [33280/60000 (55%)]\tLoss: 0.418464\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [33920/60000 (57%)]\tLoss: 0.655246\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [34560/60000 (58%)]\tLoss: 0.744008\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [35200/60000 (59%)]\tLoss: 0.497694\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [35840/60000 (60%)]\tLoss: 0.687398\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [36480/60000 (61%)]\tLoss: 0.435248\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [37120/60000 (62%)]\tLoss: 0.632875\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [37760/60000 (63%)]\tLoss: 0.661807\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [38400/60000 (64%)]\tLoss: 0.653850\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [39040/60000 (65%)]\tLoss: 0.569043\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [39680/60000 (66%)]\tLoss: 0.718136\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [40320/60000 (67%)]\tLoss: 0.595050\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [40960/60000 (68%)]\tLoss: 0.664589\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [41600/60000 (69%)]\tLoss: 0.372413\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [42240/60000 (70%)]\tLoss: 0.453466\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [42880/60000 (71%)]\tLoss: 0.381268\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [43520/60000 (72%)]\tLoss: 0.580236\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [44160/60000 (74%)]\tLoss: 0.579172\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [44800/60000 (75%)]\tLoss: 0.657730\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [45440/60000 (76%)]\tLoss: 0.468695\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [46080/60000 (77%)]\tLoss: 0.502350\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [46720/60000 (78%)]\tLoss: 0.581853\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [47360/60000 (79%)]\tLoss: 0.562236\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [48000/60000 (80%)]\tLoss: 0.440204\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [48640/60000 (81%)]\tLoss: 0.366501\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [49280/60000 (82%)]\tLoss: 0.589636\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [49920/60000 (83%)]\tLoss: 0.587362\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [50560/60000 (84%)]\tLoss: 0.459765\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [51200/60000 (85%)]\tLoss: 0.748436\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [51840/60000 (86%)]\tLoss: 0.337991\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [52480/60000 (87%)]\tLoss: 0.484137\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [53120/60000 (88%)]\tLoss: 0.345578\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [53760/60000 (90%)]\tLoss: 0.617756\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [54400/60000 (91%)]\tLoss: 0.433388\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [55040/60000 (92%)]\tLoss: 0.459494\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [55680/60000 (93%)]\tLoss: 0.383572\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [56320/60000 (94%)]\tLoss: 0.412728\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [56960/60000 (95%)]\tLoss: 0.459693\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [57600/60000 (96%)]\tLoss: 0.334113\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [58240/60000 (97%)]\tLoss: 0.526032\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [58880/60000 (98%)]\tLoss: 0.447022\n",
      "\n",
      "training-OArTxnSGg: Train Epoch: 1 [59520/60000 (99%)]\tLoss: 0.455017\n",
      "\n",
      "training-OArTxnSGg: /opt/conda/lib/python3.7/site-packages/torch/nn/_reduction.py:43: UserWarning: size_average and reduce args will be deprecated, please use reduction='sum' instead.\n",
      "\n",
      "training-OArTxnSGg:   warnings.warn(warning.format(ret))\n",
      "\n",
      "training-OArTxnSGg: \n",
      "\n",
      "training-OArTxnSGg: Test set: Average loss: 0.1991, Accuracy: 9429/10000 (94%)\n",
      "\n",
      "training-OArTxnSGg: \n",
      "\n",
      "training-OArTxnSGg: \n",
      "training-OArTxnSGg: \n",
      "training-OArTxnSGg: \n",
      "training-OArTxnSGg: \n",
      "training-OArTxnSGg: \n",
      "training-OArTxnSGg: \n",
      "training-OArTxnSGg: \n",
      "training-OArTxnSGg: \n",
      "training-OArTxnSGg: \n",
      "training-OArTxnSGg: \n",
      "training-OArTxnSGg: \n",
      "training-OArTxnSGg: \n",
      "training-OArTxnSGg: \n",
      "\n",
      "\n",
      "-----------------\n",
      "Log monitor done.\n",
      "-----------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "client.training.monitor_logs(training_run_guid_async)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After the training is complete, get the training GUID."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training GUID is: training-OArTxnSGg\n"
     ]
    }
   ],
   "source": [
    "training_details = client.training.get_details(training_run_guid_async)\n",
    "training_guid = training_details[\"entity\"][\"training_results_reference\"][\"location\"][\"model_location\"]\n",
    "print(\"Training GUID is:\", training_guid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 Cancel the training run<a id=\"cancel\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can cancel the training run by calling the method below.\n",
    "```\n",
    "client.training.cancel(training_run_guid_async)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"work\"></a>\n",
    "## 4. Work with the trained models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this sample workload, the trained model is saved as a file named\n",
    "`saved_models.pth` in the result bucket.\n",
    "The following code will fetch the model file from the bucket.\n",
    "\n",
    "**Tip:** Make sure that the training run is completed by checking it's\n",
    "status as shown earlier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# buckets[1] is the bucket to save the result data as defined above\n",
    "bucket_obj = cos.Bucket(buckets[1])\n",
    "# model file name as defined in the code \n",
    "saved_model_filename = \"saved_models.pth\"\n",
    "source_file = os.path.join(training_guid, saved_model_filename)\n",
    "bucket_obj.download_file(source_file,saved_model_filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copy the definition of the neural network as it is defined in the sample workload. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)\n",
    "        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)\n",
    "        self.conv2_drop = nn.Dropout2d()\n",
    "        self.fc1 = nn.Linear(320, 50)\n",
    "        self.fc2 = nn.Linear(50, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = F.relu(F.max_pool2d(self.conv1(x), 2))\n",
    "        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))\n",
    "        x = x.view(-1, 320)\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.dropout(x, training=self.training)\n",
    "        x = self.fc2(x)\n",
    "        return F.log_softmax(x, dim=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instantiate and load previously trained model parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<All keys matched successfully>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnist_model = Net()\n",
    "mnist_model.load_state_dict(torch.load(saved_model_filename, map_location='cpu'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download sample image files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['img_1.jpg', 'img_2.jpg', 'img_3.jpg', 'img_4.jpg', 'img_5.jpg', 'img_6.jpg', 'img_7.jpg', 'img_8.jpg', 'img_9.jpg']\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import wget\n",
    "\n",
    "images =[]\n",
    "for i in range(1,10):\n",
    "    filename = \"img_\"+str(i)+\".jpg\"\n",
    "    images.append(filename)\n",
    "    if not os.path.isfile(filename):\n",
    "        path = \"https://github.com/IBM/pytorch-on-watson-studio/raw/master/data/images/\"+filename\n",
    "        wget.download(path)\n",
    "\n",
    "print(images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the trained model to predict the digits in the sample image files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from IPython.display import display\n",
    "from PIL import Image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Python-First**\n",
    "\n",
    "PyTorch is not a Python binding into a monolithic C++ framework.\n",
    "It’s built to be deeply integrated into Python so it can\n",
    "be used with popular Python libraries.\n",
    "\n",
    "The code below shows how to convert a NumPy array to\n",
    "a PyTorch tensor using `torch.from_numpy`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAABwklEQVR4nL2Su2sUYRTFf983385zsyYmiq9CIhbBB1ooiBCIlSBBG/8Kia1NwEqwsrS0SyMIIipaig9UsPIBBhTxAUpwE3dmmc3MzhyL3SAbrT3t5Xfvuede2JDxwFkSaBkmGVUTAkyMteN43mgtCAkhhEkw23CbUAMGgATiUXSMLfGu6Yn5e3f2baYArtdFmfb7OuTTIBqdefqV2qnqsvfi2Ym/3C7rl9aLspA6n47S3DBiPWsNc19VZ5emZy4W6i1ODSoGDdbMZmdid+sjPD+YYJoZgDUDGqwjIfA5k0l/DBvrOUcLiLAwLVVxDODAGKMKL2Os7lL7/hwqG9WQM2Y4nIafwMSDXOkYAM5TDVhLP+kePpDfJpwVH9IoB5ypwfihO+fOBkfG9Vj7q+j1qSAftHWwd+Fhro4yqVNqTVpeHKZngctakfJvbzqF1FO1rvTmMKGEqJJ+vDzPsSdtfWlfW3ikNd0FwGe76aa13q98fpuVWp2Hqfv5aroRglsq1VWnKvT0wkm2Wp/jujE4/47v+Huu7G4HP6++C/sUNEqjVlYDxLRo0sDgEwAhMQ2IB01bONgJEUQkGLAQ/ONT/o9+AwaCrAyXKnAbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<PIL.Image.Image image mode=L size=28x28 at 0x7FAC48E6C940>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction for image number 1 is: 2\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAB60lEQVR4nL2SO2gUYRSFv/8xr8zuJjGYBBYf2KRQsEhEUQuxU1AstElnJ7ba2AkWFoI2aiFY2KWIWogWgpUWQlAiBgyoKIgx6+a1m+yOO//MXItdYsTer7ucc+Gew4UeFjCgAU+DB90BgIywpMAoNEbh2EoAHmgCC1j7l0YMYH2UBrQCQG2qKlSJaArlU4jk/IPGqD/3bdncllUO9DcC86tg6dOibDoASsf27j8ZNStJEfN56tF8G4AQSnD2wZKTVDptWU4Tqc8dhNgHlKcZuiIbaVprivzIxP2U7NyA6UYI4VKaLMj7j3MzT2wZ78hKdqpbmNUEx2u5yFu0D4T4u17mN0crxoDV2u5sScO9xngQR8BkXaoYbNzCcyXH/Xfrfpr7RVuIePX8xO7vOeBDOC/XK+CBxQQAN2RmKELjLG7s62JzBEdfRu6BHjZu31qiwAByu2wgBA3a4/ydukwNAj4mMmvfJvA9BRXgwtVWQzaOQl+3OyneTAwBOy4/nH4xuyrSumUpaxWRUP0QsrCaDifNPdvz2mjq6Wt3G8qugwpgLJNGUYgTScXJl2eg0Qo70EkorTytjtMMXSNOpvuTiyqUDgUoULoIk8OHivFJHs8u34ubQSooqzIh7oUPGDxzesQSYSwE0HvP/81vvQi9IwdRtQUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<PIL.Image.Image image mode=L size=28x28 at 0x7FAC48DF7C18>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction for image number 2 is: 0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAABfElEQVR4nG2RP0iVURiHn+897/nO+b5z73e9Gf4hy9wcCkEkooJAojGdvasukUNT0FhDe4GLQ9DeFOVSi0sQxQUhaGgrrpk0SHBRUT8HETkHf+vD876/8x44jQMgdwAmI4qBasBnQJVBpTG1TaDgEmDBREzBBqseWg5xsUgQAA0GwZoECpQ5gBYgMfX4gD7+slfXL67hTCIKEx97SzP5wzdrgbhtgd7qfRrHUPG1nRa6OLL5HqwKT36THAHe/rmqJbi53qMLlPHOxf4Ulszf/bEMeez5pxuM48eWtz/QZOj0MCfZzYfZnL1zI+ws8T/8TVY+6G/9rD/f3r7XIJOsBM5qydH85OC79dWb07YP5jASM6zAQn0fKKCIx7YtYerXa/AgSVsL6PfuIJgGopKMHXCdw2lQhCxmiOX6wXM8jpaFxPSh++2KwBBoEZJ3Vqv1GOSKXdnotBM2ud8BByOUz9ZeJWr3pSZffJbRf5dpQHUubNJyFHIuQ1GTTj0GLLBCcYwfTjgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<PIL.Image.Image image mode=L size=28x28 at 0x7FAC48DF7C50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction for image number 3 is: 9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAABfklEQVR4nL2Sz0rUURTHP+ece3//plFzIJAWghEFkQvtCVroSugFpK09Rft6AJGgzeADBBW0iB4hXOVCBXXThFoEyqQz97a485v6ja07q3v43u/33O/9HhiXIKqIyxTU8a/KAT866x9eqtJjohMMJ4iAgYLlBjDWjhEhghiBYbg2zgy0AsvBJsGiFDySCfiiIatBvYahX1x2n75Av0HLmHn5YW3ja4hXO45sZEEkudD1Xnz/5snTjbMfW27kzEw1wdVmp8WUcT9GKABUJWF+7sX+aSx/uuky/KKVZnrnTAWke96u0AK6/ePaSC17q7uWxBZiOGPmRuNBzy4e0YLq4cH33dvFxA/sHwE33d3PwzDfxlmdhsDUvZMVcpvd+hbfPvZ/kwyyudNX8HrvMj5fbUPtPaVFftj72IuDndUK0BrSlAab8WIQt+8A5PkIEzcAC5HO0oOrd4ex7CMaAjasYxFVKfCOjozXJckKpAUpQSikgdaNE9q41Is2rvzP+g3u71nLuWpLcgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<PIL.Image.Image image mode=L size=28x28 at 0x7FAC48DF7BE0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction for image number 4 is: 9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAABlUlEQVR4nL2SvWsUURTFf+/7zWx2Q7IEI7Kg6RQkFkr+gXSWgpVhsbARC0lhI9hK0M5aCxEsrSKCpY0aSBMWLAKxUpFNIGYnmf0w12J2FmftPcV7xeFezvlxoZT1wCyQkGgUfysAAQ1zzMI8VUVI0daAg6TqnYWl66+zz+0zDlJixTS0XvREsl/blwzgq6Pz6yJPH3ZEXmIxVU+zMVzDNx93T88noRoWUpo2Khrd3kZZRJeeP+ZwpMRc6dZWkKm0NBYd9QgduRem4hDxLevgap5dcGBh/ADkM+m3xv1ze+3T5z+GJHllsgbh9kh68n5OlTUngbJAf/dTz2fqSCyD6S7UqbVF9te1+YdQBAyr0u80maAtSCki1CF9InLXjHNqBHzqDMPQbL+ZOX52QC6MwhiNRllw4cHe70S7xYHcjBZXjBafv/xRhhdj69qm7CwrP25hwHrS5bdy8n3r3ZHI1zsKbYq1FlyksSK59OVkcLh5I6JssQ5lQKEWHn35KbuvPtxaghBtycDroo4FYwnFHboJ8/+tP/TVcadip7lNAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<PIL.Image.Image image mode=L size=28x28 at 0x7FAC48DF7CC0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction for image number 5 is: 3\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAABtElEQVR4nGWSu2uUQRTFf/O4M99j93PjY4VEDQmIREihFmKXgBhLMY35D2ysRRFBrKyDsp1YiI1CmsUuiGgjYicYxCZFxBCN65fHhmwYi2XF+fZUF86cew7nDgJoZVMFGKOIYTI7GKsU3gCSeINHEBORFpIMMGQ4dFVrvIIagMfFSu01iCVxxlRlaExiwQOQY2MhB7bHwofQWWpdv5j3YkcKh7rd7a3th274+uXFlAjUQVljw6EO9XLSt++dmDgjJ6duzL9bXN4q093QX5Jx/sf2UxR1itGrnzbC4wLlrALjioZuleGWEXCQ+vn13l0AJX3jl93NJtYnOTm++SZcHtSnSeXm54ckBgV6hFZ41qg1RwQUzXWynaMbKtuePjc+zd54doG2XJnY3ApYaKBwBlb298rO6kFYvnTs+Eyh/j/SYR69vT935El4ddohQ00WNJj5/XoOII0YD2RaVv/Mgq4N3U6Db4cHkJBVKZ96zoaPp8CCr5haGF38voCAJYm5GpbZsCYopK6GvhqytHsHhbWgq4H8tfBrLIry74XBT/Lt504lxgDlyvP33SGvPhIYy+Pe/gJoVGbHG+3E9gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<PIL.Image.Image image mode=L size=28x28 at 0x7FAC48DF7DD8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction for image number 6 is: 9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAByElEQVR4nL2SvWtTYRTGf+/73vd+tU1sSipKoHZQdC2KIjiIo4NUqIJOKuIkgoMgiGPdhDo4KFRwVxQ361Zwq4OD4tYPRRsak5jc5CY3vceh9xabP8BnPD8O5zznOZDJUVCBMEBjsAxJldAhaAr4jOxBI0a7BqAAltHhTnAtHkwCpb3AL4J60xG5fuowRfzhRnutLVGrE789ORaqPWQUeC9RPxKRlYmdms5hGxUsR+lgo8rmzEeT1V1wjGIMrkjSfRA+E1mPd+0BGAt3qnWpweWGSON2zrQD1mNqQeq1SyGcr6fNpSCDroOjGJ+X/s8LAQV1RPqS3ssdggePZVNeQxHn0ItunN4F0ApgwPTcnzJK0zKD1Xe+F6sMprBdvllJG69mA1LxmEzSdimbabXHzFpH5nDBQdlbcfLjeO5zHzxqSv8GGg0c/NLcOJ0dIQBzblW2Xvp4ENqp9ajfyc+XoM3gABO92PYszuyHyrZ9nkeBcjnaGfx+UoSLi0tpXJOn43lkSsFZEal/+/r5V9qUpiwW/o3THFuuSa/Vk24qa58WUJR3gAOu4czD75JEItXWyv4QHxPubut6cHVLklb3/olpcIdf5H/qL7SOqZFoyCiUAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<PIL.Image.Image image mode=L size=28x28 at 0x7FAC48DF7E48>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction for image number 7 is: 0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAABoElEQVR4nL2RPWsUURSGn3vv3Dsf+2GSDShZ0EhCQFBTaul/EJcg+A8ES0vBwkrs02gVNKW1RFgtLNIo+AFCIDHgx8omGjfZnZmdORa7GTP7A3yrAw/n5X3PgbEMgUUrFMopCCipjm8jiwKIdJn5QA3mgqqNMGaSOmVH0xSOCSnuPWiLbN2fPjb6J8daKpIm8ralA39y8VznZ7/zZn1XNhccRGUazLQfLzpuZvFygAHAK6Deu4ZUkkGvrgbBIOwDReTKEZFRh1yvD5Oqol+2nZnFKp7uytfTGMISM8DK1k6aiaxgJ3su3fks0kuzX8Mn4KsSW94WORCROJHBZSjD58PfIvLs0oXVb3JrbFtUub00/ePDi9WMhTNHeapFRifVCt9hwWrgoqS9+VE+9Cly38SJAdLcY3Z+31vbniM7vpsBPM8LAc63pfMSpkbIeeCHPkAtovW9K6+u1IosmpoB3NUwurGe5PFO06f4dhVofdx/133flT9y+KnZOFEwUrr5UPZykVjky0ajAjTGtgFUWNwUOZDXj+6eBYMNT77yP+svMICO2hqaMCMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<PIL.Image.Image image mode=L size=28x28 at 0x7FAC48DF7DA0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction for image number 8 is: 3\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAB0UlEQVR4nMWRPWsUURSGn/sxe2d2NrsrKqmMxKAiFouFWglW+QWJFgYJ6SzUQhHRWhu1TPwBEhAkCCJI2kSTTtAqGEREJZqEdTc7+5Wd2WMxO5DoD/BtLpyH95zz3gOZjG8BjbLpu1cW8J3GUyj9L1QW8PED8Nw+hgK0hRKEBfCzGgA6VFFf2yQBwJN4LwQonjg6tFYpvl9t7e/KIfKTr38n7a0ftZ9PT/LX0PDquyRuSSQizaUrac2hPMDZO81etCi7ydbKW9mpDmiOkMAxI9HCWSSav8CZaYkuZRkMwamHVemMh4cxuoDjttxMI+ADcyJfpgwwZJXSPE7epE4Pj2NSiyco6EKAgTL3asspzEPllch18EE7AuCuTAycHlPy9eOYH2I8SgSeF6w3pwfH6OXj3ZGRb9pCz6/bNpVnR1w3Xcji1rYXggKU8XMKcjfq8mGYbKNluaYpp6lNiMSNmQELNJPSGPfgoAHH+fVfMueyYztzuS+fThcpogr+7EpHklsEg9/DXXwuIkujw4+aEsuGfH6Q+WwRxfHqZqO7+l0k6UqncV9hDmRDCc25FzsSyfamJC+fjFLWaVelghbgumOzuXapvzjf3SjVMYnf4f/oD/qUoz9UErMWAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<PIL.Image.Image image mode=L size=28x28 at 0x7FAC48DF7D68>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction for image number 9 is: 0\n"
     ]
    }
   ],
   "source": [
    "digits = [i for i in range(10)]\n",
    "\n",
    "mnist_model.eval()\n",
    "for i, filename in enumerate(images):\n",
    "    img = Image.open(filename).resize((28, 28)).convert('L')\n",
    "    display(img)\n",
    "    data = torch.from_numpy(np.asarray(img, dtype=np.float32)[np.newaxis, np.newaxis, :, :])\n",
    "    output = mnist_model(data)\n",
    "    # get the index of the max log-probability\n",
    "    prediction = output.max(1, keepdim=True)[1]\n",
    "    print(\"Prediction for image number\", i+1, \"is:\",  digits[prediction[0,0]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Native ONNX Support**\n",
    "\n",
    "PyTorch includes native <a href=\"http://onnx.ai/\">Open Neural Network Exchange (ONNX)</a> support. \n",
    "\n",
    "The following code will export models in the standard ONNX\n",
    "format so that the models can be consumed\n",
    "by ONNX-compatible platforms, runtimes, visualizers, and more.\n",
    "\n",
    "PyTorch exports the model by running the model through the training path once\n",
    "and then save the traced model to a file using ONNX format.\n",
    "\n",
    "**Tip:** You can test the exported ONNX format model by importing and\n",
    "running it in an ONNX-comptible famework. See\n",
    "<a href=\"https://github.com/onnx/tutorials/tree/master/tutorials/PytorchTensorflowMnist.ipynb\">ONNX tutorials</a>\n",
    "for more information. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Export the trained model to ONNX\n",
    "# one black and white 28 x 28 picture will be used as the input to the model\n",
    "dummy_input = torch.randn(1, 1, 28, 28) \n",
    "\n",
    "onnx_model_filename = \"mnist.onnx\"\n",
    "torch.onnx.export(mnist_model, dummy_input,onnx_model_filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Save the ONNX model file to the result bucket."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# buckets[1] is bucket to save the result data as defined above\n",
    "bucket_obj = cos.Bucket(buckets[1])\n",
    "# model file name as defined in the code \n",
    "bucket_obj.upload_file(onnx_model_filename,onnx_model_filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You are done and can delete the training run in WML by calling the method below.\n",
    "\n",
    "```\n",
    "client.training.delete(training_run_uid_async)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<a id=\"summary\"></a>\n",
    "## 5. Summary and next steps     "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "You successfully completed this notebook! \n",
    " \n",
    "You learned how to use `watson-machine-learning-client` to train PyTorch models. \n",
    " \n",
    "Check out our <a href=\"https://dataplatform.ibm.com/docs/content/analyze-data/wml-setup.html\" target=\"_blank\" rel=\"noopener noreferrer\">Online Documentation</a> for a <a href=\"https://dataplatform.ibm.com/docs/content/analyze-data/ml-python-mnist-tutorial.html\" target=\"_blank\" rel=\"noopener noreferrer\">tutorial</a> and more samples, documentation, how-tos, and blog posts. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Citations\n",
    "Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. \"Gradient-based learning applied to document recognition.\" Proceedings of the IEEE, 86(11):2278-2324, November 1998."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References\n",
    "1. <a href=\"https://pytorch.org/\">PyTorch</a>.\n",
    "2. <a href=\"https://github.com/pytorch/examples/tree/master/mnist\">MNIST model</a> from the official PyTorch examples repository.\n",
    "3. <a href=\"https://dataplatform.cloud.ibm.com/analytics/notebooks/v2/3bd3efb8-833d-460f-b07b-fee51dd0f1af/view?access_token=6bd0ff8d807861d09e0dab0cad28ce9685711078f612fcd92bb8cf8535d089c1\">Use TensorFlow to predict handwritten digits</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Authors\n",
    "\n",
    "**Lucasz Cmielowski**, PhD, is a Automation Architect and Data Scientist at IBM with a track record of developing enterprise-level applications that substantially increase the clients' ability to turn data into actionable knowledge.\n",
    "\n",
    "**Catherine Diep** is a Solutions Architect and Performance Engineer of the Cognitive OpenTech group at IBM Silicon Valley Lab. Her current projects include deep learning related workloads that use open source frameworks and APIs such as PyTorch, TensorFlow, Keras, etc.\n",
    "\n",
    "**Simeon Monov** is a Senior Software Developer and Performance Engineer for the Cognitive OpenTech group at IBM. He is currently working in data science and machine learning related projects."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright © 2017, 2018 IBM. This notebook and its source code are released under the terms of the MIT License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"background:#F5F7FA; height:110px; padding: 2em; font-size:14px;\">\n",
    "<span style=\"font-size:18px;color:#152935;\">Love this notebook? </span>\n",
    "<span style=\"font-size:15px;color:#152935;float:right;margin-right:40px;\">Don't have an account yet?</span><br>\n",
    "<span style=\"color:#5A6872;\">Share it with your colleagues and help them discover the power of Watson Studio!</span>\n",
    "<span style=\"border: 1px solid #3d70b2;padding:8px;float:right;margin-right:40px; color:#3d70b2;\"><a href=\"https://ibm.co/wsnotebooks\" target=\"_blank\" style=\"color: #3d70b2;text-decoration: none;\">Sign Up</a></span><br>\n",
    "</div>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}