notebooks/229-distilbert-sequence-classification/229-distilbert-sequence-classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e2b748f3",
   "metadata": {},
   "source": [
    "# Sentiment Analysis with OpenVINO™\n",
    "\n",
    "**Sentiment analysis** is the use of natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and study affective states and subjective information. This notebook demonstrates how to convert and run a sequence classification model using OpenVINO. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abc41ac0",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe80a355",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import DistilBertForSequenceClassification, AutoTokenizer\n",
    "import openvino.runtime as ov\n",
    "import warnings\n",
    "from pathlib import Path\n",
    "import numpy as np\n",
    "import time\n",
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36add5c2",
   "metadata": {},
   "source": [
    "## Initializing the Model\n",
    "We will use the transformer-based [distilbert-base-uncased-finetuned-sst-2-english](https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english) model from Hugging Face."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5db803ea",
   "metadata": {},
   "outputs": [],
   "source": [
    "checkpoint = \"distilbert-base-uncased\"\n",
    "model = DistilBertForSequenceClassification.from_pretrained(\n",
    "    pretrained_model_name_or_path=checkpoint\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae70bbf5",
   "metadata": {},
   "source": [
    "## Initializing the Tokenizer\n",
    "\n",
    "Text Preprocessing cleans the text-based input data so it can be fed into the model. [Tokenization](https://towardsdatascience.com/tokenization-for-natural-language-processing-a179a891bad4) splits paragraphs and sentences into smaller units that can be more easily assigned meaning. It involves cleaning the data and assigning tokens or IDs to the words, so they are represented in a vector space where similar words have similar vectors. This helps the model understand the context of a sentence. We're making use of an [AutoTokenizer](https://huggingface.co/docs/transformers/main_classes/tokenizer) from Hugging Face, which is essentially a pretrained tokenizer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "782bbebf",
   "metadata": {},
   "outputs": [],
   "source": [
    "tokenizer = AutoTokenizer.from_pretrained(\n",
    "    pretrained_model_name_or_path=checkpoint\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b00e300",
   "metadata": {},
   "source": [
    "## Convert to ONNX\n",
    "\n",
    "**ONNX** is an open format built to represent machine learning models. ONNX defines a common set of operators - the building blocks of machine learning and deep learning models - and a common file format to enable AI developers to use models with a variety of frameworks, tools, runtimes, and compilers. We need to convert our model from PyTorch to ONNX. In order to perform the operation, we use a function [torch.onnx.export](https://pytorch.org/docs/stable/onnx.html#example-alexnet-from-pytorch-to-onnx) to [convert a Hugging Face model](https://huggingface.co/blog/convert-transformers-to-onnx#export-with-torchonnx-low-level) to its respective ONNX format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4794f066",
   "metadata": {},
   "outputs": [],
   "source": [
    "onnx_model = \"distilbert.onnx\"\n",
    "MODEL_DIR = \"model/\"\n",
    "MODEL_DIR = f\"{MODEL_DIR}\"\n",
    "onnx_model_path = Path(MODEL_DIR) / onnx_model\n",
    "dummy_model_input = tokenizer(\"This is a sample\", return_tensors=\"pt\")\n",
    "torch.onnx.export(\n",
    "    model,\n",
    "    tuple(dummy_model_input.values()),\n",
    "    f=onnx_model,\n",
    "    input_names=['input_ids', 'attention_mask'],\n",
    "    output_names=['logits'],\n",
    "    dynamic_axes={'input_ids': {0: 'batch_size', 1: 'sequence'},\n",
    "                  'attention_mask': {0: 'batch_size', 1: 'sequence'},\n",
    "                  'logits': {0: 'batch_size', 1: 'sequence'}},\n",
    "    do_constant_folding=True,\n",
    "    opset_version=13,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fff79fc",
   "metadata": {},
   "source": [
    "# Model Optimizer\n",
    "\n",
    "[Model Optimizer](https://docs.openvino.ai/latest/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html) is a cross-platform command-line tool that facilitates the transition between training and deployment environments, performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a0f48c1b",
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer_command = f'mo \\\n",
    "    --input_model {onnx_model} \\\n",
    "    --output_dir {MODEL_DIR} \\\n",
    "    --model_name {checkpoint} \\\n",
    "    --input input_ids,attention_mask \\\n",
    "    --input_shape \"[1,128],[1,128]\"'\n",
    "! $optimizer_command"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27cc074e",
   "metadata": {},
   "source": [
    "OpenVINO™ Runtime uses the [Infer Request](https://docs.openvino.ai/latest/openvino_docs_OV_UG_Infer_request.html) mechanism which allows running models on different devices in asynchronous or synchronous manners. The model graph is sent as an argument to the OpenVINO API and an inference request is created. The default inference mode is AUTO but it can be changed according to requirements and hardware available. You can explore the different inference modes and their usage [in documentation.](https://docs.openvino.ai/latest/openvino_docs_Runtime_Inference_Modes_Overview.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e31a2644",
   "metadata": {},
   "outputs": [],
   "source": [
    "warnings.filterwarnings(\"ignore\")\n",
    "core = ov.Core()\n",
    "ir_model_xml = str((Path(MODEL_DIR) / checkpoint).with_suffix(\".xml\"))\n",
    "compiled_model = core.compile_model(ir_model_xml)\n",
    "infer_request = compiled_model.create_infer_request()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "de01fccc",
   "metadata": {},
   "outputs": [],
   "source": [
    "def softmax(x):\n",
    "    \"\"\"\n",
    "    Defining a softmax function to extract\n",
    "    the prediction from the output of the IR format\n",
    "    Parameters: Logits array\n",
    "    Returns: Probabilities\n",
    "    \"\"\"\n",
    "\n",
    "    e_x = np.exp(x - np.max(x))\n",
    "    return e_x / e_x.sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e778507",
   "metadata": {},
   "source": [
    "## Inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc0c91a6",
   "metadata": {},
   "outputs": [],
   "source": [
    "def infer(input_text):\n",
    "    \"\"\"\n",
    "    Creating a generic inference function\n",
    "    to read the input and infer the result\n",
    "    into 2 classes: Positive or Negative.\n",
    "    Parameters: Text to be processed\n",
    "    Returns: Label: Positive or Negative.\n",
    "    \"\"\"\n",
    "\n",
    "    input_text = tokenizer(\n",
    "        input_text,\n",
    "        padding=\"max_length\",\n",
    "        max_length=128,\n",
    "        truncation=True,\n",
    "        return_tensors=\"np\",\n",
    "    )\n",
    "    inputs = dict(input_text)\n",
    "    label = {0: \"NEGATIVE\", 1: \"POSITIVE\"}\n",
    "    result = infer_request.infer(inputs=inputs)\n",
    "    for i in result.values():\n",
    "        probability = np.argmax(softmax(i))\n",
    "    return label[probability]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b60e79fd",
   "metadata": {},
   "source": [
    "### For a single input sentence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf976f71",
   "metadata": {},
   "outputs": [],
   "source": [
    "input_text = \"I had a wonderful day\"\n",
    "start_time = time.perf_counter()\n",
    "result = infer(input_text)\n",
    "end_time = time.perf_counter()\n",
    "total_time = end_time - start_time\n",
    "print(\"Label: \", result)\n",
    "print(\"Total Time: \", \"%.2f\" % total_time, \" seconds\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29b4d013",
   "metadata": {},
   "source": [
    "### Read from a text file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63f57d28",
   "metadata": {},
   "outputs": [],
   "source": [
    "start_time = time.perf_counter()\n",
    "with open(\"../data/text/food_reviews.txt\", \"r\") as f:\n",
    "    input_text = f.readlines()\n",
    "    for lines in input_text:\n",
    "        print(\"User Input: \", lines)\n",
    "        result = infer(lines)\n",
    "        print(\"Label: \", result, \"\\n\")\n",
    "end_time = time.perf_counter()\n",
    "total_time = end_time - start_time\n",
    "print(\"Total Time: \", \"%.2f\" % total_time, \" seconds\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  },
  "nbTranslate": {
   "displayLangs": [
    "*"
   ],
   "hotkey": "alt-t",
   "langInMainMenu": true,
   "sourceLang": "en",
   "targetLang": "fr",
   "useGoogleTranslate": true
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}