NQU_text_analysis.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/howard-haowen/NLP-demos/blob/main/NQU_text_analysis.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BkUGVmFCX8bw"
      },
      "source": [
        "# 中文分詞與分析應用"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "講者: [江豪文](howard-haowen.github.io)"
      ],
      "metadata": {
        "id": "6yLGTzn1TNkY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "![](https://github.com/howard-haowen/NLP-demos/raw/main/img/NQU_applications_of_tokens.png)"
      ],
      "metadata": {
        "id": "72MCBZpXSKot"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 目標"
      ],
      "metadata": {
        "id": "RTtGaOGFcQL3"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- <font size=\"8\">學習大量分析中文文本的工具與方法</font>\n",
        "- <font size=\"8\">了解用程式解決問題的基本方法</font>\n",
        "- <font size=\"8\">培養人文觀點的數據思維</font>"
      ],
      "metadata": {
        "id": "E2hzm3ouyjn4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 大綱"
      ],
      "metadata": {
        "id": "x4PfPGXRcWTq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- <font size=\"10\">什麼是分詞</font>\n",
        "- <font size=\"10\">如何使用程式分詞</font>\n",
        "- <font size=\"10\">大量文本分析與應用</font>\n"
      ],
      "metadata": {
        "id": "O34jvolvxSBB"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 什麼是分詞"
      ],
      "metadata": {
        "id": "q-N-z1A6ycID"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 三種常見的分詞顆粒度\n",
        "\n",
        "![](https://www.freecodecamp.org/news/content/images/size/w2000/2021/10/IMG_0079.jpg)"
      ],
      "metadata": {
        "id": "xgx0cSn4y9hJ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 如何使用程式分詞"
      ],
      "metadata": {
        "id": "A5fCd-yARKGa"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 代表性的中文分詞模型"
      ],
      "metadata": {
        "id": "Etvx59ddgm_W"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- <font size=\"8\">結巴 jieba</font>\n",
        "- <font size=\"8\">北大分詞 PKUSeg </font>\n",
        "- <font size=\"8\">[中研院 CKIP Transformers](https://ckip.iis.sinica.edu.tw/service/transformers/)</font>"
      ],
      "metadata": {
        "id": "0IU6THfTb3Fi"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### NLP框架"
      ],
      "metadata": {
        "id": "hPEUzH8TgxY_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 使用NLP框架`spaCy`整合各種模型的使用\n",
        "\n",
        "![](https://d33wubrfki0l68.cloudfront.net/3ad0582d97663a1272ffc4ccf09f1c5b335b17e9/7f49c/pipeline-fde48da9b43661abcdf62ab70a546d71.svg)"
      ],
      "metadata": {
        "id": "L-3l_YmmfjDd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "!pip install -q -U pip setuptools wheel\n",
        "!pip install -q -U spacy\n",
        "!python -m spacy download zh_core_web_sm"
      ],
      "metadata": {
        "id": "W75zNlFhxpNw",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "cellView": "form",
        "outputId": "4ac64add-e859-4b3d-f2ac-d66c62d9afa0"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[K     |████████████████████████████████| 2.1 MB 32.6 MB/s \n",
            "\u001b[K     |████████████████████████████████| 1.2 MB 55.9 MB/s \n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "ipython 7.9.0 requires jedi>=0.10, which is not installed.\u001b[0m\n",
            "\u001b[?25h\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
            "\u001b[0m2022-11-23 14:32:00.220191: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected\n",
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting zh-core-web-sm==3.4.0\n",
            "  Downloading https://github.com/explosion/spacy-models/releases/download/zh_core_web_sm-3.4.0/zh_core_web_sm-3.4.0-py3-none-any.whl (48.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m48.4/48.4 MB\u001b[0m \u001b[31m16.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: spacy<3.5.0,>=3.4.0 in /usr/local/lib/python3.7/dist-packages (from zh-core-web-sm==3.4.0) (3.4.3)\n",
            "Collecting spacy-pkuseg<0.1.0,>=0.0.27\n",
            "  Downloading spacy_pkuseg-0.0.32-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.4/2.4 MB\u001b[0m \u001b[31m57.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: wasabi<1.1.0,>=0.9.1 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (0.10.1)\n",
            "Requirement already satisfied: typing-extensions<4.2.0,>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (4.1.1)\n",
            "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (1.0.9)\n",
            "Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (2.23.0)\n",
            "Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (2.4.5)\n",
            "Requirement already satisfied: pathy>=0.3.5 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (0.8.1)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (21.3)\n",
            "Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.11.0,>=1.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (1.10.2)\n",
            "Requirement already satisfied: thinc<8.2.0,>=8.1.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (8.1.5)\n",
            "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (1.0.3)\n",
            "Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (3.3.0)\n",
            "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (3.0.8)\n",
            "Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (2.0.8)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (2.11.3)\n",
            "Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (4.64.1)\n",
            "Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (1.21.6)\n",
            "Requirement already satisfied: typer<0.8.0,>=0.3.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (0.7.0)\n",
            "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.10 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (3.0.10)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (65.6.0)\n",
            "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (2.0.7)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from catalogue<2.1.0,>=2.0.6->spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (3.10.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (3.0.9)\n",
            "Requirement already satisfied: smart-open<6.0.0,>=5.2.1 in /usr/local/lib/python3.7/dist-packages (from pathy>=0.3.5->spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (5.2.1)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (2.10)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (3.0.4)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (2022.9.24)\n",
            "Requirement already satisfied: blis<0.8.0,>=0.7.8 in /usr/local/lib/python3.7/dist-packages (from thinc<8.2.0,>=8.1.0->spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (0.7.9)\n",
            "Requirement already satisfied: confection<1.0.0,>=0.0.1 in /usr/local/lib/python3.7/dist-packages (from thinc<8.2.0,>=8.1.0->spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (0.0.3)\n",
            "Requirement already satisfied: click<9.0.0,>=7.1.1 in /usr/local/lib/python3.7/dist-packages (from typer<0.8.0,>=0.3.0->spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (7.1.2)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->spacy<3.5.0,>=3.4.0->zh-core-web-sm==3.4.0) (2.0.1)\n",
            "Installing collected packages: spacy-pkuseg, zh-core-web-sm\n",
            "Successfully installed spacy-pkuseg-0.0.32 zh-core-web-sm-3.4.0\n",
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
            "\u001b[0m\u001b[38;5;2m✔ Download and installation successful\u001b[0m\n",
            "You can now load the package via spacy.load('zh_core_web_sm')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "!git clone -l -s https://github.com/L706077/jieba-zh_TW.git jieba_tw\n",
        "%cd jieba_tw\n",
        "import jieba\n",
        "%cd ../"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uYrl_2TfWzPt",
        "outputId": "268cdf0b-b3b9-43dd-aa63-8d4d214d7284"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cloning into 'jieba_tw'...\n",
            "warning: --local is ignored\n",
            "remote: Enumerating objects: 2320, done.\u001b[K\n",
            "remote: Total 2320 (delta 0), reused 0 (delta 0), pack-reused 2320\u001b[K\n",
            "Receiving objects: 100% (2320/2320), 36.07 MiB | 20.89 MiB/s, done.\n",
            "Resolving deltas: 100% (1498/1498), done.\n",
            "/content/jieba_tw\n",
            "/content\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "import spacy\n",
        "from spacy.tokens import Doc\n",
        "\n",
        "nlp = spacy.load(\"zh_core_web_sm\")\n",
        "\n",
        "class TwTokenizer:\n",
        "    def __init__(self, vocab):\n",
        "        self.vocab = vocab\n",
        "\n",
        "    def __call__(self, text):\n",
        "        words =  jieba.lcut(text)\n",
        "        spaces = [False] * len(words)        \n",
        "        return Doc(self.vocab, words=words, spaces=spaces)\n",
        "\n",
        "nlp.tokenizer = TwTokenizer(nlp.vocab)"
      ],
      "metadata": {
        "id": "uHQbULYmBIYd"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install icecream"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WLXw0jfRLpLg",
        "outputId": "bd57b320-a311-4dc9-c1b7-1afaf589b731"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting icecream\n",
            "  Downloading icecream-2.1.3-py2.py3-none-any.whl (8.4 kB)\n",
            "Collecting executing>=0.3.1\n",
            "  Downloading executing-1.2.0-py2.py3-none-any.whl (24 kB)\n",
            "Collecting asttokens>=2.0.1\n",
            "  Downloading asttokens-2.1.0-py2.py3-none-any.whl (26 kB)\n",
            "Requirement already satisfied: pygments>=2.2.0 in /usr/local/lib/python3.7/dist-packages (from icecream) (2.6.1)\n",
            "Collecting colorama>=0.3.9\n",
            "  Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from asttokens>=2.0.1->icecream) (1.15.0)\n",
            "Installing collected packages: executing, colorama, asttokens, icecream\n",
            "Successfully installed asttokens-2.1.0 colorama-0.4.6 executing-1.2.0 icecream-2.1.3\n",
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
            "\u001b[0m"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "from icecream import ic\n",
        "\n",
        "def show_tokens(text, seperator=\"|\"):\n",
        "    tokens = [token.text for token in nlp(text)]\n",
        "    tokenized_result =  f\"{seperator}\".join(tokens)\n",
        "    ic(tokenized_result)"
      ],
      "metadata": {
        "id": "DSsR0ys7CF4_"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "TEXT = \"\"\"\n",
        "國立金門大學今天舉行建校26周年校慶典禮，校慶主題呼應26諧音「二樂」，設定為「樂觀樂群快樂前行」，包括總統與副總統都特頒賀電祝福，金門縣長楊鎮浯則是表示，金門大學從分校、學院到改制大學，一路走來，已經成為金門重要智庫，以及人才培育的搖籃，誠摯祝福金大的發展越來越好，前景越來越璀璨。\n",
        "\"\"\"\n",
        "show_tokens(TEXT)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QXr9YEkyIOeU",
        "outputId": "3272d60f-e26c-4ef8-d27f-f8a936e37616"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Building prefix dict from the default dictionary ...\n",
            "DEBUG:jieba:Building prefix dict from the default dictionary ...\n",
            "Dumping model to file cache /tmp/jieba.cache\n",
            "DEBUG:jieba:Dumping model to file cache /tmp/jieba.cache\n",
            "Loading model cost 0.929 seconds.\n",
            "DEBUG:jieba:Loading model cost 0.929 seconds.\n",
            "Prefix dict has been built succesfully.\n",
            "DEBUG:jieba:Prefix dict has been built succesfully.\n",
            "ic| tokenized_result: ('\n",
            "                      '\n",
            "                       '|國立|金門|大學|今天|舉行|建校|26|周年|校慶|典禮|，|校慶|主題|呼應|26|諧音|「|二樂|」|，|設定為|「|樂觀|樂群|快樂|前行|」|，|包括|總統|與|副總統|都|特頒|賀電|祝福|，|金門|縣長|楊鎮浯|則|是|表示|，|金門|大學|從|分校|、|學院|到|改制|大學|，|一路|走來|，|已經|成為|金門|重要|智庫|，|以及|人才|培育|的|搖籃|，|誠摯|祝福|金大|的|發展|越來越|好|，|前景|越來越|璀璨|。|\n",
            "                      ')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "!pip -q install pytextrank"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "cellView": "form",
        "id": "3hwBYWgeHcpJ",
        "outputId": "b48d1e95-ed0d-407d-d76c-b9e0e6c0e4fa"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m47.0/47.0 kB\u001b[0m \u001b[31m4.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.1/1.1 MB\u001b[0m \u001b[31m42.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m11.2/11.2 MB\u001b[0m \u001b[31m73.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m965.4/965.4 kB\u001b[0m \u001b[31m49.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "ipython 7.9.0 requires jedi>=0.10, which is not installed.\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
            "\u001b[0m"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import pytextrank\n",
        "nlp.add_pipe(\"textrank\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JqG90-7AIAqZ",
        "outputId": "628464df-1775-40c6-db97-53fee41d087e"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<pytextrank.base.BaseTextRankFactory at 0x7f278662a490>"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "nlp.analyze_pipes(pretty=True)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wn7e2V48KV7d",
        "outputId": "b9e1cd9e-93da-4ff2-8d9b-73d17c189608"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[1m\n",
            "============================= Pipeline Overview =============================\u001b[0m\n",
            "\n",
            "#   Component         Assigns               Requires   Scores             Retokenizes\n",
            "-   ---------------   -------------------   --------   ----------------   -----------\n",
            "0   tok2vec           doc.tensor                                          False      \n",
            "                                                                                     \n",
            "1   tagger            token.tag                        tag_acc            False      \n",
            "                                                                                     \n",
            "2   parser            token.dep                        dep_uas            False      \n",
            "                      token.head                       dep_las                       \n",
            "                      token.is_sent_start              dep_las_per_type              \n",
            "                      doc.sents                        sents_p                       \n",
            "                                                       sents_r                       \n",
            "                                                       sents_f                       \n",
            "                                                                                     \n",
            "3   attribute_ruler                                                       False      \n",
            "                                                                                     \n",
            "4   ner               doc.ents                         ents_f             False      \n",
            "                      token.ent_iob                    ents_p                        \n",
            "                      token.ent_type                   ents_r                        \n",
            "                                                       ents_per_type                 \n",
            "                                                                                     \n",
            "5   textrank                                                              False      \n",
            "\n",
            "\u001b[38;5;2m✔ No problems found.\u001b[0m\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'summary': {'tok2vec': {'assigns': ['doc.tensor'],\n",
              "   'requires': [],\n",
              "   'scores': [],\n",
              "   'retokenizes': False},\n",
              "  'tagger': {'assigns': ['token.tag'],\n",
              "   'requires': [],\n",
              "   'scores': ['tag_acc'],\n",
              "   'retokenizes': False},\n",
              "  'parser': {'assigns': ['token.dep',\n",
              "    'token.head',\n",
              "    'token.is_sent_start',\n",
              "    'doc.sents'],\n",
              "   'requires': [],\n",
              "   'scores': ['dep_uas',\n",
              "    'dep_las',\n",
              "    'dep_las_per_type',\n",
              "    'sents_p',\n",
              "    'sents_r',\n",
              "    'sents_f'],\n",
              "   'retokenizes': False},\n",
              "  'attribute_ruler': {'assigns': [],\n",
              "   'requires': [],\n",
              "   'scores': [],\n",
              "   'retokenizes': False},\n",
              "  'ner': {'assigns': ['doc.ents', 'token.ent_iob', 'token.ent_type'],\n",
              "   'requires': [],\n",
              "   'scores': ['ents_f', 'ents_p', 'ents_r', 'ents_per_type'],\n",
              "   'retokenizes': False},\n",
              "  'textrank': {'assigns': [],\n",
              "   'requires': [],\n",
              "   'scores': [],\n",
              "   'retokenizes': False}},\n",
              " 'problems': {'tok2vec': [],\n",
              "  'tagger': [],\n",
              "  'parser': [],\n",
              "  'attribute_ruler': [],\n",
              "  'ner': [],\n",
              "  'textrank': []},\n",
              " 'attrs': {'token.tag': {'assigns': ['tagger'], 'requires': []},\n",
              "  'token.head': {'assigns': ['parser'], 'requires': []},\n",
              "  'doc.sents': {'assigns': ['parser'], 'requires': []},\n",
              "  'doc.ents': {'assigns': ['ner'], 'requires': []},\n",
              "  'token.dep': {'assigns': ['parser'], 'requires': []},\n",
              "  'token.ent_iob': {'assigns': ['ner'], 'requires': []},\n",
              "  'token.ent_type': {'assigns': ['ner'], 'requires': []},\n",
              "  'doc.tensor': {'assigns': ['tok2vec'], 'requires': []},\n",
              "  'token.is_sent_start': {'assigns': ['parser'], 'requires': []}}}"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 大量文本分析與應用"
      ],
      "metadata": {
        "id": "Rychxr1cIqYP"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- <font size=\"8\">準備文本</font>\n",
        "- <font size=\"8\">文本前處理 </font>\n",
        "- <font size=\"8\">文本類別統計</font>\n",
        "- <font size=\"8\">topK高頻詞</font>\n",
        "    - <font size=\"8\">依關鍵詞過濾</font>\n",
        "    - <font size=\"8\">依詞類過濾</font>\n",
        "    - <font size=\"8\">依命名實體類別過濾</font>"
      ],
      "metadata": {
        "id": "3bbZJ0tth5La"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 準備文本"
      ],
      "metadata": {
        "id": "DmUGvNtpRTta"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "import requests\n",
        "import pandas as pd\n",
        "pd.options.plotting.backend = \"plotly\"\n",
        "\n",
        "def fetch_data(keyword, limit='100'):\n",
        "    payload = {'limit': limit, \n",
        "               'offset': '0',\n",
        "               'keyword': keyword}\n",
        "    url = f\"https://api10.daoyidh.com/yhl/zh-tw/search/keyword/1.0\"\n",
        "    resp = requests.get(url, params=payload)\n",
        "    data = resp.json().get(\"data\")\n",
        "    df = pd.DataFrame(data)\n",
        "    return df"
      ],
      "metadata": {
        "cellView": "form",
        "id": "K-cYaYr7Uh2Y"
      },
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "KEYWORD = \"金門大學\"\n",
        "df = fetch_data(KEYWORD)\n",
        "df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 988
        },
        "id": "DJ0Gy1GiZtM5",
        "outputId": "6abb4179-bab8-4822-9ffd-1e1316cbb883"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "          date     artId author                                    label  \\\n",
              "0   2022-06-11  ART34335    陳長慶  珍惜活在當下的每一個時光 ──《浯鄉縮影：陳長慶散文集2014~2021》自序   \n",
              "1   2022-05-18  ART34425    魏千宸                          首次訪金，首次穿梭特色金大校園   \n",
              "2   2022-05-18  ART34424    洪春柳              認識自我 有取有捨 ──訪金中輔導老師李瓊芳談生活小道   \n",
              "3   2022-05-06  ART34478    陳為學                                  金嶝親橋一線牽   \n",
              "4   2022-03-29  ART34641    黃克全                  知人論世和以意逆志——談楊樹清、陳慶瀚與吳鈞堯   \n",
              "..         ...       ...    ...                                      ...   \n",
              "95  2015-05-21  ART45069    陳麗玉                                   話說后宅洋樓   \n",
              "96  2015-05-09  ART45110    蔡宜庭                               DOBGFamily   \n",
              "97  2015-05-08  ART45112    李金振                            眾志成城共創金大奇蹟（下）   \n",
              "98  2015-05-08  ART45113    莊舒惠                         建立和諧的人際關係──非暴力溝通   \n",
              "99  2015-05-07  ART45116    李金振                            眾志成城共創金大奇蹟（上）   \n",
              "\n",
              "   dataset                                            content  \n",
              "0     副刊文學  《浯鄉縮影》是我從2014至2021年，陸續在報刊發表的散文作品。即使只有少數的二十三篇，然...  \n",
              "1     副刊文學  首次乘著飛機盤桓在這座古稱浯洲的上空，正當飛機下了雲層，地表上漸漸清晰之際，首先映入眼簾的是...  \n",
              "2     副刊文學  輔導的精義：在一段時間裡，陪伴一個人，盡力地幫一個人認識自己。輔導學生認識自我，肯定自我。輔...  \n",
              "3     副刊文學  金門大學前院長陳益源教授於2018年4月29日，在金門日報的「浯江夜話」發表他借調金門後的第...  \n",
              "4     副刊文學  楊樹清竟是一個不折不扣的阜內思。我閱讀波赫士這篇小說時有兩張臉重疊著，意思是以下的敘述主體可...  \n",
              "..     ...                                                ...  \n",
              "95    副刊文學  洋樓的滄桑與興修_x000D_\\n  「王金城洋樓」於民國21年興建完工，當時因王金鎖等兄弟...  \n",
              "96    副刊文學  對我們身邊很多人來說，金門離島亂遠的，到底是國內還是國外傻傻分不清楚，十分不解大部分的我們怎...  \n",
              "97    副刊文學  五、從落榜生到榜首_x000D_\\n辦學的最終目的是得天下英才而教育之。本校創設之單純目的，...  \n",
              "98    副刊文學  在我們的日常生活中難免會與他人發生爭執，歸究其原因，也許是我們習慣以先入為主的觀念做比較與批...  \n",
              "99    副刊文學  1997年，教育部長吳京院士徵詢交待我一個不平凡的使命，為金門籌設第一所高等學府。這件突如其...  \n",
              "\n",
              "[100 rows x 6 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-f3735369-3847-4a7f-8d4b-ecc4535cadc6\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>date</th>\n",
              "      <th>artId</th>\n",
              "      <th>author</th>\n",
              "      <th>label</th>\n",
              "      <th>dataset</th>\n",
              "      <th>content</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2022-06-11</td>\n",
              "      <td>ART34335</td>\n",
              "      <td>陳長慶</td>\n",
              "      <td>珍惜活在當下的每一個時光 ──《浯鄉縮影：陳長慶散文集2014~2021》自序</td>\n",
              "      <td>副刊文學</td>\n",
              "      <td>《浯鄉縮影》是我從2014至2021年，陸續在報刊發表的散文作品。即使只有少數的二十三篇，然...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2022-05-18</td>\n",
              "      <td>ART34425</td>\n",
              "      <td>魏千宸</td>\n",
              "      <td>首次訪金，首次穿梭特色金大校園</td>\n",
              "      <td>副刊文學</td>\n",
              "      <td>首次乘著飛機盤桓在這座古稱浯洲的上空，正當飛機下了雲層，地表上漸漸清晰之際，首先映入眼簾的是...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2022-05-18</td>\n",
              "      <td>ART34424</td>\n",
              "      <td>洪春柳</td>\n",
              "      <td>認識自我 有取有捨 ──訪金中輔導老師李瓊芳談生活小道</td>\n",
              "      <td>副刊文學</td>\n",
              "      <td>輔導的精義：在一段時間裡，陪伴一個人，盡力地幫一個人認識自己。輔導學生認識自我，肯定自我。輔...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2022-05-06</td>\n",
              "      <td>ART34478</td>\n",
              "      <td>陳為學</td>\n",
              "      <td>金嶝親橋一線牽</td>\n",
              "      <td>副刊文學</td>\n",
              "      <td>金門大學前院長陳益源教授於2018年4月29日，在金門日報的「浯江夜話」發表他借調金門後的第...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2022-03-29</td>\n",
              "      <td>ART34641</td>\n",
              "      <td>黃克全</td>\n",
              "      <td>知人論世和以意逆志——談楊樹清、陳慶瀚與吳鈞堯</td>\n",
              "      <td>副刊文學</td>\n",
              "      <td>楊樹清竟是一個不折不扣的阜內思。我閱讀波赫士這篇小說時有兩張臉重疊著，意思是以下的敘述主體可...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>95</th>\n",
              "      <td>2015-05-21</td>\n",
              "      <td>ART45069</td>\n",
              "      <td>陳麗玉</td>\n",
              "      <td>話說后宅洋樓</td>\n",
              "      <td>副刊文學</td>\n",
              "      <td>洋樓的滄桑與興修_x000D_\\n  「王金城洋樓」於民國21年興建完工，當時因王金鎖等兄弟...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>96</th>\n",
              "      <td>2015-05-09</td>\n",
              "      <td>ART45110</td>\n",
              "      <td>蔡宜庭</td>\n",
              "      <td>DOBGFamily</td>\n",
              "      <td>副刊文學</td>\n",
              "      <td>對我們身邊很多人來說，金門離島亂遠的，到底是國內還是國外傻傻分不清楚，十分不解大部分的我們怎...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>97</th>\n",
              "      <td>2015-05-08</td>\n",
              "      <td>ART45112</td>\n",
              "      <td>李金振</td>\n",
              "      <td>眾志成城共創金大奇蹟（下）</td>\n",
              "      <td>副刊文學</td>\n",
              "      <td>五、從落榜生到榜首_x000D_\\n辦學的最終目的是得天下英才而教育之。本校創設之單純目的，...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>98</th>\n",
              "      <td>2015-05-08</td>\n",
              "      <td>ART45113</td>\n",
              "      <td>莊舒惠</td>\n",
              "      <td>建立和諧的人際關係──非暴力溝通</td>\n",
              "      <td>副刊文學</td>\n",
              "      <td>在我們的日常生活中難免會與他人發生爭執，歸究其原因，也許是我們習慣以先入為主的觀念做比較與批...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>99</th>\n",
              "      <td>2015-05-07</td>\n",
              "      <td>ART45116</td>\n",
              "      <td>李金振</td>\n",
              "      <td>眾志成城共創金大奇蹟（上）</td>\n",
              "      <td>副刊文學</td>\n",
              "      <td>1997年，教育部長吳京院士徵詢交待我一個不平凡的使命，為金門籌設第一所高等學府。這件突如其...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>100 rows × 6 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f3735369-3847-4a7f-8d4b-ecc4535cadc6')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-f3735369-3847-4a7f-8d4b-ecc4535cadc6 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-f3735369-3847-4a7f-8d4b-ecc4535cadc6');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "sample_text = df.at[1, 'content']\n",
        "sample_text"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 408
        },
        "id": "T4WxdYSzB9rH",
        "outputId": "7c50c2b4-800e-44dc-d368-1ddf7032d4e7"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'首次乘著飛機盤桓在這座古稱浯洲的上空，正當飛機下了雲層，地表上漸漸清晰之際，首先映入眼簾的是行駛在海上的一艘艘貨船，宛若風中小筏，乘風而前欲往之處。飛機隨一秒秒的時間靠近金門島上，一幢幢紅屋瓦舍屋頂漸現，空中彷彿流竄著一絲絲閩風，空中散發出與台灣不同的氣氛，像是穿越時光與空間，穿越到當年陶淵明所述的桃花源，心中滿滿是說不出的感動及雀躍，正因飛機即將降落的是我未來即將要生活多年的地方─金門。\\n\\xa0\\xa0剛出尚義機場，薰風迎面而來，或許是風獅爺的庇佑，矗立在風頭的島鄉竟風不比號稱風城的故土新竹，浯洲的風是柔情的風、是舒適的風。在金門夏日的蒼穹湛藍無雲，透徹的天際映著蒼茫的海水，大武山特立天際，一排排蒼翠的大樹，短綴著金門獨特的紅土。讓酷熱的夏日，有可以在大樹下休憩的機會。或許是拜當年軍管之賜，雖令浯島居民曾經歷生活不自由的日子，卻意外地將金門最具價值的文化資產，原汁原味的保存下來。\\n\\xa0\\xa0金門的王宅十八間、得月樓等建築聞名全台，位於金寧盤山的金門大學卻意外的令我驚豔，其中幢幢閩式風情的校舍，不只是照片上風格在地化特別的風味，當場觀看，龐大的建築量體，紅磚外牆，黑瓦燕脊，鑲著閩式特有的建築裝飾，富文化特色的校園建築，動人的令我無法自已，有著閩式建築小巧精緻的裝飾特色，又有建築大型量體的氣派，綜合教學大樓正立面上兩旁八角式的處理，增強正立面的氣勢。圖資大樓旁的高台讓人聯想當初聚落為海防而建的得月樓，一棟棟風格相似的校舍宛若一個小型聚落，讓我不禁讚嘆當年設計校舍的建築師是如何發揮此種巧思，也許是金門的建築特色太讓人目眩神迷了吧!\\n\\xa0\\xa0在後現代全球國際主義風潮下的建築，大多都走向幾何建構的極簡風，大多建築乍看之下都有屬於各自的建築設計巧思，卻鮮有應屬於當地特有的建築特色，金門大學所採當地閩風建築為風格建設校舍，我個人認為是台灣各大學中最具有特色的其中之一，雖沒有台大等老校有校內古蹟加持，卻在一片追求新穎、炫麗的現代建築中走出屬於自己的一條路。\\n\\xa0\\xa0許多現代建築師想要創造屬於東方風格建築的風氣，卻常敗筆在大量體建築物與東方傳統小巧結構風格衝突，常造成太過具象化的建築詭譎，像是只是扣著琉璃大盤帽的西方建築，我覺得有時與其將北方傳統大型建築特色硬融入當今建築，不妨參考金大校舍，將閩南地區特有的紅磚、板瓦融入建物中，配合著古代的窗櫺，在大型立方體中貫穿著由數百年、甚至千年先祖所留下的建築藝術融入進現代民居中。我相信在這樣的建築中會更有故事、會更能展現屬於我們自己的文化，讓我們後代的子孫們，不管身在何處，都能認知到屬於我們自己先祖的一脈血緣。'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### ✍ 練習"
      ],
      "metadata": {
        "id": "sEp08cGLxeWe"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 在KEYWORD的引號內輸入一段文字\n",
        "- 刪除頭尾的 \"\"\" 之後執行"
      ],
      "metadata": {
        "id": "3EkkcnV6x6mM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "KEYWORD = \"\"\n",
        "df = fetch_data(KEYWORD)\n",
        "df\n",
        "\"\"\""
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "doDfwB4WxjbZ",
        "outputId": "ba1e2513-2bc9-4423-b9a0-a78d35d07078"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'\\nKEYWORD = \"\"\\ndf = fetch_data(KEYWORD)\\ndf\\n'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 上面執行成功後，刪除下面頭尾的 \"\"\" 之後執行"
      ],
      "metadata": {
        "id": "OMlBQ150yUBb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "sample_text = df.at[1, 'content']\n",
        "sample_text\n",
        "\"\"\""
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "x0KfLXkuyK9I",
        "outputId": "d62c4cbf-7250-466b-996d-e4580f9ed9eb"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"\\nsample_text = df.at[1, 'content']\\nsample_text\\n\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 文本前處理"
      ],
      "metadata": {
        "id": "HhNCEL43p-F1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "def preprocess_df(df):\n",
        "    columns = ['date', 'author']\n",
        "    new_df = df.copy()[columns]\n",
        "    new_df['text'] = df['label'] + ' ' + df['content']\n",
        "    return new_df"
      ],
      "metadata": {
        "id": "a4vbBo22Jfuz",
        "cellView": "form"
      },
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "new_df = preprocess_df(df)\n",
        "new_df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 641
        },
        "id": "qyjwQzIGKxEj",
        "outputId": "99bef3e5-c94e-4573-89e1-2868072b7129"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "          date author                                               text\n",
              "0   2022-06-11    陳長慶  珍惜活在當下的每一個時光 ──《浯鄉縮影：陳長慶散文集2014~2021》自序 《浯鄉縮影》...\n",
              "1   2022-05-18    魏千宸  首次訪金，首次穿梭特色金大校園 首次乘著飛機盤桓在這座古稱浯洲的上空，正當飛機下了雲層，地表...\n",
              "2   2022-05-18    洪春柳  認識自我 有取有捨 ──訪金中輔導老師李瓊芳談生活小道 輔導的精義：在一段時間裡，陪伴一個人...\n",
              "3   2022-05-06    陳為學  金嶝親橋一線牽 金門大學前院長陳益源教授於2018年4月29日，在金門日報的「浯江夜話」發表...\n",
              "4   2022-03-29    黃克全  知人論世和以意逆志——談楊樹清、陳慶瀚與吳鈞堯 楊樹清竟是一個不折不扣的阜內思。我閱讀波赫士...\n",
              "..         ...    ...                                                ...\n",
              "95  2015-05-21    陳麗玉  話說后宅洋樓 洋樓的滄桑與興修_x000D_\\n  「王金城洋樓」於民國21年興建完工，當時...\n",
              "96  2015-05-09    蔡宜庭  DOBGFamily 對我們身邊很多人來說，金門離島亂遠的，到底是國內還是國外傻傻分不清楚，...\n",
              "97  2015-05-08    李金振  眾志成城共創金大奇蹟（下） 五、從落榜生到榜首_x000D_\\n辦學的最終目的是得天下英才而...\n",
              "98  2015-05-08    莊舒惠  建立和諧的人際關係──非暴力溝通 在我們的日常生活中難免會與他人發生爭執，歸究其原因，也許是...\n",
              "99  2015-05-07    李金振  眾志成城共創金大奇蹟（上） 1997年，教育部長吳京院士徵詢交待我一個不平凡的使命，為金門籌...\n",
              "\n",
              "[100 rows x 3 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-cf04622f-f79d-4543-aec7-5e27bb4a47b6\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>date</th>\n",
              "      <th>author</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2022-06-11</td>\n",
              "      <td>陳長慶</td>\n",
              "      <td>珍惜活在當下的每一個時光 ──《浯鄉縮影：陳長慶散文集2014~2021》自序 《浯鄉縮影》...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2022-05-18</td>\n",
              "      <td>魏千宸</td>\n",
              "      <td>首次訪金，首次穿梭特色金大校園 首次乘著飛機盤桓在這座古稱浯洲的上空，正當飛機下了雲層，地表...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2022-05-18</td>\n",
              "      <td>洪春柳</td>\n",
              "      <td>認識自我 有取有捨 ──訪金中輔導老師李瓊芳談生活小道 輔導的精義：在一段時間裡，陪伴一個人...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2022-05-06</td>\n",
              "      <td>陳為學</td>\n",
              "      <td>金嶝親橋一線牽 金門大學前院長陳益源教授於2018年4月29日，在金門日報的「浯江夜話」發表...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2022-03-29</td>\n",
              "      <td>黃克全</td>\n",
              "      <td>知人論世和以意逆志——談楊樹清、陳慶瀚與吳鈞堯 楊樹清竟是一個不折不扣的阜內思。我閱讀波赫士...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>95</th>\n",
              "      <td>2015-05-21</td>\n",
              "      <td>陳麗玉</td>\n",
              "      <td>話說后宅洋樓 洋樓的滄桑與興修_x000D_\\n  「王金城洋樓」於民國21年興建完工，當時...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>96</th>\n",
              "      <td>2015-05-09</td>\n",
              "      <td>蔡宜庭</td>\n",
              "      <td>DOBGFamily 對我們身邊很多人來說，金門離島亂遠的，到底是國內還是國外傻傻分不清楚，...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>97</th>\n",
              "      <td>2015-05-08</td>\n",
              "      <td>李金振</td>\n",
              "      <td>眾志成城共創金大奇蹟（下） 五、從落榜生到榜首_x000D_\\n辦學的最終目的是得天下英才而...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>98</th>\n",
              "      <td>2015-05-08</td>\n",
              "      <td>莊舒惠</td>\n",
              "      <td>建立和諧的人際關係──非暴力溝通 在我們的日常生活中難免會與他人發生爭執，歸究其原因，也許是...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>99</th>\n",
              "      <td>2015-05-07</td>\n",
              "      <td>李金振</td>\n",
              "      <td>眾志成城共創金大奇蹟（上） 1997年，教育部長吳京院士徵詢交待我一個不平凡的使命，為金門籌...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>100 rows × 3 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-cf04622f-f79d-4543-aec7-5e27bb4a47b6')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-cf04622f-f79d-4543-aec7-5e27bb4a47b6 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-cf04622f-f79d-4543-aec7-5e27bb4a47b6');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "\n",
        "def preprocess_texts(texts):\n",
        "    PUNCT_SYM = [\"PUNCT\", \"SYM\"]\n",
        "\n",
        "    clean_texts = []    \n",
        "    # nlp.pipe() is more efficient than nlp()\n",
        "    for doc in nlp.pipe(texts, disable=[\"tok2vec\", \"tagger\", \"parser\", \"ner\", \"textrank\"]):\n",
        "        tokens = [tok for tok in doc if tok.pos_ not in PUNCT_SYM]\n",
        "        tokens = [tok.text for tok in tokens if (\n",
        "                    not tok.is_stop \n",
        "                    and not tok.is_punct\n",
        "                    and not tok.is_currency\n",
        "                    and not tok.is_space\n",
        "                    and not tok.like_num\n",
        "                    and not tok.like_url\n",
        "                    and not tok.like_email\n",
        "                )\n",
        "        ]\n",
        "        clean_text = \" \".join(tokens)\n",
        "        clean_texts.append(clean_text)\n",
        "    return clean_texts"
      ],
      "metadata": {
        "id": "wvAi49UXImeA",
        "cellView": "form"
      },
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "clean_text = preprocess_texts([sample_text])\n",
        "clean_text"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4eJq0tBTJQWV",
        "outputId": "0a2eb1c3-30d7-40c6-e3a9-86306117bec5"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['首 次 乘著 飛機 盤桓 這 座 古稱 浯洲 上空 正當 飛機 雲層 地表 漸漸 清晰 之際 映入 眼簾 行駛 海上 艘 艘 貨船 宛若 風中 小筏 乘風 前 欲 往之處 飛機 隨 一秒 秒 時間 靠近 金門島 幢幢 紅屋 瓦舍 屋頂 漸現 空中 彷彿 流竄 著 絲絲 閩 風 空中 散發出 與 台灣 氣氛 穿越 時光 與 空間 穿越 當年 陶淵明 所述 桃花源 心中 滿滿 說 不出 感動 雀躍 正 飛機 即將 降落 未來 即將 生活 多年 地方 ─ 金門 剛出 尚義 機場 薰風 迎面 來 或許 風獅爺 庇佑 矗立 風頭 島鄉 竟風 號稱 風城 故土 新竹 浯洲 風是 柔情 風 舒適 風 金門 夏日 蒼穹 湛藍 無 雲 透徹 天際 映著 蒼茫 海水 大武山 特立 天際 排排 蒼翠 大樹 短綴 著 金門 獨特 紅土 讓 酷熱 夏日 大樹 休憩 機會 或許 是拜 當年 軍管 賜 雖 令 浯島 居民 經歷 生活 自由 日子 卻 意外 將 金門 具 價值 文化 資產 原汁 原味 保存 下來 金門 王宅 十八間 月樓 建築 聞名 全台 位於 金寧 盤山 金門 大學 卻 意外 令 驚豔 中 幢幢 閩式 風情 校舍 照片 風格 地化 特別 風味 當場 觀看 龐大 建築 量體 紅磚 外牆 黑瓦 燕脊 鑲 著 閩式 特有 建築 裝飾 富 文化 特色 校園 建築 動人 令 無法自已 著 閩式 建築 小巧 精緻 裝飾 特色 建築 大型 量體 氣派 綜合 教學 大樓 正 立面 兩 旁 角式 處理 增強 正 立面 氣勢 圖資 大樓 旁 高台 讓 聯想 當初 聚落 為 海防 而建 月樓 棟 棟 風格 校舍 宛若 個 小型 聚落 讓 不禁 讚嘆 當年 設計 校舍 建築師 發揮 種 巧思 也許 金門 建築 特色 太 讓 目眩神迷 後現代 全球 國際 主義 風潮 建築 走向 幾何 建構 極簡 風 建築 乍看 之下 屬於 建築 設計 巧思 卻 鮮有 應 屬於 當地 特有 建築 特色 金門 大學 採 當地 閩 風 建築 為 風格 建設 校舍 個 認為 台灣 大學 中 特色 中 雖 沒有 台大 老校 校內 古蹟 加持 卻 追求 新穎 炫麗 現代 建築 中 走出 屬於 條路 許多 現代 建築師 想要 創造 屬於 東方 風格 建築 風氣 卻 敗筆 量體 建築物 與 東方 傳統 小巧 結構 風格 衝突 太過 具象化 建築 詭譎 扣著 琉璃 大盤帽 西方 建築 覺得 有時 與 將 北方 傳統 大型 建築 特色 硬 融入 當今 建築 參考 金大 校舍 將 閩南 地區 特有 紅磚 板瓦 融入 建物 中 配合 著 古代 窗櫺 大型 立方體 中 貫穿 著 數百 年 千年 先祖 留下 建築 藝術 融入 進 現代 民居 中 這樣 建築 中會 故事 會 展現 屬於 我們 文化 讓 我們 後代 子孫 們 身 何處 認知到 屬於 我們 先祖 一脈 血緣']"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "TEXTS = new_df['text']\n",
        "new_df['clean_text'] = preprocess_texts(TEXTS)\n",
        "new_df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 815
        },
        "id": "VB3Y9-UiLifM",
        "outputId": "c947062a-c6a4-4731-8f3b-85c2c64d34fb"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "          date author                                               text  \\\n",
              "0   2022-06-11    陳長慶  珍惜活在當下的每一個時光 ──《浯鄉縮影：陳長慶散文集2014~2021》自序 《浯鄉縮影》...   \n",
              "1   2022-05-18    魏千宸  首次訪金，首次穿梭特色金大校園 首次乘著飛機盤桓在這座古稱浯洲的上空，正當飛機下了雲層，地表...   \n",
              "2   2022-05-18    洪春柳  認識自我 有取有捨 ──訪金中輔導老師李瓊芳談生活小道 輔導的精義：在一段時間裡，陪伴一個人...   \n",
              "3   2022-05-06    陳為學  金嶝親橋一線牽 金門大學前院長陳益源教授於2018年4月29日，在金門日報的「浯江夜話」發表...   \n",
              "4   2022-03-29    黃克全  知人論世和以意逆志——談楊樹清、陳慶瀚與吳鈞堯 楊樹清竟是一個不折不扣的阜內思。我閱讀波赫士...   \n",
              "..         ...    ...                                                ...   \n",
              "95  2015-05-21    陳麗玉  話說后宅洋樓 洋樓的滄桑與興修_x000D_\\n  「王金城洋樓」於民國21年興建完工，當時...   \n",
              "96  2015-05-09    蔡宜庭  DOBGFamily 對我們身邊很多人來說，金門離島亂遠的，到底是國內還是國外傻傻分不清楚，...   \n",
              "97  2015-05-08    李金振  眾志成城共創金大奇蹟（下） 五、從落榜生到榜首_x000D_\\n辦學的最終目的是得天下英才而...   \n",
              "98  2015-05-08    莊舒惠  建立和諧的人際關係──非暴力溝通 在我們的日常生活中難免會與他人發生爭執，歸究其原因，也許是...   \n",
              "99  2015-05-07    李金振  眾志成城共創金大奇蹟（上） 1997年，教育部長吳京院士徵詢交待我一個不平凡的使命，為金門籌...   \n",
              "\n",
              "                                           clean_text  \n",
              "0   珍惜 活在 當下 個 時光 ─ ─ 浯鄉 縮影 陳 長慶 散文集 自序 浯鄉 縮影 從 年 ...  \n",
              "1   首 次 訪金 首 次 穿梭 特色 金大 校園 首 次 乘著 飛機 盤桓 這 座 古稱 浯洲 ...  \n",
              "2   認識 自我 有取 有捨 ─ ─ 訪金 中 輔導 老師 李 瓊芳 談 生活 小道 輔導 精義 ...  \n",
              "3   金 嶝 親 橋 一線 牽 金門 大學 前院長 陳益源 教授 於 年 月 日 金門 日報 浯江...  \n",
              "4   知人 論世 以意 逆志 談 楊 樹清 陳慶 瀚與 吳 鈞堯 楊 樹清 竟是 個 不折不扣 的...  \n",
              "..                                                ...  \n",
              "95  話 說 后宅 洋樓 洋樓 滄桑 與 興修 x000D 王 金城 洋樓 於 民國 年 興建 完...  \n",
              "96  DOBGFamily 對 我們 身邊 人來 說 金門 離島 亂遠 國內 還是 國外 傻傻 分...  \n",
              "97  眾志成城 共創 金大 奇蹟 從 落榜生 榜首 x000D 辦學 最終 目的 天下 英才 教育...  \n",
              "98  建立 和諧 人際 關係 ─ ─ 非暴力 溝通 我們 日常 生活 中 難免 會 與 發生 爭執...  \n",
              "99  眾志成城 共創 金大 奇蹟 年 教育部長 吳京 院士 徵詢 交待 個 平凡 使命 為 金門 ...  \n",
              "\n",
              "[100 rows x 4 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-51b683f6-8eb6-406e-9ce2-6263d7b1e65c\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>date</th>\n",
              "      <th>author</th>\n",
              "      <th>text</th>\n",
              "      <th>clean_text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2022-06-11</td>\n",
              "      <td>陳長慶</td>\n",
              "      <td>珍惜活在當下的每一個時光 ──《浯鄉縮影：陳長慶散文集2014~2021》自序 《浯鄉縮影》...</td>\n",
              "      <td>珍惜 活在 當下 個 時光 ─ ─ 浯鄉 縮影 陳 長慶 散文集 自序 浯鄉 縮影 從 年 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2022-05-18</td>\n",
              "      <td>魏千宸</td>\n",
              "      <td>首次訪金，首次穿梭特色金大校園 首次乘著飛機盤桓在這座古稱浯洲的上空，正當飛機下了雲層，地表...</td>\n",
              "      <td>首 次 訪金 首 次 穿梭 特色 金大 校園 首 次 乘著 飛機 盤桓 這 座 古稱 浯洲 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2022-05-18</td>\n",
              "      <td>洪春柳</td>\n",
              "      <td>認識自我 有取有捨 ──訪金中輔導老師李瓊芳談生活小道 輔導的精義：在一段時間裡，陪伴一個人...</td>\n",
              "      <td>認識 自我 有取 有捨 ─ ─ 訪金 中 輔導 老師 李 瓊芳 談 生活 小道 輔導 精義 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2022-05-06</td>\n",
              "      <td>陳為學</td>\n",
              "      <td>金嶝親橋一線牽 金門大學前院長陳益源教授於2018年4月29日，在金門日報的「浯江夜話」發表...</td>\n",
              "      <td>金 嶝 親 橋 一線 牽 金門 大學 前院長 陳益源 教授 於 年 月 日 金門 日報 浯江...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2022-03-29</td>\n",
              "      <td>黃克全</td>\n",
              "      <td>知人論世和以意逆志——談楊樹清、陳慶瀚與吳鈞堯 楊樹清竟是一個不折不扣的阜內思。我閱讀波赫士...</td>\n",
              "      <td>知人 論世 以意 逆志 談 楊 樹清 陳慶 瀚與 吳 鈞堯 楊 樹清 竟是 個 不折不扣 的...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>95</th>\n",
              "      <td>2015-05-21</td>\n",
              "      <td>陳麗玉</td>\n",
              "      <td>話說后宅洋樓 洋樓的滄桑與興修_x000D_\\n  「王金城洋樓」於民國21年興建完工，當時...</td>\n",
              "      <td>話 說 后宅 洋樓 洋樓 滄桑 與 興修 x000D 王 金城 洋樓 於 民國 年 興建 完...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>96</th>\n",
              "      <td>2015-05-09</td>\n",
              "      <td>蔡宜庭</td>\n",
              "      <td>DOBGFamily 對我們身邊很多人來說，金門離島亂遠的，到底是國內還是國外傻傻分不清楚，...</td>\n",
              "      <td>DOBGFamily 對 我們 身邊 人來 說 金門 離島 亂遠 國內 還是 國外 傻傻 分...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>97</th>\n",
              "      <td>2015-05-08</td>\n",
              "      <td>李金振</td>\n",
              "      <td>眾志成城共創金大奇蹟（下） 五、從落榜生到榜首_x000D_\\n辦學的最終目的是得天下英才而...</td>\n",
              "      <td>眾志成城 共創 金大 奇蹟 從 落榜生 榜首 x000D 辦學 最終 目的 天下 英才 教育...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>98</th>\n",
              "      <td>2015-05-08</td>\n",
              "      <td>莊舒惠</td>\n",
              "      <td>建立和諧的人際關係──非暴力溝通 在我們的日常生活中難免會與他人發生爭執，歸究其原因，也許是...</td>\n",
              "      <td>建立 和諧 人際 關係 ─ ─ 非暴力 溝通 我們 日常 生活 中 難免 會 與 發生 爭執...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>99</th>\n",
              "      <td>2015-05-07</td>\n",
              "      <td>李金振</td>\n",
              "      <td>眾志成城共創金大奇蹟（上） 1997年，教育部長吳京院士徵詢交待我一個不平凡的使命，為金門籌...</td>\n",
              "      <td>眾志成城 共創 金大 奇蹟 年 教育部長 吳京 院士 徵詢 交待 個 平凡 使命 為 金門 ...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>100 rows × 4 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-51b683f6-8eb6-406e-9ce2-6263d7b1e65c')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-51b683f6-8eb6-406e-9ce2-6263d7b1e65c button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-51b683f6-8eb6-406e-9ce2-6263d7b1e65c');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 練習 ✍"
      ],
      "metadata": {
        "id": "SKv3MLgJy6_G"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 刪除頭尾的 \"\"\" 之後執行"
      ],
      "metadata": {
        "id": "FKg39tl4yj01"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "new_df = preprocess_df(df)\n",
        "TEXTS = new_df['text']\n",
        "new_df['clean_text'] = preprocess_texts(TEXTS)\n",
        "new_df\n",
        "\"\"\""
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "2d5aU38Tym-z",
        "outputId": "0df91532-37ec-4316-cd5c-b36f8eecccda"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"\\nnew_df = preprocess_df(df)\\nTEXTS = new_df['text']\\nnew_df['clean_text'] = preprocess_texts(TEXTS)\\nnew_df\\n\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 文本類別統計"
      ],
      "metadata": {
        "id": "4RrL392YNKtd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "CATEGORY = 'author'\n",
        "cat_counts = new_df[CATEGORY].value_counts()\n",
        "cat_counts"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LM2SrsG_MIoD",
        "outputId": "0145d464-7958-42e1-fac8-8f0d04fde6c6"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "陳長慶    8\n",
              "陳慶元    5\n",
              "陳為學    4\n",
              "李金振    4\n",
              "吳如明    3\n",
              "      ..\n",
              "洪明燦    1\n",
              "魏千宸    1\n",
              "陳秀竹    1\n",
              "楊天厚    1\n",
              "莊舒惠    1\n",
              "Name: author, Length: 69, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "cat_counts.plot.bar()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 542
        },
        "id": "ZE3j_n7ANf5c",
        "outputId": "736e5cc9-55c2-4887-9ea2-76a701134957"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script src=\"https://cdn.plot.ly/plotly-2.8.3.min.js\"></script>                <div id=\"e178125e-42f3-4a7a-b61e-b0086cc018e4\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"e178125e-42f3-4a7a-b61e-b0086cc018e4\")) {                    Plotly.newPlot(                        \"e178125e-42f3-4a7a-b61e-b0086cc018e4\",                        [{\"alignmentgroup\":\"True\",\"hovertemplate\":\"variable=author<br>index=%{x}<br>value=%{y}<extra></extra>\",\"legendgroup\":\"author\",\"marker\":{\"color\":\"#636efa\",\"pattern\":{\"shape\":\"\"}},\"name\":\"author\",\"offsetgroup\":\"author\",\"orientation\":\"v\",\"showlegend\":true,\"textposition\":\"auto\",\"x\":[\"\\u9673\\u9577\\u6176\",\"\\u9673\\u6176\\u5143\",\"\\u9673\\u70ba\\u5b78\",\"\\u674e\\u91d1\\u632f\",\"\\u5433\\u5982\\u660e\",\"\\u8523\\u7acb\\u6069\",\"\\u5f90\\u66fc\\u742a\",\"\\u805e\\u5ba3\",\"\\u6771\\u745e\",\"\\u9ec3\\u5947\",\"\\u90ce\\u82f1\",\"\\u9ec3\\u79c0\\u6cbb\",\"\\u9ec3\\u514b\\u5168\",\"\\u912d\\u6101\\u4e88\",\"\\u6d2a\\u6625\\u67f3\",\"\\u502a\\u632f\\u91d1\",\"\\u9673\\u9298\\u78fb\",\"\\u5289\\u7b2c\\u7d05\",\"\\u9673\\u9e97\\u7389\",\"\\u8521\\u4e00\\u8c6a\",\"\\u5510\\u632f\\u745c\",\"\\u9673\\u570b\\u8208\",\"\\u694a\\u8085\\u4fca\",\"\\u90b1\\u82f1\\u7f8e\",\"\\u7fc1\\u7dad\\u7490\",\"\\u5bd2\\u7389\",\"\\u5c0f\\u5e73\",\"\\u6797\\u4f73\\u7469\",\"\\u90b1\\u6bd3\\u96ef\",\"\\u738b\\u4e16\\u52e4\",\"\\u859b\\u7d20\\u74ca\",\"\\u963f\\u7c73\\u5e2b\",\"\\u9673\\u662b\\u6f54\",\"\\u8521\\u5b9c\\u5ead\",\"\\u5409\\u58eb\\u535a\",\"\\u912d\\u5b8f\\u5ead\",\"\\u5b6b\\u5b87\\u5fb7\",\"\\u91d1\\u5b50\",\"\\u91ce\\u592b\",\"\\u674e\\u70b7\\u70fd\",\"\\u674e\\u798f\\u5357\",\"\\u9ec3\\u7b71\\u84c1\",\"\\u8ac7\\u5b9c\\u82b3\",\"\\u984f\\u654f\\u4e98\",\"\\u65b9\\u5b9a\\u582f\",\"\\u61ff\\u53c3\",\"\\u5f35\\u706b\\u6728\",\"\\u90c1\\u52db\",\"\\u5289\\u9999\\u862d\",\"\\u6e58\\u592b\\u4eba\",\"\\u4f55\\u570b\\u7965\",\"\\u859b\\u627f\\u6cf0\",\"\\u674e\\u5fd7\\u6f84\",\"\\u5433\\u5bb6\\u7bb4\",\"\\u8521\\u6dd1\\u8ce2\",\"\\u694a\\u5efa\\u8c50\",\"\\u9ec3\\u6069\\u4e9e\",\"\\u694a\\u6a39\\u6e05\",\"\\u7267\\u7f8a\\u5973\",\"\\u9673\\u66f8\\u6bc5\",\"\\u856d\\u6c38\\u7246\",\"\\u9673\\u6c38\\u5bcc\",\"\\u8521\\u6e05\\u5176\",\"\\u5433\\u7db1\\u7acb\",\"\\u6d2a\\u660e\\u71e6\",\"\\u9b4f\\u5343\\u5bb8\",\"\\u9673\\u79c0\\u7af9\",\"\\u694a\\u5929\\u539a\",\"\\u838a\\u8212\\u60e0\"],\"xaxis\":\"x\",\"y\":[8,5,4,4,3,3,2,2,2,2,2,2,2,2,2,2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1],\"yaxis\":\"y\",\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"geo\":{\"bgcolor\":\"white\",\"lakecolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"white\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"light\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"ternary\":{\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"index\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"value\"}},\"legend\":{\"title\":{\"text\":\"variable\"},\"tracegroupgap\":0},\"margin\":{\"t\":60},\"barmode\":\"relative\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('e178125e-42f3-4a7a-b61e-b0086cc018e4');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "cat_counts.plot.area()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 542
        },
        "id": "07FGLG5jN8_O",
        "outputId": "2434264d-366d-476a-c870-080c23085008"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script src=\"https://cdn.plot.ly/plotly-2.8.3.min.js\"></script>                <div id=\"ea5f6da9-1615-45d6-83fb-32081e14829d\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"ea5f6da9-1615-45d6-83fb-32081e14829d\")) {                    Plotly.newPlot(                        \"ea5f6da9-1615-45d6-83fb-32081e14829d\",                        [{\"hovertemplate\":\"variable=author<br>index=%{x}<br>value=%{y}<extra></extra>\",\"legendgroup\":\"author\",\"line\":{\"color\":\"#636efa\"},\"marker\":{\"symbol\":\"circle\"},\"mode\":\"lines\",\"name\":\"author\",\"orientation\":\"v\",\"showlegend\":true,\"stackgroup\":\"1\",\"x\":[\"\\u9673\\u9577\\u6176\",\"\\u9673\\u6176\\u5143\",\"\\u9673\\u70ba\\u5b78\",\"\\u674e\\u91d1\\u632f\",\"\\u5433\\u5982\\u660e\",\"\\u8523\\u7acb\\u6069\",\"\\u5f90\\u66fc\\u742a\",\"\\u805e\\u5ba3\",\"\\u6771\\u745e\",\"\\u9ec3\\u5947\",\"\\u90ce\\u82f1\",\"\\u9ec3\\u79c0\\u6cbb\",\"\\u9ec3\\u514b\\u5168\",\"\\u912d\\u6101\\u4e88\",\"\\u6d2a\\u6625\\u67f3\",\"\\u502a\\u632f\\u91d1\",\"\\u9673\\u9298\\u78fb\",\"\\u5289\\u7b2c\\u7d05\",\"\\u9673\\u9e97\\u7389\",\"\\u8521\\u4e00\\u8c6a\",\"\\u5510\\u632f\\u745c\",\"\\u9673\\u570b\\u8208\",\"\\u694a\\u8085\\u4fca\",\"\\u90b1\\u82f1\\u7f8e\",\"\\u7fc1\\u7dad\\u7490\",\"\\u5bd2\\u7389\",\"\\u5c0f\\u5e73\",\"\\u6797\\u4f73\\u7469\",\"\\u90b1\\u6bd3\\u96ef\",\"\\u738b\\u4e16\\u52e4\",\"\\u859b\\u7d20\\u74ca\",\"\\u963f\\u7c73\\u5e2b\",\"\\u9673\\u662b\\u6f54\",\"\\u8521\\u5b9c\\u5ead\",\"\\u5409\\u58eb\\u535a\",\"\\u912d\\u5b8f\\u5ead\",\"\\u5b6b\\u5b87\\u5fb7\",\"\\u91d1\\u5b50\",\"\\u91ce\\u592b\",\"\\u674e\\u70b7\\u70fd\",\"\\u674e\\u798f\\u5357\",\"\\u9ec3\\u7b71\\u84c1\",\"\\u8ac7\\u5b9c\\u82b3\",\"\\u984f\\u654f\\u4e98\",\"\\u65b9\\u5b9a\\u582f\",\"\\u61ff\\u53c3\",\"\\u5f35\\u706b\\u6728\",\"\\u90c1\\u52db\",\"\\u5289\\u9999\\u862d\",\"\\u6e58\\u592b\\u4eba\",\"\\u4f55\\u570b\\u7965\",\"\\u859b\\u627f\\u6cf0\",\"\\u674e\\u5fd7\\u6f84\",\"\\u5433\\u5bb6\\u7bb4\",\"\\u8521\\u6dd1\\u8ce2\",\"\\u694a\\u5efa\\u8c50\",\"\\u9ec3\\u6069\\u4e9e\",\"\\u694a\\u6a39\\u6e05\",\"\\u7267\\u7f8a\\u5973\",\"\\u9673\\u66f8\\u6bc5\",\"\\u856d\\u6c38\\u7246\",\"\\u9673\\u6c38\\u5bcc\",\"\\u8521\\u6e05\\u5176\",\"\\u5433\\u7db1\\u7acb\",\"\\u6d2a\\u660e\\u71e6\",\"\\u9b4f\\u5343\\u5bb8\",\"\\u9673\\u79c0\\u7af9\",\"\\u694a\\u5929\\u539a\",\"\\u838a\\u8212\\u60e0\"],\"xaxis\":\"x\",\"y\":[8,5,4,4,3,3,2,2,2,2,2,2,2,2,2,2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1],\"yaxis\":\"y\",\"type\":\"scatter\"}],                        {\"template\":{\"data\":{\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"geo\":{\"bgcolor\":\"white\",\"lakecolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"white\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"light\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"ternary\":{\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"index\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"value\"}},\"legend\":{\"title\":{\"text\":\"variable\"},\"tracegroupgap\":0},\"margin\":{\"t\":60}},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('ea5f6da9-1615-45d6-83fb-32081e14829d');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 練習 ✍"
      ],
      "metadata": {
        "id": "6MBxtS1zzU9Z"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 刪除頭尾的 \"\"\" 之後執行"
      ],
      "metadata": {
        "id": "T1WIlw8NzETe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "CATEGORY = 'author'\n",
        "cat_counts = new_df[CATEGORY].value_counts()\n",
        "cat_counts.plot.bar()\n",
        "\"\"\""
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "TkEyneK_zFUQ",
        "outputId": "e4fc3220-058f-4074-899b-c2e0be40db59"
      },
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"\\nCATEGORY = 'author'\\ncat_counts = new_df[CATEGORY].value_counts()\\ncat_counts.plot.bar()\\n\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### topK高頻詞"
      ],
      "metadata": {
        "id": "udKxK6DzUTa8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "import itertools\n",
        "from collections import Counter\n",
        "\n",
        "def count_words(df, text_col):\n",
        "    nested_words = [row.split() for row in df[text_col]]\n",
        "    flat_words = itertools.chain(*nested_words)\n",
        "    counter = Counter(flat_words)\n",
        "    return counter"
      ],
      "metadata": {
        "cellView": "form",
        "id": "RYj3nqAtV70a"
      },
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "counter = count_words(new_df, 'clean_text')"
      ],
      "metadata": {
        "id": "vG_C5lLIYjuS"
      },
      "execution_count": 27,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "topK = 50\n",
        "counter.most_common(topK)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vKR6bBeMYrqa",
        "outputId": "c67991b3-7855-4877-e811-f8008a0a7bc5"
      },
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('金門', 1542),\n",
              " ('與', 726),\n",
              " ('個', 705),\n",
              " ('這', 620),\n",
              " ('大學', 465),\n",
              " ('我們', 459),\n",
              " ('x000D', 410),\n",
              " ('著', 390),\n",
              " ('說', 347),\n",
              " ('年', 338),\n",
              " ('中', 334),\n",
              " ('為', 312),\n",
              " ('讓', 282),\n",
              " ('楊', 281),\n",
              " ('還', 251),\n",
              " ('從', 250),\n",
              " ('對', 249),\n",
              " ('後', 248),\n",
              " ('來', 235),\n",
              " ('時', 235),\n",
              " ('陳', 222),\n",
              " ('會', 213),\n",
              " ('文學', 200),\n",
              " ('於', 200),\n",
              " ('老師', 197),\n",
              " ('兩', 196),\n",
              " ('次', 196),\n",
              " ('沒有', 187),\n",
              " ('卻', 181),\n",
              " ('種', 177),\n",
              " ('並', 172),\n",
              " ('位', 159),\n",
              " ('台灣', 154),\n",
              " ('將', 146),\n",
              " ('博士', 146),\n",
              " ('因為', 145),\n",
              " ('歷史', 145),\n",
              " ('月', 137),\n",
              " ('校長', 137),\n",
              " ('前', 136),\n",
              " ('─', 135),\n",
              " ('想', 134),\n",
              " ('他們', 134),\n",
              " ('教育', 124),\n",
              " ('生活', 119),\n",
              " ('忠禮', 117),\n",
              " ('民國', 111),\n",
              " ('妳', 111),\n",
              " ('歲', 109),\n",
              " ('裡', 107)]"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "def plot_topK_words(counter, topK=50):\n",
        "    counter_df = pd.DataFrame(counter.most_common(topK), columns =['word', 'count'])\n",
        "    plot = counter_df.set_index('word').plot.bar()\n",
        "    return plot"
      ],
      "metadata": {
        "cellView": "form",
        "id": "Q41kCBIhhhKu"
      },
      "execution_count": 29,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "plot_topK_words(counter, topK=50)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 542
        },
        "id": "8loNcxdMhsb8",
        "outputId": "eb1d759a-7dc8-40e5-c3fa-675ef8bc54b9"
      },
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script src=\"https://cdn.plot.ly/plotly-2.8.3.min.js\"></script>                <div id=\"c4c2b743-1c92-44d7-aa8c-afb5c0fb44bb\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"c4c2b743-1c92-44d7-aa8c-afb5c0fb44bb\")) {                    Plotly.newPlot(                        \"c4c2b743-1c92-44d7-aa8c-afb5c0fb44bb\",                        [{\"alignmentgroup\":\"True\",\"hovertemplate\":\"variable=count<br>word=%{x}<br>value=%{y}<extra></extra>\",\"legendgroup\":\"count\",\"marker\":{\"color\":\"#636efa\",\"pattern\":{\"shape\":\"\"}},\"name\":\"count\",\"offsetgroup\":\"count\",\"orientation\":\"v\",\"showlegend\":true,\"textposition\":\"auto\",\"x\":[\"\\u91d1\\u9580\",\"\\u8207\",\"\\u500b\",\"\\u9019\",\"\\u5927\\u5b78\",\"\\u6211\\u5011\",\"x000D\",\"\\u8457\",\"\\u8aaa\",\"\\u5e74\",\"\\u4e2d\",\"\\u70ba\",\"\\u8b93\",\"\\u694a\",\"\\u9084\",\"\\u5f9e\",\"\\u5c0d\",\"\\u5f8c\",\"\\u4f86\",\"\\u6642\",\"\\u9673\",\"\\u6703\",\"\\u6587\\u5b78\",\"\\u65bc\",\"\\u8001\\u5e2b\",\"\\u5169\",\"\\u6b21\",\"\\u6c92\\u6709\",\"\\u537b\",\"\\u7a2e\",\"\\u4e26\",\"\\u4f4d\",\"\\u53f0\\u7063\",\"\\u5c07\",\"\\u535a\\u58eb\",\"\\u56e0\\u70ba\",\"\\u6b77\\u53f2\",\"\\u6708\",\"\\u6821\\u9577\",\"\\u524d\",\"\\u2500\",\"\\u60f3\",\"\\u4ed6\\u5011\",\"\\u6559\\u80b2\",\"\\u751f\\u6d3b\",\"\\u5fe0\\u79ae\",\"\\u6c11\\u570b\",\"\\u59b3\",\"\\u6b72\",\"\\u88e1\"],\"xaxis\":\"x\",\"y\":[1542,726,705,620,465,459,410,390,347,338,334,312,282,281,251,250,249,248,235,235,222,213,200,200,197,196,196,187,181,177,172,159,154,146,146,145,145,137,137,136,135,134,134,124,119,117,111,111,109,107],\"yaxis\":\"y\",\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"geo\":{\"bgcolor\":\"white\",\"lakecolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"white\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"light\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"ternary\":{\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"word\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"value\"}},\"legend\":{\"title\":{\"text\":\"variable\"},\"tracegroupgap\":0},\"margin\":{\"t\":60},\"barmode\":\"relative\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('c4c2b743-1c92-44d7-aa8c-afb5c0fb44bb');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "def show_keyterm_in_counter(counter, keyterm):\n",
        "    for term, count in counter.items():\n",
        "        if keyterm in term:\n",
        "            ic(term, count)"
      ],
      "metadata": {
        "cellView": "form",
        "id": "xhGof90waVmT"
      },
      "execution_count": 31,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "show_keyterm_in_counter(counter, \"金門\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Jg4XhdSsacGx",
        "outputId": "8d88ac27-d1e0-4cf9-f908-204d92168cee"
      },
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "ic| term: '金門', count: 1542\n",
            "ic| term: '金門島', count: 11\n",
            "ic| term: '金門縣', count: 72\n",
            "ic| term: '金門人', count: 73\n",
            "ic| term: '金門籍', count: 4\n",
            "ic| term: '金門味', count: 1\n",
            "ic| term: '金門城', count: 2\n",
            "ic| term: '金門湖', count: 1\n",
            "ic| term: '金門鎮', count: 4\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "def write_counter2txt(counter, filename):\n",
        "    with open(f'{filename}.txt', 'w') as f:\n",
        "        for term in sorted(counter, key=counter.get, reverse=True):\n",
        "            f.write(f\"{term}\\t{counter[term]}\\n\")"
      ],
      "metadata": {
        "cellView": "form",
        "id": "6HylZ_p3jhbu"
      },
      "execution_count": 33,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "write_counter2txt(counter, 'clean_text')"
      ],
      "metadata": {
        "id": "O8ZRXhAtj0e2"
      },
      "execution_count": 34,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 練習 ✍"
      ],
      "metadata": {
        "id": "_yIuNZ-jzwn-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 修改`topK`的數字\n",
        "- 刪除頭尾的 \"\"\" 之後執行"
      ],
      "metadata": {
        "id": "oCPPdh4Ezu4e"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "topK = 50\n",
        "counter = count_words(new_df, 'clean_text')\n",
        "counter.most_common(topK)\n",
        "\"\"\""
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "GJKRL8Biz0f2",
        "outputId": "1f8f464e-ceed-4975-f453-2218873e9651"
      },
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"\\ntopK = 50\\ncounter = count_words(new_df, 'clean_text')\\ncounter.most_common(topK)\\n\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 上面執行成功後，刪除下面三個的#符號之後，依序執行，`金門`可以換成你感興趣的其他詞"
      ],
      "metadata": {
        "id": "XmDh9bdA028t"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#plot_topK_words(counter, topK=topK)"
      ],
      "metadata": {
        "id": "j4zNaOg40Lpy"
      },
      "execution_count": 36,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#show_keyterm_in_counter(counter, \"金門\")"
      ],
      "metadata": {
        "id": "pg2DVCwm0UHQ"
      },
      "execution_count": 37,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#write_counter2txt(counter, 'clean_text')"
      ],
      "metadata": {
        "id": "av6BrtRP0QqM"
      },
      "execution_count": 38,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 依關鍵詞過濾"
      ],
      "metadata": {
        "id": "j10CKh2LIlTa"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Text Rank是常見的關鍵詞抽取演算法\n",
        "![](https://upload.wikimedia.org/wikipedia/commons/1/1e/Websites_interlinking_to_illustrate_PageRank_percents.png)"
      ],
      "metadata": {
        "id": "SBptU_uaa-4f"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "doc = nlp(sample_text)\n",
        "for phrase in doc._.phrases:\n",
        "    ic(phrase)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mppbNvtyInac",
        "outputId": "190058e4-d8db-4676-fd79-9a50c2dc80e2"
      },
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "ic| phrase: Phrase(text='北方傳統', chunks=[北方傳統], count=1, rank=0.475632100758785)\n",
            "ic| phrase: Phrase(text='千年', chunks=[千年], count=1, rank=0.35672407556908875)\n",
            "ic| phrase: Phrase(text='台灣', chunks=[台灣, 台灣], count=2, rank=0.35672407556908875)\n",
            "ic| phrase: Phrase(text='大樓', chunks=[大樓, 大樓], count=2, rank=0.35672407556908875)\n",
            "ic| phrase: Phrase(text='建築師', chunks=[建築師], count=1, rank=0.35672407556908875)\n",
            "ic| phrase: Phrase(text='金門', chunks=[金門], count=1, rank=0.35672407556908875)\n",
            "ic| phrase: Phrase(text='閩式', chunks=[閩式, 閩式, 閩式], count=3, rank=0.35672407556908875)\n",
            "ic| phrase: Phrase(text='衝突', chunks=[衝突], count=1, rank=0.2951087819086472)\n",
            "ic| phrase: Phrase(text='台大', chunks=[台大], count=1, rank=0.12640379149569736)\n",
            "ic| phrase: Phrase(text='大武山', chunks=[大武山], count=1, rank=0.12640379149569736)\n",
            "ic| phrase: Phrase(text='王宅', chunks=[王宅], count=1, rank=0.12640379149569736)\n",
            "ic| phrase: Phrase(text='閩', chunks=[閩], count=1, rank=0.12640379149569736)\n",
            "ic| phrase: Phrase(text='一秒秒', chunks=[一秒秒], count=1, rank=0.0)\n",
            "ic| phrase: Phrase(text='十八間', chunks=[十八間], count=1, rank=0.0)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tr = doc._.textrank\n",
        "tr.plot_keyphrases()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 385
        },
        "id": "48pwxyihMveG",
        "outputId": "8d29dd33-a605-4cd1-87f9-cfa2b4a4146f"
      },
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "\n",
              "<div id=\"altair-viz-13bf41637c5444d5a95b8aa17e636726\"></div>\n",
              "<script type=\"text/javascript\">\n",
              "  var VEGA_DEBUG = (typeof VEGA_DEBUG == \"undefined\") ? {} : VEGA_DEBUG;\n",
              "  (function(spec, embedOpt){\n",
              "    let outputDiv = document.currentScript.previousElementSibling;\n",
              "    if (outputDiv.id !== \"altair-viz-13bf41637c5444d5a95b8aa17e636726\") {\n",
              "      outputDiv = document.getElementById(\"altair-viz-13bf41637c5444d5a95b8aa17e636726\");\n",
              "    }\n",
              "    const paths = {\n",
              "      \"vega\": \"https://cdn.jsdelivr.net/npm//vega@5?noext\",\n",
              "      \"vega-lib\": \"https://cdn.jsdelivr.net/npm//vega-lib?noext\",\n",
              "      \"vega-lite\": \"https://cdn.jsdelivr.net/npm//vega-lite@4.17.0?noext\",\n",
              "      \"vega-embed\": \"https://cdn.jsdelivr.net/npm//vega-embed@6?noext\",\n",
              "    };\n",
              "\n",
              "    function maybeLoadScript(lib, version) {\n",
              "      var key = `${lib.replace(\"-\", \"\")}_version`;\n",
              "      return (VEGA_DEBUG[key] == version) ?\n",
              "        Promise.resolve(paths[lib]) :\n",
              "        new Promise(function(resolve, reject) {\n",
              "          var s = document.createElement('script');\n",
              "          document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
              "          s.async = true;\n",
              "          s.onload = () => {\n",
              "            VEGA_DEBUG[key] = version;\n",
              "            return resolve(paths[lib]);\n",
              "          };\n",
              "          s.onerror = () => reject(`Error loading script: ${paths[lib]}`);\n",
              "          s.src = paths[lib];\n",
              "        });\n",
              "    }\n",
              "\n",
              "    function showError(err) {\n",
              "      outputDiv.innerHTML = `<div class=\"error\" style=\"color:red;\">${err}</div>`;\n",
              "      throw err;\n",
              "    }\n",
              "\n",
              "    function displayChart(vegaEmbed) {\n",
              "      vegaEmbed(outputDiv, spec, embedOpt)\n",
              "        .catch(err => showError(`Javascript Error: ${err.message}<br>This usually means there's a typo in your chart specification. See the javascript console for the full traceback.`));\n",
              "    }\n",
              "\n",
              "    if(typeof define === \"function\" && define.amd) {\n",
              "      requirejs.config({paths});\n",
              "      require([\"vega-embed\"], displayChart, err => showError(`Error loading script: ${err.message}`));\n",
              "    } else {\n",
              "      maybeLoadScript(\"vega\", \"5\")\n",
              "        .then(() => maybeLoadScript(\"vega-lite\", \"4.17.0\"))\n",
              "        .then(() => maybeLoadScript(\"vega-embed\", \"6\"))\n",
              "        .catch(showError)\n",
              "        .then(() => displayChart(vegaEmbed));\n",
              "    }\n",
              "  })({\"config\": {\"view\": {\"continuousWidth\": 400, \"continuousHeight\": 300}}, \"data\": {\"name\": \"data-903ff1d7272fcc0786d901a840b3bf45\"}, \"mark\": \"bar\", \"encoding\": {\"color\": {\"field\": \"count\", \"type\": \"quantitative\"}, \"tooltip\": [{\"field\": \"text\", \"type\": \"nominal\"}, {\"field\": \"rank\", \"type\": \"quantitative\"}, {\"field\": \"count\", \"type\": \"quantitative\"}], \"x\": {\"field\": \"index\", \"type\": \"quantitative\"}, \"y\": {\"field\": \"rank\", \"type\": \"quantitative\"}}, \"title\": \"Keyphrase profile of the document\", \"$schema\": \"https://vega.github.io/schema/vega-lite/v4.17.0.json\", \"datasets\": {\"data-903ff1d7272fcc0786d901a840b3bf45\": [{\"index\": 0, \"text\": \"\\u5317\\u65b9\\u50b3\\u7d71\", \"count\": 1, \"rank\": 0.475632100758785}, {\"index\": 1, \"text\": \"\\u5343\\u5e74\", \"count\": 1, \"rank\": 0.35672407556908875}, {\"index\": 2, \"text\": \"\\u53f0\\u7063\", \"count\": 2, \"rank\": 0.35672407556908875}, {\"index\": 3, \"text\": \"\\u5927\\u6a13\", \"count\": 2, \"rank\": 0.35672407556908875}, {\"index\": 4, \"text\": \"\\u5efa\\u7bc9\\u5e2b\", \"count\": 1, \"rank\": 0.35672407556908875}, {\"index\": 5, \"text\": \"\\u91d1\\u9580\", \"count\": 1, \"rank\": 0.35672407556908875}, {\"index\": 6, \"text\": \"\\u95a9\\u5f0f\", \"count\": 3, \"rank\": 0.35672407556908875}, {\"index\": 7, \"text\": \"\\u885d\\u7a81\", \"count\": 1, \"rank\": 0.2951087819086472}, {\"index\": 8, \"text\": \"\\u53f0\\u5927\", \"count\": 1, \"rank\": 0.12640379149569736}, {\"index\": 9, \"text\": \"\\u5927\\u6b66\\u5c71\", \"count\": 1, \"rank\": 0.12640379149569736}, {\"index\": 10, \"text\": \"\\u738b\\u5b85\", \"count\": 1, \"rank\": 0.12640379149569736}, {\"index\": 11, \"text\": \"\\u95a9\", \"count\": 1, \"rank\": 0.12640379149569736}, {\"index\": 12, \"text\": \"\\u4e00\\u79d2\\u79d2\", \"count\": 1, \"rank\": 0.0}, {\"index\": 13, \"text\": \"\\u5341\\u516b\\u9593\", \"count\": 1, \"rank\": 0.0}]}}, {\"mode\": \"vega-lite\"});\n",
              "</script>"
            ],
            "text/plain": [
              "alt.Chart(...)"
            ]
          },
          "metadata": {},
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "def collect_keyterms(texts):\n",
        "    all_keyterms = []    \n",
        "    for doc in nlp.pipe(texts):\n",
        "        phrases = [phrase.text for phrase in doc._.phrases]\n",
        "\n",
        "        if phrases:\n",
        "            keyterms = \" \".join(phrases)\n",
        "        else:\n",
        "            keyterms = \"NONE\"\n",
        "            \n",
        "        all_keyterms.append(keyterms)\n",
        "    return all_keyterms"
      ],
      "metadata": {
        "cellView": "form",
        "id": "3FcoSOuLNYoK"
      },
      "execution_count": 41,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "new_df['keyterm'] = collect_keyterms(TEXTS)\n",
        "new_df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "kRMaJ5MlQk_L",
        "outputId": "2b040842-ff7e-4cb1-c087-f21aa2ebe2e5"
      },
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "          date author                                               text  \\\n",
              "0   2022-06-11    陳長慶  珍惜活在當下的每一個時光 ──《浯鄉縮影：陳長慶散文集2014~2021》自序 《浯鄉縮影》...   \n",
              "1   2022-05-18    魏千宸  首次訪金，首次穿梭特色金大校園 首次乘著飛機盤桓在這座古稱浯洲的上空，正當飛機下了雲層，地表...   \n",
              "2   2022-05-18    洪春柳  認識自我 有取有捨 ──訪金中輔導老師李瓊芳談生活小道 輔導的精義：在一段時間裡，陪伴一個人...   \n",
              "3   2022-05-06    陳為學  金嶝親橋一線牽 金門大學前院長陳益源教授於2018年4月29日，在金門日報的「浯江夜話」發表...   \n",
              "4   2022-03-29    黃克全  知人論世和以意逆志——談楊樹清、陳慶瀚與吳鈞堯 楊樹清竟是一個不折不扣的阜內思。我閱讀波赫士...   \n",
              "..         ...    ...                                                ...   \n",
              "95  2015-05-21    陳麗玉  話說后宅洋樓 洋樓的滄桑與興修_x000D_\\n  「王金城洋樓」於民國21年興建完工，當時...   \n",
              "96  2015-05-09    蔡宜庭  DOBGFamily 對我們身邊很多人來說，金門離島亂遠的，到底是國內還是國外傻傻分不清楚，...   \n",
              "97  2015-05-08    李金振  眾志成城共創金大奇蹟（下） 五、從落榜生到榜首_x000D_\\n辦學的最終目的是得天下英才而...   \n",
              "98  2015-05-08    莊舒惠  建立和諧的人際關係──非暴力溝通 在我們的日常生活中難免會與他人發生爭執，歸究其原因，也許是...   \n",
              "99  2015-05-07    李金振  眾志成城共創金大奇蹟（上） 1997年，教育部長吳京院士徵詢交待我一個不平凡的使命，為金門籌...   \n",
              "\n",
              "                                           clean_text  \\\n",
              "0   珍惜 活在 當下 個 時光 ─ ─ 浯鄉 縮影 陳 長慶 散文集 自序 浯鄉 縮影 從 年 ...   \n",
              "1   首 次 訪金 首 次 穿梭 特色 金大 校園 首 次 乘著 飛機 盤桓 這 座 古稱 浯洲 ...   \n",
              "2   認識 自我 有取 有捨 ─ ─ 訪金 中 輔導 老師 李 瓊芳 談 生活 小道 輔導 精義 ...   \n",
              "3   金 嶝 親 橋 一線 牽 金門 大學 前院長 陳益源 教授 於 年 月 日 金門 日報 浯江...   \n",
              "4   知人 論世 以意 逆志 談 楊 樹清 陳慶 瀚與 吳 鈞堯 楊 樹清 竟是 個 不折不扣 的...   \n",
              "..                                                ...   \n",
              "95  話 說 后宅 洋樓 洋樓 滄桑 與 興修 x000D 王 金城 洋樓 於 民國 年 興建 完...   \n",
              "96  DOBGFamily 對 我們 身邊 人來 說 金門 離島 亂遠 國內 還是 國外 傻傻 分...   \n",
              "97  眾志成城 共創 金大 奇蹟 從 落榜生 榜首 x000D 辦學 最終 目的 天下 英才 教育...   \n",
              "98  建立 和諧 人際 關係 ─ ─ 非暴力 溝通 我們 日常 生活 中 難免 會 與 發生 爭執...   \n",
              "99  眾志成城 共創 金大 奇蹟 年 教育部長 吳京 院士 徵詢 交待 個 平凡 使命 為 金門 ...   \n",
              "\n",
              "                                              keyterm  \n",
              "0   胡志明市人文社科大學文學系 道範顏馨五桂聯芳 陳長慶短篇小說集 五位教授 凡塵悲歌 島嶼天青...  \n",
              "1      北方傳統 千年 台灣 大樓 建築師 金門 閩式 衝突 台大 大武山 王宅 閩 一秒秒 十八間  \n",
              "2   金門大學閩南研究所 金大研究助理 金中輔導室偏重 金中輔導室 慈橋社是高中 培育出 年齡層 ...  \n",
              "3   二話不說 今天 今日 傍晚 夏 張小燕 稟報 來自大嶝島 今年三月初 西元1989年張 過世...  \n",
              "4   高雄義守大學 蜻蜓哲學家 過程中心論 工程系 幾乎 悲劇性 楊 現代性 邏輯性 立竿見影 小...  \n",
              "..                                                ...  \n",
              "95  承澤祖德基業 三間 九十七 二樓 十一人 十五日 天錫借 日軍 明年 曾祖母 王仁 王金 過...  \n",
              "96                 蔡厝古道 古戰場 台灣海峽 明天 海邊 金沙 沒關係 西門町 一 十  \n",
              "97  金門分部爭取獨立 凌晨二點 南京大學 四川大學 親愛的同仁 2010 中央 八年 台灣 國際...  \n",
              "98      金門大學 今天 會 沙發 衝突 Rosenberg 盧森堡 靜下心 一 二 第一步 第二步  \n",
              "99  金門縣政府 台大研究所 為了金門 徵詢交待 二舍 今天 十棟 夏 天大 教育部 清晨 為什麼...  \n",
              "\n",
              "[100 rows x 5 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-7981d092-d39f-4087-9fe3-13e469b99425\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>date</th>\n",
              "      <th>author</th>\n",
              "      <th>text</th>\n",
              "      <th>clean_text</th>\n",
              "      <th>keyterm</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2022-06-11</td>\n",
              "      <td>陳長慶</td>\n",
              "      <td>珍惜活在當下的每一個時光 ──《浯鄉縮影：陳長慶散文集2014~2021》自序 《浯鄉縮影》...</td>\n",
              "      <td>珍惜 活在 當下 個 時光 ─ ─ 浯鄉 縮影 陳 長慶 散文集 自序 浯鄉 縮影 從 年 ...</td>\n",
              "      <td>胡志明市人文社科大學文學系 道範顏馨五桂聯芳 陳長慶短篇小說集 五位教授 凡塵悲歌 島嶼天青...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2022-05-18</td>\n",
              "      <td>魏千宸</td>\n",
              "      <td>首次訪金，首次穿梭特色金大校園 首次乘著飛機盤桓在這座古稱浯洲的上空，正當飛機下了雲層，地表...</td>\n",
              "      <td>首 次 訪金 首 次 穿梭 特色 金大 校園 首 次 乘著 飛機 盤桓 這 座 古稱 浯洲 ...</td>\n",
              "      <td>北方傳統 千年 台灣 大樓 建築師 金門 閩式 衝突 台大 大武山 王宅 閩 一秒秒 十八間</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2022-05-18</td>\n",
              "      <td>洪春柳</td>\n",
              "      <td>認識自我 有取有捨 ──訪金中輔導老師李瓊芳談生活小道 輔導的精義：在一段時間裡，陪伴一個人...</td>\n",
              "      <td>認識 自我 有取 有捨 ─ ─ 訪金 中 輔導 老師 李 瓊芳 談 生活 小道 輔導 精義 ...</td>\n",
              "      <td>金門大學閩南研究所 金大研究助理 金中輔導室偏重 金中輔導室 慈橋社是高中 培育出 年齡層 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2022-05-06</td>\n",
              "      <td>陳為學</td>\n",
              "      <td>金嶝親橋一線牽 金門大學前院長陳益源教授於2018年4月29日，在金門日報的「浯江夜話」發表...</td>\n",
              "      <td>金 嶝 親 橋 一線 牽 金門 大學 前院長 陳益源 教授 於 年 月 日 金門 日報 浯江...</td>\n",
              "      <td>二話不說 今天 今日 傍晚 夏 張小燕 稟報 來自大嶝島 今年三月初 西元1989年張 過世...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2022-03-29</td>\n",
              "      <td>黃克全</td>\n",
              "      <td>知人論世和以意逆志——談楊樹清、陳慶瀚與吳鈞堯 楊樹清竟是一個不折不扣的阜內思。我閱讀波赫士...</td>\n",
              "      <td>知人 論世 以意 逆志 談 楊 樹清 陳慶 瀚與 吳 鈞堯 楊 樹清 竟是 個 不折不扣 的...</td>\n",
              "      <td>高雄義守大學 蜻蜓哲學家 過程中心論 工程系 幾乎 悲劇性 楊 現代性 邏輯性 立竿見影 小...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>95</th>\n",
              "      <td>2015-05-21</td>\n",
              "      <td>陳麗玉</td>\n",
              "      <td>話說后宅洋樓 洋樓的滄桑與興修_x000D_\\n  「王金城洋樓」於民國21年興建完工，當時...</td>\n",
              "      <td>話 說 后宅 洋樓 洋樓 滄桑 與 興修 x000D 王 金城 洋樓 於 民國 年 興建 完...</td>\n",
              "      <td>承澤祖德基業 三間 九十七 二樓 十一人 十五日 天錫借 日軍 明年 曾祖母 王仁 王金 過...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>96</th>\n",
              "      <td>2015-05-09</td>\n",
              "      <td>蔡宜庭</td>\n",
              "      <td>DOBGFamily 對我們身邊很多人來說，金門離島亂遠的，到底是國內還是國外傻傻分不清楚，...</td>\n",
              "      <td>DOBGFamily 對 我們 身邊 人來 說 金門 離島 亂遠 國內 還是 國外 傻傻 分...</td>\n",
              "      <td>蔡厝古道 古戰場 台灣海峽 明天 海邊 金沙 沒關係 西門町 一 十</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>97</th>\n",
              "      <td>2015-05-08</td>\n",
              "      <td>李金振</td>\n",
              "      <td>眾志成城共創金大奇蹟（下） 五、從落榜生到榜首_x000D_\\n辦學的最終目的是得天下英才而...</td>\n",
              "      <td>眾志成城 共創 金大 奇蹟 從 落榜生 榜首 x000D 辦學 最終 目的 天下 英才 教育...</td>\n",
              "      <td>金門分部爭取獨立 凌晨二點 南京大學 四川大學 親愛的同仁 2010 中央 八年 台灣 國際...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>98</th>\n",
              "      <td>2015-05-08</td>\n",
              "      <td>莊舒惠</td>\n",
              "      <td>建立和諧的人際關係──非暴力溝通 在我們的日常生活中難免會與他人發生爭執，歸究其原因，也許是...</td>\n",
              "      <td>建立 和諧 人際 關係 ─ ─ 非暴力 溝通 我們 日常 生活 中 難免 會 與 發生 爭執...</td>\n",
              "      <td>金門大學 今天 會 沙發 衝突 Rosenberg 盧森堡 靜下心 一 二 第一步 第二步</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>99</th>\n",
              "      <td>2015-05-07</td>\n",
              "      <td>李金振</td>\n",
              "      <td>眾志成城共創金大奇蹟（上） 1997年，教育部長吳京院士徵詢交待我一個不平凡的使命，為金門籌...</td>\n",
              "      <td>眾志成城 共創 金大 奇蹟 年 教育部長 吳京 院士 徵詢 交待 個 平凡 使命 為 金門 ...</td>\n",
              "      <td>金門縣政府 台大研究所 為了金門 徵詢交待 二舍 今天 十棟 夏 天大 教育部 清晨 為什麼...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>100 rows × 5 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7981d092-d39f-4087-9fe3-13e469b99425')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-7981d092-d39f-4087-9fe3-13e469b99425 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-7981d092-d39f-4087-9fe3-13e469b99425');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "counter = count_words(new_df, 'keyterm')\n",
        "plot_topK_words(counter, topK=50)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 542
        },
        "id": "1xJ7JLLRVqef",
        "outputId": "f99af835-f3f3-44ec-f97a-4f202e486aad"
      },
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script src=\"https://cdn.plot.ly/plotly-2.8.3.min.js\"></script>                <div id=\"39323672-5c2a-472f-8a71-2aeea8353716\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"39323672-5c2a-472f-8a71-2aeea8353716\")) {                    Plotly.newPlot(                        \"39323672-5c2a-472f-8a71-2aeea8353716\",                        [{\"alignmentgroup\":\"True\",\"hovertemplate\":\"variable=count<br>word=%{x}<br>value=%{y}<extra></extra>\",\"legendgroup\":\"count\",\"marker\":{\"color\":\"#636efa\",\"pattern\":{\"shape\":\"\"}},\"name\":\"count\",\"offsetgroup\":\"count\",\"orientation\":\"v\",\"showlegend\":true,\"textposition\":\"auto\",\"x\":[\"\\u91d1\\u9580\",\"\\u4e00\",\"\\u4e09\",\"\\u53f0\\u7063\",\"\\u70ba\\u4e86\",\"\\u56db\",\"\\u4e8c\",\"\\u53f0\\u5317\",\"\\u4eca\\u5929\",\"\\u674e\",\"\\u4e94\",\"\\u91d1\\u57ce\",\"\\u8521\",\"\\u4e00\\u5929\",\"\\u738b\",\"\\u516d\",\"\\u5927\\u6a13\",\"\\u4e09\\u5e74\",\"\\u694a\",\"\\u7f8e\",\"\\u65b0\\u52a0\\u5761\",\"\\u798f\\u5efa\",\"\\u4eca\\u5e74\",\"\\u91d1\\u9580\\u7e23\",\"\\u7b2c\",\"\\u5357\\u6d0b\",\"\\u9ad8\\u96c4\",\"\\u65e5\",\"20\",\"\\u2500\",\"\\u5341\\u5e74\",\"\\u4e0b\\u5348\",\"\\u516b\",\"3\",\"\\u53bb\\u5e74\",\"\\u65e5\\u672c\",\"\\u6c11\\u570b\",\"\\u884c\\u653f\\u9662\",\"6\",\"\\u4e09\\u5927\",\"\\u4eca\\u65e5\",\"\\u4e03\",\"\\u53f0\",\"\\u4e2d\\u6587\",\"10\",\"\\u767e\\u5e74\",\"\\u5370\\u5c3c\",\"\\u91d1\\u9580\\u6587\\u85dd\",\"\\u8b8a\\u9077\",\"1\"],\"xaxis\":\"x\",\"y\":[70,57,47,46,39,31,25,25,20,20,20,17,16,16,16,15,14,14,13,13,13,12,12,12,12,12,11,11,11,11,11,10,10,9,9,9,9,8,8,8,8,8,8,8,8,8,8,7,7,7],\"yaxis\":\"y\",\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"geo\":{\"bgcolor\":\"white\",\"lakecolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"white\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"light\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"ternary\":{\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"word\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"value\"}},\"legend\":{\"title\":{\"text\":\"variable\"},\"tracegroupgap\":0},\"margin\":{\"t\":60},\"barmode\":\"relative\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('39323672-5c2a-472f-8a71-2aeea8353716');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "show_keyterm_in_counter(counter, \"金門\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "j8bjYnN8aq_R",
        "outputId": "fd7a4fa2-f86f-4bd8-c97a-54028e9c013f"
      },
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "ic| term: '金門前鋒報', count: 2\n",
            "ic| term: '金門文藝', count: 7\n",
            "ic| term: '金門日報', count: 6\n",
            "ic| term: '金門', count: 70\n",
            "ic| term: '金門大學閩南研究所', count: 1\n",
            "ic| term: '金門青少年文學獎', count: 1\n",
            "ic| term: '金門學習指導中心承辦金門縣社區', count: 1\n",
            "ic| term: '金門學習指導中心', count: 1\n",
            "ic| term: '金門學習指導中心設計', count: 1\n",
            "ic| term: '金門成人教育侃侃而談', count: 1\n",
            "ic| term: '金門成立中心', count: 1\n",
            "ic| term: '金門高中', count: 5\n",
            "ic| term: '金門縣', count: 12\n",
            "ic| term: '金門成人教育的', count: 1\n",
            "ic| term: '金門日報‧鄉訊', count: 1\n",
            "ic| term: '海上仙洲金門', count: 1\n",
            "ic| term: '金門島嶼邊緣', count: 2\n",
            "ic| term: '金門學', count: 1\n",
            "ic| term: '金門報導', count: 2\n",
            "ic| term: '金門縣文化局', count: 3\n",
            "ic| term: '敬悼金門', count: 1\n",
            "ic| term: '金門籍', count: 1\n",
            "ic| term: '金門日報總編輯陳國興', count: 1\n",
            "ic| term: '台南金門同鄉會陳瑞華理事長', count: 1\n",
            "ic| term: '金門傀儡戲團蔡', count: 1\n",
            "ic| term: '金門文化局', count: 4\n",
            "ic| term: '金門日報．', count: 1\n",
            "ic| term: '金門縣政府台北服務處', count: 1\n",
            "ic| term: '金門縣政府', count: 6\n",
            "ic| term: '台南市金門同鄉會理事長陳瑞華、副理事長何柏霆', count: 1\n",
            "ic| term: '金門文藝》革新', count: 1\n",
            "ic| term: '金門縣政府營利事業登記證（65年', count: 1\n",
            "ic| term: '金門縣宗族文化研究協會', count: 2\n",
            "ic| term: '金門縣志', count: 2\n",
            "ic| term: '金門生命線', count: 2\n",
            "ic| term: '金門高粱酒', count: 3\n",
            "ic| term: '金門的家鄉味', count: 1\n",
            "ic| term: '金門傳統音樂館館長', count: 1\n",
            "ic| term: '浯江衍派～金門', count: 1\n",
            "ic| term: '金門社會教育館', count: 1\n",
            "ic| term: '金門叢書', count: 1\n",
            "ic| term: '金門先賢史蹟研究', count: 1\n",
            "ic| term: '金門黃氏宗親會', count: 1\n",
            "ic| term: '金門人東向與南向', count: 1\n",
            "ic| term: '金門學術研究中心', count: 1\n",
            "ic| term: '金門交通旅遊局', count: 1\n",
            "ic| term: '金門大學發展', count: 1\n",
            "ic| term: '金門學人聯誼會', count: 1\n",
            "ic| term: '金門料羅灣', count: 1\n",
            "ic| term: '把北風留給金門', count: 1\n",
            "ic| term: '金門大橋」興建', count: 1\n",
            "ic| term: '金門設計教學筆記', count: 1\n",
            "ic| term: '高雄應用科技大學金門分部', count: 1\n",
            "ic| term: '金門分部', count: 1\n",
            "ic| term: '金門文學國際學術研討會', count: 1\n",
            "ic| term: '金門民間業者', count: 1\n",
            "ic| term: '金門事', count: 1\n",
            "ic| term: '金門憶舊', count: 1\n",
            "ic| term: '君自故鄉來：一些金門人', count: 1\n",
            "ic| term: '金門海峽', count: 1\n",
            "ic| term: '金門永世太平', count: 1\n",
            "ic| term: '金門現代文學史', count: 1\n",
            "ic| term: '金門文學史三書', count: 1\n",
            "ic| term: '「顯影」中金門', count: 1\n",
            "ic| term: '香港金門同鄉會', count: 1\n",
            "ic| term: '金門文化獎', count: 1\n",
            "ic| term: '金門大學工業工程', count: 1\n",
            "ic| term: '金門縣僑務辦公室', count: 1\n",
            "ic| term: '世界金門日', count: 1\n",
            "ic| term: '金門期間曾念過幾年', count: 1\n",
            "ic| term: '金門大學舉行丹斯里', count: 1\n",
            "ic| term: '金門大學榮譽講座教授', count: 1\n",
            "ic| term: '福建金門湖', count: 1\n",
            "ic| term: '金門東海岸', count: 1\n",
            "ic| term: '金門家扶中心', count: 2\n",
            "ic| term: '南方沿海金門', count: 1\n",
            "ic| term: '金門寺廟', count: 1\n",
            "ic| term: '金門志', count: 2\n",
            "ic| term: '現為金門法院', count: 1\n",
            "ic| term: '勸建金門', count: 1\n",
            "ic| term: '金門‧馬祖', count: 1\n",
            "ic| term: '金門鎮', count: 1\n",
            "ic| term: '金門歷史', count: 1\n",
            "ic| term: '金門縣空軍後備', count: 1\n",
            "ic| term: '金門大學理工', count: 1\n",
            "ic| term: '金門顏厝', count: 1\n",
            "ic| term: '清金門鎮', count: 1\n",
            "ic| term: '金門大學側門', count: 1\n",
            "ic| term: '金門縣生命線', count: 1\n",
            "ic| term: '金門日報伴我成長', count: 1\n",
            "ic| term: '金門當兵', count: 1\n",
            "ic| term: '金門鄉訊人物誌', count: 1\n",
            "ic| term: '金門管理控股公司', count: 1\n",
            "ic| term: '飲酒金門行', count: 1\n",
            "ic| term: '金門西園', count: 1\n",
            "ic| term: '金門分部爭取獨立', count: 1\n",
            "ic| term: '金門大學', count: 1\n",
            "ic| term: '為了金門', count: 1\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "write_counter2txt(counter, 'keyterm')"
      ],
      "metadata": {
        "id": "6d2ZSe9kmXLL"
      },
      "execution_count": 45,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 練習 ✍"
      ],
      "metadata": {
        "id": "0YgQSI801cMP"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 修改`topK`的數字\n",
        "- 刪除頭尾的 \"\"\" 之後執行](https://)"
      ],
      "metadata": {
        "id": "W5nKBHbP1bTw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "topK = 50\n",
        "new_df['keyterm'] = collect_keyterms(TEXTS)\n",
        "counter = count_words(new_df, 'keyterm')\n",
        "counter.most_common(topK)\n",
        "\"\"\""
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "QmzkADmS1gfl",
        "outputId": "caca6d06-ce94-4ccd-9b4c-bec680716d8f"
      },
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"\\ntopK = 50\\nnew_df['keyterm'] = collect_keyterms(TEXTS)\\ncounter = count_words(new_df, 'keyterm')\\ncounter.most_common(topK)\\n\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 上面執行成功後，刪除下面三個的#符號之後，依序執行，`金門`可以換成你感興趣的其他詞"
      ],
      "metadata": {
        "id": "hWtQkX1v2Asq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#plot_topK_words(counter, topK=topK)"
      ],
      "metadata": {
        "id": "f91ddPQu168N"
      },
      "execution_count": 47,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#show_keyterm_in_counter(counter, \"金門\")"
      ],
      "metadata": {
        "id": "m3DgN2G917g5"
      },
      "execution_count": 48,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#write_counter2txt(counter, 'keyterm')"
      ],
      "metadata": {
        "id": "jO3vkeMV1-XO"
      },
      "execution_count": 49,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 依詞類過濾"
      ],
      "metadata": {
        "id": "YJVVUk8dgWMe"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 中文[Treebank的詞類標記](https://www.sketchengine.eu/chinese-penn-treebank-part-of-speech-tagset/)"
      ],
      "metadata": {
        "id": "bOxMGX5Fg5OD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "def collect_tags(texts, tag_labels):\n",
        "    all_pos = []    \n",
        "    # nlp.pipe() is more efficient than nlp()\n",
        "    for doc in nlp.pipe(texts):\n",
        "        pos_texts = [tok.text for tok in doc if tok.tag_ in tag_labels]\n",
        "        pos_texts = \" \".join(pos_texts)\n",
        "        all_pos.append(pos_texts)\n",
        "    return all_pos"
      ],
      "metadata": {
        "cellView": "form",
        "id": "COyaltLHahHB"
      },
      "execution_count": 50,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "TAG_LABELS = [\"NN\", \"NR\"]\n",
        "tag_text = collect_tags([sample_text], TAG_LABELS)\n",
        "tag_text"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OFu0Cm13a3A2",
        "outputId": "91a3a09d-2102-4312-b224-7ec6613c5833"
      },
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['盤桓 這 古稱 浯洲 上空 正當 雲層 地表 之際 行駛 海上 艘 宛若 小筏 往之處 時間 金門島 紅屋 瓦舍 屋頂 漸現 空中 彷彿 絲絲 閩 風 空中 散發出 台灣 氣氛 時光 空間 陶淵明 桃花源 心中 說 感動 雀躍 未來 即將 地方 金門 剛出 尚義 機場 薰風 或許 風獅爺 庇佑 風頭 竟風 故土 新竹 浯洲 風是 柔情 風 舒適 風 金門 夏日 蒼穹 雲 天際 映著 蒼茫 海水 大武山 天際 大樹 短綴 金門 獨特 紅土 讓 夏日 大樹 休憩 機會 或許 軍管 賜 浯島 居民 日子 金門 價值 文化 資產 原味 保存 下來 金門 王宅 建築 聞名 全台 位於 金寧 金門 大學 幢幢 風情 校舍 照片 風格 地化 風味 當場 觀看 建築 量體 外牆 黑瓦 閩式 建築 裝飾 文化 特色 校園 動人 建築 小巧 裝飾 特色 量體 氣派 綜合 教學 大樓 立面 兩 角式 處理 立面 氣勢 圖資 大樓 高台 人 聯想 聚落 海防 風格 校舍 聚落 校舍 建築師 也許 金門 建築 特色 人 後現代 全球 風潮 建築 幾何 極簡 風 屬於 建築 設計 屬於 當地 建築 特色 金門 大學 閩 風 建築 風格 校舍 人 台灣 大學 特色 台大 老校 校內 古蹟 新穎 炫麗 建築 屬於 條路 建築師 屬於 風格 風氣 量體 建築物 傳統 結構 風格 具象化 建築 扣著 大盤帽 西方 建築 覺得 北方 傳統 建築 特色 建築 參考 校舍 閩南 地區 紅磚 板瓦 建物 古代 窗櫺 立方體 數百 千年 建築 進 現代 民居 這樣 建築 中會 故事 會 屬於 我們 文化 我們 子孫 身 何處 認知到 屬於 我們 血緣']"
            ]
          },
          "metadata": {},
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "new_df['tag'] = collect_tags(TEXTS, TAG_LABELS)\n",
        "new_df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "WnczpGwghf3H",
        "outputId": "55be878e-4e9d-43f4-9d7e-7697fa2a0db9"
      },
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "          date author                                               text  \\\n",
              "0   2022-06-11    陳長慶  珍惜活在當下的每一個時光 ──《浯鄉縮影：陳長慶散文集2014~2021》自序 《浯鄉縮影》...   \n",
              "1   2022-05-18    魏千宸  首次訪金，首次穿梭特色金大校園 首次乘著飛機盤桓在這座古稱浯洲的上空，正當飛機下了雲層，地表...   \n",
              "2   2022-05-18    洪春柳  認識自我 有取有捨 ──訪金中輔導老師李瓊芳談生活小道 輔導的精義：在一段時間裡，陪伴一個人...   \n",
              "3   2022-05-06    陳為學  金嶝親橋一線牽 金門大學前院長陳益源教授於2018年4月29日，在金門日報的「浯江夜話」發表...   \n",
              "4   2022-03-29    黃克全  知人論世和以意逆志——談楊樹清、陳慶瀚與吳鈞堯 楊樹清竟是一個不折不扣的阜內思。我閱讀波赫士...   \n",
              "..         ...    ...                                                ...   \n",
              "95  2015-05-21    陳麗玉  話說后宅洋樓 洋樓的滄桑與興修_x000D_\\n  「王金城洋樓」於民國21年興建完工，當時...   \n",
              "96  2015-05-09    蔡宜庭  DOBGFamily 對我們身邊很多人來說，金門離島亂遠的，到底是國內還是國外傻傻分不清楚，...   \n",
              "97  2015-05-08    李金振  眾志成城共創金大奇蹟（下） 五、從落榜生到榜首_x000D_\\n辦學的最終目的是得天下英才而...   \n",
              "98  2015-05-08    莊舒惠  建立和諧的人際關係──非暴力溝通 在我們的日常生活中難免會與他人發生爭執，歸究其原因，也許是...   \n",
              "99  2015-05-07    李金振  眾志成城共創金大奇蹟（上） 1997年，教育部長吳京院士徵詢交待我一個不平凡的使命，為金門籌...   \n",
              "\n",
              "                                           clean_text  \\\n",
              "0   珍惜 活在 當下 個 時光 ─ ─ 浯鄉 縮影 陳 長慶 散文集 自序 浯鄉 縮影 從 年 ...   \n",
              "1   首 次 訪金 首 次 穿梭 特色 金大 校園 首 次 乘著 飛機 盤桓 這 座 古稱 浯洲 ...   \n",
              "2   認識 自我 有取 有捨 ─ ─ 訪金 中 輔導 老師 李 瓊芳 談 生活 小道 輔導 精義 ...   \n",
              "3   金 嶝 親 橋 一線 牽 金門 大學 前院長 陳益源 教授 於 年 月 日 金門 日報 浯江...   \n",
              "4   知人 論世 以意 逆志 談 楊 樹清 陳慶 瀚與 吳 鈞堯 楊 樹清 竟是 個 不折不扣 的...   \n",
              "..                                                ...   \n",
              "95  話 說 后宅 洋樓 洋樓 滄桑 與 興修 x000D 王 金城 洋樓 於 民國 年 興建 完...   \n",
              "96  DOBGFamily 對 我們 身邊 人來 說 金門 離島 亂遠 國內 還是 國外 傻傻 分...   \n",
              "97  眾志成城 共創 金大 奇蹟 從 落榜生 榜首 x000D 辦學 最終 目的 天下 英才 教育...   \n",
              "98  建立 和諧 人際 關係 ─ ─ 非暴力 溝通 我們 日常 生活 中 難免 會 與 發生 爭執...   \n",
              "99  眾志成城 共創 金大 奇蹟 年 教育部長 吳京 院士 徵詢 交待 個 平凡 使命 為 金門 ...   \n",
              "\n",
              "                                              keyterm  \\\n",
              "0   胡志明市人文社科大學文學系 道範顏馨五桂聯芳 陳長慶短篇小說集 五位教授 凡塵悲歌 島嶼天青...   \n",
              "1      北方傳統 千年 台灣 大樓 建築師 金門 閩式 衝突 台大 大武山 王宅 閩 一秒秒 十八間   \n",
              "2   金門大學閩南研究所 金大研究助理 金中輔導室偏重 金中輔導室 慈橋社是高中 培育出 年齡層 ...   \n",
              "3   二話不說 今天 今日 傍晚 夏 張小燕 稟報 來自大嶝島 今年三月初 西元1989年張 過世...   \n",
              "4   高雄義守大學 蜻蜓哲學家 過程中心論 工程系 幾乎 悲劇性 楊 現代性 邏輯性 立竿見影 小...   \n",
              "..                                                ...   \n",
              "95  承澤祖德基業 三間 九十七 二樓 十一人 十五日 天錫借 日軍 明年 曾祖母 王仁 王金 過...   \n",
              "96                 蔡厝古道 古戰場 台灣海峽 明天 海邊 金沙 沒關係 西門町 一 十   \n",
              "97  金門分部爭取獨立 凌晨二點 南京大學 四川大學 親愛的同仁 2010 中央 八年 台灣 國際...   \n",
              "98      金門大學 今天 會 沙發 衝突 Rosenberg 盧森堡 靜下心 一 二 第一步 第二步   \n",
              "99  金門縣政府 台大研究所 為了金門 徵詢交待 二舍 今天 十棟 夏 天大 教育部 清晨 為什麼...   \n",
              "\n",
              "                                                  tag  \n",
              "0   時光 浯鄉 縮影 長慶 散文集 自序 縮影 陸續 報刊 散文 作品 少數 心血 結晶 腦力 ...  \n",
              "1   訪金 穿梭 特色 金大 校園 盤桓 這 古稱 浯洲 上空 正當 雲層 地表 之際 行駛 海上...  \n",
              "2   認識 訪金 老師 李 生活 輔導 精義 人 人 認識 認識 能力 極限 李 金中 校園 情感...  \n",
              "3   金 橋 牽 金門 大學 陳益源 教授 月 金門 日報 夜話 金門 力作 來自 兩 家書 文章...  \n",
              "4   知人 論世 逆志 談 楊 瀚與 鈞堯 楊 波赫士 小說 兩 張 臉 重疊 意思 主體 阜 內...  \n",
              "..                                                ...  \n",
              "95  話 說 后宅 洋樓 洋樓 滄桑 x000D 王 金城 洋樓 民國 當時 王金 兄弟 十一人 ...  \n",
              "96  人來 說 金門 離島 國內 還是 傻傻 部分 我們 會 台灣 本島 機會 金門 x000D ...  \n",
              "97  眾志成城 共創 從 榜首 x000D 最終 目的 天下 英才 本校 創設 單純 目的 金門 ...  \n",
              "98  和諧 人際 關係 溝通 我們 生活 難免 會 發生 原因 也許 習慣 先入為主 觀念 比較 ...  \n",
              "99  眾志成城 共創 教育部長 吳京 院士 交待 使命 金門 籌設 學府 這 吩咐 闊別 餘年 遊...  \n",
              "\n",
              "[100 rows x 6 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-dc5fa9b9-a8db-484e-bccc-6c11b3937cae\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>date</th>\n",
              "      <th>author</th>\n",
              "      <th>text</th>\n",
              "      <th>clean_text</th>\n",
              "      <th>keyterm</th>\n",
              "      <th>tag</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2022-06-11</td>\n",
              "      <td>陳長慶</td>\n",
              "      <td>珍惜活在當下的每一個時光 ──《浯鄉縮影：陳長慶散文集2014~2021》自序 《浯鄉縮影》...</td>\n",
              "      <td>珍惜 活在 當下 個 時光 ─ ─ 浯鄉 縮影 陳 長慶 散文集 自序 浯鄉 縮影 從 年 ...</td>\n",
              "      <td>胡志明市人文社科大學文學系 道範顏馨五桂聯芳 陳長慶短篇小說集 五位教授 凡塵悲歌 島嶼天青...</td>\n",
              "      <td>時光 浯鄉 縮影 長慶 散文集 自序 縮影 陸續 報刊 散文 作品 少數 心血 結晶 腦力 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2022-05-18</td>\n",
              "      <td>魏千宸</td>\n",
              "      <td>首次訪金，首次穿梭特色金大校園 首次乘著飛機盤桓在這座古稱浯洲的上空，正當飛機下了雲層，地表...</td>\n",
              "      <td>首 次 訪金 首 次 穿梭 特色 金大 校園 首 次 乘著 飛機 盤桓 這 座 古稱 浯洲 ...</td>\n",
              "      <td>北方傳統 千年 台灣 大樓 建築師 金門 閩式 衝突 台大 大武山 王宅 閩 一秒秒 十八間</td>\n",
              "      <td>訪金 穿梭 特色 金大 校園 盤桓 這 古稱 浯洲 上空 正當 雲層 地表 之際 行駛 海上...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2022-05-18</td>\n",
              "      <td>洪春柳</td>\n",
              "      <td>認識自我 有取有捨 ──訪金中輔導老師李瓊芳談生活小道 輔導的精義：在一段時間裡，陪伴一個人...</td>\n",
              "      <td>認識 自我 有取 有捨 ─ ─ 訪金 中 輔導 老師 李 瓊芳 談 生活 小道 輔導 精義 ...</td>\n",
              "      <td>金門大學閩南研究所 金大研究助理 金中輔導室偏重 金中輔導室 慈橋社是高中 培育出 年齡層 ...</td>\n",
              "      <td>認識 訪金 老師 李 生活 輔導 精義 人 人 認識 認識 能力 極限 李 金中 校園 情感...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2022-05-06</td>\n",
              "      <td>陳為學</td>\n",
              "      <td>金嶝親橋一線牽 金門大學前院長陳益源教授於2018年4月29日，在金門日報的「浯江夜話」發表...</td>\n",
              "      <td>金 嶝 親 橋 一線 牽 金門 大學 前院長 陳益源 教授 於 年 月 日 金門 日報 浯江...</td>\n",
              "      <td>二話不說 今天 今日 傍晚 夏 張小燕 稟報 來自大嶝島 今年三月初 西元1989年張 過世...</td>\n",
              "      <td>金 橋 牽 金門 大學 陳益源 教授 月 金門 日報 夜話 金門 力作 來自 兩 家書 文章...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2022-03-29</td>\n",
              "      <td>黃克全</td>\n",
              "      <td>知人論世和以意逆志——談楊樹清、陳慶瀚與吳鈞堯 楊樹清竟是一個不折不扣的阜內思。我閱讀波赫士...</td>\n",
              "      <td>知人 論世 以意 逆志 談 楊 樹清 陳慶 瀚與 吳 鈞堯 楊 樹清 竟是 個 不折不扣 的...</td>\n",
              "      <td>高雄義守大學 蜻蜓哲學家 過程中心論 工程系 幾乎 悲劇性 楊 現代性 邏輯性 立竿見影 小...</td>\n",
              "      <td>知人 論世 逆志 談 楊 瀚與 鈞堯 楊 波赫士 小說 兩 張 臉 重疊 意思 主體 阜 內...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>95</th>\n",
              "      <td>2015-05-21</td>\n",
              "      <td>陳麗玉</td>\n",
              "      <td>話說后宅洋樓 洋樓的滄桑與興修_x000D_\\n  「王金城洋樓」於民國21年興建完工，當時...</td>\n",
              "      <td>話 說 后宅 洋樓 洋樓 滄桑 與 興修 x000D 王 金城 洋樓 於 民國 年 興建 完...</td>\n",
              "      <td>承澤祖德基業 三間 九十七 二樓 十一人 十五日 天錫借 日軍 明年 曾祖母 王仁 王金 過...</td>\n",
              "      <td>話 說 后宅 洋樓 洋樓 滄桑 x000D 王 金城 洋樓 民國 當時 王金 兄弟 十一人 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>96</th>\n",
              "      <td>2015-05-09</td>\n",
              "      <td>蔡宜庭</td>\n",
              "      <td>DOBGFamily 對我們身邊很多人來說，金門離島亂遠的，到底是國內還是國外傻傻分不清楚，...</td>\n",
              "      <td>DOBGFamily 對 我們 身邊 人來 說 金門 離島 亂遠 國內 還是 國外 傻傻 分...</td>\n",
              "      <td>蔡厝古道 古戰場 台灣海峽 明天 海邊 金沙 沒關係 西門町 一 十</td>\n",
              "      <td>人來 說 金門 離島 國內 還是 傻傻 部分 我們 會 台灣 本島 機會 金門 x000D ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>97</th>\n",
              "      <td>2015-05-08</td>\n",
              "      <td>李金振</td>\n",
              "      <td>眾志成城共創金大奇蹟（下） 五、從落榜生到榜首_x000D_\\n辦學的最終目的是得天下英才而...</td>\n",
              "      <td>眾志成城 共創 金大 奇蹟 從 落榜生 榜首 x000D 辦學 最終 目的 天下 英才 教育...</td>\n",
              "      <td>金門分部爭取獨立 凌晨二點 南京大學 四川大學 親愛的同仁 2010 中央 八年 台灣 國際...</td>\n",
              "      <td>眾志成城 共創 從 榜首 x000D 最終 目的 天下 英才 本校 創設 單純 目的 金門 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>98</th>\n",
              "      <td>2015-05-08</td>\n",
              "      <td>莊舒惠</td>\n",
              "      <td>建立和諧的人際關係──非暴力溝通 在我們的日常生活中難免會與他人發生爭執，歸究其原因，也許是...</td>\n",
              "      <td>建立 和諧 人際 關係 ─ ─ 非暴力 溝通 我們 日常 生活 中 難免 會 與 發生 爭執...</td>\n",
              "      <td>金門大學 今天 會 沙發 衝突 Rosenberg 盧森堡 靜下心 一 二 第一步 第二步</td>\n",
              "      <td>和諧 人際 關係 溝通 我們 生活 難免 會 發生 原因 也許 習慣 先入為主 觀念 比較 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>99</th>\n",
              "      <td>2015-05-07</td>\n",
              "      <td>李金振</td>\n",
              "      <td>眾志成城共創金大奇蹟（上） 1997年，教育部長吳京院士徵詢交待我一個不平凡的使命，為金門籌...</td>\n",
              "      <td>眾志成城 共創 金大 奇蹟 年 教育部長 吳京 院士 徵詢 交待 個 平凡 使命 為 金門 ...</td>\n",
              "      <td>金門縣政府 台大研究所 為了金門 徵詢交待 二舍 今天 十棟 夏 天大 教育部 清晨 為什麼...</td>\n",
              "      <td>眾志成城 共創 教育部長 吳京 院士 交待 使命 金門 籌設 學府 這 吩咐 闊別 餘年 遊...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>100 rows × 6 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-dc5fa9b9-a8db-484e-bccc-6c11b3937cae')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-dc5fa9b9-a8db-484e-bccc-6c11b3937cae button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-dc5fa9b9-a8db-484e-bccc-6c11b3937cae');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 52
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "counter = count_words(new_df, 'tag')\n",
        "plot_topK_words(counter, topK=100)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 542
        },
        "id": "mVIAOYGZhvN3",
        "outputId": "40a477a6-a75b-45f5-a000-afbbbcc608ae"
      },
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script src=\"https://cdn.plot.ly/plotly-2.8.3.min.js\"></script>                <div id=\"a7a77e1a-f862-4acb-8567-4b08b14552c7\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"a7a77e1a-f862-4acb-8567-4b08b14552c7\")) {                    Plotly.newPlot(                        \"a7a77e1a-f862-4acb-8567-4b08b14552c7\",                        [{\"alignmentgroup\":\"True\",\"hovertemplate\":\"variable=count<br>word=%{x}<br>value=%{y}<extra></extra>\",\"legendgroup\":\"count\",\"marker\":{\"color\":\"#636efa\",\"pattern\":{\"shape\":\"\"}},\"name\":\"count\",\"offsetgroup\":\"count\",\"orientation\":\"v\",\"showlegend\":true,\"textposition\":\"auto\",\"x\":[\"\\u91d1\\u9580\",\"\\u9019\",\"\\u4eba\",\"x000D\",\"\\u5927\\u5b78\",\"\\u6211\\u5011\",\"\\u8aaa\",\"\\u6587\\u5b78\",\"\\u694a\",\"\\u8001\\u5e2b\",\"\\u53f0\\u7063\",\"\\u535a\\u58eb\",\"\\u5169\",\"\\u56e0\\u70ba\",\"\\u6821\\u9577\",\"\\u6708\",\"\\u6703\",\"\\u8005\",\"\\u6559\\u80b2\",\"\\u6587\\u5316\",\"\\u6c11\\u570b\",\"\\u5148\\u751f\",\"\\u6559\\u6388\",\"\\u751f\\u6d3b\",\"\\u674e\",\"\\u5730\\u65b9\",\"\\u793e\\u6703\",\"\\u5fe0\\u79ae\",\"\\u767c\\u5c55\",\"\\u570b\\u7acb\",\"\\u6545\\u4e8b\",\"\\u7236\\u89aa\",\"\\u9109\\u89aa\",\"\\u5de5\\u4f5c\",\"\\u958b\\u59cb\",\"\\u4e4b\\u5f8c\",\"\\u6bcd\\u89aa\",\"\\u5247\",\"\\u91d1\\u5927\",\"\\u6b77\\u53f2\",\"\\u4f86\",\"\\u6d3b\\u52d5\",\"\\u6587\\u7ae0\",\"\\u91d1\\u9580\\u4eba\",\"\\u53f0\\u5317\",\"\\u5efa\\u7bc9\",\"\\u4ec0\\u9ebc\",\"\\u91d1\\u9580\\u7e23\",\"\\u9084\",\"\\u7cbe\\u795e\",\"\\u4e2d\\u5fc3\",\"\\u9662\\u9577\",\"\\u4e3b\\u4efb\",\"\\u4f5c\\u5bb6\",\"\\u6642\\u9593\",\"\\u5b78\\u6821\",\"\\u5bb6\",\"\\u8207\",\"\\u751f\\u547d\",\"\\u96d6\\u7136\",\"\\u5b78\\u751f\",\"\\u4eba\\u751f\",\"\\u5e7e\",\"\\u91d1\\u57ce\",\"\\u7576\\u6642\",\"\\u7814\\u7a76\",\"\\u6b72\",\"\\u5bb6\\u9109\",\"\\u4f5c\\u54c1\",\"\\u8a18\\u61b6\",\"\\u738b\",\"\\u571f\\u5730\",\"\\u670b\\u53cb\",\"\\u65e5\\u5831\",\"\\u8521\",\"\\u653f\\u5e9c\",\"\\u9084\\u662f\",\"\\u53c3\\u52a0\",\"\\u4e8b\",\"\\u4e26\",\"\\u6700\\u5f8c\",\"\\u6a5f\\u6703\",\"\\u89ba\\u5f97\",\"\\u81fa\\u7063\",\"\\u6d0b\\u6a13\",\"\\u4eba\\u6587\",\"\\u50b3\\u7d71\",\"\\u5b69\\u5b50\",\"\\u767c\\u73fe\",\"\\u65bc\\u662f\",\"\\u9019\\u662f\",\"\\u8d8a\\u5357\",\"\\u73fe\\u5728\",\"\\u4e16\\u754c\",\"\\u5bb6\\u5ead\",\"\\u8ad6\\u6587\",\"\\u5c07\",\"\\u74b0\\u5883\",\"\\u70ba\\u4e86\",\"\\u5927\\u9678\"],\"xaxis\":\"x\",\"y\":[1534,533,524,398,389,324,259,200,200,192,154,146,139,138,137,134,126,126,120,106,106,104,102,98,96,91,86,86,85,85,85,83,83,83,80,80,79,78,78,77,76,74,73,73,72,70,70,70,69,69,67,66,66,66,65,64,63,62,62,62,61,61,61,60,59,59,59,58,56,56,56,55,54,54,54,53,53,52,51,51,50,49,49,49,49,48,48,48,48,47,47,46,46,45,44,43,42,42,42,42],\"yaxis\":\"y\",\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"geo\":{\"bgcolor\":\"white\",\"lakecolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"white\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"light\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"ternary\":{\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"word\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"value\"}},\"legend\":{\"title\":{\"text\":\"variable\"},\"tracegroupgap\":0},\"margin\":{\"t\":60},\"barmode\":\"relative\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('a7a77e1a-f862-4acb-8567-4b08b14552c7');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "show_keyterm_in_counter(counter, \"金門\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "84nxwBPcaxlc",
        "outputId": "11422d59-fc33-4736-8ea9-454e6e03a825"
      },
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "ic| term: '金門', count: 1534\n",
            "ic| term: '金門島', count: 11\n",
            "ic| term: '金門縣', count: 70\n",
            "ic| term: '金門人', count: 73\n",
            "ic| term: '金門籍', count: 4\n",
            "ic| term: '金門味', count: 1\n",
            "ic| term: '金門城', count: 2\n",
            "ic| term: '金門湖', count: 1\n",
            "ic| term: '金門鎮', count: 4\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "write_counter2txt(counter, 'tag')"
      ],
      "metadata": {
        "id": "nCuyStiimk5q"
      },
      "execution_count": 55,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 練習 ✍"
      ],
      "metadata": {
        "id": "rF-tIhnB2NDE"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 修改`topK`的數字\n",
        "- 將`TAG_LABELS`改成其他[Treebank的詞類標記](https://www.sketchengine.eu/chinese-penn-treebank-part-of-speech-tagset/)\n",
        "- 刪除頭尾的 \"\"\" 之後執行\n"
      ],
      "metadata": {
        "id": "dE46WGwt2QFL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "topK = 100\n",
        "TAG_LABELS = [\"NN\", \"NR\"]\n",
        "new_df['tag'] = collect_tags(TEXTS, TAG_LABELS)\n",
        "counter = count_words(new_df, 'tag')\n",
        "counter.most_common(topK)\n",
        "\"\"\""
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 71
        },
        "id": "LTYyKgtt2PQy",
        "outputId": "7693f2b2-409e-4b32-cbe7-bdf5c6f8f4a5"
      },
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'\\ntopK = 100\\nTAG_LABELS = [\"NN\", \"NR\"]\\nnew_df[\\'tag\\'] = collect_tags(TEXTS, TAG_LABELS)\\ncounter = count_words(new_df, \\'tag\\')\\ncounter.most_common(topK)\\n'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 上面執行成功後，刪除下面三個的#符號之後，依序執行，`金門`可以換成你感興趣的其他詞"
      ],
      "metadata": {
        "id": "PjtlT5Pp3F7l"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#plot_topK_words(counter, topK=topK)"
      ],
      "metadata": {
        "id": "c5ojrFFd3JxW"
      },
      "execution_count": 57,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#show_keyterm_in_counter(counter, \"金門\")"
      ],
      "metadata": {
        "id": "QtbFWnxp3Lxt"
      },
      "execution_count": 58,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#write_counter2txt(counter, 'tag')"
      ],
      "metadata": {
        "id": "4De5eNUs3TJm"
      },
      "execution_count": 59,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 依命名實體類別過濾"
      ],
      "metadata": {
        "id": "Lq5kE8SMqt9L"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 使用命名實體來過濾詞組\n",
        "![](https://editor.analyticsvidhya.com/uploads/19617Intro%20image.jpg)"
      ],
      "metadata": {
        "id": "2J8QEzY_fsAE"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- spaCy模型所支援的[命名實體類別](https://spacy.io/models/zh)"
      ],
      "metadata": {
        "id": "bhDKZfQpwpsl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title\n",
        "def collect_ner(texts, ner_labels):\n",
        "    all_ners = []    \n",
        "    for doc in nlp.pipe(texts, disable=[\"tok2vec\", \"tagger\", \"attribute_ruler\", \"tagger\"]):\n",
        "        ner_texts = [ent.text for ent in doc.ents if ent.label_ in ner_labels]\n",
        "        ner_texts = \" \".join(ner_texts)\n",
        "        all_ners.append(ner_texts)\n",
        "    return all_ners"
      ],
      "metadata": {
        "cellView": "form",
        "id": "0WuHbqIGk2o6"
      },
      "execution_count": 60,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "NER_LABELS = [\"EVENT\", \"FAC\", \"LOC\", \"PERSON\", \"WORK_OF_ART\"]\n",
        "ner_text = collect_ner([sample_text], NER_LABELS)\n",
        "ner_text"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z2vDThp7aCqq",
        "outputId": "00ec4b01-3149-4860-d8ef-cbbd5d72f34f"
      },
      "execution_count": 61,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['閩 金門 大武山 王宅 北方傳統']"
            ]
          },
          "metadata": {},
          "execution_count": 61
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "new_df['ner'] = collect_ner(TEXTS, NER_LABELS)\n",
        "new_df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "IuCKUkqzpsMt",
        "outputId": "6cfa9c08-855b-4d76-9273-84a998218fa2"
      },
      "execution_count": 62,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "          date author                                               text  \\\n",
              "0   2022-06-11    陳長慶  珍惜活在當下的每一個時光 ──《浯鄉縮影：陳長慶散文集2014~2021》自序 《浯鄉縮影》...   \n",
              "1   2022-05-18    魏千宸  首次訪金，首次穿梭特色金大校園 首次乘著飛機盤桓在這座古稱浯洲的上空，正當飛機下了雲層，地表...   \n",
              "2   2022-05-18    洪春柳  認識自我 有取有捨 ──訪金中輔導老師李瓊芳談生活小道 輔導的精義：在一段時間裡，陪伴一個人...   \n",
              "3   2022-05-06    陳為學  金嶝親橋一線牽 金門大學前院長陳益源教授於2018年4月29日，在金門日報的「浯江夜話」發表...   \n",
              "4   2022-03-29    黃克全  知人論世和以意逆志——談楊樹清、陳慶瀚與吳鈞堯 楊樹清竟是一個不折不扣的阜內思。我閱讀波赫士...   \n",
              "..         ...    ...                                                ...   \n",
              "95  2015-05-21    陳麗玉  話說后宅洋樓 洋樓的滄桑與興修_x000D_\\n  「王金城洋樓」於民國21年興建完工，當時...   \n",
              "96  2015-05-09    蔡宜庭  DOBGFamily 對我們身邊很多人來說，金門離島亂遠的，到底是國內還是國外傻傻分不清楚，...   \n",
              "97  2015-05-08    李金振  眾志成城共創金大奇蹟（下） 五、從落榜生到榜首_x000D_\\n辦學的最終目的是得天下英才而...   \n",
              "98  2015-05-08    莊舒惠  建立和諧的人際關係──非暴力溝通 在我們的日常生活中難免會與他人發生爭執，歸究其原因，也許是...   \n",
              "99  2015-05-07    李金振  眾志成城共創金大奇蹟（上） 1997年，教育部長吳京院士徵詢交待我一個不平凡的使命，為金門籌...   \n",
              "\n",
              "                                           clean_text  \\\n",
              "0   珍惜 活在 當下 個 時光 ─ ─ 浯鄉 縮影 陳 長慶 散文集 自序 浯鄉 縮影 從 年 ...   \n",
              "1   首 次 訪金 首 次 穿梭 特色 金大 校園 首 次 乘著 飛機 盤桓 這 座 古稱 浯洲 ...   \n",
              "2   認識 自我 有取 有捨 ─ ─ 訪金 中 輔導 老師 李 瓊芳 談 生活 小道 輔導 精義 ...   \n",
              "3   金 嶝 親 橋 一線 牽 金門 大學 前院長 陳益源 教授 於 年 月 日 金門 日報 浯江...   \n",
              "4   知人 論世 以意 逆志 談 楊 樹清 陳慶 瀚與 吳 鈞堯 楊 樹清 竟是 個 不折不扣 的...   \n",
              "..                                                ...   \n",
              "95  話 說 后宅 洋樓 洋樓 滄桑 與 興修 x000D 王 金城 洋樓 於 民國 年 興建 完...   \n",
              "96  DOBGFamily 對 我們 身邊 人來 說 金門 離島 亂遠 國內 還是 國外 傻傻 分...   \n",
              "97  眾志成城 共創 金大 奇蹟 從 落榜生 榜首 x000D 辦學 最終 目的 天下 英才 教育...   \n",
              "98  建立 和諧 人際 關係 ─ ─ 非暴力 溝通 我們 日常 生活 中 難免 會 與 發生 爭執...   \n",
              "99  眾志成城 共創 金大 奇蹟 年 教育部長 吳京 院士 徵詢 交待 個 平凡 使命 為 金門 ...   \n",
              "\n",
              "                                              keyterm  \\\n",
              "0   胡志明市人文社科大學文學系 道範顏馨五桂聯芳 陳長慶短篇小說集 五位教授 凡塵悲歌 島嶼天青...   \n",
              "1      北方傳統 千年 台灣 大樓 建築師 金門 閩式 衝突 台大 大武山 王宅 閩 一秒秒 十八間   \n",
              "2   金門大學閩南研究所 金大研究助理 金中輔導室偏重 金中輔導室 慈橋社是高中 培育出 年齡層 ...   \n",
              "3   二話不說 今天 今日 傍晚 夏 張小燕 稟報 來自大嶝島 今年三月初 西元1989年張 過世...   \n",
              "4   高雄義守大學 蜻蜓哲學家 過程中心論 工程系 幾乎 悲劇性 楊 現代性 邏輯性 立竿見影 小...   \n",
              "..                                                ...   \n",
              "95  承澤祖德基業 三間 九十七 二樓 十一人 十五日 天錫借 日軍 明年 曾祖母 王仁 王金 過...   \n",
              "96                 蔡厝古道 古戰場 台灣海峽 明天 海邊 金沙 沒關係 西門町 一 十   \n",
              "97  金門分部爭取獨立 凌晨二點 南京大學 四川大學 親愛的同仁 2010 中央 八年 台灣 國際...   \n",
              "98      金門大學 今天 會 沙發 衝突 Rosenberg 盧森堡 靜下心 一 二 第一步 第二步   \n",
              "99  金門縣政府 台大研究所 為了金門 徵詢交待 二舍 今天 十棟 夏 天大 教育部 清晨 為什麼...   \n",
              "\n",
              "                                                  tag  \\\n",
              "0   時光 浯鄉 縮影 長慶 散文集 自序 縮影 陸續 報刊 散文 作品 少數 心血 結晶 腦力 ...   \n",
              "1   訪金 穿梭 特色 金大 校園 盤桓 這 古稱 浯洲 上空 正當 雲層 地表 之際 行駛 海上...   \n",
              "2   認識 訪金 老師 李 生活 輔導 精義 人 人 認識 認識 能力 極限 李 金中 校園 情感...   \n",
              "3   金 橋 牽 金門 大學 陳益源 教授 月 金門 日報 夜話 金門 力作 來自 兩 家書 文章...   \n",
              "4   知人 論世 逆志 談 楊 瀚與 鈞堯 楊 波赫士 小說 兩 張 臉 重疊 意思 主體 阜 內...   \n",
              "..                                                ...   \n",
              "95  話 說 后宅 洋樓 洋樓 滄桑 x000D 王 金城 洋樓 民國 當時 王金 兄弟 十一人 ...   \n",
              "96  人來 說 金門 離島 國內 還是 傻傻 部分 我們 會 台灣 本島 機會 金門 x000D ...   \n",
              "97  眾志成城 共創 從 榜首 x000D 最終 目的 天下 英才 本校 創設 單純 目的 金門 ...   \n",
              "98  和諧 人際 關係 溝通 我們 生活 難免 會 發生 原因 也許 習慣 先入為主 觀念 比較 ...   \n",
              "99  眾志成城 共創 教育部長 吳京 院士 交待 使命 金門 籌設 學府 這 吩咐 闊別 餘年 遊...   \n",
              "\n",
              "                                                  ner  \n",
              "0   浯鄉縮影：陳長慶散文集 浯鄉縮影 變遷 慈母駕鶴西 黎光長 陳長慶短篇 黎光長 陳長慶短篇 ...  \n",
              "1                                    閩 金門 大武山 王宅 北方傳統  \n",
              "2   李 李 李 培育出 李 李福 李 李福 蔡 培養出 李 李 李 變遷 李 李 李 李 李 讀...  \n",
              "3                         來自大嶝島 林 夏 李榮 清樂 土生土長 張小燕 清樂  \n",
              "4       悲劇性 幾乎 楊 高雄義守大學 蜻蜓哲學家 Josef Johann 邏輯性 太人性的小鎮  \n",
              "..                                                ...  \n",
              "95  王金 王 王金 王 南洋 秀菊 王 王 南洋 變成 王仁 王仁 王仁 王仁 王仁 曾祖母 王...  \n",
              "96                                        金沙 古戰場 蔡厝古道  \n",
              "97  郭台銘 董事長 四川大學 南京大學 青黃不接 幾乎 淵源 青黃不接 無可諱言 為了 史學家 ...  \n",
              "98                            會 金門大學 Rosenberg 沙發 靜下心  \n",
              "99    夏 即早 李國鼎 夏 金門 幾乎 籌備 為什麼 朱熹 朱熹 朱 淵遠 環保署 孔廟 二舍 二舍  \n",
              "\n",
              "[100 rows x 7 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-6b17d9e7-ff0c-40bb-bfdf-3f4a54ffe0ff\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>date</th>\n",
              "      <th>author</th>\n",
              "      <th>text</th>\n",
              "      <th>clean_text</th>\n",
              "      <th>keyterm</th>\n",
              "      <th>tag</th>\n",
              "      <th>ner</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2022-06-11</td>\n",
              "      <td>陳長慶</td>\n",
              "      <td>珍惜活在當下的每一個時光 ──《浯鄉縮影：陳長慶散文集2014~2021》自序 《浯鄉縮影》...</td>\n",
              "      <td>珍惜 活在 當下 個 時光 ─ ─ 浯鄉 縮影 陳 長慶 散文集 自序 浯鄉 縮影 從 年 ...</td>\n",
              "      <td>胡志明市人文社科大學文學系 道範顏馨五桂聯芳 陳長慶短篇小說集 五位教授 凡塵悲歌 島嶼天青...</td>\n",
              "      <td>時光 浯鄉 縮影 長慶 散文集 自序 縮影 陸續 報刊 散文 作品 少數 心血 結晶 腦力 ...</td>\n",
              "      <td>浯鄉縮影：陳長慶散文集 浯鄉縮影 變遷 慈母駕鶴西 黎光長 陳長慶短篇 黎光長 陳長慶短篇 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2022-05-18</td>\n",
              "      <td>魏千宸</td>\n",
              "      <td>首次訪金，首次穿梭特色金大校園 首次乘著飛機盤桓在這座古稱浯洲的上空，正當飛機下了雲層，地表...</td>\n",
              "      <td>首 次 訪金 首 次 穿梭 特色 金大 校園 首 次 乘著 飛機 盤桓 這 座 古稱 浯洲 ...</td>\n",
              "      <td>北方傳統 千年 台灣 大樓 建築師 金門 閩式 衝突 台大 大武山 王宅 閩 一秒秒 十八間</td>\n",
              "      <td>訪金 穿梭 特色 金大 校園 盤桓 這 古稱 浯洲 上空 正當 雲層 地表 之際 行駛 海上...</td>\n",
              "      <td>閩 金門 大武山 王宅 北方傳統</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2022-05-18</td>\n",
              "      <td>洪春柳</td>\n",
              "      <td>認識自我 有取有捨 ──訪金中輔導老師李瓊芳談生活小道 輔導的精義：在一段時間裡，陪伴一個人...</td>\n",
              "      <td>認識 自我 有取 有捨 ─ ─ 訪金 中 輔導 老師 李 瓊芳 談 生活 小道 輔導 精義 ...</td>\n",
              "      <td>金門大學閩南研究所 金大研究助理 金中輔導室偏重 金中輔導室 慈橋社是高中 培育出 年齡層 ...</td>\n",
              "      <td>認識 訪金 老師 李 生活 輔導 精義 人 人 認識 認識 能力 極限 李 金中 校園 情感...</td>\n",
              "      <td>李 李 李 培育出 李 李福 李 李福 蔡 培養出 李 李 李 變遷 李 李 李 李 李 讀...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2022-05-06</td>\n",
              "      <td>陳為學</td>\n",
              "      <td>金嶝親橋一線牽 金門大學前院長陳益源教授於2018年4月29日，在金門日報的「浯江夜話」發表...</td>\n",
              "      <td>金 嶝 親 橋 一線 牽 金門 大學 前院長 陳益源 教授 於 年 月 日 金門 日報 浯江...</td>\n",
              "      <td>二話不說 今天 今日 傍晚 夏 張小燕 稟報 來自大嶝島 今年三月初 西元1989年張 過世...</td>\n",
              "      <td>金 橋 牽 金門 大學 陳益源 教授 月 金門 日報 夜話 金門 力作 來自 兩 家書 文章...</td>\n",
              "      <td>來自大嶝島 林 夏 李榮 清樂 土生土長 張小燕 清樂</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2022-03-29</td>\n",
              "      <td>黃克全</td>\n",
              "      <td>知人論世和以意逆志——談楊樹清、陳慶瀚與吳鈞堯 楊樹清竟是一個不折不扣的阜內思。我閱讀波赫士...</td>\n",
              "      <td>知人 論世 以意 逆志 談 楊 樹清 陳慶 瀚與 吳 鈞堯 楊 樹清 竟是 個 不折不扣 的...</td>\n",
              "      <td>高雄義守大學 蜻蜓哲學家 過程中心論 工程系 幾乎 悲劇性 楊 現代性 邏輯性 立竿見影 小...</td>\n",
              "      <td>知人 論世 逆志 談 楊 瀚與 鈞堯 楊 波赫士 小說 兩 張 臉 重疊 意思 主體 阜 內...</td>\n",
              "      <td>悲劇性 幾乎 楊 高雄義守大學 蜻蜓哲學家 Josef Johann 邏輯性 太人性的小鎮</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>95</th>\n",
              "      <td>2015-05-21</td>\n",
              "      <td>陳麗玉</td>\n",
              "      <td>話說后宅洋樓 洋樓的滄桑與興修_x000D_\\n  「王金城洋樓」於民國21年興建完工，當時...</td>\n",
              "      <td>話 說 后宅 洋樓 洋樓 滄桑 與 興修 x000D 王 金城 洋樓 於 民國 年 興建 完...</td>\n",
              "      <td>承澤祖德基業 三間 九十七 二樓 十一人 十五日 天錫借 日軍 明年 曾祖母 王仁 王金 過...</td>\n",
              "      <td>話 說 后宅 洋樓 洋樓 滄桑 x000D 王 金城 洋樓 民國 當時 王金 兄弟 十一人 ...</td>\n",
              "      <td>王金 王 王金 王 南洋 秀菊 王 王 南洋 變成 王仁 王仁 王仁 王仁 王仁 曾祖母 王...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>96</th>\n",
              "      <td>2015-05-09</td>\n",
              "      <td>蔡宜庭</td>\n",
              "      <td>DOBGFamily 對我們身邊很多人來說，金門離島亂遠的，到底是國內還是國外傻傻分不清楚，...</td>\n",
              "      <td>DOBGFamily 對 我們 身邊 人來 說 金門 離島 亂遠 國內 還是 國外 傻傻 分...</td>\n",
              "      <td>蔡厝古道 古戰場 台灣海峽 明天 海邊 金沙 沒關係 西門町 一 十</td>\n",
              "      <td>人來 說 金門 離島 國內 還是 傻傻 部分 我們 會 台灣 本島 機會 金門 x000D ...</td>\n",
              "      <td>金沙 古戰場 蔡厝古道</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>97</th>\n",
              "      <td>2015-05-08</td>\n",
              "      <td>李金振</td>\n",
              "      <td>眾志成城共創金大奇蹟（下） 五、從落榜生到榜首_x000D_\\n辦學的最終目的是得天下英才而...</td>\n",
              "      <td>眾志成城 共創 金大 奇蹟 從 落榜生 榜首 x000D 辦學 最終 目的 天下 英才 教育...</td>\n",
              "      <td>金門分部爭取獨立 凌晨二點 南京大學 四川大學 親愛的同仁 2010 中央 八年 台灣 國際...</td>\n",
              "      <td>眾志成城 共創 從 榜首 x000D 最終 目的 天下 英才 本校 創設 單純 目的 金門 ...</td>\n",
              "      <td>郭台銘 董事長 四川大學 南京大學 青黃不接 幾乎 淵源 青黃不接 無可諱言 為了 史學家 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>98</th>\n",
              "      <td>2015-05-08</td>\n",
              "      <td>莊舒惠</td>\n",
              "      <td>建立和諧的人際關係──非暴力溝通 在我們的日常生活中難免會與他人發生爭執，歸究其原因，也許是...</td>\n",
              "      <td>建立 和諧 人際 關係 ─ ─ 非暴力 溝通 我們 日常 生活 中 難免 會 與 發生 爭執...</td>\n",
              "      <td>金門大學 今天 會 沙發 衝突 Rosenberg 盧森堡 靜下心 一 二 第一步 第二步</td>\n",
              "      <td>和諧 人際 關係 溝通 我們 生活 難免 會 發生 原因 也許 習慣 先入為主 觀念 比較 ...</td>\n",
              "      <td>會 金門大學 Rosenberg 沙發 靜下心</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>99</th>\n",
              "      <td>2015-05-07</td>\n",
              "      <td>李金振</td>\n",
              "      <td>眾志成城共創金大奇蹟（上） 1997年，教育部長吳京院士徵詢交待我一個不平凡的使命，為金門籌...</td>\n",
              "      <td>眾志成城 共創 金大 奇蹟 年 教育部長 吳京 院士 徵詢 交待 個 平凡 使命 為 金門 ...</td>\n",
              "      <td>金門縣政府 台大研究所 為了金門 徵詢交待 二舍 今天 十棟 夏 天大 教育部 清晨 為什麼...</td>\n",
              "      <td>眾志成城 共創 教育部長 吳京 院士 交待 使命 金門 籌設 學府 這 吩咐 闊別 餘年 遊...</td>\n",
              "      <td>夏 即早 李國鼎 夏 金門 幾乎 籌備 為什麼 朱熹 朱熹 朱 淵遠 環保署 孔廟 二舍 二舍</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>100 rows × 7 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6b17d9e7-ff0c-40bb-bfdf-3f4a54ffe0ff')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-6b17d9e7-ff0c-40bb-bfdf-3f4a54ffe0ff button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-6b17d9e7-ff0c-40bb-bfdf-3f4a54ffe0ff');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 62
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "counter = count_words(new_df, 'ner')\n",
        "plot_topK_words(counter, topK=100)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 542
        },
        "id": "8oHdS6uiqPLd",
        "outputId": "54dc92c9-d798-4e95-9d86-f97b89a27ffc"
      },
      "execution_count": 63,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script src=\"https://cdn.plot.ly/plotly-2.8.3.min.js\"></script>                <div id=\"57e4d3db-a84b-4c13-b3bc-7cd43c61a689\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"57e4d3db-a84b-4c13-b3bc-7cd43c61a689\")) {                    Plotly.newPlot(                        \"57e4d3db-a84b-4c13-b3bc-7cd43c61a689\",                        [{\"alignmentgroup\":\"True\",\"hovertemplate\":\"variable=count<br>word=%{x}<br>value=%{y}<extra></extra>\",\"legendgroup\":\"count\",\"marker\":{\"color\":\"#636efa\",\"pattern\":{\"shape\":\"\"}},\"name\":\"count\",\"offsetgroup\":\"count\",\"orientation\":\"v\",\"showlegend\":true,\"textposition\":\"auto\",\"x\":[\"\\u91d1\\u9580\",\"\\u674e\",\"\\u70ba\\u4e86\",\"\\u8521\",\"\\u738b\",\"\\u674e\\u91d1\",\"\\u8463\\u4e8b\\u9577\",\"\\u694a\",\"\\u6797\",\"\\u859b\\u627f\\u6cf0\",\"\\u2500\",\"\\u5357\\u6d0b\",\"\\u9673\\u9577\\u6176\\u77ed\\u7bc7\",\"\\u9675\\u6c34\\u6e56\",\"\\u843d\\u756a\",\"\\u5357\\u5c71\",\"\\u5b54\\u5b50\",\"\\u6797\\u7fbd\",\"\\u6731\\u71b9\",\"\\u5e6b\\u5fd9\",\"\\u738b\\u4ec1\",\"Quemoy\",\"\\u4f5c\\u54c1\\u5c55\",\"\\u5716\\u66f8\\u9928\",\"\\u90ed\",\"\\u738b\\u570b\",\"\\u606c\\u5eb5\\u907a\\u7a3f\",\"\\u8b8a\\u9077\",\"\\u5e7e\\u4e4e\",\"\\u6d2a\",\"\\u963f\\u4f2f\",\"\\u8b8a\\u6210\",\"\\u9ece\\u5149\\u9577\",\"\\u96e2\\u958b\",\"\\u674e\\u70b7\\u70fd\",\"\\u4f55\\u570b\\u5091\",\"\\u9675\\u6c34\\u8a8d\\u771f\\u60c5\",\"\\u95a9\\u5357\\u8a9e\",\"\\u6703\",\"\\u9ad8\\u7956\\u7236\",\"\\u5468\",\"\\u4e59\\u672c\",\"\\u4e2d\\u570b\",\"\\u984f\",\"\\u6c11\\u570b\",\"\\u96fb\\u5f71\",\"\\u738b\\u570b\\u57fa\",\"\\u57f7\\u884c\\u9577\",\"\\u6a21\\u7bc4\\u8857\",\"\\u904e\\u5f80\",\"\\u590f\",\"\\u6731\\u5b50\",\"\\u793e\\u5340\\u5831\",\"\\u96dc\\u8ca8\\u5e97\",\"\\u738b\\u632f\\u6f22\",\"\\u8b80\\u8005\",\"\\u962e\",\"\\u6797\\u91d1\\u751f\",\"\\u5927\\u6a13\",\"\\u674e\\u5b50\\u6046\",\"\\u77f3\\u66c9\\u6953\",\"\\u6587\\u5b78\\u53f2\",\"\\u90ed\\u53f0\\u9298\",\"\\u738b\\u59d3\",\"\\u738b\\u91d1\",\"\\u7236\\u89aa\\u7684\\u9ad8\\u7cb1\\u9152\",\"\\u6cc9\\u6e90\",\"\\u5d14\",\"\\u7368\\u7acb\",\"\\u592a\\u5e73\\u6d0b\",\"\\u5929\\u5802\\u4e4b\\u8def\",\"\\u6e21\",\"\\u5c0f\\u8a18\\u8005\\u7368\\u767d\",\"\\u7368\\u4e00\\u7121\\u4e8c\",\"\\u96fb\\u8996\\u53f0\",\"\\u738b\\u5148\",\"\\u5931\\u53bb\\u7684\\u6625\\u5929\",\"\\u6c76\\u6e90\\u5bae\",\"\\u6c5f\\u5357\",\"\\u50d1\\u5c45\\u5730\",\"\\u806f\\u7d61\",\"\\u8b80\\u5230\",\"\\u7c4c\\u5099\\u6703\",\"\\u6975\\u5176\",\"\\u99ac\\u7e3d\\u7d71\",\"\\u4e2d\\u5c71\\u5927\\u5b78\",\"\\u5510\\u654f\\u9054\",\"\\u5357\\u6d77\",\"\\u6148\\u6e56\",\"\\u4eca\\u65e5\\u4e4b\",\"\\u7109\",\"\\u5343\\u91cc\",\"\\u4faf\\u5b5d\\u8ce2\",\"\\u9ece\",\"\\u91d1\\u57ce\\u9152\\u5ee0\",\"\\u5510\",\"\\u91d1\\u57ce\\u93ae\",\"\\u5c48\\u539f\",\"\\u91d1\\u6e56\\u93ae\",\"\\u738b\\u91d1\\u5e73\"],\"xaxis\":\"x\",\"y\":[97,85,70,53,47,28,26,24,16,15,14,13,12,12,12,11,11,11,10,10,10,9,8,8,8,8,8,7,7,7,7,7,6,6,6,6,6,6,5,5,5,5,5,5,5,5,5,5,5,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3],\"yaxis\":\"y\",\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"geo\":{\"bgcolor\":\"white\",\"lakecolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"white\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"light\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"ternary\":{\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"word\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"value\"}},\"legend\":{\"title\":{\"text\":\"variable\"},\"tracegroupgap\":0},\"margin\":{\"t\":60},\"barmode\":\"relative\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('57e4d3db-a84b-4c13-b3bc-7cd43c61a689');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "show_keyterm_in_counter(counter, \"金門\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VZpoVR_0ZuCn",
        "outputId": "c1cea5e4-cb78-423d-e05e-4a79d192174a"
      },
      "execution_count": 64,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "ic| term: '金門', count: 97\n",
            "ic| term: '金門成立中心', count: 1\n",
            "ic| term: '海上仙洲金門', count: 1\n",
            "ic| term: '敬悼金門', count: 1\n",
            "ic| term: '金門縣志', count: 2\n",
            "ic| term: '金門生命線', count: 1\n",
            "ic| term: '浯江衍派～金門', count: 2\n",
            "ic| term: '把北風留給金門', count: 1\n",
            "ic| term: '金門學人聯誼會', count: 1\n",
            "ic| term: '金門人東向與南向', count: 1\n",
            "ic| term: '金門縣', count: 2\n",
            "ic| term: '君自故鄉來：一些金門人', count: 1\n",
            "ic| term: '金門海峽', count: 1\n",
            "ic| term: '「顯影」中金門', count: 1\n",
            "ic| term: '世界金門日', count: 2\n",
            "ic| term: '金門期間曾念過幾年', count: 1\n",
            "ic| term: '金門大學舉行丹斯里', count: 1\n",
            "ic| term: '南方沿海金門', count: 1\n",
            "ic| term: '金門鎮', count: 2\n",
            "ic| term: '金門大學理工', count: 1\n",
            "ic| term: '清金門鎮', count: 1\n",
            "ic| term: '金門顏厝', count: 1\n",
            "ic| term: '金門縣生命線', count: 1\n",
            "ic| term: '飲酒金門行', count: 1\n",
            "ic| term: '金門西園', count: 1\n",
            "ic| term: '金門大學', count: 1\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "write_counter2txt(counter, 'ner')"
      ],
      "metadata": {
        "id": "LHwllyP4delQ"
      },
      "execution_count": 65,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 練習 ✍"
      ],
      "metadata": {
        "id": "u-QHYm-V31Dg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 修改`topK`的數字\n",
        "- 將`NER_LABELS`改成其他spaCy模型所支援的[命名實體類別](https://spacy.io/models/zh)\n",
        "- 刪除頭尾的 \"\"\" 之後執行"
      ],
      "metadata": {
        "id": "92c_Q_xV39B0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "topK = 100\n",
        "NER_LABELS = [\"EVENT\", \"FAC\", \"LOC\", \"PERSON\", \"WORK_OF_ART\"]\n",
        "new_df['ner'] = collect_ner(TEXTS, NER_LABELS)\n",
        "counter = count_words(new_df, 'ner')\n",
        "counter.most_common(topK)\n",
        "\"\"\""
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 71
        },
        "id": "DtOXnnnu34Ac",
        "outputId": "59ccc953-80d2-48b6-c378-bd5507f7cfad"
      },
      "execution_count": 66,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'\\ntopK = 100\\nNER_LABELS = [\"EVENT\", \"FAC\", \"LOC\", \"PERSON\", \"WORK_OF_ART\"]\\nnew_df[\\'ner\\'] = collect_ner(TEXTS, NER_LABELS)\\ncounter = count_words(new_df, \\'ner\\')\\ncounter.most_common(topK)\\n'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 66
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 上面執行成功後，刪除下面三個的#符號之後，依序執行，`金門`可以換成你感興趣的其他詞"
      ],
      "metadata": {
        "id": "49Jr0pUL4_f4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#plot_topK_words(counter, topK=topK)"
      ],
      "metadata": {
        "id": "jnt2g-hj5AUN"
      },
      "execution_count": 67,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#show_keyterm_in_counter(counter, \"金門\")"
      ],
      "metadata": {
        "id": "DsebRGr05Csw"
      },
      "execution_count": 68,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#write_counter2txt(counter, 'tag')"
      ],
      "metadata": {
        "id": "K0n0yHYv5GFb"
      },
      "execution_count": 69,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "- 全部執行成功後，比較四個`txt`詞表內的詞頻差異"
      ],
      "metadata": {
        "id": "cwAXC05S5Jzt"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 結語"
      ],
      "metadata": {
        "id": "9Wn0XqROQ6eI"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "![](https://www.azquotes.com/picture-quotes/quote-the-secret-of-getting-ahead-is-getting-started-the-secret-of-getting-started-is-breaking-mark-twain-39-93-88.jpg)"
      ],
      "metadata": {
        "id": "DQ_9EmJiQ8Bs"
      }
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.5"
    },
    "colab": {
      "provenance": [],
      "include_colab_link": true
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}