2024_Notebook_6_extra_parsing.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/TurkuNLP/ATP_kurssi/blob/master/2024_Notebook_6_extra_parsing.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Parsing with Trankit\n",
        "\n",
        "Let's try to produce conllu data by using a parser! Here we test parsing with Trankit which uses UD annotations. Trankit is available at https://trankit.readthedocs.io/\n",
        "\n",
        "For more information on Trankit, see the link.\n",
        "\n",
        "Note that we need Python to use Trankit. Therefore, this is optional, but if you are interested and/or have some extra time, feel free to try!\n"
      ],
      "metadata": {
        "id": "kDDtytgVnwW9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#N.B. This may take some time\n",
        "\n",
        "! pip install trankit"
      ],
      "metadata": {
        "id": "pjTQnn9doSR1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "! git clone https://github.com/nlp-uoregon/trankit\n",
        "%cd trankit\n",
        "! pip install -e ."
      ],
      "metadata": {
        "id": "C60sOKuKoWQ5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "! pip install trankit==1.1.0\n",
        "! pip install transformers"
      ],
      "metadata": {
        "id": "OTShhh5EoaTE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from trankit import Pipeline\n",
        "from trankit import trankit2conllu\n",
        "\n",
        "p = Pipeline('english') # other languages possible as well!"
      ],
      "metadata": {
        "id": "hzvUD10Youp1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# a non-empty string to process, which can be a document or a paragraph with multiple sentences\n",
        "#within the ''' ''' comes the text you want to parse\n",
        "\n",
        "doc_text = '''Hello! This is Trankit.'''\n",
        "\n",
        "# perform all tasks on the input\n",
        "all = p(doc_text)\n",
        "all_conllu = trankit2conllu(all)\n",
        "print(all_conllu)"
      ],
      "metadata": {
        "id": "waAWrrF8o-eP"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We also provide a script that can parse from standard input. Let's try it!"
      ],
      "metadata": {
        "id": "qXZQDY-x-18H"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "! git clone https://github.com/TurkuNLP/ATP_kurssi.git"
      ],
      "metadata": {
        "id": "BeuyR1DbvDzN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "! echo \"This is Trankit!\" | python3 ATP_kurssi/scripts/parse.py"
      ],
      "metadata": {
        "id": "7Kv3V84Dx1_Q"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}