intro2lt_assignment_2.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "intro2lt_assignment_2.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/TurkuNLP/intro-to-nlp/blob/master/intro2lt_assignment_2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h3>Introduction</h3>\n",
        "\n",
        "This notebook is the template for **project assignment 2**. Please base your submission on this template. Further information on submission is at the end of the notebook.\n",
        "\n",
        "In this assignment, we analyze the dataset created by our annotations from last week. There are three exercises in this assignment:\n",
        "\n",
        "*   Reading in the dataset through a Python reader\n",
        "*   Looking at the basic statistics (e.g. number of examples, label distribution) of the dataset\n",
        "*   Obtaining the vocabulary size and the word frequency list (top n words)\n",
        "*   Annotation agreement (extra task, not mandatory)\n",
        "\n",
        "We provide textual guidelines for the tasks and your task is to turn these instructions into code. In many cases, there are many ways a task can be achieved. Our tips and instructions are for beginning coders, and if you know what you are doing, you can choose whatever methods that you are comfortable with. Please ask for help in case you run into issues.\n",
        "\n",
        "### Contributions\n",
        "\n",
        "In case the assignment is completed as **group work**, list all students and their constributios here.\n",
        "\n",
        "* Names:\n",
        "* Contributions: "
      ],
      "metadata": {
        "id": "3mvdzdpexOZu"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h3>Data reading</h3> \n",
        "\n",
        "(1) **Download the data**: The annotated targeted sentiment dataset can be downloaded from the GitHub repository [`TurkuNLP/sentiment-target-corpus`](https://github.com/TurkuNLP/sentiment-target-corpus). To download the data, you can use the terminal command `wget <url>`, where `url` is the address of the **raw** file you want to download (Go to the Git repo > click the file > click raw). In the Colab environment, prepend `!` to a terminal command. After downloanding, take a look at the data. It should have five columns separated by tabs and look like:\n",
        "\n",
        "```\n",
        "#ID  text    majority    agreement   annotations\n",
        "comments2015d-45552-1.1     <TARGET>Kuopionlahden</TARGET> vierasvenelaituri (kuten nimi oikeasti on) on mielestämme tosi kiva.    positive    0.6 positive,neither,positive,neither,positive\n",
        "comments2014a-469816-3.1    Käytetyssä <TARGET>Meganessa</TARGET> yksi etu verrattuna vaikkapa Skoda Octaviaan on se että Meganen voi ostaa jonkun vuoden uudempana. positive    1.0 positive,positive,positive,positive,positive\n",
        "comments2007d-182055-6.1    Monet heistä ovat olleet kymmeniä vuosia <TARGET>Finnairin</TARGET> palveluksessa ja nyt tulee kenkää.  negative    0.8 negative,negative,negative,neither,negative\n",
        "...\n",
        "```\n",
        "\n",
        "(2) **Load the data**: load the data into the environment using Python. You can, for example, read the data line by line, and split on tabs (recommended for beginners). Remember to exclude the header line for your further analysis. While in this exercise we provide clean text data, which do not include tabs and new lines inside the text field, it's safe to rely on these. In case you have more complicated text data (e.g. data includes escaped tabs and new lines), a reader (e.g. the [csv reader module](https://docs.python.org/3/library/csv.html)) is a better option.\n",
        "\n",
        "(3) **Verify that the data has been loaded**: Print out a small number of examples to verify that the data has been loaded successfully and is in the format you expect it to be.\n",
        "\n"
      ],
      "metadata": {
        "id": "DNZ3yCmG1z7s"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "69NPuN5huvDW"
      },
      "outputs": [],
      "source": [
        "# Your code here"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h3>Basic statistics</h3>\n",
        "\n",
        "Explore the dataset. For example\n",
        "\n",
        "*   How many examples are there? (tip: use `len()`)\n",
        "*   What are the labels and how often do the labels occur? (tip: use [`Counter`](https://docs.python.org/3/library/collections.html#collections.Counter))\n",
        "    * `majority` column is a majority vote over all individual annotations and it can be considered as the main label for the example. `annotations` column lists all labels annotated by individual annotators, and these can be used for example to study annotation agreement."
      ],
      "metadata": {
        "id": "y7-Xr5gXE4MW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Your code here"
      ],
      "metadata": {
        "id": "jhtd9awU3ES5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h3>Vocabulary size and word frequency list</h3>\n",
        "\n",
        "*   How big is the vocabulary?\n",
        "*   What are the most frequent words?\n",
        "\n",
        "For this task, the text will have to be tokenized. Please refer to the code in the video Text Segmentation from Topic 1. You can either use Regex or UDPipe. In the material of the upcoming week, we will cover the use of sklearn vectorizer, which has in-built tokenization so that it does not have to be done separately."
      ],
      "metadata": {
        "id": "-YFHxBLVFOfs"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Your code here"
      ],
      "metadata": {
        "id": "Zd-99kCXByi-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h3>Annotation agreement (extra task)</h3>\n",
        "Based on the labels in the file, how many examples were annotated more than once? How many of them have perfect agreement?"
      ],
      "metadata": {
        "id": "WeSKIESKWYVY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Your code here"
      ],
      "metadata": {
        "id": "yg7tw2L5XAli"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<h3>Submission information</h3>\n",
        "\n",
        "**Please submit a pdf version of the notebook to Moodle.** You can obtain pdf on Colab through File > Print > Save to pdf. Please check that the submission is of reasonable quality. e.g. All the cells are executed and their outputs are visible. The word frequency list is truncated to only the top n words (you can decide what n will be here, as long as the submission will not result in a long document)."
      ],
      "metadata": {
        "id": "mb47GSKrEOdL"
      }
    }
  ]
}