2024_Notebook_10.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyPc0Is3nFsydg21QTW5PGM2",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/TurkuNLP/ATP_kurssi/blob/master/2024_Notebook_10.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **For loops**\n",
        "\n",
        "*   with for loops we can execute the same command on several items with just one command\n",
        "\n",
        "\n",
        "> e.g., in\n",
        "\n",
        "\n",
        "> ```\n",
        "for f in cat dog mouse ; do echo $f ; done\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "> we execute the command `echo` on all the strings `cat` `dog` `mouse` with just one command, instead of running the commands\n",
        "\n",
        "\n",
        "> `echo cat`\n",
        "\n",
        "\n",
        "> `echo dog`\n",
        "\n",
        "\n",
        "> `echo mouse`\n",
        "\n",
        "\n",
        "> separately.\n",
        "\n",
        "\n",
        "*   the item on which the command is executed in a for loop can be a file\n",
        "   \n",
        "\n",
        "*   this means that we can **execute the same command to multiple files** with one command\n",
        "*   this is practical, since with for loops we can also execute **scripts** to multiple files with just one command\n",
        "\n",
        "\n",
        "*   for for loops we need **regular expressions**\n",
        "\n",
        "**In practise**\n",
        "\n",
        "\n",
        "*   for loop begins with\n",
        "\n",
        ">  `for f`\n",
        "\n",
        "\n",
        "\n",
        "where `f` stands for the variable on which the command is executed (you can think of it as 'file': 'for file in the folder')\n",
        "\n",
        "\n",
        "*   then the folder with the files\n",
        "\n",
        "\n",
        "> all files in the folder\n",
        "\n",
        "\n",
        "> `for f in folder/*`\n",
        "\n",
        "\n",
        "> files ending with csv\n",
        "\n",
        "\n",
        "> `for f in folder/*csv`\n",
        "\n",
        "\n",
        "> files starting with myfi\n",
        "\n",
        "\n",
        "\n",
        "> `for f in folder/myfi*`\n",
        "\n",
        "\n",
        "\n",
        "*   OR if you are in the folder already\n",
        "\n",
        "\n",
        "> all files in the folder\n",
        "\n",
        "\n",
        "> `for f in *`\n",
        "\n",
        "\n",
        "> all files ending with .txt\n",
        "\n",
        "\n",
        "> `for f in *.txt`\n",
        "\n",
        "\n",
        "*   commands that are executed for each file are put between `;`\n",
        "*   the command starts with `do`\n",
        "*   the `$f` refers to the file the command is executed on\n",
        "*   the loop ends with `done`\n",
        "\n",
        "\n",
        "\n",
        "> `for f in folder/* ; do echo $f ; done`\n",
        "\n",
        "*   there can be anything between `;` (pipes, scripts, ...)\n",
        "\n",
        "\n",
        "\n",
        "> `for f in folder/* ; do cat $f | egrep \"snow\" ; done`\n",
        "\n",
        "\n",
        "> `for f in folder/* ; do cat $f | ./script.sh ; done`\n",
        "\n",
        "\n",
        "**Directing output in a file**\n",
        "\n",
        "\n",
        "\n",
        "*   the output can be directed into a file\n",
        "\n",
        "\n",
        "\n",
        "> in the end (all output in **one file**)\n",
        "\n",
        "\n",
        "\n",
        "> `for f in folder/* ; do cat $f | egrep \"snow\" ; done > output.txt`\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "> within the command (each output in **separate files**)\n",
        "\n",
        "\n",
        "\n",
        "> `for f in folder/* ; do cat $f | ./script.sh > $f-output.txt ; done`\n",
        "\n",
        "\n",
        "**Several commands with for loop**\n",
        "\n",
        "*   use several `;` if you wish to execute several commands\n",
        "*   e.g., here print file names and lines with \"snow\" in the files\n",
        "\n",
        "\n",
        "\n",
        "> `for f in folder/* ; do echo $f ; cat $ | egrep \"snow\" ; done`\n",
        "\n",
        "\n",
        "**Examples**\n",
        "\n",
        "*   print **file names and word count**\n",
        "\n",
        "```\n",
        "for f in folder/*txt ; do echo $f ; cat $f | wc -w ; done\n",
        "\n",
        "```\n",
        "\n",
        "*   print **word count only**\n",
        "\n",
        "```\n",
        "for f in folder/*txt ; do cat $f | wc -w ; done\n",
        "\n",
        "```\n",
        "\n",
        "*   print **number of file names only**\n",
        "\n",
        "```\n",
        "for f in folder/*txt ; do echo $f | wc -w ; done\n",
        "\n",
        "```\n",
        "\n",
        "*   print **file names only**\n",
        "\n",
        "```\n",
        "for f in folder/*txt ; do echo $f ; done\n",
        "\n",
        "```\n",
        "\n",
        "**NOTE**\n",
        "\n",
        "\n",
        "\n",
        "*   back reference also uses `$`, but works as usual within a for loop\n",
        "\n",
        "\n",
        "\n",
        "```\n",
        "for f in *.txt ; do echo $f ; cat $f | perl -pe 's/([[:punct:]])/\\n$1/g' ; done\n",
        "```\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "*   for all files ending in `.txt`\n",
        "\n",
        "\n",
        "1.   print the file name\n",
        "2.   print the file and replace all punctuations with a new line and the punctuation in question (each punctuation is put on a new line)\n",
        "\n",
        "\n",
        "*   for loops can be used for looping also other items than just files, and the item after `for` (e.g., above: `for f`) doesn't have to be `f`; remember though that it has to match the reference next to `$` (so if you use `for n` make sure you use `$n`)\n",
        "\n",
        "\n",
        "\n",
        "```\n",
        "for n in cat dog mouse ; do echo $n ; done\n",
        "```\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "---------\n",
        "\n",
        "**Exercise 1**\n",
        "\n",
        "Practise using for loops.\n",
        "\n",
        "\n",
        "\n",
        "1.   Print all file names in a folder using for loop.\n",
        "2.   Print the file names and word count of each file in a folder.\n",
        "3.   Grep and print all perl scripts in your .sh files in a folder. Print also the .sh file names.\n",
        "\n",
        "\n",
        "\n",
        "**Exercise 2**\n",
        "\n",
        "More for loops.\n",
        "\n",
        "1.   Run one of your scripts from Notebook9 on two similar files (meaning that the script works equally well on both files) in your folder using a for loop. Print the names and the output in one file.\n",
        "2.   What's the difference if you execute the for loop with `do echo $f` or without it?\n",
        "3.   Copy the files cricket.csv and icehockey.csv from /home/mavela/data-2022/\n",
        "4.   Make a script that counts the words in a file. Exclude numbers and punctuation. Run the script on the cricket and ice hockey files using for loops and direct the output into separate files.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "YqVmG0E_dx-0"
      }
    }
  ]
}