009_Summing-Up.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://www.hydroffice.org/epom/\"><img src=\"images/000_000_epom_logo.png\" alt=\"ePOM\" title=\"Open ePOM home page\" align=\"center\" width=\"12%\" alt=\"Python logo\\\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://piazza.com/class/js5dnu0q39n6qe\"><img src=\"images/help.png\" alt=\"ePOM\" title=\"Ask questions on Piazza.com\" align=\"right\" width=\"10%\" alt=\"Piazza.com\\\"></a>\n",
    "# Summing-Up"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the last notebook of this collection. We will not introduce significant new concepts, but we will apply what has been discussed in the past notebooks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will do this by creating a class that holds the data and the functions for reading and writing data in a format that is more complex than previously encountered."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the text content of the `ctd.txt` file in the `data` folder:\n",
    "\n",
    "![ctd_txt](images/010_000_ctd_txt.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see in the above image, the first four rows contain some metadata describing when and where the data were collected."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Starting from the fifth row, the file has a structure of four columns, with observations of depth, sound speed, temperature, and salinity."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In ocean mapping, it is common to collect multiple oceanographic measurements using a [CTD instrument](https://en.wikipedia.org/wiki/CTD_(instrument)). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Class Creation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As done in the [A Class as a Data Container notebook](008_A_Class_as_a_Data_Container.ipynb), we will first create a class with the `init(self)` special method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "class CTDData:\n",
    "    \"\"\"A class for CTD data\"\"\"\n",
    "    \n",
    "    def __init__(self):\n",
    "        self.metadata = dict()        \n",
    "        self.depth_values = list()\n",
    "        self.ss_values = list()        \n",
    "        self.temp_values = list()\n",
    "        self.sal_values = list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above class is richer in attributes than the previous ones that we created. We need to accommodate the metadata and the four columns representing different types of observations. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Path Retrieval"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will retrieve the full path to the `ctd.txt` file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_data_paths():\n",
    "    data_paths = list()\n",
    "    cur_folder = os.path.abspath(os.path.curdir)\n",
    "    data_folder = os.path.join(cur_folder, \"data\")\n",
    "    data_filenames = os.listdir(data_folder)\n",
    "    \n",
    "    for data_filename in data_filenames:\n",
    "        data_path = os.path.join(data_folder, data_filename)\n",
    "        data_paths.append(data_path)\n",
    "    \n",
    "    data_paths.sort()  # sort in alphabetical order\n",
    "    \n",
    "    return data_paths\n",
    "\n",
    "retrieved_paths = get_data_paths()\n",
    "input_path = retrieved_paths[0]  # 0 because is the first file in our data folder\n",
    "print(\"input path: \" + input_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a Function to Read the CTD File"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly to what we did in the [Read a Text File section](006_Read_and_Write_Text_Files.ipynb#Read-a-Text-File), we will define a function to read the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def read_ctd_data(path, data):\n",
    "    # check whether the passed file does not exist\n",
    "    if not os.path.exists(path):\n",
    "        raise RuntimeError(\"Unable to locate %s\" % (path, ))\n",
    "\n",
    "    # read the file content\n",
    "    ctd_file = open(path)\n",
    "    ctd_content = ctd_file.read()\n",
    "    ctd_file.close()\n",
    "\n",
    "    ctd_lines = ctd_content.splitlines()\n",
    "    count = 0  # initialize the counter for the number of rows read \n",
    "    for ctd_line in ctd_lines:\n",
    "\n",
    "        if count < 4: # metadata \n",
    "            meta_pair = ctd_line.split()  # 'tokenize' the string\n",
    "            data.metadata[meta_pair[0]] = meta_pair[1]  # use the tokens as keys and values\n",
    "\n",
    "        else:  # measures\n",
    "            measures = ctd_line.split()\n",
    "            data.depth_values.append(float(measures[0]))\n",
    "            data.ss_values.append(float(measures[1]))\n",
    "            data.temp_values.append(float(measures[2]))\n",
    "            data.sal_values.append(float(measures[3]))\n",
    "\n",
    "        count += 1  # the result is equal to writing: count = count + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above code we use the [`str.split()`](https://docs.python.org/3.6/library/stdtypes.html?#str.split) method. \n",
    "\n",
    "This method returns a list of the words in a string by splitting it using a delimiter (e.g., `\":\"`) passed as a parameter. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "time_str = \"14:02:39\"\n",
    "time_list = time_str.split(\":\")\n",
    "print(\"The resulting list after splitting time_str is: %s\" % (time_list, ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In case that a delimiter parameter is **not** specified (as we did for the measures section of the code), the following splitting algorithm is applied: *\"runs of consecutive whitespace are regarded as a single separator, and the result will contain no empty strings at the start or end if the string has leading or trailing whitespace.\"* This sentence comes from the official Python documentation! We will leave the interpretation to you... To help you, look at the example below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample_str = \"0.003  1501.09    3.7610       25.0900\"\n",
    "sample_list = sample_str.split()\n",
    "print(\"The resulting list after splitting sample_str is: %s\" % (sample_list, ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading the Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is now time to create an instance (**instantiate**) our `CTDData` class and to call our `read` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ctd_data = CTDData()\n",
    "read_ctd_data(input_path, ctd_data)\n",
    "print(\"The metadata are: %s\" % (ctd_data.metadata, ))\n",
    "print(\"Nr. of samples: %s\" % (len(ctd_data.depth_values), ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data are now **loaded in memory**. \n",
    "\n",
    "We can check the success of this operation by printing depths and sound speed values. We will do this by accessing the values by index with the help of the [`range()`](https://docs.python.org/3.6/library/stdtypes.html?#range) type.\n",
    "\n",
    "A `range()` with an integer value as single parameter represents a sequence of numbers ranging from 0 up to (but not including) the value passed as a parameter. In the code below, we use `range` with `10`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for value in range(10):\n",
    "    print(\"Current range value: %s\" % (value, ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus, we can use `range()` with the number of loaded samples to print all the values in the `depth_values` and `ss_values` lists preceded by the corresponding index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nr_of_samples = len(ctd_data.depth_values)\n",
    "\n",
    "for index in range(nr_of_samples):\n",
    "    print(\"%s %.3f %.2f\" % (index, ctd_data.depth_values[index], ctd_data.ss_values[index]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It worked! We have been able to read a complex file format."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img align=\"left\" width=\"6%\" style=\"padding-right:10px; padding-top:10px;\" src=\"images/refs.png\">\n",
    "\n",
    "## Useful References"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* [The official Python 3.6 documentation](https://docs.python.org/3.6/index.html)\n",
    "* [CTD instrument](https://en.wikipedia.org/wiki/CTD_(instrument))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img align=\"left\" width=\"5%\" style=\"padding-right:10px;\" src=\"images/email.png\">\n",
    "\n",
    "*For issues or suggestions related to this notebook, write to: epom@ccom.unh.edu*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--NAVIGATION-->\n",
    "[< A Class as a Data Container](008_A_Class_as_a_Data_Container.ipynb) | [Contents](index.ipynb) | [Congrats >](congrats.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}