TBFY_Crosslingual_Example.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "TBFY-Crosslingual-Example.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Otbiu9WVd7Pf",
        "colab_type": "text"
      },
      "source": [
        "# How to explore contracts in the European Union using TBFY Tools"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2eIjzGeCeQzJ",
        "colab_type": "text"
      },
      "source": [
        "Our goal is to find **organizations** related to a particular **procurement**. This procurement is described by **free text**, and the organizations we are looking for are those that have participated in **tenders** related to that text."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DD_iyPRYWIXu",
        "colab_type": "code",
        "outputId": "b293ecf7-5d09-4848-eac3-91c41a12774e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "source": [
        "procurement_text = \"Lappalto ha per oggetto la fornitura di apparecchiature conformi alla normativa ECAC Standard 3 con velocita di movimento del nastro interno di almeno 05 msec.  per la rilevazione automatica di esplosivi EDS nei bagagli da stiva dei passeggeri in transito presso gli Aeroporti Milano Linate e Milano Malpensa.  Rientrano altresi nell'oggetto dell'appalto l'installazione e la manutenzione per un periodo di 10 dieci anni a decorrere dalla data di emissione del certificato di verifica di conformita delle apparecchiature fornite.\"\n",
        "print (\"procurement text: \", procurement_text)"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "procurement text:  Lappalto ha per oggetto la fornitura di apparecchiature conformi alla normativa ECAC Standard 3 con velocita di movimento del nastro interno di almeno 05 msec.  per la rilevazione automatica di esplosivi EDS nei bagagli da stiva dei passeggeri in transito presso gli Aeroporti Milano Linate e Milano Malpensa.  Rientrano altresi nell'oggetto dell'appalto l'installazione e la manutenzione per un periodo di 10 dieci anni a decorrere dalla data di emissione del certificato di verifica di conformita delle apparecchiature fornite.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x8xwxMhSfnC0",
        "colab_type": "text"
      },
      "source": [
        "Below we will see how we can address this problem using TBFY's Search-API and KG-API services:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wxW9_risfqlh",
        "colab_type": "text"
      },
      "source": [
        "## Search for related texts across languages"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Aw0bQG5xfxMn",
        "colab_type": "text"
      },
      "source": [
        "We can use the [Search API](https://github.com/TBFY/search-API) service to search for tenders related to that procurement text:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LrPegcXii3Ic",
        "colab_type": "code",
        "outputId": "bc93fab3-f18e-4323-f2f4-7098bdc5ef2e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 207
        }
      },
      "source": [
        "import requests\n",
        "\n",
        "def retrieve_documents(request):\n",
        "  base_url = 'https://tbfy.librairy.linkeddata.es/search-api'\n",
        "  resp = requests.post(base_url+'/items', json=request)\n",
        "  if resp.status_code != 200:\n",
        "    # This means something went wrong.\n",
        "    print('POST /items/ {}'.format(resp.status_code))\n",
        "    pass\n",
        "  return resp.json()    \n",
        "  \n",
        "\n",
        "def search_documents(request):\n",
        "  i=1\n",
        "  for document in retrieve_documents(request):\n",
        "    print('[{}] {}'.format(i, document))\n",
        "    i+=1  \n",
        "\n",
        "request = {\"source\": \"tender\", \"size\": \"10\", \"text\" : procurement_text }\n",
        "search_documents(request)"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] {'id': 'ocds-0c46vo-0133-029814-2019_029814-2019_td', 'name': 'Procedura negoziata per laffidamento, attraverso la stipula di accordo quadro, della fornitura e manutenzione di apparecchiature per il controllo radiogeno dei bagagli da stiva', 'score': 2546.525390625}\n",
            "[2] {'id': 'ocds-0c46vo-0020-llaol_331_llaol_331', 'name': 'L19031-T-DS Metal Fabrication Framework for supply of goods and services for London Luton Airport', 'score': 2537.8173828125}\n",
            "[3] {'id': 'ocds-0c46vo-0133-053934-2019_053934-2019_td', 'name': 'Automated Tray Return System, Central Image Processing and Directed Search Equipment Framework', 'score': 2467.199462890625}\n",
            "[4] {'id': 'ocds-0c46vo-0133-016465-2019_016465-2019_td', 'name': 'Servicios de operacin de aeronaves', 'score': 2467.199462890625}\n",
            "[5] {'id': 'ocds-0c46vo-0133-025225-2019_025225-2019_td', 'name': 'Provision of Airspace Redesign for Manchester, East Midlands and Stansted Airports', 'score': 2467.199462890625}\n",
            "[6] {'id': 'ocds-0c46vo-0133-036773-2019_036773-2019_td', 'name': 'Supporto industriale di articoli aeronautici vari imbarcati sugli aeromobili T-339 ed TH/OH-500', 'score': 2467.199462890625}\n",
            "[7] {'id': 'ocds-0c46vo-0133-059850-2019_059850-2019_td', 'name': '20182035: Repuestos de sistemas de seguridad de las aeronaves del Ejrcito del Aire', 'score': 2467.199462890625}\n",
            "[8] {'id': 'ocds-0c46vo-0102-CSL_2019_dr_i7sUCyp_CSL_2019_dr_i7sUCyp', 'name': 'Maintenance aronautique sur moteurs Pratt et Whitney et accessoires associs.', 'score': 2464.411376953125}\n",
            "[9] {'id': 'ocds-0c46vo-0133-037160-2019_037160-2019_td', 'name': 'Suministro de hasta 5 autobuses de 15 metros para prestar servicio en la lnea aeropuerto', 'score': 2464.411376953125}\n",
            "[10] {'id': 'ocds-0c46vo-0133-037359-2019_037359-2019_td', 'name': 'Boat Tender Outer Namdal', 'score': 2464.411376953125}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UZsMKLalkdd5",
        "colab_type": "text"
      },
      "source": [
        "The above search was tuned to retrieve documents describing tenders (i.e `source = tender`, that are related to a given text (i.e. `text='procurement_text'`), and only the top 10 were retrieved (i.e. `size=10`).\n",
        "\n",
        "Each search result contains the document identifier, the name and a numerical value to illustrate the level of relationship to the reference text (e.g. `'score': 2794.13`).\n",
        "\n",
        "The search service can be currently filtered by the following parameters: `size`, `source`, `terms`, `name` and `lang`. **All of them can be freely combined.**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f8tNmTFAmorz",
        "colab_type": "text"
      },
      "source": [
        "#### Filtering by language"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dog3IZr-mqvv",
        "colab_type": "text"
      },
      "source": [
        "The language parameter `lang` follows the [ISO 639-1 Code](https://www.iso.org/iso-639-language-codes.html). The service currently supports the following languages: English (`en`), Spanish (`es`), French(`fr`), Italian(`it`) and Portuguese(`pt`).\n",
        "\n",
        "Let's now search for related documents written in Spanish:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EPEg_80VmI_1",
        "colab_type": "code",
        "outputId": "89943d77-e1fd-4b12-8bb6-eb2f7f419c0f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 207
        }
      },
      "source": [
        "request = {\"source\": \"tender\", \"size\": \"10\", \"text\" : procurement_text, \"lang\": \"es\" }\n",
        "search_documents(request)"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] {'id': 'ocds-0c46vo-0133-016465-2019_016465-2019_td', 'name': 'Servicios de operacin de aeronaves', 'score': 2467.199462890625}\n",
            "[2] {'id': 'ocds-0c46vo-0133-059850-2019_059850-2019_td', 'name': '20182035: Repuestos de sistemas de seguridad de las aeronaves del Ejrcito del Aire', 'score': 2467.199462890625}\n",
            "[3] {'id': 'ocds-0c46vo-0133-037160-2019_037160-2019_td', 'name': 'Suministro de hasta 5 autobuses de 15 metros para prestar servicio en la lnea aeropuerto', 'score': 2464.411376953125}\n",
            "[4] {'id': 'ocds-0c46vo-0203-3041_1-2019_3041_1%2F2019', 'name': 'Ayuntamiento de Lumbier>Alcaldia del Ayuntamiento de Lumbier', 'score': 2464.411376953125}\n",
            "[5] {'id': 'ocds-0c46vo-0133-043009-2019_043009-2019_td', 'name': 'Contratacin del suministro de tubera multicapa de polipropileno para sistema de climatizacin en la obra del edificio situado en la plaza del Marqus de Salamanca, 8, de Madrid', 'score': 2464.411376953125}\n",
            "[6] {'id': 'ocds-0c46vo-0133-043863-2019_043863-2019_td', 'name': 'Servicios de transporte areo sometidos a obligaciones de servicio pblico en la ruta area Menorca-Madrid', 'score': 2464.411376953125}\n",
            "[7] {'id': 'ocds-0c46vo-0133-044728-2019_044728-2019_td', 'name': 'Pliego para la cesin de cuatro (4) locales para la actividad de restauracin en el Aeropuerto de Sevilla', 'score': 2464.411376953125}\n",
            "[8] {'id': 'ocds-0c46vo-0133-044729-2019_044729-2019_td', 'name': 'Cesin de dos (2) locales para la actividad de restauracin en el Aeropuerto de Sevilla, C/SVQ/040/18', 'score': 2464.411376953125}\n",
            "[9] {'id': 'ocds-0c46vo-0203-2018-2366_2018%2F2366', 'name': 'Entitats municipals de Catalunya>Consell Comarcal de la Selva', 'score': 2464.411376953125}\n",
            "[10] {'id': 'ocds-0c46vo-0313-1812051158245ACD4050_1812051158245ACD4050', 'name': 'Servicio de gestin de 20 plazas mixtas de proteccin en recursos habitacionales a menores extranjeros no acompaados y 5 plazas mixtas de valoracin y acogida para menores extranjeros no acompaados', 'score': 2464.411376953125}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fsmWBWi1oP1j",
        "colab_type": "text"
      },
      "source": [
        "#### Filtering by source"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xaRhxzkjoTTM",
        "colab_type": "text"
      },
      "source": [
        "Currently, both the tender descriptions retrieved from the TBFY Knowledge-Graph (i.e. `source=tender`) and the legal texts published in the JRC-Acquis dataset (i.e. `source=jrc`) are available through the web service . \n",
        "\n",
        "If the `source` parameter is missing, no filtering is done:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0NnDwetWpNAy",
        "colab_type": "code",
        "outputId": "57c6fefe-d364-44b0-c25e-1d95477864e9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 207
        }
      },
      "source": [
        "request = {\"size\": \"10\", \"text\" : procurement_text }\n",
        "search_documents(request)"
      ],
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] {'id': 'ocds-0c46vo-0133-029814-2019_029814-2019_td', 'name': 'Procedura negoziata per laffidamento, attraverso la stipula di accordo quadro, della fornitura e manutenzione di apparecchiature per il controllo radiogeno dei bagagli da stiva', 'score': 2546.525390625}\n",
            "[2] {'id': 'ocds-0c46vo-0020-llaol_331_llaol_331', 'name': 'L19031-T-DS Metal Fabrication Framework for supply of goods and services for London Luton Airport', 'score': 2537.8173828125}\n",
            "[3] {'id': 'jrc32004R0261-it', 'name': \"Regolamento (CE) n. 261/2004 del Parlamento europeo e del Consiglio, dell'11 febbraio 2004, che istituisce regole comuni in materia di compensazione ed assistenza ai passeggeri in caso di negato imbarco, di cancellazione del volo o di ritardo prolungato e che abroga il regolamento (CEE) n. 295/91 (Testo rilevante ai fini del SEE) - Dichiarazione della Commissione\", 'score': 2473.11962890625}\n",
            "[4] {'id': 'ocds-0c46vo-0133-053934-2019_053934-2019_td', 'name': 'Automated Tray Return System, Central Image Processing and Directed Search Equipment Framework', 'score': 2467.199462890625}\n",
            "[5] {'id': 'ocds-0c46vo-0133-016465-2019_016465-2019_td', 'name': 'Servicios de operacin de aeronaves', 'score': 2467.199462890625}\n",
            "[6] {'id': 'ocds-0c46vo-0133-025225-2019_025225-2019_td', 'name': 'Provision of Airspace Redesign for Manchester, East Midlands and Stansted Airports', 'score': 2467.199462890625}\n",
            "[7] {'id': 'ocds-0c46vo-0133-036773-2019_036773-2019_td', 'name': 'Supporto industriale di articoli aeronautici vari imbarcati sugli aeromobili T-339 ed TH/OH-500', 'score': 2467.199462890625}\n",
            "[8] {'id': 'ocds-0c46vo-0133-059850-2019_059850-2019_td', 'name': '20182035: Repuestos de sistemas de seguridad de las aeronaves del Ejrcito del Aire', 'score': 2467.199462890625}\n",
            "[9] {'id': 'jrc52006XC0808_01-en', 'name': 'Commission communication pursuant to Article 4(1)(a) of Council Regulation (EEC) 2408/92 — Modification by the United Kingdom of public service obligations in respect of scheduled air services between Glasgow-Campbeltown, Glasgow-Tiree and Glasgow-Barra (Glasgow Airport means Glasgow International Airport)   (Text with EEA relevance)', 'score': 2464.411376953125}\n",
            "[10] {'id': 'jrc52006XC0628_02-en', 'name': 'Commission notice pursuant to Article 4(1)(a) of Council Regulation (EEC) No 2408/92 — Imposition of a public service obligation in respect of scheduled air services within Italy   (Text with EEA relevance)', 'score': 2464.411376953125}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m_zaY4h9rHPM",
        "colab_type": "text"
      },
      "source": [
        "We can search through tenders (i.e. `source=tender`):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4vKxiQ9ErT-k",
        "colab_type": "code",
        "outputId": "971f7adb-dd3d-4d66-fc34-48691d638473",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 207
        }
      },
      "source": [
        "request = {\"source\":\"tender\", \"size\": \"10\", \"text\" : procurement_text }\n",
        "search_documents(request)"
      ],
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] {'id': 'ocds-0c46vo-0133-029814-2019_029814-2019_td', 'name': 'Procedura negoziata per laffidamento, attraverso la stipula di accordo quadro, della fornitura e manutenzione di apparecchiature per il controllo radiogeno dei bagagli da stiva', 'score': 2546.525390625}\n",
            "[2] {'id': 'ocds-0c46vo-0020-llaol_331_llaol_331', 'name': 'L19031-T-DS Metal Fabrication Framework for supply of goods and services for London Luton Airport', 'score': 2537.8173828125}\n",
            "[3] {'id': 'ocds-0c46vo-0133-053934-2019_053934-2019_td', 'name': 'Automated Tray Return System, Central Image Processing and Directed Search Equipment Framework', 'score': 2467.199462890625}\n",
            "[4] {'id': 'ocds-0c46vo-0133-016465-2019_016465-2019_td', 'name': 'Servicios de operacin de aeronaves', 'score': 2467.199462890625}\n",
            "[5] {'id': 'ocds-0c46vo-0133-025225-2019_025225-2019_td', 'name': 'Provision of Airspace Redesign for Manchester, East Midlands and Stansted Airports', 'score': 2467.199462890625}\n",
            "[6] {'id': 'ocds-0c46vo-0133-036773-2019_036773-2019_td', 'name': 'Supporto industriale di articoli aeronautici vari imbarcati sugli aeromobili T-339 ed TH/OH-500', 'score': 2467.199462890625}\n",
            "[7] {'id': 'ocds-0c46vo-0133-059850-2019_059850-2019_td', 'name': '20182035: Repuestos de sistemas de seguridad de las aeronaves del Ejrcito del Aire', 'score': 2467.199462890625}\n",
            "[8] {'id': 'ocds-0c46vo-0102-CSL_2019_dr_i7sUCyp_CSL_2019_dr_i7sUCyp', 'name': 'Maintenance aronautique sur moteurs Pratt et Whitney et accessoires associs.', 'score': 2464.411376953125}\n",
            "[9] {'id': 'ocds-0c46vo-0133-037160-2019_037160-2019_td', 'name': 'Suministro de hasta 5 autobuses de 15 metros para prestar servicio en la lnea aeropuerto', 'score': 2464.411376953125}\n",
            "[10] {'id': 'ocds-0c46vo-0133-037359-2019_037359-2019_td', 'name': 'Boat Tender Outer Namdal', 'score': 2464.411376953125}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "haEv_FLArYdk",
        "colab_type": "text"
      },
      "source": [
        "Or among European Union legal texts (i.e `source=jrc`):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F56HIIAyrg64",
        "colab_type": "code",
        "outputId": "2ae7b25c-2aa8-43b6-8b92-67a7376189c4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 207
        }
      },
      "source": [
        "request = {\"source\":\"jrc\", \"size\": \"10\", \"text\" : procurement_text }\n",
        "search_documents(request)"
      ],
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] {'id': 'jrc32004R0261-it', 'name': \"Regolamento (CE) n. 261/2004 del Parlamento europeo e del Consiglio, dell'11 febbraio 2004, che istituisce regole comuni in materia di compensazione ed assistenza ai passeggeri in caso di negato imbarco, di cancellazione del volo o di ritardo prolungato e che abroga il regolamento (CEE) n. 295/91 (Testo rilevante ai fini del SEE) - Dichiarazione della Commissione\", 'score': 2473.11962890625}\n",
            "[2] {'id': 'jrc52006XC0808_01-en', 'name': 'Commission communication pursuant to Article 4(1)(a) of Council Regulation (EEC) 2408/92 — Modification by the United Kingdom of public service obligations in respect of scheduled air services between Glasgow-Campbeltown, Glasgow-Tiree and Glasgow-Barra (Glasgow Airport means Glasgow International Airport)   (Text with EEA relevance)', 'score': 2464.411376953125}\n",
            "[3] {'id': 'jrc52006XC0628_02-en', 'name': 'Commission notice pursuant to Article 4(1)(a) of Council Regulation (EEC) No 2408/92 — Imposition of a public service obligation in respect of scheduled air services within Italy   (Text with EEA relevance)', 'score': 2464.411376953125}\n",
            "[4] {'id': 'jrc52006XC0729_06-en', 'name': 'Amendment of public service obligations imposed in respect of scheduled air services on routes within Greece in accordance with Council Regulation (EEC) No 2408/92   (Text with EEA relevance)', 'score': 2464.411376953125}\n",
            "[5] {'id': 'jrc52006XC0729_04-en', 'name': 'Guidelines for the allocation of limited air traffic rights', 'score': 2464.411376953125}\n",
            "[6] {'id': 'jrc52006XC0620_02-en', 'name': 'Public service obligation   (Text with EEA relevance)', 'score': 2464.411376953125}\n",
            "[7] {'id': 'jrcC2006#294#52-en', 'name': 'Case C-396/06: Reference for a preliminary ruling from the Østre Landsret (Denmark) lodged on 21 September 2006 — Eivind F. Kramme v SAS Scandinavian Airlines Danmark A/S', 'score': 2464.411376953125}\n",
            "[8] {'id': 'jrc52006XC1114_01-en', 'name': 'Lifting by France of the public service obligations for scheduled air services between Strasbourg and Munich   (Text with EEA relevance)', 'score': 2464.411376953125}\n",
            "[9] {'id': 'jrc52006XC1214_04-en', 'name': 'Commission notice pursuant to Article 4(1)(a) of Council Regulation (EEC) No 2408/92 — Imposition of a public service obligation on scheduled air services within Italy   (Text with EEA relevance)', 'score': 2464.411376953125}\n",
            "[10] {'id': 'jrc52006XC1118_06-en', 'name': 'Commission notice pursuant to Article 4(1)(a) of Council Regulation (EEC) No 2408/92 — Amendment of public service obligations imposed on certain scheduled air services within Portugal   (Text with EEA relevance)', 'score': 2464.411376953125}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9Q3_Y47yruUk",
        "colab_type": "text"
      },
      "source": [
        "In any case, the search can also be filtered by a particular language, for example French (i.e. `lang=fr`):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Zc6gn87vr39E",
        "colab_type": "code",
        "outputId": "47473b8c-a92d-4187-ad90-5ba440a53647",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 207
        }
      },
      "source": [
        "request = {\"lang\": \"fr\", \"source\":\"jrc\", \"size\": \"10\", \"text\" : procurement_text }\n",
        "search_documents(request)"
      ],
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] {'id': 'jrc32006R0730-fr', 'name': \"Règlement (CE) n o  730/2006 de la Commission du  11 mai 2006  sur la classification de l’espace aérien et l’accès aux vols effectués selon les règles de vol à vue au-dessus du niveau de vol 195   (Texte présentant de l'intérêt pour l'EEE)\", 'score': 2464.411376953125}\n",
            "[2] {'id': 'jrc52005DC0046-fr', 'name': \"Communication de la Commission au Parlement européen et au Conseil - Renforcer les droits des passagers au sein de l'Union européenne  /* COM/2005/0046 final */\", 'score': 2464.411376953125}\n",
            "[3] {'id': 'jrc31999R1083-fr', 'name': \"Règlement (CE) n° 1083/1999 de la Commission, du 26 mai 1999, modifiant le règlement (CEE) n° 1617/93 concernant l'application de l'article 85, paragraphe 3, du traité à certaines catégories d'accords, de décisions ou de pratiques concertées ayant pour objet la planification conjointe et la coordination des horaires, l'exploitation de services en commun, les consultations tarifaires pour le transport de passagers et de fret sur les services aériens réguliers et la réparation des créneaux horaires dans les aéroports\", 'score': 2464.411376953125}\n",
            "[4] {'id': 'jrc52005AE1059-fr', 'name': \"Avis du Comité économique et social européen sur la  Proposition de règlement du Parlement européen et du Conseil concernant les droits des personnes à mobilité réduite lorsqu'elles font des voyages aériens [COM(2005)\\xa047 final — 07/2005 (COD)]\", 'score': 2464.411376953125}\n",
            "[5] {'id': 'jrc52005DC0078-fr', 'name': \"Communication de la Commission - Une politique communautaire dans le domaine de l'aviation civile à l’égard de la République populaire de Chine - Renforcer la coopération et ouvrir des marchés  /* COM/2005/0078 final */\", 'score': 2464.411376953125}\n",
            "[6] {'id': 'jrc52005DC0406-fr', 'name': 'Communication de la Commission - Resserrer les relations avec le Chili dans le domaine de l’aviation  /* COM/2005/0406 final */', 'score': 2464.411376953125}\n",
            "[7] {'id': 'jrc52005DC0407-fr', 'name': 'Communication de la Commission - Développer une politique communautaire dans le domaine de l’aviation civile à l’égard de la Nouvelle-Zélande  /* COM/2005/0407 final */', 'score': 2464.411376953125}\n",
            "[8] {'id': 'jrc52005DC0408-fr', 'name': 'Communication de la Commission - Définir une politique communautaire en matière d’aviation civile vis-à-vis de l’Australie  /* COM/2005/0408 final */', 'score': 2464.411376953125}\n",
            "[9] {'id': 'jrc52004XC1104_01-fr', 'name': 'JRC-ACQUIS 52004XC1104(01) French', 'score': 2464.411376953125}\n",
            "[10] {'id': 'jrc52004IR0063-fr', 'name': \"Avis d'initiative du Comité des régions sur «Les compagnies aériennes low cost et le développement territorial»\", 'score': 2464.411376953125}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X_vR_6tyvvDy",
        "colab_type": "text"
      },
      "source": [
        "#### Filtering by Terms\n",
        "\n",
        "You can also filter by documents containing certain words. Just add the `terms` parameter with the exact sequence of words you want the documents to contain. Please be careful with upper and lower case. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "46rRoQd6wF1i",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 139
        },
        "outputId": "82ab6b1f-c0c2-44e7-9f44-784f6ec5b243"
      },
      "source": [
        "request = { \"terms\": \"explosive\", \"source\":\"tender\", \"size\": \"10\", \"text\": procurement_text}\n",
        "search_documents(request)"
      ],
      "execution_count": 50,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] {'id': 'ocds-0c46vo-0022-FEB285089_FEB285089', 'name': 'Supply ECAC Std 3 Explosive Detection System to Inverness Airport', 'score': 2464.411376953125}\n",
            "[2] {'id': 'ocds-0c46vo-0022-FEB345537_FEB345537', 'name': 'Supply ECAC Std 3 Explosive Detection System to Inverness Airport', 'score': 2464.411376953125}\n",
            "[3] {'id': 'ocds-0c46vo-0020-iomg_679_iomg_679', 'name': 'Quick Quote - Isle of Man Airport EDS3 Enabling Works', 'score': 374.5386657714844}\n",
            "[4] {'id': 'ocds-0c46vo-0133-092864-2019_092864-2019_td', 'name': 'National Supply of Pollution Control Equipment', 'score': 73.40591430664062}\n",
            "[5] {'id': 'ocds-0c46vo-0064-19-21447_19-21447', 'name': \"Mission de matrise d'oeuvre pour la construction du dpt de bus  Romans sur Isre(26)Appel  candidature\", 'score': 2.7879698276519775}\n",
            "[6] {'id': 'ocds-0c46vo-0133-059040-2019_059040-2019_td', 'name': 'Supply of Explosive Detector to Reinforcing Aviation Security at Rafik HarirI International Airport', 'score': 2.7879698276519775}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rWo1Z-cCw3f9",
        "colab_type": "text"
      },
      "source": [
        "#### Filtering by Name"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gD6Zm2GGx97p",
        "colab_type": "text"
      },
      "source": [
        "You can also filter by document name, or part of the name. Again, be careful with upper and lower case.  Just use the `name` parameter\n",
        "\n",
        "The special character `*` expresses any word in that position. So if you want its name to start with `Security` the filter should be `name=Security*`:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0sW_Wm0-yG9k",
        "colab_type": "code",
        "outputId": "a0226de0-d417-43f3-886a-5317b04d145c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 88
        }
      },
      "source": [
        "request = { \"name\" : \"Security*\", \"source\":\"tender\", \"size\": \"10\", \"text\": procurement_text}\n",
        "search_documents(request)"
      ],
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] {'id': 'ocds-0c46vo-0133-082181-2019_082181-2019_td', 'name': 'Security Scanner Framework', 'score': 2464.411376953125}\n",
            "[2] {'id': 'ocds-0c46vo-0049-54321-Tender-1682cd8a029-7bd5593f2bf083c9_54321-Tender-1682cd8a029-7bd5593f2bf083c9', 'name': 'Securityleistungen', 'score': 120.09278106689453}\n",
            "[3] {'id': 'ocds-0c46vo-0133-007837-2019_007837-2019_td', 'name': 'Security Sector Mapping in Jordan', 'score': 120.09278106689453}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OTO71E6gsKrY",
        "colab_type": "text"
      },
      "source": [
        "But if you want it to contain `Security` in any position, the filter should be `name=*Security*`:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gK9iEVyhsUN2",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 207
        },
        "outputId": "49b38aca-9142-4000-df11-349604e3b25d"
      },
      "source": [
        "request = { \"name\" : \"*Security*\", \"source\":\"tender\", \"size\": \"10\", \"text\": procurement_text}\n",
        "search_documents(request)"
      ],
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] {'id': 'ocds-0c46vo-0133-082181-2019_082181-2019_td', 'name': 'Security Scanner Framework', 'score': 2464.411376953125}\n",
            "[2] {'id': 'ocds-0c46vo-0133-127531-2019_127531-2019_td', 'name': 'Provision of Air Traffic Services Provider and Aviation Security Liability Insurance (DCA 2/2018)', 'score': 2464.411376953125}\n",
            "[3] {'id': 'ocds-0c46vo-0009-DN387730-1_DN387730-1', 'name': 'Broadmoor Hospital Security System Maintenance Service Provision', 'score': 371.7507019042969}\n",
            "[4] {'id': 'ocds-0c46vo-0133-031957-2019_031957-2019_td', 'name': 'Scottish Enterprise  Overseas and International Travel, Accommodation and Travel Security Services', 'score': 371.7507019042969}\n",
            "[5] {'id': 'ocds-0c46vo-0049-54321-Tender-1682cd8a029-7bd5593f2bf083c9_54321-Tender-1682cd8a029-7bd5593f2bf083c9', 'name': 'Securityleistungen', 'score': 120.09278106689453}\n",
            "[6] {'id': 'ocds-0c46vo-0133-007837-2019_007837-2019_td', 'name': 'Security Sector Mapping in Jordan', 'score': 120.09278106689453}\n",
            "[7] {'id': 'ocds-0c46vo-0133-018120-2019_018120-2019_td', 'name': 'Train Cab Security Improvements', 'score': 2.7879698276519775}\n",
            "[8] {'id': 'ocds-0c46vo-0045-83488_83488', 'name': 'ITB Procurement of two perpetual Licenses: 1. Splunk Enterprise (Core) and 2. Splunk Enterprise Security (application- SIEM)', 'score': 2.7879698276519775}\n",
            "[9] {'id': 'ocds-0c46vo-0133-059040-2019_059040-2019_td', 'name': 'Supply of Explosive Detector to Reinforcing Aviation Security at Rafik HarirI International Airport', 'score': 2.7879698276519775}\n",
            "[10] {'id': 'ocds-0c46vo-0133-026862-2019_026862-2019_td', 'name': 'Grndungsmanager fr das Projekt IT-Security Startup Inkubator', 'score': 2.1330511569976807}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "itptWW0R0fKv",
        "colab_type": "text"
      },
      "source": [
        "## Alignment with the TBFY Knowledge Graph"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rN7JKxdN0qK3",
        "colab_type": "text"
      },
      "source": [
        "Once we have the list of related documents, we can directly use their `id` and `source` attributes to read the resource in the Knowledge-Graph.\n",
        "\n",
        "The document identifiers (i.e. `id`) in the Search-API are the same as the resources (i.e. `source`) in the KG-API.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uUVM8CkP1aB9",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "ab8fa758-2a73-4d72-eaa6-8833de7831e3"
      },
      "source": [
        "def retrieve_tenders(request):\n",
        "  documents = retrieve_documents(request)\n",
        "  tenders = []\n",
        "  for document in documents:\n",
        "    id = document['id']\n",
        "    tender = requests.get('https://tbfy.librairy.linkeddata.es/kg-api/tender/'+id).json()\n",
        "    tenders.append(tender)\n",
        "  return tenders\n",
        "\n",
        "def search_tenders(request):\n",
        "  tenders = retrieve_tenders(request)\n",
        "  i=1\n",
        "  for tender in tenders:\n",
        "    print('[{}] [{}] \"{} ({})\"'.format(i,tender['status'], tender['title'], tender['id']))    \n",
        "    i+=1\n",
        "\n",
        "print(\"Search for tenders in KG is ready\")"
      ],
      "execution_count": 85,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Search for tenders in KG is ready\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xLOrNF8T08pB",
        "colab_type": "text"
      },
      "source": [
        "#### Reading tender status"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s2dkF9w82W4n",
        "colab_type": "text"
      },
      "source": [
        "We can easily read the `status` of the tenders in the KG from the documents found in the Search-API:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "53lvmnZf2Y61",
        "colab_type": "code",
        "outputId": "884178df-cc86-4ddd-8340-6651d898214d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 207
        }
      },
      "source": [
        "request = { \"source\":\"tender\",\"size\": \"10\", \"text\": procurement_text}\n",
        "search_tenders(request)"
      ],
      "execution_count": 86,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] [complete] \"Procedura negoziata per laffidamento, attraverso la stipula di accordo quadro, della fornitura e manutenzione di apparecchiature per il controllo radiogeno dei bagagli da stiva (ocds-0c46vo-0133-029814-2019_029814-2019_td)\"\n",
            "[2] [active] \"L19031-T-DS Metal Fabrication Framework for supply of goods and services for London Luton Airport (ocds-0c46vo-0020-llaol_331_llaol_331)\"\n",
            "[3] [complete] \"Automated Tray Return System, Central Image Processing and Directed Search Equipment Framework (ocds-0c46vo-0133-053934-2019_053934-2019_td)\"\n",
            "[4] [planned] \"Servicios de operacin de aeronaves (ocds-0c46vo-0133-016465-2019_016465-2019_td)\"\n",
            "[5] [complete] \"Provision of Airspace Redesign for Manchester, East Midlands and Stansted Airports (ocds-0c46vo-0133-025225-2019_025225-2019_td)\"\n",
            "[6] [complete] \"Supporto industriale di articoli aeronautici vari imbarcati sugli aeromobili T-339 ed TH/OH-500 (ocds-0c46vo-0133-036773-2019_036773-2019_td)\"\n",
            "[7] [complete] \"20182035: Repuestos de sistemas de seguridad de las aeronaves del Ejrcito del Aire (ocds-0c46vo-0133-059850-2019_059850-2019_td)\"\n",
            "[8] [complete] \"Maintenance aronautique sur moteurs Pratt et Whitney et accessoires associs. (ocds-0c46vo-0102-CSL_2019_dr_i7sUCyp_CSL_2019_dr_i7sUCyp)\"\n",
            "[9] [complete] \"Suministro de hasta 5 autobuses de 15 metros para prestar servicio en la lnea aeropuerto (ocds-0c46vo-0133-037160-2019_037160-2019_td)\"\n",
            "[10] [complete] \"Boat Tender Outer Namdal (ocds-0c46vo-0133-037359-2019_037359-2019_td)\"\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cckMdwZhwFdH",
        "colab_type": "text"
      },
      "source": [
        "And add or remove the filters we've seen before:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qXf96ReEwGwi",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 207
        },
        "outputId": "fc22866d-f5e5-4a53-debb-a0f739a8417d"
      },
      "source": [
        "request = { \"lang\":\"it\", \"source\":\"tender\",\"size\": \"10\", \"text\": procurement_text}\n",
        "search_tenders(request)"
      ],
      "execution_count": 87,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] [complete] \"Procedura negoziata per laffidamento, attraverso la stipula di accordo quadro, della fornitura e manutenzione di apparecchiature per il controllo radiogeno dei bagagli da stiva (ocds-0c46vo-0133-029814-2019_029814-2019_td)\"\n",
            "[2] [complete] \"Supporto industriale di articoli aeronautici vari imbarcati sugli aeromobili T-339 ed TH/OH-500 (ocds-0c46vo-0133-036773-2019_036773-2019_td)\"\n",
            "[3] [complete] \"Servizi di vigilanza, sicurezza e controllo presso l'aeroporto Guglielmo Marconi di Bologna (ocds-0c46vo-0133-051622-2019_051622-2019_td)\"\n",
            "[4] [complete] \"Supporto logistico ai velivoli cacciabombardieri AM-X (ocds-0c46vo-0133-039182-2019_039182-2019_td)\"\n",
            "[5] [complete] \"Servizio di copertura assicurativa dei rischi aeronautici connessi al trasporto aereo di Stato, di Governo e per ragioni umanitarie per l'anno 2019 (ocds-0c46vo-0133-048329-2019_048329-2019_td)\"\n",
            "[6] [complete] \"Procedura negoziata per fornitura, installazione monitor FIDS, comprensiva del servizio di manutenzione preventiva correttiva e straordinaria e dei dispositivi periferici  aeroporti Linate e Malpensa. (ocds-0c46vo-0133-022495-2019_022495-2019_td)\"\n",
            "[7] [complete] \"Ampliamento aerostazione lotto 4A. (ocds-0c46vo-0133-010349-2019_010349-2019_td)\"\n",
            "[8] [complete] \"Procedura aperta per laffidamento in subappalto di servizi di handling c/o laeroporto di Comiso  CIG:75291757C2. La procedura sar condotta nel rispetto dei principi di economicit, di efficacia, di imparzialit, di parit di trattamento, di trasparenza e di proporzionalit nonch, altres, in ossequio agli ulteriori principi previsti dallart. 30, del D.Lgs. n. 50/2016. (ocds-0c46vo-0133-008309-2019_008309-2019_td)\"\n",
            "[9] [complete] \"Servizi ingegneria opere area nord (ocds-0c46vo-0332-6881_6881)\"\n",
            "[10] [complete] \"Realizzazione di un drone per attivit di rilievo specialistiche e manutentive (ocds-0c46vo-0332-100_100)\"\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Sb9Xt-Nf1F2U",
        "colab_type": "text"
      },
      "source": [
        "#### Getting suppliers of related tenders"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2xgK_XZk1c61",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "09701d02-2581-49ec-d627-b07c1fa01971"
      },
      "source": [
        "def get_contracting_process(tender_id):\n",
        "  resp = requests.get('https://tbfy.librairy.linkeddata.es/kg-api/tender/'+tender_id+\"/contractingProcess\")\n",
        "  if resp.status_code != 200:\n",
        "    # This means something went wrong.\n",
        "    print('GET /tender/contracting_process {}'.format(resp.status_code))\n",
        "    pass\n",
        "  return resp.json() \n",
        "\n",
        "def get_buyer(contracting_process_id):\n",
        "  resp = requests.get('https://tbfy.librairy.linkeddata.es/kg-api/contractingProcess/'+contracting_process_id+\"/buyer\")\n",
        "  if resp.status_code != 200:\n",
        "    # This means something went wrong.\n",
        "    print('GET /contracting_process {}'.format(resp.status_code))\n",
        "    pass\n",
        "  return resp.json() \n",
        "\n",
        "def search_buyers(request):\n",
        "  tenders = retrieve_tenders(request)\n",
        "  i = 1\n",
        "  for tender in tenders:\n",
        "    contracting_processes = get_contracting_process(tender['id'])\n",
        "    for contracting_process in contracting_processes:\n",
        "      buyers = get_buyer(contracting_process['id'])\n",
        "      for buyer in buyers:\n",
        "        print('[{}] {}'.format(i,buyer['legalName']))\n",
        "    i+=1\n",
        "\n",
        "print(\"auxiliar functions ready\")"
      ],
      "execution_count": 92,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "auxiliar functions ready\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EPDUczZ91KoY",
        "colab_type": "text"
      },
      "source": [
        "We will now move through the KG to identify the buyers that have participated in these tenders:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8tPaNZA42maO",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        },
        "outputId": "9cf873b8-4dac-41b6-fb6e-1395a383991b"
      },
      "source": [
        "request = { \"source\":\"tender\",\"size\": \"10\", \"text\": procurement_text}\n",
        "search_buyers(request)"
      ],
      "execution_count": 93,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] Societ per Azioni Esercizi Aeroportuali S.E.A.\n",
            "[2] London Luton Airport\n",
            "[3] Manchester Airport Plc\n",
            "[4] Servicio Murciano de Salud\n",
            "[5] Manchester Airport Plc\n",
            "[6] Ministero della Difesa  Aeronautica Militare  Ufficio generale Centro di responsabilit amministrativa\n",
            "[7] Direccin de Adquisiciones del Mando de Apoyo Logstico del Ejrcito del Aire\n",
            "[8] ENAC Ecole Nationale de l'Aviation Civile\n",
            "[9] Transportes Urbanos de Sevilla, S. A. M.\n",
            "[10] AtB AS\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b6OWS2Iz895m",
        "colab_type": "text"
      },
      "source": [
        "And we can filter through any of the filters seen above, for example language:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IGDsp_Qs9DMt",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        },
        "outputId": "3a9ec1e2-9425-4048-98df-85a0212965c0"
      },
      "source": [
        "request = { \"lang\":\"en\", \"source\":\"tender\",\"size\": \"10\", \"text\": procurement_text}\n",
        "search_buyers(request)"
      ],
      "execution_count": 94,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] London Luton Airport\n",
            "[2] Manchester Airport Plc\n",
            "[3] Manchester Airport Plc\n",
            "[4] AtB AS\n",
            "[5] Finavia Oyj\n",
            "[6] Westminster City Council\n",
            "[7] London Borough of Waltham Forest\n",
            "[8] London Borough of Lewisham\n",
            "[9] TIMR KOMMUN\n",
            "[10] Cambridgeshire County Council\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}