Move eg task folder (#87)

* Add example-tasks from desktop

* Update example tasks docs

* Update docs

README.md

@@ -23,15 +23,18 @@ https://www.figma.com/file/JkjN5hBQYfCFEMpc2QS4eH/MECAnywhere-Architecture-Diagr
 
 - did-go
     - Decentalized Identity (DID) service written in Golang
-- Tower
-    - Hosted on the edge to provide task forwarding for end users
-    - [README](tower/README.md)
+- example-tasks
+    - Example tasks for the MECAnywhere platform
+    - [README to build tasks](example-tasks/README.md)
 - Pymeca actors
     - Template actors that use pymeca library to interact with the smart contracts. These come in the form of CLI.
     - [README to setup](pymeca-actors/README.md)
     - User guides in docs
+- Tower
+    - Hosted on the edge to provide task forwarding for end users
+    - [README to setup](tower/README.md)
 
-# Quick Start (local)
+# Quick Start (local development)
 1. Configure keys/env variables (see [Configuration -> Secret keys](#secret-keys))
 
 2. Pull submodules with `git submodule update --init --recursive` in `pymeca` folder.

docs/source/pages/user_guides/example_tasks.rst

@@ -0,0 +1,12 @@
+.. _example-tasks:
+
+Building tasks
+==============
+
+This guide will walk you through the development of a task container that a task developer can upload.
+Example tasks are provided in `example tasks README <https://github.com/sbip-sg/mec_anywhere/tree/main/example-tasks>`__.
+Each folder must contain all of the files listed in the structure below.
+
+.. include:: ../../../../example-tasks/README.md
+    :parser: myst_parser.sphinx_
+    :start-after: ## Structure

docs/source/pages/user_guides/guides.rst

@@ -8,3 +8,4 @@ User Guides
    host_guide
    task_dev_guide
    tower_guide
+   example_tasks

docs/source/pages/user_guides/task_dev_guide.rst

@@ -16,19 +16,18 @@ Introduction
 The task developer is responsible for creating tasks that can be
 executed by hosts. This guide will walk you through the process of
 setting up and managing a task developer using the CLI. To see how to
-develop tasks, check out
-https://github.com/sbip-sg/mec_anywhere_desktop/tree/main/sdk
+develop tasks, check out :ref:`example-tasks`.
 
 Installation
 ------------
 
 1. To install the task developer, you will need to have Docker installed
    on your machine. You can download Docker from the `official Docker
    website <https://www.docker.com/get-started>`__.
-2. Start and serve the task developer container. (see pymeca-actors
-   readme)
-3. Ensure that you have a valid connection to IPFS for uploading (see
-   https://docs.ipfs.tech/install/). You can also use the docker image
+2. Start and serve the task developer container. (see `pymeca-actors
+   readme <https://github.com/sbip-sg/mec_anywhere/blob/main/pymeca-actors/README.md>`__)
+3. Ensure that you have a valid connection to IPFS for uploading. (see
+   `IPFS installation docs <https://docs.ipfs.tech/install/>`__) You can also use the docker image
    from IPFS following instructions in pymeca-actors readme.
 
 Usage

example-tasks/.gitignore

@@ -0,0 +1,136 @@
+/out/
+output.png
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+tmp/
+.idea
+result.png

example-tasks/README.md

@@ -0,0 +1,104 @@
+# Example task containers
+
+This folder contains example task containers showing how to build your own task container. 
+
+## Examples
+
+Folder | Description
+--- | ---
+/python-template | A simple Python task container that returns the json input as output.
+/knn | A task container that uses the K-Nearest Neighbors algorithm to classify data.
+/stablediffusion | A task container that uses a prompt to generate an image.
+/sgx-task | A basic SGX container that runs code in a Trusted Execution Environment.
+
+
+## Structure
+
+Each folder must contain all of the files listed in the structure as follows:
+
+```
+- folder
+  - Dockerfile
+  - src
+  - description.txt
+  - name.txt
+  - example_input.bin
+  - example_output.bin
+  - config.json
+
+```
+
+## Building a task container
+
+### 1. App that handles requests
+
+The app will receive a post request at `localhost:8080\` and return the result. The input format is any json object and the output format is a string. After the app starts up, print `"meca-init-done"`.
+
+For example, we create a flask app:
+
+```python
+  from flask import Flask, request
+  import json
+
+  app = Flask(__name__)
+
+  @app.route('/', methods=['POST'])
+  def hello_world():
+      data = request.json
+      return data['input']
+
+  print("meca-init-done")
+```
+
+### 2. Dockerfile
+
+Expose port 8080 to the app of the request handler.
+
+Following the flask app created above, the Dockerfile may look like this:
+
+```dockerfile
+FROM python:3.9-slim
+
+WORKDIR /app
+
+COPY flask_app.py flask_app.py
+
+EXPOSE 8080
+
+CMD ["python", "-m", "flask", "--app", "flask_app.py", "run", "--host=0.0.0.0", "--port=8080"]
+```
+
+### 3. config.json fields
+
+The `config.json` file contains the configuration of the task, which it will use to build its container, with the following default fields:
+
+DEFAULTS:
+
+```json
+{
+  "resource": {
+    "cpu": 1,
+    "mem": 128,
+    "use_gpu": false, 
+    "gpu_count": 0
+  }
+}
+```
+
+field | description
+--- | ---
+`resource.cpu` | The number of CPUs to allocate to the task. This field will limit the users that can run the task to those with the same or more CPUs available.
+`resource.mem` | The amount of memory to allocate to the task in MB. This field will limit the users that can run the task to those with the same or more memory available.
+`resource.use_gpu` | Whether the task requires a GPU to run.
+`resource.gpu_count` | The number of GPUs to allocate to the task, if `use_gpu` is true.
+
+### 4. Describe your task
+
+Fill in the `description.txt` and `name.txt` files with the description and name of your task.
+In the `example_input.bin` and `example_output.bin` files, provide examples of the input and output of your task exactly as they are read in the body of a HTTP request and reply.
+
+### 5. Using and uploading the container
+
+Build the image and push it to IPFS via the MECA CLI as a task developer. You will be compensated with the task fee that you list for each task executed by a MECA client.
+
+Test your task folder structure by running the test function in the MECA CLI.

example-tasks/knn/Dockerfile

@@ -0,0 +1,14 @@
+FROM python:3.9-slim
+
+WORKDIR /app
+
+COPY /src/requirements.txt requirements.txt
+
+RUN pip install torch==2.0.1+cpu --index-url https://download.pytorch.org/whl/cpu
+RUN pip install -r requirements.txt
+
+COPY /src/app.py app.py
+
+EXPOSE 8080
+
+CMD ["python", "-m", "flask", "--app", "app.py", "run", "--host=0.0.0.0", "--port=8080"]

example-tasks/knn/config.json

@@ -0,0 +1,6 @@
+{
+  "resource": {
+    "cpu": 2,
+    "mem": 1024
+  }
+}

example-tasks/knn/description.txt

@@ -0,0 +1 @@
+This app serves HTTP requests to run K-nearest neighbors algorithm on a chunked dataset in a distributed fashion.

example-tasks/knn/example_input.bin

@@ -0,0 +1 @@
+{"dataset":[[1,2],[3,4],[5,6],[7,8],[9,10]],"point":[2,3],"k":2,"num_processes":2}

example-tasks/knn/example_output.bin

@@ -0,0 +1 @@
+[0, 1]

example-tasks/knn/name.txt

@@ -0,0 +1 @@
+knn