Skip to content

Commit

Permalink
add brief introduction and advanced configuration documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
@StrubT
StrubT committed Sep 17, 2016
1 parent a235472 commit dff94af
Show file tree
Hide file tree
Showing 5 changed files with 132 additions and 73 deletions.
96 changes: 23 additions & 73 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This is the code of the **Executor** component of the project **Progressor - The

These instructions are written for *Ubuntu* 16.04 LTS.

The goal is to install the Executor in `/opt/Executor/` and run it as a Docker container.
The goal is to install the **Executor** in `/opt/Executor/` and run it as a Docker container.

### Server Setup

Expand All @@ -33,105 +33,55 @@ The goal is to install the Executor in `/opt/Executor/` and run it as a Docker c
environment=KOTLIN_HOME="/kotlinc"
```

* If you need to override the default settings, please refer to the detailed [configuration information](doc/ExecutorConfig.md).

1. Make *Supervisor* start when the server boots: `sudo systemctl enable supervisor`
1. Start the service by executing `sudo service supervisor start`
1. Reload *Supervisor* configuration by running `sudo supervisorctl reread`
1. and `sudo supervisorctl update`

### Uploading the Executor JAR and Building the Docker Image

1. If you built the Executor jar from this repo, rename it to `ProgressorExecutor.jar` and upload it to your server in `/opt/Executor/`, otherwise download the pre-compiled one:
1. If you built the **Executor** *JAR* from this repo, rename it to `ProgressorExecutor.jar` and upload it to your server in `/opt/Executor/`, otherwise download the pre-compiled one:
1. `cd /opt/Executor/`
1. `wget https://github.com/Progressor/ProgressorMeteor/raw/master/bin/ProgressorExecutor.jar`
1. If you modified the [*Dockerfile*](src/main/docker/Dockerfile), upload it to your server in `/opt/Executor/`, otherwise download the pre-compiled one:
1. `cd /opt/Executor`
1. `wget https://raw.githubusercontent.com/Progressor/ProgressorExecutor/master/src/main/docker/Dockerfile`
1. On the server, still in `/opt/Executor`, build the *Docker* image used by the Executor by running `docker build -t progressor/executor .`
1. On the server, still in `/opt/Executor`, build the *Docker* image used by the **Executor** by running `docker build -t progressor/executor .`

The *Executor* will use this image to create one container per request.
The **Executor** will use this image to create one container per request.
After the request has been processed, the container is stopped and immediately deleted.

That way we make sure the different requests never interfere with each other.

# Building the Executor from source
## Maven
### Building the Executor From Source

This repository contains a [*Maven*](https://maven.apache.org/) project created using [*IntelliJ IDEA*](https://www.jetbrains.com/idea/).

Executor won't work properly if Apache Thrift files have not been generated. Luckily, Maven generates these files for us when you build the project, so before you run the Executor run it as a Maven build: *mvn -clean -package*
### Dependencies
This project has four *Maven* dependencies:
1. [org.apache.thrift:libthrift:0.9.3](http://mvnrepository.com/artifact/org.apache.thrift/libthrift/0.9.3)
for *Apache Thrift*
1. [org.json:json:20151123](http://mvnrepository.com/artifact/org.json/json/20151123)
for *JSON* decoding
1. [org.slf4j:slf4j-simple:1.7.13](http://mvnrepository.com/artifact/org.slf4j/slf4j-simple/1.7.13)
for logging
1. [org.testng:testng:RELEASE](http://mvnrepository.com/artifact/org.testng/testng)
for unit tests
# Docker
[*Docker*](https://www.docker.com/) is active by default on Linux distributions. On Windows it is deactivated by default, since you're not able to use the Executor with *Docker* on Windows.
The created *Docker* image Tag is named `progressor/executor`.
If you decide not to use *Docker*, use the following `-docker false`.

Be aware that not using *Docker* requires you to install all the compilers of the languages you want to support on your server.
If you use *Docker* you need to install the compilers in your *Docker* image. To do that, you need to adjust the Dockerfile and rebuild your image.

# Programming Languages

The **Executor** currently supports five programming languages.

To use the languages, the following compilers (and other tools) need to be installed and available in the **PATH**.

1. *Java*: `javac` and `java`
1. *C/C++*: `g++`
1. *C#*: `csc` (Windows) or `msc` (Linux)
1. *Kotlin*: `kotlinc` and `kotlin`
1. *Python*: `python` (Windows)

As already mentioned, if you are using *Docker* these compilers need to be installed inside the *Docker* image via the Dockerfile.
Java is the only exception, since it is needed inside the *Docker* image as well as on the server to run the executor.

### Java

Support for *Java SE* version 8 is required.

The *Java Developer Toolkit* (JDK) can be downloaded from the [official *Oracle* downloads page](http://www.oracle.com/technetwork/java/javase/downloads/).

### C/C++

This projects targets the [*GNU Compiler Collection* (GCC)](https://gcc.gnu.org/).
Support for *C++11* is required.
*Maven* can easily be [downloaded](https://maven.apache.org/download.cgi) and [installed](https://maven.apache.org/install.html) on any platform.
But it is also included in many IDEs like [*IntelliJ IDEA*](https://www.jetbrains.com/idea/), [*NetBeans*](https://netbeans.org/) and others.

* For Linux, install *g++* using `apt-get install g++`
* For Windows, the following packages are available:
* [*MinGW*](http://www.mingw.org/), which can be downloaded from [*sourceforge*](https://sourceforge.net/projects/mingw/files/).
* A [x64 version](http://mingw-w64.org/) is available on a [dedicated site](http://mingw-w64.org/doku.php/download/win-builds).
* [*Cygwin*](http://sourceware.org/cygwin/), which can be downloaded from their home page.
The dependencies are outlined in a [separate document](doc/ExecutorDependencies.md).

### C# #
You can build the project by executing `mvn clean package -Dmaven.test.skip=true` which will create the executable `target/ProgressorExecutor-1.0-jar-with-dependencies.jar`.

* On Windows, the C# compiler is already included in most recent Windows operatiog systems. Make sure that the *PATH* environment variable is set to `C:\WINDOWS\Microsoft.NET\Framework\v[your version number]\csc.exe.`
* If this directory does not exist, you can download the [*.NET Core*](https://www.microsoft.com/net/download)
* On Linux, [*Mono*](http://www.mono-project.com/) can be downloaded on their [download page](http://www.mono-project.com/download/).
This statement will do the following:

#### Kotlin
1. It will clean the workspace and remove any files left over from previous builds.
1. It will generate *Java*-specific [*Apache Thrift*](https://thrift.apache.org/) code based on the language-independent interface definition in `src/main/thrift/executor.thrift`.
1. It will skip the tests as you may not have properly set up your development machine to test all supported programming languages.
1. It will build the application.
1. It will generate two *JAR* files: one slim version (without the dependencies included) and one with all the dependencies included which you can then copy and use wherever you like.

For [*Kotlin*](http://kotlinlang.org/), a [stand-alone compiler](http://kotlinlang.org/docs/tutorials/command-line.html) can be downloaded from [*GitHub*](https://github.com/JetBrains/kotlin/releases/latest).
## Introduction

### Python
The **Executor** is a *Java* application that receives requests from the **Progressor** web application using *Apache Thrift*.

* On Linux, Python 3 install using *apt-get install python3*
* On Windows, download [*Python 3*](https://www.python.org/downloads/release/python-351/) and install it.
The actual application logic is completely separated from the *Apache Thrift* layer and has its own public API that can be used to [extend](doc/ExecutorConfig.md#extensibility) the **Executor**.

## Extensibility
![public Executor API](doc/images/ExecutorAPI.png)

The **Executor** uses a [ServiceLoader](http://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) to load the different code executors at runtime.
Additionally, the project contains two abstract classes that can be reused to easily implement a new programming language.

New languages can be supported by simply implementing the service *ch.bfh.progressor.executor.api.CodeExecutor* and making it discoverable by the **Executor**.
Additional information can be found in the [Java SE API Specification](http://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html).
![abstract Code Executors](doc/images/ExecutorCodeExecutor.png)
22 changes: 22 additions & 0 deletions doc/ExecutorConfig.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Progressor - Advanced Executor Configuration Options

## Command-Line Arguments

The **Executor** accepts a number of command-line arguments:

* `-p port` or `-port port` will override the default port (9001).
* `-c [false|no]` or `-cleanup [false|no]` will prevent temporary files (e.g. generated code files, compiled executables or *Docker* containers) from being deleted.
* `-d [true|yes]` or `-docker [true|yes]` has no effect because because cleanup is enabled by default.
* `-d [false|no]` or `-docker [false|no]` will disable *Docker* support on Linux.
* `-d [true|yes]` or `-docker [true|yes]` has no effect because because *Docker* is enabled by default on Linux.
If invoked on Windows, the application will abort because this feature is not yet supported.
* `-t [true|yes]` or `-test [true|yes]` will run a test client instead of the server (request handler).
* `-t [false|no]` or `-test [false|no]` has no effect because server is the standard mode.
* `-h hostname` or `-host hostname` can be used to test an instance different from the local one.

## Extensibility

The **Executor** uses a [ServiceLoader](http://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) to load the different code executors at runtime.

New languages can be supported by simply implementing the service *ch.bfh.progressor.executor.api.CodeExecutor* and making it discoverable by the **Executor**.
Additional information can be found in the [Java SE API Specification](http://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html).
87 changes: 87 additions & 0 deletions doc/ExecutorDependencies.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Progressor - Executor Dependencies

This project has five *Maven* dependencies:

1. [org.apache.thrift:libthrift:0.9.3](http://mvnrepository.com/artifact/org.apache.thrift/libthrift/0.9.3)
for *Apache Thrift*
1. [org.json:json:20151123](http://mvnrepository.com/artifact/org.json/json/20151123)
for *JSON* decoding
1. [org.slf4j:slf4j-simple:1.7.13](http://mvnrepository.com/artifact/org.slf4j/slf4j-simple/1.7.13)
for logging
1. [org.testng:testng:RELEASE](http://mvnrepository.com/artifact/org.testng/testng)
for unit tests
1. [commons-io:commons-io:2.5](http://mvnrepository.com/artifact/commons-io/commons-io/2.5)
for proper *UTF-8* support on streams

## Docker

[*Docker*](https://www.docker.com/) support is activated by default on Linux distributions. On any other platform, *Docker* is not yet supported.
The created *Docker* image is tagged `progressor/executor`.
If you decide not to use *Docker*, use the following `-docker false`.

Be aware that not using *Docker* requires you to install all the compilers of the languages you want to support on your server.
If you use *Docker* you need to install the compilers in your *Docker* image. To do that, you need to adjust the *Dockerfile* and rebuild your image.

## Programming Languages

The **Executor** currently supports five programming languages.

To use the languages, the following compilers (and other tools) need to be installed and available in the **PATH**.

1. *Java*: `javac` and `java`
1. *C/C++*: `g++`
1. *C#*: `csc` (Windows) or `msc` and `mono` (Linux)
1. *Python*: `python` (Windows)
1. *JavaScript* ([Node.js](https://nodejs.org/)): `node`
1. *PHP*: `php`
1. *Kotlin*: `kotlinc` and `kotlin` (these scripts are actually simulated until a security issue can be solved with *Java 9*)
1. *VB.NET*: `vbc` (Windows) or `vbnc` and `mono` (Linux)

As already mentioned, if you are using *Docker* these compilers need to be installed inside the *Docker* image via the *Dockerfile*.
Java is the only exception, since it is needed inside the *Docker* image as well as on the server to run the **Executor**.

### Java

Support for *Java 8* is required.

The *Java Developer Toolkit* (JDK) can be downloaded from the [official *Oracle* downloads page](http://www.oracle.com/technetwork/java/javase/downloads/).

### C/C++

This projects targets the [*GNU Compiler Collection* (GCC)](https://gcc.gnu.org/).
Support for *C++11* is required.

* For Linux, install *g++* using `apt-get install g++`
* For Windows, the following packages are available:
* [*MinGW*](http://www.mingw.org/), which can be downloaded from [*sourceforge*](https://sourceforge.net/projects/mingw/files/).
* A [x64 version](http://mingw-w64.org/) is available on a [dedicated site](http://mingw-w64.org/doku.php/download/win-builds).
* [*Cygwin*](http://sourceware.org/cygwin/), which can be downloaded from their home page.

### C# #

* On Windows, the C# compiler is already included in most recent Windows operating systems. Make sure that the *PATH* environment variable is set to `C:\WINDOWS\Microsoft.NET\Framework\v[your version number]\csc.exe.`
* If this directory does not exist, you can download the [*.NET Core*](https://www.microsoft.com/net/download).
* On Linux, [*Mono*](http://www.mono-project.com/) can be downloaded on their [download page](http://www.mono-project.com/download/).

### Python

* On Linux, Python 3 install using *apt-get install python3*
* On Windows, download [*Python 3*](https://www.python.org/downloads/release/python-351/) and install it.

### JavaScript

Install [Node.js](https://nodejs.org/) version *6.x.x* as any version below will not fully comply with *ES6*.

### PHP

Install [PHP](http://php.net/downloads.php) ([Windows](http://windows.php.net/download#php-7.0)) version 7.x.x.

### Kotlin

For [*Kotlin*](http://kotlinlang.org/), a [stand-alone compiler](http://kotlinlang.org/docs/tutorials/command-line.html) can be downloaded from [*GitHub*](https://github.com/JetBrains/kotlin/releases/latest).

### VB.NET

* On Windows, the VB.NET compiler is already included in most recent Windows operating systems. Make sure that the *PATH* environment variable is set to `C:\WINDOWS\Microsoft.NET\Framework\v[your version number]\vbc.exe.`
* If this directory does not exist, you can download the [*.NET Core*](https://www.microsoft.com/net/download).
* On Linux, [*Mono*](http://www.mono-project.com/) can be downloaded on their [download page](http://www.mono-project.com/download/).
Binary file added doc/images/ExecutorAPI.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/images/ExecutorCodeExecutor.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit dff94af

Please sign in to comment.