Merge pull request #3 from EEESlab/develop

Develop to master

docker/README.md

@@ -0,0 +1,71 @@
+# ExaMon Docker Setup
+This setup will install all server-side components of the ExaMon framework:
+
+ - MQTT broker and Db connector
+ - Grafana
+ - KairosDB
+ - Cassandra
+
+## Prerequisites
+Since Cassandra is the component that requires the majority of resources, you can find more details about the suggested hardware configuration of the system that will host the services here:
+
+[Hardware Configuration](https://cassandra.apache.org/doc/latest/operating/hardware.html#:~:text=While%20Cassandra%20can%20be%20made,at%20least%2032GB%20of%20RAM)
+
+To install all the services needed by ExaMon we will use Docker and Docker Compose:
+
+[Install Docker and Docker Compose](https://docs.docker.com/engine/installation/).
+
+
+## Setup
+
+### Clone the Git repository
+
+First you will need to clone the Git repository:
+
+```bash
+git clone https://github.com/EEESlab/examon
+cd examon/docker/
+```
+
+### Create Docker Services
+
+Once you have the above setup, you need to create the Docker services:
+
+```bash
+docker-compose up -d
+```
+
+This will build the Docker images and fetch some prebuilt images and then start the services. You can refer to the `docker-compose.yml` file to see the full configuration. 
+
+### Configure Grafana
+
+Log in to the Grafana server using your browser and the default credentials:
+
+http://localhost:3000
+
+Follow the normal procedure for adding a new data source (KairosDB):
+
+[Add a Datasource](https://grafana.com/docs/grafana/latest/datasources/add-a-data-source/)
+
+Fill out the form with the following settings:
+
+ - Type: `KairosDB`  
+ - Name: `kairosdb` 
+ - Url: http://kairosdb:8083 
+ - Access: `Proxy`
+
+## Usage Examples
+
+### Collecting data using the "pmu_pub" plugin
+Once all Docker services are running (can be started either by `docker-compose up -d` or `docker-compose start`), the MQTT broker is available at `TEST_SERVER` port `1883` where `TEST_SERVER` is the address of the server where the services run.
+
+To test the installation we can use the `pmu_pub` plugin available in the `publishers/pmu_pub` folder of  this project.
+
+After having installed and configured it on one or more test nodes we can start the data collection running for example:
+
+```bash
+[root@testnode00]$ ./pmu_pub -b TEST_SERVER -p 1883 -t org/myorg -s 1 run
+```
+If everything went well, the data are available both through the Grafana interface and using the `examon-client`. 
+
+

docker/cassandra/Dockerfile

@@ -0,0 +1 @@
+FROM cassandra:3.0.19

docker/docker-compose.yml

@@ -0,0 +1,83 @@
+version: "3.8"
+
+networks:
+  examon_net:
+    driver: bridge
+
+services:
+  examon:
+    image: examonhpc/examon:0.1.0
+    logging:
+      driver: json-file
+      options:
+        max-size: "10m"
+        max-file: "1"
+    restart: always
+    ports:
+      - "1883:1883"
+    networks:
+      - examon_net
+    environment:
+      - EX_KAIROSDB_HOST=kairosdb
+    depends_on:
+      - kairosdb
+
+  grafana:
+    build:
+      context: ./grafana
+    image: examonhpc/grafana:5.4.3 
+    logging:
+      driver: json-file
+      options:
+        max-size: "10m"
+        max-file: "1"
+    restart: always
+    ports:
+      - "3000:3000"
+    networks:
+      - examon_net
+    environment:
+      - GF_SECURITY_ADMIN_PASSWORD=Password
+    depends_on:
+      - kairosdb
+
+  kairosdb:
+    build:
+      context: ./kairosdb
+    image: examonhpc/kairosdb:1.2.1
+    logging:
+      driver: json-file
+      options:
+        max-size: "10m"
+        max-file: "1"
+    restart: always
+    ports:
+      - "8083:8083"
+    networks:
+      - examon_net
+    environment:      
+      - CASSANDRA_HOST_LIST=cassandra:9042
+    depends_on:
+      - cassandra
+
+  cassandra:
+    build:
+      context: ./cassandra
+    image: examonhpc/cassandra:3.0.19
+    logging:
+      driver: json-file
+      options:
+        max-size: "10m"
+        max-file: "1"
+    restart: always
+    ports:
+      - "9042:9042"
+    volumes:
+      - ${PWD}/cassandra_volume:/var/lib/cassandra
+    networks:
+      - examon_net
+    environment:
+      - CASSANDRA_CLUSTER_NAME=examon
+      - CASSANDRA_BROADCAST_ADDRESS=cassandra
+      - CASSANDRA_START_RPC=true
+      - CASSANDRA_LISTEN_ADDRESS=auto

docker/examon/Dockerfile

@@ -0,0 +1 @@
+FROM examonhpc/examon:0.1.0

docker/grafana/Dockerfile

@@ -0,0 +1,3 @@
+FROM grafana/grafana:5.4.3
+
+COPY ./kairosdb-datasource /var/lib/grafana/plugins/kairosdb-datasource

docker/grafana/LICENSE

@@ -0,0 +1,14 @@
+Copyright 2014-2016 Torkel Ödegaard, Raintank Inc.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you
+may not use this file except in compliance with the License. You may
+obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+implied. See the License for the specific language governing
+permissions and limitations under the License.
+

docker/grafana/README.md

@@ -0,0 +1,106 @@
+# Grafana Docker image
+
+This project builds a Docker image with the latest master build of Grafana.
+
+## Running your Grafana container
+
+Start your container binding the external port `3000`.
+
+```
+docker run -d --name=grafana -p 3000:3000 grafana/grafana
+```
+
+Try it out, default admin user is admin/admin.
+
+## Configuring your Grafana container
+
+All options defined in conf/grafana.ini can be overriden using environment
+variables by using the syntax `GF_<SectionName>_<KeyName>`.
+For example:
+
+```
+docker run \
+  -d \
+  -p 3000:3000 \
+  --name=grafana \
+  -e "GF_SERVER_ROOT_URL=http://grafana.server.name" \
+  -e "GF_SECURITY_ADMIN_PASSWORD=secret" \
+  grafana/grafana
+```
+
+More information in the grafana configuration documentation: http://docs.grafana.org/installation/configuration/
+
+## Grafana container with persistent storage (recommended)
+
+```
+# create /var/lib/grafana as persistent volume storage
+docker run -d -v /var/lib/grafana --name grafana-storage busybox:latest
+
+# start grafana
+docker run \
+  -d \
+  -p 3000:3000 \
+  --name=grafana \
+  --volumes-from grafana-storage \
+  grafana/grafana
+```
+
+## Installing plugins for Grafana 3
+
+Pass the plugins you want installed to docker with the `GF_INSTALL_PLUGINS` environment variable as a comma seperated list. This will pass each plugin name to `grafana-cli plugins install ${plugin}`.
+
+```
+docker run \
+  -d \
+  -p 3000:3000 \
+  --name=grafana \
+  -e "GF_INSTALL_PLUGINS=grafana-clock-panel,grafana-simple-json-datasource" \
+  grafana/grafana
+```
+
+## Running specific version of Grafana
+
+```
+# specify right tag, e.g. 2.6.0 - see Docker Hub for available tags
+docker run \
+  -d \
+  -p 3000:3000 \
+  --name grafana \
+  grafana/grafana:2.6.0
+```
+
+## Configuring AWS credentials for CloudWatch support
+
+```
+docker run \
+  -d \
+  -p 3000:3000 \
+  --name=grafana \
+  -e "GF_AWS_PROFILES=default" \
+  -e "GF_AWS_default_ACCESS_KEY_ID=YOUR_ACCESS_KEY" \
+  -e "GF_AWS_default_SECRET_ACCESS_KEY=YOUR_SECRET_KEY" \
+  -e "GF_AWS_default_REGION=us-east-1" \
+  grafana/grafana
+```
+
+You may also specify multiple profiles to `GF_AWS_PROFILES` (e.g.
+`GF_AWS_PROFILES=default another`).
+
+Supported variables:
+
+- `GF_AWS_${profile}_ACCESS_KEY_ID`: AWS access key ID (required).
+- `GF_AWS_${profile}_SECRET_ACCESS_KEY`: AWS secret access  key (required).
+- `GF_AWS_${profile}_REGION`: AWS region (optional).
+
+## Changelog
+
+### v4.2.0
+* Plugins are now installed into ${GF_PATHS_PLUGINS}
+* Building the container now requires a full url to the deb package instead of just version
+
+### v4.0.0-beta2
+* Plugins dir (`/var/lib/grafana/plugins`) is no longer a separate volume
+
+### v3.1.1
+* Make it possible to install specific plugin version https://github.com/grafana/grafana-docker/issues/59#issuecomment-260584026
+

docker/grafana/kairosdb-datasource/.gitignore

@@ -0,0 +1 @@
+.idea

docker/grafana/kairosdb-datasource/Gruntfile.js

@@ -0,0 +1,19 @@
+'use strict';
+module.exports = function(grunt) {
+grunt.loadNpmTasks('grunt-typescript');
+
+grunt.initConfig({
+typescript: {
+    base: {
+      src: ['*.ts'],
+      dest: '.',
+      options: {
+        module: 'amd'
+      }      
+    }
+  }
+});
+
+grunt.registerTask('default', ['typescript']);
+
+}