Merge branch 'feature/vscode' of github.com:saagie/technologies-commu…

…nity into feature/vscode

README.md

@@ -5,11 +5,10 @@
 [![GitHub release date](https://img.shields.io/github/release-date/saagie/technologies-community?style=for-the-badge&color=blue)][releases]  
 
 
-[![Issues](https://img.shields.io/github/issues-raw/saagie/technologies-community?style=for-the-badge&color=blue)][issues]
+[![Issues](https://img.shields.io/github/issues-raw/saagie/technologies-community?style=for-the-badge&color=green)][issues]
 
-[![License](https://img.shields.io/github/license/saagie/technologies-community?style=for-the-badge&color=blue)][license]
-
-[![Contributors](https://img.shields.io/github/contributors/saagie/technologies-community?style=for-the-badge&color=blue)][contributors]
+[![License](https://img.shields.io/github/license/saagie/technologies-community?style=for-the-badge&color=black)][license]
+[![Contributors](https://img.shields.io/github/contributors/saagie/technologies-community?style=for-the-badge&color=black)][contributors]
 
 [releases]: https://github.com/saagie/technologies-community/releases
 [contributors]: https://github.com/saagie/technologies-community/graphs/contributors
@@ -24,9 +23,17 @@ All these technologies shared on this Github repository are supported by its cre
 Before contributing to this repository, make sure your technology does not already exist in the [Saagie Official Technlogy Repository](https://github.com/saagie/technologies) nor is planned in [the oficial repository kanban](https://github.com/saagie/technologies/projects).
 
 
-## CONTENTS
+## Technologies
 
 This repository contains community job and application technologies.
+
+|  | Technology |Type |Description|
+| --- | --- | --- |--- |
+|<img src="https://upload.wikimedia.org/wikipedia/commons/f/f3/Apache_Spark_logo.svg" width="30" height="30"> | **Spark History Server**| App | Monitoring tool that displays information about completed Spark applications 
+|<img src="https://upload.wikimedia.org/wikipedia/en/a/a1/Grafana_logo.svg" width="30" height="30"> |**Grafana**| App| Open-source platform for monitoring and observability
+|<img src="https://www.openanalytics.eu/shinyproxy/logo.png" width="30" height="30"> |**ShinyProxy**| App| Open source platform to deploy Shiny apps
+|<img src="https://upload.wikimedia.org/wikipedia/commons/b/b3/Terminalicon2.png" width="30" height="30"> |**TTYD**| App| Interactive Bash with hadoop commands
+|<img src="https://www.metabase.com/images/logo.svg" width="30" height="30"> |**Metabase**| App| Open source Business Intelligence server
 
 ### Job technologies
 

technologies/app/grafana/README.md

@@ -1,4 +1,5 @@
 # Grafana
+![Docker Image Size (tag)](https://img.shields.io/docker/image-size/saagie/grafana/7.3.4?label=v7.3.4%20image%20size&style=for-the-badge)
 
 ## Description
 This directory contains version of Grafana contenairized and customized for Saagie Platform.

technologies/app/metabase/README.md

@@ -0,0 +1,28 @@
+# Metabase
+
+![Docker Image Size (tag)](https://img.shields.io/docker/image-size/saagie/metabase/0.38?label=v0.38%20image%20size&style=for-the-badge)
+
+## Description
+This directory contains version of Metabase contenairized and customized for Saagie Platform.
+See Metabase official documentation for more information https://www.metabase.com/docs/latest/
+
+## How to build in local
+
+Inside the `metabase-x.y` folder corresponding to your version, run :
+```
+docker build -t saagie/metabase:<version> .
+docker push saagie/metabase:<version>
+```
+
+## Job/App specific information
+Default admin credentials are to be created during first login.
+
+## Configuration
+This version comes with a local H2 table to store Metabase internal data. You can choose to use an external MySQL / PostgreSQL table to do so. This can be configured (among other parameters) through environment variables. Follow this [documentation](https://www.metabase.com/docs/latest/operations-guide/environment-variables.html) for more information.
+
+## Known issues
+When you first log in, after having createad your user, you will be redirected to your Saagie Projects instead of the Metabase homepage. Simply return to your Saagie App and try to open the Metabase url (this issue only happens during first login).
+
+## Configure Athena
+When configuring Athena in Metabase, you'll need to add the following connection string:
+UseResultsetStreaming=0

technologies/app/metabase/metabase-0.38/Dockerfile

@@ -0,0 +1,30 @@
+FROM metabase/metabase:v0.38.1
+
+USER root
+
+# Installing Nginx
+RUN apk update \
+    && apk add nginx \
+    && rm /etc/nginx/conf.d/default.conf \
+    && mkdir -p /run/nginx
+
+COPY server.conf /etc/nginx/conf.d/metabase.conf
+
+# Installing Drivers
+RUN mkdir /app/plugins && chmod 777 /app/plugins
+COPY assets/impala.metabase-driver.jar /app/plugins/impala.metabase-driver.jar
+COPY assets/ImpalaJDBC41.jar /app/plugins/ImpalaJDBC41.jar
+COPY assets/athena.metabase-driver.jar /app/plugins/athena.metabase-driver.jar
+COPY assets/AthenaJDBC42_2.0.13.jar /app/plugins/AthenaJDBC42_2.0.13.jar
+COPY assets/log4j2.xml /metabase.db/log4j2.xml
+
+ENV MB_DB_TYPE h2
+ENV MB_DB_FILE /metabase-data
+ENV MB_PLUGINS_DIR /app/plugins
+ENV JAVA_OPTS "-Dlog4j.configurationFile=file:///metabase.db/log4j2.xml"
+ENV MB_DB_CONNECTION_TIMEOUT_MS 30000
+
+EXPOSE 80
+ADD entrypoint.sh /entrypoint.sh
+
+ENTRYPOINT ["/entrypoint.sh"]

technologies/app/metabase/metabase-0.38/assets/log4j2.xml

@@ -0,0 +1,24 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<Configuration>
+  <Appenders>
+    <Console name="STDOUT" target="SYSTEM_OUT">
+      <PatternLayout pattern="%d %p %c{2} :: %m%n">
+        <replace regex=":basic-auth \\[.*\\]" replacement=":basic-auth [redacted]"/>
+      </PatternLayout>
+    </Console>
+  </Appenders>
+
+  <Loggers>
+    <Logger name="metabase" level="WARN"/>
+    <Logger name="metabase-enterprise" level="WARN"/>
+    <Logger name="metabase.plugins" level="WARN"/>
+    <Logger name="metabase.core" level="INFO"/>
+    <Logger name="metabase.server.middleware" level="WARN"/>
+    <Logger name="metabase.query-processor.async" level="WARN"/>
+    <Logger name="com.mchange" level="ERROR"/>
+
+    <Root level="WARN">
+      <AppenderRef ref="STDOUT"/>
+    </Root>
+  </Loggers>
+</Configuration>

technologies/app/metabase/metabase-0.38/entrypoint.sh

@@ -0,0 +1,3 @@
+#!/bin/bash
+sed -i 's:SAAGIE_BASE_PATH:'"$SAAGIE_BASE_PATH"':g' /etc/nginx/conf.d/metabase.conf
+nginx && /app/run_metabase.sh

technologies/app/metabase/metabase-0.38/server.conf

@@ -0,0 +1,22 @@
+map $http_connection $upgrade_requested {
+  default upgrade;
+  '' close;
+}
+server {
+  listen 80 default_server;
+  root /usr/share/nginx/html;
+  index index.html index.htm;
+
+  location SAAGIE_BASE_PATH {
+    proxy_http_version 1.1;
+    proxy_set_header Upgrade $http_upgrade;
+    proxy_set_header Connection $upgrade_requested;
+    proxy_read_timeout 20d;
+
+    rewrite ^SAAGIE_BASE_PATH/(.*)$ /$1 break;
+    rewrite ^SAAGIE_BASE_PATH$ / break;
+    proxy_pass http://localhost:3000;
+    proxy_redirect http://localhost:3000/ $scheme://$hostSAAGIE_BASE_PATH/;
+    proxy_redirect https://localhost:3000/ $scheme://$hostSAAGIE_BASE_PATH/;
+  }
+}

technologies/app/metabase/metadata.yaml

@@ -0,0 +1,25 @@
+version: v1
+type: APP
+id: metabase
+label: Metabase
+baseline: "The fastest, easiest way to share data and analytics inside your company"
+description: "The fastest, easiest way to share data and analytics inside your company"
+available: true
+icon: dataset
+backgroundColor: "#5ea4e3"
+customFlags: []
+contexts:
+  - id: metabase-0.38
+    label: Metabase 0.38
+    releaseNotes: ""
+    available: true
+    trustLevel: experimental
+    ports:
+      - port: 80
+        name: Metabase
+        rewriteUrl: false
+        basePath: SAAGIE_BASE_PATH
+    volumes: ["/metabase-data"]
+    dockerInfo:
+      image: saagie/metabase
+      version: 0.38

technologies/app/shiny-proxy/README.md

@@ -0,0 +1,41 @@
+# ShinyProxy
+
+![Docker Image Size (tag)](https://img.shields.io/docker/image-size/saagie/shiny-proxy/2.4.3?label=v2.4.3%20image%20size&style=for-the-badge)
+
+## Description
+This directory contains version of ShinyProxy containerized and customized for Saagie Platform.
+See ShinyProxy official documentation for more information https://shinyproxy.io/documentation/
+
+## How to build in local
+
+```
+docker build -t saagie/shinyproxy-<version> .
+docker push saagie/shinyproxy-<version>
+```
+
+
+## Job/App specific information
+In order to run it, you must specifiy through an environment variable `SHINYPROXY_CONF_URL` where your application.yml configuration file is stored (can be either WebHDFS or S3). Examples  : 
+* `export SHINYPROXY_CONF_URL=http://<your hdfs namenode url>:50070/webhdfs/v1/path/to/application.yml?op=OPEN&user.name=my.username`
+* `export SHINYPROXY_CONF_URL=s3://path/to/application.yml`
+
+A template for this `application.yml` file is provided in this repository. In order to be compatible within Saagie, you must be careful about keeping the `context-path: SAAGIE_BASE_PATH` entry, mandatory for Saagie. To configure ShinyProxy based ou your requirements, follow the [official documentation](https://shinyproxy.io/documentation/configuration)
+
+## Customize ShinyProxy
+
+This Docker image relies on 2 forks from Open Analytics open source projects.
+
+https://github.com/saagie/shinyproxy
+https://github.com/saagie/containerproxy
+
+If you want to customize these implementations, just pull the `containerproxy`, make your changes and build it with the following command : 
+
+
+```
+mvn -U clean install
+```
+
+Do the same for the `shinyproxy` repository (make sure to update your `pom.xml` to refer to the previously built containerproxy jar).
+
+The build will result in a single `.jar` file that is made available in the `target` directory, you can replace the **shinyproxy-2.4.3-saagie** of this repository with this new jar.
+

technologies/app/shiny-proxy/metadata.yaml

@@ -0,0 +1,24 @@
+version: v1
+type: APP
+id: shiny-proxy
+label: ShinyProxy
+baseline: "ShinyProxy is a novel, open source platform to deploy Shiny apps for the enterprise or larger organizations"
+description: "ShinyProxy is a novel, open source platform to deploy Shiny apps for the enterprise or larger organizations"
+available: true
+icon: generic-app
+backgroundColor: "#32a6d3"
+customFlags: []
+contexts:
+  - id: shiny-proxy-2.4.3
+    label: ShinyProxy 2.4.3
+    releaseNotes: ""
+    available: true
+    trustLevel: experimental
+    ports:
+      - port: 8080
+        name: ShinyProxy
+        rewriteUrl: false
+        basePath: SAAGIE_BASE_PATH
+    dockerInfo:
+      image: saagie/shiny-proxy
+      version: 2.4.3

technologies/app/shiny-proxy/shiny-proxy-2.4.3/Dockerfile

@@ -0,0 +1,17 @@
+FROM openjdk:8-jre
+
+#AWS CLI needed when application.yml is stored on S3
+RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" \
+    && unzip awscliv2.zip \
+    && ./aws/install
+
+RUN mkdir -p /opt/shinyproxy/
+
+COPY shinyproxy-2.4.3-saagie.jar /opt/shinyproxy/shinyproxy.jar
+COPY run-app.sh /opt/shinyproxy/
+COPY templates /opt/shinyproxy/templates
+COPY application.yml /opt/shinyproxy/application.yml
+
+EXPOSE 8080
+WORKDIR /opt/shinyproxy/
+ENTRYPOINT ["sh", "/opt/shinyproxy/run-app.sh"]

technologies/app/shiny-proxy/shiny-proxy-2.4.3/application.yml

@@ -0,0 +1,30 @@
+server:
+  useForwardHeaders: true
+  servlet:
+    context-path: SAAGIE_BASE_PATH
+proxy:
+  title: Shiny Proxy for Saagie
+  logo-url: https://www.saagie.com/wp-content/uploads/2020/06/[email protected]
+  template-path: ./templates/3colcards
+  authentication: simple
+  admin-groups: scientists
+  users:
+    - name: jack
+      password: password
+      groups: scientists
+    - name: jeff
+      password: password
+      groups: mathematicians
+
+  specs:
+  - id: saagie-demo
+    display-name: Saagie
+    container-proxy-managed: false
+    container-app-url: http://www.saagie.com
+    access-groups: ["scientists","mathematicians"]
+
+logging:
+  file:
+    shinyproxy.log
+  level:
+    root: info

technologies/app/shiny-proxy/shiny-proxy-2.4.3/run-app.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+
+case "$SHINYPROXY_CONF_URL" in 
+    s3*) 
+        echo "[INFO] - Fetching application.yml file from S3"
+        aws s3 cp $SHINYPROXY_CONF_URL /opt/shinyproxy/application.yml;;
+    http*)
+        echo "[INFO] - Fetching application.yml file"
+        wget $SHINYPROXY_CONF_URL -O /opt/shinyproxy/application.yml;;
+    *)
+        echo "[WARN] - application.yml configuration file cannot be fetched : protocol not compatible => choose between s3 or http"
+        echo "[INFO] - Sample application.yml file will be used"
+esac
+
+sed -i 's#SAAGIE_BASE_PATH#'"$SAAGIE_BASE_PATH"'#g' /opt/shinyproxy/application.yml
+java -jar /opt/shinyproxy/shinyproxy.jar

technologies/app/shiny-proxy/shiny-proxy-2.4.3/templates/3colcards/app.html

@@ -0,0 +1,64 @@
+<!--
+
+    ShinyProxy
+
+    Copyright (C) 2016-2019 Open Analytics
+
+    ===========================================================================
+
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the Apache License as published by
+    The Apache Software Foundation, either version 2 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    Apache License for more details.
+
+    You should have received a copy of the Apache License
+    along with this program.  If not, see <http://www.apache.org/licenses/>
+
+-->
+<!DOCTYPE html>
+<html xmlns:th="http://www.thymeleaf.org"
+	xmlns:sec="http://www.thymeleaf.org/thymeleaf-extras-springsecurity4">
+<head lang="en">
+	<title th:text="${appTitle}"></title>
+	<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+	<meta name="viewport" content="width=device-width, initial-scale=1"/>
+	<link rel="stylesheet" media="screen" th:href="@{${bootstrapCss}}" />
+	<link rel="stylesheet" media="screen" th:href="@{/css/default.css}"/>
+	<script th:src="@{${jqueryJs}}"></script>
+	<script th:src="@{${bootstrapJs}}"></script>
+</head>
+<body>
+	<div th:replace="../fragments/navbar :: navbar"></div>
+
+    <iframe id="shinyframe" th:src="${container}" width="100%" frameBorder="0"></iframe>
+    <div class="loading"><div class="loading-txt">Launching <span th:text="${appTitle}"></span>...</div></div>
+
+	<script type="text/javascript" th:inline="javascript">
+		function setShinyframeHeight() {
+			$('#shinyframe').css('height', ($(window).height())+'px');
+		}
+		window.addEventListener("load", setShinyframeHeight);
+		window.addEventListener("resize",  setShinyframeHeight);
+
+		$(window).on('load', function() {
+			var source = $("#shinyframe").attr("src");
+			if (source == "") {
+				$(".loading").show();
+				$.post(window.location.pathname + window.location.search, function(response) {
+					$("#shinyframe").attr("src", response.containerPath);
+					$(".loading").fadeOut("slow");
+				}).fail(function(request) {
+					var newDoc = document.open("text/html", "replace");
+					newDoc.write(request.responseText);
+					newDoc.close();
+				});
+			}
+		});
+	</script>
+</body>
+</html>