[teraslice] Add Teraslice helm chart (#3834)

This PR makes the following changes:

- Adds `teraslice` helm chart
- Adds helmfile with basic configuration
- This allows users to quickly try the helm chart with `opensearch1` and
start a simple job

Ref to issue ##3828

---------

Co-authored-by: Austin Godber <[email protected]>

examples/helm/README.md

@@ -0,0 +1,135 @@
+# Starting teraslice using helm and helmfile
+
+## Prerequisites
+
+The following dependencies are required to successfully deploy a basic instance of Teraslice and interact with its API. The examples provided use Homebrew (brew) and Node.js's (npm) for installation.
+
+- Docker
+- [Helm](https://helm.sh/docs/intro/install/)
+    - `brew install helm`
+- [helmfile](https://formulae.brew.sh/formula/helmfile)
+    - `brew install helmfile`
+- [Kubectl](https://kubernetes.io/docs/reference/kubectl/)
+    - `brew install kubectl`
+- [Kind](https://kind.sigs.k8s.io/) - Kubernetes in Docker
+    - `brew install kind`
+- [curl](https://formulae.brew.sh/formula/curl) - Command-line tool for making HTTP requests
+    - `brew install curl`
+- [teraslice-cli](https://www.npmjs.com/package/teraslice-cli) - A CLI tool for managing Teraslice
+    - `npm install -g teraslice-cli`
+
+### Initial Setup
+
+First you're going to want to be in the correct directory. Starting in the top level of the teraslice directory:
+
+```bash
+cd ./examples/helm
+```
+
+### Step 1: Creating a Kind Cluster
+
+Create a single node Kubernetes cluster by running the following command:
+
+```bash
+kind create cluster --config kindConfig.yaml
+```
+
+### Step 2: Building the Teraslice Docker Image
+
+Build the teraslice docker image using the following command. Ensure the image is tagged correctly to match the intended version `dev-nodev22.9.0` in this example:
+
+```bash
+docker build -t ghcr.io/terascope/teraslice:dev-nodev22.9.0 ../../.
+```
+
+### Step 3: Loading the Teraslice Docker Image into the Kind Cluster
+
+Load the Teraslice Docker image, built above, into the Kind cluster's control plane:
+
+```bash
+kind load docker-image --name k8s-env ghcr.io/terascope/teraslice:dev-nodev22.9.0
+```
+
+### Step 4: Verifying the Image Load
+
+Confirm that the teraslice image has been successfully loaded into the cluster. The following command lists the images available in the clusters control plane:
+
+```bash
+docker exec -it k8s-env-control-plane crictl images
+```
+
+### Step 5: Verifying the Kubernetes Resource Configuration
+
+Generate a preview of the Kubernetes resources that will be deployed. This step ensures that the `helmfile` is configured correctly:
+
+```bash
+helmfile diff
+```
+
+### Step 6: Deploying Resources
+
+Lastly if there were no errors with the `diff` command, deploy teraslice and opensearch into the cluster by running:
+
+```bash
+helmfile sync
+```
+
+### Step 7: Deploying Assets
+
+The example job requires the `standard-assets` and `elasticsearch-assets` to be available in the cluster for successful execution. Use the `teraslice-cli` tool to deploy these assets:
+
+```bash
+teraslice-cli assets deploy localhost terascope/standard-assets
+```
+
+```bash
+teraslice-cli assets deploy localhost terascope/elasticsearch-assets
+```
+
+### Step 8: Submitting and Starting a Test Job
+
+This example job generates `10,000` records and writes them to an Opensearch index named `random-data-1`. Submit the job to the Teraslice API using the following command:
+
+```bash
+curl -XPOST 'localhost:5678/v1/jobs' -H "Content-Type: application/json" -d '{
+    "name": "data-to-es",
+    "lifecycle": "once",
+    "workers": 1,
+    "assets": [
+        "standard",
+        "elasticsearch"
+    ],
+    "operations": [
+        {
+            "_op": "data_generator",
+            "size": 10000
+        },
+        {
+            "_op": "elasticsearch_bulk",
+            "size": 10000,
+            "index": "random-data-1"
+        }
+    ]
+}'
+
+```
+
+### Step 9: Viewing results in opensearch
+
+Once the job completes, query Opensearch to verify that the documents have been written successfully. Use the following command to view the index information:
+
+```bash
+curl 'localhost:9200/_cat/indices?v&h=index,status,docs.count,docs.deleted,store.size,pri.store.size'
+```
+
+Results:
+
+```bash
+index                        status docs.count docs.deleted store.size pri.store.size
+teraslice__assets            open            2            0      2.8mb          2.8mb
+teraslice__state-2024.11     open            1            0     28.8kb         28.8kb
+teraslice__ex                open            1            0     49.1kb         49.1kb
+teraslice__jobs              open            1            0      5.6kb          5.6kb
+random-data-1                open        10000            0        7mb            7mb
+teraslice__analytics-2024.11 open            4            0     23.9kb         23.9kb
+```

examples/helm/default-values.yaml

@@ -0,0 +1,10 @@
+opensearch1:
+  # If false, opensearch1 will be excluded on helmfile sync
+  enabled: true
+  version: 1.3.14
+  esJavaOpts: -Xmx512M -Xms512M
+  memoryLimit: 100Mi
+  persistentVolumeSize: 8Gi
+  # The number of replicas
+  instances: 1
+

examples/helm/helmfile.yaml

@@ -0,0 +1,49 @@
+environments:
+  default:
+    values:
+      - default-values.yaml
+---
+
+repositories:
+  - name: opensearch
+    url: https://opensearch-project.github.io/helm-charts/
+
+helmDefaults:
+  wait: true
+
+releases:
+  - name: opensearch1
+    namespace: ts-dev1
+    version: 2.17.1
+    chart: opensearch/opensearch
+    values:
+      - ./templates/os1.yaml.gotmpl
+
+  - name: teraslice
+    namespace: ts-dev1
+    version: 0.8.12
+    chart: ../../helm/teraslice/
+    needs:
+      - ts-dev1/{{ .Values | get "ts-dev1.stateCluster" "opensearch1" }}
+    values:
+      - terafoundation:
+          connectors:
+            elasticsearch-next:
+              default:
+                node:
+                  - "http://opensearch1.ts-dev1:9200"
+        service:
+          nodePort: 30678
+          type: NodePort
+        master:
+          teraslice:
+            kubernetes_namespace: ts-dev1
+            cluster_manager_type: kubernetesV2
+            asset_storage_connection_type: elasticsearch-next
+        worker:
+          teraslice:
+            kubernetes_namespace: ts-dev1
+            cluster_manager_type: kubernetesV2
+            asset_storage_connection_type: elasticsearch-next
+        image:
+          tag: dev-nodev22.9.0

examples/helm/kindConfig.yaml

@@ -0,0 +1,10 @@
+kind: Cluster
+name: k8s-env
+apiVersion: kind.x-k8s.io/v1alpha4
+nodes:
+- role: control-plane
+  extraPortMappings:
+  - containerPort: 30678 # Map internal teraslice api service to host port
+    hostPort: 5678
+  - containerPort: 30921 # Map internal opensearch1 service to host port
+    hostPort: 9200

examples/helm/templates/os1.yaml.gotmpl

@@ -0,0 +1,36 @@
+replicas: {{ .Values | get "opensearch1.instances" 1 }}
+
+{{- if eq (.Values | get "opensearch1.instances" 1) 1 }}
+singleNode: true
+{{- else }}
+singleNode: false
+{{- end }}
+
+image:
+    tag: {{ .Values | get "opensearch1.version" "1.3.14" }}
+
+service:
+    type: NodePort
+    port: 9200
+    nodePort: 30921
+
+config:
+    opensearch.yml:
+        plugins:
+            security:
+                disabled: true
+        {{- if eq (.Values | get "opensearch1.instances" 1) 1 }}
+        discovery.type: single-node
+        {{- end }}
+
+clusterName: opensearch1-cluster
+
+masterService: "opensearch1"
+
+resources:
+    requests:
+        cpu: "1000m"
+        memory: {{ .Values | get "opensearch1.memoryLimit" "100Mi" }}
+
+persistence:
+    size: {{ .Values | get "opensearch1.persistentVolumeSize" "8Gi" }}

helm/teraslice/Chart.yaml

@@ -0,0 +1,11 @@
+apiVersion: v2
+name: teraslice
+description: Teraslice -  Distributed computing platform for processing JSON data
+type: application
+version: 0.8.12
+appVersion: v2.7.0
+sources:
+  - https://github.com/terascope/teraslice
+keywords:
+  - teraslice
+  - elasticsearch

helm/teraslice/README.md

@@ -0,0 +1,4 @@
+
+# Teraslice Helm Chart
+
+https://terascope.github.io/teraslice/docs/configuration/clustering-k8s