Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: architecture #2

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
**/*.tgz
_site/
Gemfile
Gemfile.lock
84 changes: 76 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,82 @@
# Yellow Stack

## Charts Usage
## Introduction

[Helm](https://helm.sh) must be installed to use the charts. Please refer to
Helm's [documentation](https://helm.sh/docs) to get started.
Welcome to the **Yellow Stack** repository. The primary goal of this repository is to host Helm charts for a comprehensive applications stack deployment, streamlining the process of deploying and managing applications in Kubernetes environments. This repository aims to provide a centralized resource for all necessary deployment assets and instructions.

Once Helm has been set up correctly, add the repo as follows:
As the project evolves, we anticipate expanding the scope of this repository to include additional tools, utilities, and documentation to support a wider range of deployment scenarios and operational tasks.

helm repo add yellow-stack https://layer-3.github.io/stack
## Architecture

If you had already added this repo earlier, run `helm repo update` to retrieve
the latest versions of the packages. You can then run `helm search repo
yellow-stack` to see the charts.
The **Yellow Stack** is designed as a multi-layered architecture that facilitates interaction with blockchain smart contracts, application services, and data storage. This architecture ensures a separation of concerns, allowing for modular development and scalability.

The architecture diagram illustrates the interconnectivity and data flow between each layer and component within the Yellow Stack, ensuring a cohesive and scalable infrastructure.

![Yellow Stack Architecture](assets/architecture.png)

### User Layer

- **Terminal Front-End**: The user interface that clients interact with, providing a seamless and responsive experience to access the functionalities offered by the Terminal application.

### App Layer

Central to user interaction, this layer contains the applications that provide the stack's core functionality:

- **Terminal**: Acts as the interface to the blockchain, allowing users to perform actions on smart contracts and retrieve data.
- **Clearport A & B**: Represent instances of the Clearport application that handle specific business logic, interfacing with the blockchain layer and possibly serving distinct user segments or functions.
- **NeoDAX**: An application possibly geared towards data exchange or analytics, interfacing with the blockchain for data that may involve trading or financial analysis.

#### Servers and Communication

The system leverages different server types to handle communication protocols:

- **WS (WebSocket Server)**: Facilitates real-time, bi-directional communication between the client's front-end and the server for a reactive user interface.
- **UG (User gRPC Server)**: Employs gRPC for efficient, low-latency communication between user interfaces and services, enhancing the responsiveness of user-related interactions.
- **PG (Peer-to-peer gRPC Server)**: Utilizes gRPC for peer-to-peer communication within the distributed network, crucial for blockchain operations and data synchronization.

### Blockchain Layer

This layer contains the smart contracts that our applications interact with:

- **ERC20 Contracts**: Token contracts compliant with the ERC20 standard, used to issue and manage fungible tokens within the ecosystem.
- **ClearSync Contracts**: Custom smart contracts tailored for synchronization processes across different components and blockchain state.
- **Smart Wallet Contracts**: Contracts that enable advanced wallet functionalities like automated transactions and enhanced security protocols.

### Storage Layer

Responsible for data persistence and management, the storage layer comprises:

- **InfluxDB**: A specialized time-series database optimized for high-write loads and real-time analytics, storing time-stamped data from various applications.
- **PostgreSQL**: A robust and flexible relational database managing structured data, with powerful query capabilities to support complex data retrieval needs.

## Clearport

The Clearport application is designed to [...description of what Clearport does and its role in the stack...]. To deploy Clearport using the provided Helm chart, follow these steps:

1. [...Step 1...]
2. [...Step 2...]
3. [...Additional steps...]

## Neodax

Neodax is a [...description of Neodax, its purpose, and how it integrates with the rest of the stack...]. Deployment instructions are as follows:

1. [...Step 1...]
2. [...Step 2...]
3. [...Additional steps...]

## Terminal

Terminal serves as [...description of Terminal's functionality and its importance in the stack...]. To deploy Terminal, use the following steps:

1. [...Step 1...]
2. [...Step 2...]
3. [...Additional steps...]

<!-- ## Contributing

We welcome contributions to the Yellow Stack repository! If you have suggestions, improvements, or new features to add, please feel free to submit an issue or a pull request. For more details on how to contribute, please refer to our CONTRIBUTING.md file. -->

<!-- ## License

This project is licensed under the [...License Name...], which means [...brief description of the license and what it allows users to do...]. -->
Binary file added assets/architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
96 changes: 96 additions & 0 deletions docs/deployment.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# Deployment

## Overview

This document outlines the process for deploying our application stack on Kubernetes. The deployment procedure includes establishing the required infrastructure, configuring the applications according to best practices, and verifying that the stack is functional and ready for production use.

## Prerequisites

To successfully deploy the stack, the following prerequisites must be in place:

- **Ingress Controller**: An Ingress Controller such as Nginx is necessary to manage external access to the services in the cluster.
- **Cert-Manager**: This component automates the management and issuance of SSL certificates to secure communication.
- **InfluxDB ^1.8.10**: For production environments, it is advisable to configure a multi-node setup with a minimum of two nodes for redundancy and high availability.
- **PostgreSQL ^13.7.0**: A PostgreSQL database should be prepared to handle structured data storage requirements.

## Configuration

Configuration of the stack is facilitated through Helm charts. Each chart includes a `values.yaml` file where configuration parameters are set.

### Values Configuration

Copy the [values.yaml](../charts/stack/values.yaml) file to review and modify the configuration options. It is imperative to adjust only the entries marked with the `# CHANGEME` commentary.

#### Node Ports

Initially, determine the node ports required for external access to gRPC endpoints by listing currently utilized ports:

```sh
kubectl get svc --all-namespaces -o go-template='{{range .items}}{{range.spec.ports}}{{if .nodePort}}{{.nodePort}}{{"\n"}}{{end}}{{end}}{{end}}' | sort
```

#### Database Hostnames

Update the hostnames for PostgreSQL and InfluxDB services appropriately. For multiple InfluxDB nodes, delineate the `neodax.secrets.FINEX_INFLUX_HOST` as a comma-separated list of hosts.

#### Database Passwords

Assign unique, self-generated passwords for the databases. Clearport A & B and NeoDAX will have distinct database users, which are generated upon the initial deployment.

#### Clearport B Configuration

Clearport B primarily functions in responder mode and thus requires initial operator. Set the operator’s name in the `clearportB.responderMode.name` configuration field.

#### Component Domains

Specify unique domain names for each component in the `<component>.externalHostname` field to ensure distinct access points.

### Secrets Management

Create two Kubernetes secrets within the deployment namespace:

This secret should store the credentials for the root database user, which must possess the privileges required to create new users and databases.
```sh
kubectl create secret generic db-credentials \
-n "<namespace>" \
--from-literal DATABASE_ROOT_USER="<root_user>" \
--from-literal DATABASE_ROOT_PASSWORD="<root_password>"
```

This secret should include details for the blockchain node, bundler, and the private key associated with the broker operator wallet.
```sh
kubectl create secret generic db-credentials \
-n "<namespace>" \
--from-literal BLOCKCHAIN_RPC="<node_ws_url>" \
--from-literal USEROP_CLIENT_PROVIDER_URL="<node_rpc_url>" \
--from-literal USEROP_CLIENT_BUNDLER_URL="<bundler_rpc_url>" \
--from-literal SIGNER_LOCAL_PRIVATE_KEY="<broker_private_key>"
```

Ensure that these secrets are securely managed and that access to them is restricted according to the principles of least privilege and need-to-know basis. After setting up the necessary configurations and secrets, proceed with the Helm chart deployment to instantiate the stack on your Kubernetes cluster.

## Installation

First, add the Yellow Stack Helm repository to your local Helm installation:
```sh
helm repo add yellow-stack https://layer-3.github.io/stack
```

Next, install the stack chart with the following command:
```sh
helm install "<release>" yellow-stack/stack \
-n "<namespace>" \
-f "<path_to_values.yaml>"
```

Replace the placeholders with the desired release name, namespace, and the path to the modified `values.yaml` file. The deployment process will begin, and the stack components will be initialized according to the specified configuration.

## Verification

After the deployment process is complete, verify that the stack is operational by checking the status of the pods and TLS certificates:
```sh
kubectl get pods -n "<namespace>"
kubectl get cert -n "<namespace>"
```

If the pods are running and the certificates are issued, the stack is successfully deployed and ready for use. Proceed to test the functionality of the applications and verify that the stack is operational.