game-ci · cloudymax · Nov 16, 2023 · Jun 3, 2023 · Jun 3, 2023 · Jun 3, 2023
diff --git a/docs/12-self-hosting/01-overview.mdx b/docs/12-self-hosting/01-overview.mdx
@@ -0,0 +1,97 @@
+---
+toc_max_heading_level: 4
+---
+
+# About Self-Hosting
+
+Projects often encounter a constraint or requirement which makes free-tier hosted CI/CD runners
+insufficient for their needs. In these cases hosting your own CI/CD runner can be a viable
+alternative to premium-tier services or subscriptions. Self-hosting may also provide access to
+resources that are simply not available on many CI/CD services such as GPUs, faster drives, and
+newer CPU models.
+
+This guide will cover basic methods for hosting CI/CD runners on Bare-Metal, Virtual Machines, or
+using Cloud Runner. Containerized hosts will not be discussed because of their inherent reliance on
+insecure practices such as
+[Docker-in-Docker](http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/),
+[Privileged Containers](https://www.trendmicro.com/en_th/research/19/l/why-running-a-privileged-container-in-docker-is-a-bad-idea.html)
+and the additional tooling required to mitigate those risks such as
+[Kaniko](https://github.com/GoogleContainerTools/kaniko) or
+[Kata Containers](https://katacontainers.io/).
+
+## 📚 Prerequisite Knowledge
+
+Users of this guide should already be familiar with the Linux command-line, Shell scripting, and
+have a basic grasp of CI/CD concepts. For users who are not familiar with these concepts, we have
+included some resources for you to start your learning journey below.
+
+- [Techworld with Nana](https://www.youtube.com/@TechWorldwithNana)
+- [DevOps Toolkit](https://www.youtube.com/@DevOpsToolkit)
+- [Introduction to Bash Scripting](https://itsfoss.com/bash-scripting-tutorial/)
+
+## 📋 Constraints
+
+There are many ways to self-host CI/CD runners, and which one is best for you will depend on your
+own situation and constraints. For the purpose of this guide we will make the following assumptions:
+
+- 💻 User already has their own hardware
+- 💸 Budget for new hardware, software, or services is $0
+- 🛠️ FOSS tools should be prioritized where possible
+- 📜 We define `Self-Hosting` in this context to refer to a user taking responsibility for the
+  operating-system level configuration and life-cycle-management of a given compute resource (metal,
+  on-prem, cloud VM, VPS etc...)
+
+## ⚠️ Security Disclaimer
+
+This guide strives to maintain a balance between convenience and security for the sake of usability.
+The examples included in this guide are intended for use with on-prem hardware without public IP
+addresses accessible from external networks. Security is a constantly moving target which requires
+continuous effort to maintain. Users should conduct their own security review before using the
+following techniques on production or public systems.
+
+## ⚡️ Power Costs
+
+Hosting your own runners also comes with an increase in power consumption. This will vary based on
+the hardware you use and the prices of energy in your area. Below are some useful resources for
+discovering the potential energy costs of self-hosting.
+
+- https://outervision.com/power-supply-calculator
+- https://energyusecalculator.com/electricity_computer.htm
+
+## 💻 System Requirements
+
+This guide is tested on devices which meet the following requirements:
+
+- x86 or amd64 processor
+- Ubuntu 22.04 LTS Server or Debian 12 Bookworm
+- Root access to the operating system
+- Network connectivity
+
+## 📎 Quick Links:
+
+### Host Creation
+
+"Host Creation" in this context is the process of installing an operating system onto a piece of
+physical hardware, or the creation and configuration of virtualised compute resources.
+
+- [Bare-Metal](./03-host-creation/02-bare-metal.mdx)
+- [Virtual Machines using Multipass](./03-host-creation/02-multipass.mdx)
+- [Virtual Machines using QEMU](./03-host-creation/03-QEMU/01-overview.mdx)
+
+### Host Provisioning
+
+"Provisioning" here refers to the process of installing additional resources onto, and the
+configuration of your host beyond installing the base operating-system. Both manual and declarative
+workflows are supported.
+
+- [Manual Ubuntu 22.04 Setup](./04-host-provisioning/02-ubuntu-setup.mdx)
+- [Manual Debian 12 Setup](./04-host-provisioning/01-debian-setup.mdx)
+- [Declarative provisioning via Cloud-Init](./04-host-provisioning/03-cloud-init/01-about.mdx)
+
+### Runner Application Installation
+
+Once your host has been provisioned, you will then need to install the appropriate runner
+application. The guides below will walk you through that process.
+
+- [Github Actions](./05-runner-application-installation/02-github-actions.mdx)
+- [GitLab Pipelines](./05-runner-application-installation/01-gitlab-pipelines.mdx)
diff --git a/docs/12-self-hosting/02-host-types.mdx b/docs/12-self-hosting/02-host-types.mdx
@@ -0,0 +1,129 @@
+import Virtualisation from '/assets/images/Virtualization.drawio.png';
+import Metal from '/assets/images/Metal.drawio.png';
+import Docker from '/assets/images/DockerHost.drawio.png';
+import Kubernetes from '/assets/images/kubernetes.drawio.png';
+import Layers from '/assets/images/k8s-layers.drawio.png';
+import Layer0 from '/assets/images/k8s-layer0.drawio.png';
+import Layer1 from '/assets/images/k8s-layer1.drawio.png';
+import Layer2 from '/assets/images/k8s-layer2.drawio.png';
+import Layers01 from '/assets/images/k8s-layers01.drawio.png';
+import Layers012 from '/assets/images/k8s-layers012.drawio.png';
+
+# Types of Hosts
+
+## Bare-Metal
+
+"Bare Metal" means that your host OS is running directly on a piece of hardware without any
+virtualisation. This reduces the complexity of deployment at the cost of increased time and effort
+for re-provisioning the host.
+
+<div style={{ textAlign: 'center' }}>
+  <a target="\_blank" href={require('/assets/images/Metal.drawio.png').default}>
+    <img src={Metal} width="500" />
+  </a>
+</div>
+<br />
+
+## Virtual Machines
+
+Virtual Machines are a software-defined layer of abstraction atop a Bare-Metal host which makes
+deployments more consistent and easier to manage declaratively. This greatly reduces the difficulty
+of re-deployment and creates the conditions required for securely running multiple guests within the
+same physical host. Virtual Machines can also be used to create hosts that run different operating
+systems (Windows, MacOS) or architectures (ARM) than the host machine. This added functionality
+comes at the cost of added complexity, a slight performance penalty, and you need to already have a
+Bare-Metal host on which to run the VMs.
+
+<div style={{ textAlign: 'center' }}>
+  <a target="\_blank" href={require('/assets/images/Virtualization.drawio.png').default}>
+    <img src={Virtualisation} width="500" />
+  </a>
+</div>
+<br />
+
+Additional Reading:
+
+- [A Study of Performance and Security Across the Virtualization Spectrum](https://repository.tudelft.nl/islandora/object/uuid:34b3732e-2960-4374-94a2-1c1b3f3c4bd5/datastream/OBJ/download) -
+  Vincent van Rijn
+- [Hyper-converged infrastructure](https://en.wikipedia.org/wiki/Hyper-converged_infrastructure) -
+  Wikipedia
+- [Rethinking the PC](https://www.computerworld.com/article/3518849/rethinking-the-pc-why-virtual-machines-should-replace-operating-systems.html) -
+  Rob Enderle
+
+## Containers
+
+Containers are built on 'cgroups' (control groups), which are a feature of the Linux kernel that
+limits monitors, and isolates the resource usage of a collection of processes. This means that
+running containers on Linux is very lightweight form of virtualisation. However, on other operating
+systems which do not use the Linux kernel, a Linux virtual machine or translation-layer must be
+created to run containers. The manner by which each Operating System resolves this issue varies
+greatly as shown below. Because of this variance, the self-hosting documentation targets Linux as a
+means of avoiding excess complexity.
+
+<div style={{ textAlign: 'center' }}>
+  <a target="\_blank" href={require('/assets/images/DockerHost.drawio.png').default}>
+    <img src={Docker} width="500" />
+  </a>
+</div>
+<br />
+
+Additional Reading:
+
+- [The Mental Model Of Docker Container Shipping](https://bernhardwenzel.com/2022/the-mental-model-of-docker-container-shipping/) -
+  Bernhard Wenzel
+- [Why is Docker-in-Docker considered bad?](https://devops.stackexchange.com/questions/676/why-is-docker-in-docker-considered-bad)
+- [Why it is recommended to run only one process in a container?](https://devops.stackexchange.com/questions/447/why-it-is-recommended-to-run-only-one-process-in-a-container)
+
+## Kubernetes (Cloud Runner)
+
+Kubernetes is somewhat of a combination of all other host types. Since it is an API, it must be
+installed on an existing host (called a "Node") which is usually either a VM or physical device. A
+Kubernetes "Cluster" is usually made up of 3 or more nodes - though you can have as few as one, or
+as many 5,000 per cluster.
+
+<br />
+<div style={{ textAlign: 'center' }}>
+  <a target="\_blank" href={require('/assets/images/k8s-layer0.drawio.png').default}>
+    <img src={Layer0} width="700" />
+  </a>
+  <br />
+</div>
+<br />
+
+Once installed, Kubernetes creates
+[standardised interfaces](https://matt-rickard.com/kubernetes-interfaces) to control the hardware &
+software components of the underlying nodes (networking, storage, GPUs, CPU cores etc...) as well as
+a distributed key-value store which facilitates communication between all nodes in the cluster.
+
+<br />
+<div style={{ textAlign: 'center' }}>
+  <a target="\_blank" href={require('/assets/images/k8s-layers01.drawio.png').default}>
+    <img src={Layers01} width="750" />
+  </a>
+  <br />
+</div>
+<br />
+
+With the underlying hardware abstracted into a generic pool of resources, Kubernetes is then able to
+re-compose those assets into isolated environments called "Namespaces" where it deploys
+containerised workloads in groups called "Pods". This layer of Kubernetes is very similar to a
+typical container host but with many more features for multi-tenancy, security, and life-cycle
+management.
+
+<br />
+<div style={{ textAlign: 'center' }}>
+  <a target="\_blank" href={require('/assets/images/k8s-layers012.drawio.png').default}>
+    <img src={Layers012} width="800" />
+  </a>
+</div>
+<br />
+
+Additional Reading:
+
+- [Kubernetes Components](https://kubernetes.io/docs/concepts/overview/components/) - kubernetes.io
+- [A visual guide to Kubernetes networking fundamentals](https://opensource.com/article/22/6/kubernetes-networking-fundamentals) -
+  Nived Velayudhan
+- [Thinking about the complexity of the Kubernetes ecosystem](https://erkanerol.github.io/post/complexity-of-kubernetes/) -
+  Erkan Erol
+- [Ephemeral, Idempotent and Immutable Infrastructure ](https://cloudnativenow.com/topics/ephemeral-idempotent-and-immutable-infrastructure/) -
+  Marc Hornbeek
diff --git a/docs/12-self-hosting/03-host-creation/02-bare-metal.mdx b/docs/12-self-hosting/03-host-creation/02-bare-metal.mdx
@@ -0,0 +1,41 @@
+import Metal from '/assets/images/Metal.drawio.png';
+
+# Bare-Metal
+
+The Host is the computer that will execute the runner application. This can be a desktop computer,
+laptop, Virtual Machine, or VPS from a cloud provider.
+
+<div style={{ textAlign: 'center' }}>
+  <a target="\_blank" href={require('/assets/images/Metal.drawio.png').default}>
+    <img src={Metal} width="500" />
+  </a>
+</div>
+<br />
+
+## If your host is a local machine:
+
+For a local machine you will need to perform a clean installation of the operating system. This
+means creating a bootable USB drive from an ISO file, booting the machine from the USB drive, and
+installing the OS. Links to download an official Live ISO file and installation guides are provided
+below. If you would like to create a custom ISO, try [PXEless](https://github.com/cloudymax/pxeless)
+or [Cubic](https://github.com/PJ-Singh-001/Cubic).
+
+### Ubuntu
+
+- Download the Ubuntu 22.04 LTS
+  [server installer](https://ftp.snt.utwente.nl/pub/os/linux/ubuntu-releases/22.04.3/ubuntu-22.04.3-live-server-amd64.iso)
+- [Guide: Install Ubuntu 22.04 LTS on a local machine](https://ostechnix.com/install-ubuntu-server/)
+
+### Debian
+
+- Download the Debian 12
+  [installation image](https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-12.1.0-amd64-DVD-1.iso)
+- [Guide: Install Debian on a local system](https://www.linuxtechi.com/how-to-install-debian-11-bullseye/)
+
+## If your host is a virtual-machine:
+
+If you are using a VPS or VM, the OS should already be installed and admin user should already
+exist. Follow the appropriate guide in the provisioning section for your operating-system.
+
+- [Ubuntu 22.04](../04-host-provisioning/02-ubuntu-setup.mdx)
+- [Debian 12](../04-host-provisioning/01-debian-setup.mdx)