Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add self-hosting documentation #419

Merged
merged 66 commits into from
Nov 16, 2023
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
a0084b0
Update README.md
cloudymax Jun 3, 2023
9ad4023
Add files via upload
cloudymax Jun 3, 2023
fc851bc
remove extra line break
cloudymax Jun 3, 2023
89f9165
Update README.md
cloudymax Jun 4, 2023
3368605
Update README.md
cloudymax Jun 4, 2023
6c784e3
Update README.md
cloudymax Jun 4, 2023
6e5fa83
Update README.md
cloudymax Jun 4, 2023
d821fc1
Update README.md
cloudymax Jun 4, 2023
f010069
fix formatting
cloudymax Jun 15, 2023
c4e5ba4
change link format from html to markdown
cloudymax Jun 15, 2023
0bb7abe
missed a link
cloudymax Jun 15, 2023
1f64e53
Run prettier
cloudymax Jun 15, 2023
df4ad52
run prettier with yarn instead of npx?
cloudymax Jun 15, 2023
3a2a071
fix codeblock formatting
cloudymax Jun 24, 2023
5a17baf
run prettier formatting
cloudymax Jun 24, 2023
f2db0be
Merge branch 'main' into main
davidmfinol Jul 4, 2023
1b2525d
adding self-hosting docs
cloudymax Sep 10, 2023
4ce6f92
Merge branch 'self-hosting-docs' into self-hosting2
cloudymax Sep 10, 2023
76ad692
revise language based on feedback, add instructions for runner applic…
cloudymax Sep 22, 2023
b628a6c
start adding QEMU
Oct 9, 2023
a85e4ad
add images to vm setup docs, separate out qemu types
cloudymax Oct 10, 2023
d3f5aef
add new qemu dir
cloudymax Oct 10, 2023
b76a19a
finish up Windows and add macos
Oct 11, 2023
d0a8e5a
run prettier on readme
Oct 11, 2023
68b76c7
add images to windows, cleanup formatting, fix spelling errors
Oct 15, 2023
8e695eb
change organasation structure and fix typos
Oct 16, 2023
a009b59
re-roganize, add cloud-init details, dogfood testing
Oct 18, 2023
26c8702
Fix spellign errors, dogfooding, add advanced qemu config guides
Oct 18, 2023
48dd2fd
testing some style stuff that the linter doesnt like
Oct 18, 2023
98d1867
testing some style stuff that the linter doesnt like
Oct 18, 2023
349074b
changing the syle to abuse titles less
Oct 18, 2023
a3b86e7
changing the syle to abuse titles less
Oct 18, 2023
1fa9ff5
changing the syle to abuse titles less
Oct 18, 2023
b0c9d86
changing the syle to abuse titles less
Oct 18, 2023
a601556
changing the syle to abuse titles less
Oct 18, 2023
03fe6fc
cleanup dangling :
Oct 18, 2023
0d54d45
Formatting pass, add new diagrams
Oct 28, 2023
b415d27
center images and lint
Oct 28, 2023
7847591
proof-reading and docgfood pass on k8s docs
Oct 28, 2023
60715a3
add info on creating secrets, minor formatting
Oct 29, 2023
8134d3f
move k8s to new branch to unblock the rest of the docs
Nov 1, 2023
7881aa3
re-center images
Nov 1, 2023
2f5404a
add paddig above ssh image
Nov 1, 2023
20bbf9e
change image indent so linter wont wreck the spacing
Nov 1, 2023
236e54e
remove text artifact inserted by linter
Nov 1, 2023
15671f4
center images
Nov 1, 2023
7d464a4
add disclaimers to advanced configs, add more examples to windows
Nov 1, 2023
f94cbd7
remove text artifact introduced by linter and minor formatting fix
Nov 1, 2023
c1398c5
Merge branch 'main' into self-hosting2
cloudymax Nov 1, 2023
2be1df0
add missing image file
Nov 9, 2023
c3f23a4
Update docs/12-self-hosting/01-overview.mdx
cloudymax Nov 9, 2023
bd90456
Update docs/12-self-hosting/03-host-creation/03-QEMU/01-overview.mdx
cloudymax Nov 9, 2023
10e6d78
Update docs/12-self-hosting/03-host-creation/03-QEMU/01-overview.mdx
cloudymax Nov 9, 2023
8bbf93a
Update docs/12-self-hosting/03-host-creation/03-QEMU/03-linux-liveiso…
cloudymax Nov 9, 2023
e6c6986
fix issue with metal diagram failing to render as a svg, its a png now
Nov 9, 2023
ffb2654
remove accidental file addition
Nov 9, 2023
1289fc1
fix formatting
Nov 9, 2023
194f438
Update docs/12-self-hosting/03-host-creation/03-QEMU/03-linux-liveiso…
cloudymax Nov 9, 2023
0ffff35
add power usage blurb
Nov 15, 2023
b7d5010
add doublequotes to VNC invocation
Nov 15, 2023
1b6d057
add sections to overview
Nov 15, 2023
a2d56f7
add not on hackintosh legality
Nov 16, 2023
bdcf1f7
add note about remote access security to MacOS
Nov 16, 2023
f1338de
fix typo in link
Nov 16, 2023
a958137
fix linux path for multipass
Nov 16, 2023
8f74d7e
Merge branch 'main' into self-hosting2
davidmfinol Nov 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,18 @@ You may also view the markdown files directly at the following links:

## Building the Docs Site


Built with [Docusaurus 2](https://docusaurus.io/).


This project has a hard dependancy on Node16. You will need to make sure that you have the proper
version of node installed to avoid errors.


- [Install NodeJS v18 on Ubuntu](https://joshtronic.com/2022/04/24/how-to-install-nodejs-18-on-ubuntu-2004-lts/)
- [Install NodeJS v18 using brew](https://apple.stackexchange.com/a/207883)


This project relies on yarn for package management. You will need to install yarn in order to build
and test the documentation site.

Expand Down
83 changes: 83 additions & 0 deletions docs/12-self-hosting/01-getting-started.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
toc_max_heading_level: 4
---

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please only use this configuration to change the default if you really need it. Recommend getting rid of them for all files unless you can not use better headings and need to override it.

# Self-Hosting

Projects often encounter a constraint or requirement which make free-tier hosted CI/CD instances
insufficient for their needs. In these cases hosting your own CI/CD runner can be a viable solution
to premium-tier services or subscriptions. Self-hosting may also provide access to resources that
are simply not available on many CI/CD services such as GPUs, faster drives, and newer CPU models.

There are many ways to self-host CI/CD runners, and which one is best for you will depend on your on
situation and constraints. For the purpose of this guide we will make the following assumptions:

- User already has their own hardware 💻
- Budget is $0 💸
- FOSS tools should be prioritized where possible 🛠️
- We define `Self-Hosting` in this context to refer to a user taking responsibilty for the
operating-system level configuration and life-cycle-management of a given compute resource (metal,
on-prem, cloud VM, VPS etc...) 📜
cloudymax marked this conversation as resolved.
Show resolved Hide resolved

## Requirements

This guide is tested for compatibility on devics which meet the following requirements:

- x86 or amd64 processor
- Ubuntu 22.04 LTS Server or Debian 12 Bookworm
- root access
- Network connectivity
- Nvidia GPU (required for GPU Acceleration)
cloudymax marked this conversation as resolved.
Show resolved Hide resolved

## Finding a Host

The "Host" is the computer which will execute the runner program. This can be a desktop computer,
cloudymax marked this conversation as resolved.
Show resolved Hide resolved
laptop, Virtual Machine, or VPS from a cloud provider. If your host is cloud-instance then the OS
and admin user should already exist. Move on to the provisioning section for further instructions.
If your host is a local machine, perform a clean installation of the operating system using the
cloudymax marked this conversation as resolved.
Show resolved Hide resolved
following guides:

Ubuntu:

- Download the Ubuntu 22.04 LTS server installer
[HERE](https://ftp.snt.utwente.nl/pub/os/linux/ubuntu-releases/22.04.3/ubuntu-22.04.3-live-server-amd64.iso)
cloudymax marked this conversation as resolved.
Show resolved Hide resolved
- [Install Ubuntu 22.04 LTS on a local machine](https://ostechnix.com/install-ubuntu-server/)

Debian:

- Download the Debain 12 installer
[HERE](https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-12.1.0-amd64-DVD-1.iso)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Download the Debain 12 installer
[HERE](https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-12.1.0-amd64-DVD-1.iso)
- Download the Debian 12 [installer](https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-12.1.0-amd64-DVD-1.iso)

Note the typo in Debian

- [Install Debian on a local system](https://www.linuxtechi.com/how-to-install-debian-11-bullseye/)

## Provisioning the Host

Once you can sign into your host, follow the appropriate guide to complete the provisioning process.
Note that the Nvidia driver and container-toolkit installation are only required for GPU
acceleration. Skip those steps if you do require that functionality.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Once you can sign into your host, follow the appropriate guide to complete the provisioning process.
Note that the Nvidia driver and container-toolkit installation are only required for GPU
acceleration. Skip those steps if you do require that functionality.
Once you can sign in on your Host, follow the appropriate guide to provision it.
Note that the Nvidia driver and container-toolkit installations are only required for GPU
acceleration. Skip those steps if you do not require that functionality.


- [Ubuntu 22.04 Setup](./ubuntu-setup)

- [Debian 12 Setup](./debian-setup)

## Splitting the Host

You may find yourself with the requirement for multiple runners, runners of differing
configurations, or different operating systems but only have access to a single Host. In this case,
you can split your Host into multiple runners using virtualization. If this applies to you, complete
the provisioning step and then follow the [Virtual Machines Guide](./virtual-machines).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case people don't navigate back here, might it be useful to simplify this text into a single line and append it to the bottom of the provisioning steps? For example:

If you want to run multiple runners on this host, proceed with the [Virtual Machines guide](./virtual-machines).

Also, since guides are a first class citizen on this website it doesn't really need to be a keyword and use lowercase guide instead.


## Installing the Runner Software
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Installing the Runner Software
## Installing the Runner


- [Github Actions](./github-actions)

- [GitLab Pipelines](./gitlab-pipelines)

- CircleCI (ToDo)

- Argo Workflows (ToDo)

- Jenkins (ToDo)

- Ansible/Semaphore/AWX (ToDo)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can convert this from technical to functional language by using words like "not yet supported"


## Update Workflows
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Update Workflows
## Updating workflows

Loading