diff --git a/1.0.0/img/boot-components.png b/1.0.0/img/boot-components.png new file mode 100644 index 0000000..024503f Binary files /dev/null and b/1.0.0/img/boot-components.png differ diff --git a/1.0.0/img/certificate-ca-sign.png b/1.0.0/img/certificate-ca-sign.png new file mode 100644 index 0000000..42a0843 Binary files /dev/null and b/1.0.0/img/certificate-ca-sign.png differ diff --git a/1.0.0/img/certificate-encryption.png b/1.0.0/img/certificate-encryption.png new file mode 100644 index 0000000..f22550a Binary files /dev/null and b/1.0.0/img/certificate-encryption.png differ diff --git a/1.0.0/img/favicon.png b/1.0.0/img/favicon.png new file mode 100644 index 0000000..7667a25 Binary files /dev/null and b/1.0.0/img/favicon.png differ diff --git a/1.0.0/img/jump-host.png b/1.0.0/img/jump-host.png new file mode 100644 index 0000000..4a05d29 Binary files /dev/null and b/1.0.0/img/jump-host.png differ diff --git a/1.0.0/img/keypair-encryption.png b/1.0.0/img/keypair-encryption.png new file mode 100644 index 0000000..8e4e571 Binary files /dev/null and b/1.0.0/img/keypair-encryption.png differ diff --git a/1.0.0/img/keypair-signature.png b/1.0.0/img/keypair-signature.png new file mode 100644 index 0000000..8720502 Binary files /dev/null and b/1.0.0/img/keypair-signature.png differ diff --git a/1.0.0/img/multiple-namespaces-mode.png b/1.0.0/img/multiple-namespaces-mode.png new file mode 100644 index 0000000..17c8346 Binary files /dev/null and b/1.0.0/img/multiple-namespaces-mode.png differ diff --git a/1.0.0/img/single-namespace-mode.png b/1.0.0/img/single-namespace-mode.png new file mode 100644 index 0000000..7868414 Binary files /dev/null and b/1.0.0/img/single-namespace-mode.png differ diff --git a/1.0.0/index.html b/1.0.0/index.html new file mode 100644 index 0000000..ff46e18 --- /dev/null +++ b/1.0.0/index.html @@ -0,0 +1,3399 @@ + + + + + + + + + + +Boot - Solution to launch lightweight test and development environments + + + + + + + +
+
+
+
+

This reference for version: 1.0.0

+
+
+

Boot is a solution to launch lightweight test and development environments in Kubernetes.

+
+
+
+
+

1. Getting Started

+
+
+

1.1. Quick Start Guide

+
+ + + + + +
+ + +
+

This section shows how to install Boot limited to 1 CPU. Detailed information on installing a license key allowing to activate more CPUs is shown in Installing License section.

+
+
+
+
+

1.1.1. Installing to Kubernetes

+
+
Prerequisites
+
+
    +
  1. +

    Running Kubernetes cluster.

    +
    2. +
  2. +

    kubectl client installed and pointing to the cluster

    +
    3. +
  3. +

    A pair of SSH keys which is often stored in ~/.ssh/ directory as id_rsa (private key) and id_rsa.pub (public key) files. If you don’t have such files - use ssh-keygen command to generate them. To install Boot you will only need public key file which usually looks like this:

    +
    +
    +
    $ cat ~/.ssh/id_rsa.pub
+ssh-rsa AAAAB3N... some-comment
    +
    +
    +
    4. +
  4. +

    If you are starting Boot in Kubernetes cluster deployed on workstation with minikube tool - use the following commands to start Kubernetes:

    +
    +
    Minikube startup under Linux
    +
    +
    $ minikube start --driver kvm2
    +
    +
    +
    +
    Minikube startup under MacOS
    +
    +
    $ minikube start --driver=hyperkit
    +
    +
    +
    +
    Minikube startup under Windows
    +
    +
    $ DISM /Online /Enable-Feature /All /FeatureName:Microsoft-Hyper-V # Enable Hyper-V
+$ minikube start --driver=hyperv
    +
    +
    +
    5. +
+
+
+
+
Install Boot with Helm
+
+ + + + + +
+ + +Helm chart is the recommended Boot installation way. Steps below require Helm 3 and will not work with older releases. +
+
+
+

We deliver already packed and published Helm charts, so installing Boot with Helm is straightforward:

+
+
+
    +
  1. +

    Prepare a values.yaml file with Helm chart settings:

    +
    +
    +
    boot:
+  jumphost:
+    authorizedKeys:
+      - ssh-rsa AAAAB3N... some-comment # Insert at least one public key contents here
    +
    +
    +
    2. +
  2. +

    Add Aerokube charts repository:

    +
    +
    +
    $ helm repo add aerokube https://charts.aerokube.com/
+$ helm repo update
    +
    +
    +
    3. +
  3. +

    To list available Boot versions type:

    +
    +
    +
    $ helm search repo aerokube --versions
    +
    +
    +
    4. +
  4. +

    Create a namespace:

    +
    +
    +
    $ kubectl create namespace boot
    +
    +
    +
    5. +
  5. +

    To install or upgrade Boot type:

    +
    +
    +
    $ helm upgrade --install -f values.yaml -n boot boot aerokube/boot
    +
    +
    +
    6. +
  6. +

    Boot chart has a lot of different configuration parameters that can be listed as follows:

    +
    +
    +
    $ helm show values aerokube/boot
    +
    +
    +
    +

    To change one of these parameters - add an override to values.yaml file and reapply Helm chart again.

    +
    +
    7. +
  7. +

    Detect Boot load balancer IP address. Usually this address can be obtained from Boot service:

    +
    +
    Get Boot service IP address
    +
    +
    $ kubectl get svc boot -n boot
+NAME   TYPE           CLUSTER-IP       EXTERNAL-IP    PORT(S)    AGE
+boot   LoadBalancer   10.107.117.163   192.168.64.5   2222/TCP   15d
    +
    +
    +
    +

    IP address is indicated in EXTERNAL-IP column. On Minikube you need to explicitly add IP address to Boot service with minikube ip command output:

    +
    +
    +
    Patch Boot service with minikube IP address
    +
    +
    $ kubectl patch svc boot -n boot --patch "{\"spec\":{\"externalIPs\":[\"$(minikube ip)\"]}}"
    +
    +
    +
    +

    On Windows - you may need to insert minikube ip output manually, because $() expression expansion may not work.

    +
    +
    8. +
  8. +

    Make sure there is a domain name pointing to IP address. On Minikube - simply add an entry to hosts file:

    +
    +
    +
    $ sudo echo "$(minikube ip) boot.aerokube.local" >> /etc/hosts
    +
    +
    +
    +

    On Windows you may need to update hosts file manually.

    +
    +
    9. +
  9. +

    Configure your SSH client to use Boot. To do that add the following lines to ~/.ssh/config file:

    +
    +
    +
    $ cat ~/.ssh/config
+# ... some lines here
+Host bootjump
+    IdentityFile ~/.ssh/id_rsa
+    HostName boot.aerokube.local
+    Port 2222
+Host *.vm.boot.svc.cluster.local
+    IdentityFile ~/.ssh/id_rsa
+    ProxyJump jump@bootjump
    +
    +
    +
    10. +
  10. +

    Create your first virtual machine. To do that create a YAML file with virtual machine definition:

    +
    +
    +
    $ cat ~/vm.yaml
+apiVersion: boot.aerokube.com/v1
+kind: VirtualMachine
+metadata:
+  name: my-vm
+  namespace: boot
+spec:
+  os: ubuntu
+  authorizedKeys:
+    - ssh-rsa AAAAB3N... some-comment # The same public key contents here too
    +
    +
    +
    +

    Then apply this file as follows:

    +
    +
    +
    +
    $ kubectl create -f vm.yaml
    +
    +
    +
    11. +
  11. +

    Check SSH connectivity to virtual machine:

    +
    +
    +
    $ ssh root@my-vm.vm.boot.svc.cluster.local
+# Some welcome text goes here...
+root@my-vm:~# # You now have a root access to created virtual machine
    +
    +
    +
    12. +
+
+
+
+
+
+

1.2. Architecture

+
+

1.2.1. Boot Components

+
+
Boot Components
+

boot components

+
+
+

Boot cluster consists of several components:

+
+
+
    +
  1. +

    One or more Boot application instances. Their main purpose is to start/stop virtual machines and to control access to created virtual machines via SSH. Boot is usually accessible on some SSH port, e.g. 2222.

    +
    2. +
  2. +

    One or more Boot UI application instances (currently under development). Boot UI collects information from Boot and visualizes it. It is usually available on HTTP port 8080.

    +
    3. +
  3. +

    Running virtual machine pods and respective virtual machine custom resources.

    +
    4. +
+
+
+
+

1.2.2. Boot Pod Contents

+
+

Every Boot pod contains several containers with different purpose.

+
+ + ++++ + + + + + + + + + + + + + + + + + + + + +
Table 1. Boot Pod Containers
NamePurpose

boot

Starts and stops virtual machines

jumphost

Regular SSH server with secure configuration working as SSH jump host

reloader

During Boot updates waits for configured period of time for SSH connections to virtual machines to close or explicitly closes them after that time

+
+
+

1.2.3. Multiple Namespaces Support

+
+

Boot can launch virtual machines in any number of Kubernetes namespaces. By default, Boot itself and all launched virtual machines are running in the same Kubernetes namespace. This is suitable if only one team is using Boot or you don’t need to limit virtual machines consumption for different Boot users.

+
+
+
Boot with Multiple Namespaces Configured
+

multiple-namespaces-mode

+
+
+

It is easy to configure Boot to run in one namespace and to launch virtual machines in separate namespaces. Multiple namespaces are mainly needed when you want to control computing resources, virtual machine operating system versions or network access rules (network policies) available for every team. For example, user 1 virtual machines will be running in team 1 namespace and user 2 virtual machines will be running in team 2 namespace. How to configure multiple namespaces is described here.

+
+
+
+
+

1.3. Required Permissions

+
+

Boot requires a limited set of permissions and should work with default Kubernetes settings. Boot can launch virtual machines in any number of Kubernetes namespaces. All required roles are automatically created with Helm chart. The following table shows Boot permissions in the namespace where it is running:

+
+ + ++++ + + + + + + + + + + +
Table 2. Required Permissions for the Namespace where Boot runs

Permission

Purpose

To get, watch and list and patch Boot custom resources in boot.aerokube.com API group

These custom resources store Boot configuration and are cluster-wide, so a ClusterRole is needed for these resources.

+
+

For every user namespace Boot needs the following permissions:

+
+ + ++++ + + + + + + + + + + + + + + +
Table 3. Required Boot Permissions For User Namespace

Permission

Purpose

To get, watch, list, create, delete, update and patch pods

Used to manipulate pods with virtual machines

To get, watch, list, create, delete, update and patch config maps

Used to pass users and groups to virtual machines pods

+
+
+
+
+

2. Main Features

+
+
+

2.1. Virtual Machines

+
+

The main Boot feature is creating lightweight virtual machines. Such virtual machines can be used for:

+
+
+
    +
  • +

    Functional software testing, i.e. testing that implemented software features work as expected.

    +
    • +
  • +

    Penetration testing, i.e. testing that given software distribution does not contain security vulnerabilities.

    +
    • +
  • +

    Performance testing, i.e. testing how software performs in terms of responsiveness and stability under a particular workload.

    +
    • +
  • +

    Software development. This allows you to have more computing resources (CPUs, memory, disk) than a regular developer workstation allows.

    +
    • +
+
+
+

Like traditional virtual machines, Boot virtual machines are accessed using SSH. That means that you can use any standard tools working on top of SSH to update virtual machine state:

+
+
+ +
+
+

2.1.1. Listing Virtual Machines

+
+

Boot virtual machines are native Kubernetes objects, so listing them can be done with standard kubectl commands:

+
+
+
Listing virtual machines
+
+
$ kubectl get vm -n boot
+NAME    FQDN                              IP           OS            CPU         MEM     STATUS    UPTIME
+my-vm   my-vm.vm.boot.svc.cluster.local   172.17.0.4   alpine:3.18   250m/128m   512Mi   Running   4s
+
+
+
+

You can see virtual machine name, fully qualified domain name and IP address to access it, used operating system image, CPU and memory limits and requests (limits/requests or simply limits if they have the same value), status and uptime.

+
+
+
+

2.1.2. Creating Virtual Machines

+
+
Option 1. Using kubectl and YAML manifest
+
+

Creating a virtual machine is as easy as creating any other Kubernetes resource.

+
+
+
    +
  1. +

    Create a file with virtual machine YAML definition:

    +
    +
    +
    $ cat vm.yaml
+apiVersion: boot.aerokube.com/v1
+kind: VirtualMachine
+metadata:
+  name: my-vm
+  namespace: boot
+spec:
+  os: ubuntu
+  authorizedKeys: # Use one or more SSH public keys here
+    - ssh-rsa AAAAB3N... some-key
+    - ssh-rsa AAAAB3N... another-key
    +
    +
    +
    2. +
  2. +

    Create virtual machine using YAML definition:

    +
    +
    +
    $ kubectl create -f ~/vm.yaml
+virtualmachine.boot.aerokube.com/my-vm created
    +
    +
    +
    3. +
+
+
+

Now you can list virtual machines and obtain fully qualified domain name to access this virtual machine. How to access virtual machines is described in Accessing Virtual Machines section.

+
+
+
+
Option 2. Using Boot plugin for kubectl
+
+

The second possible way of creating Boot virtual machines is using Boot plugin for kubectl.

+
+
+

The simplest way of installing Boot plugin is using Krew package manager:

+
+
+
Installing Boot plugin with Krew
+
+
$ kubectl krew index add aerokube https://github.com/aerokube/krew-index.git
+$ kubectl krew install aerokube/vm
+
+
+
+

Another possibility is to download plugin distribution archive and unpack in to a directory in PATH:

+
+
+
Installing Boot plugin manually on Mac
+
+
$ curl -o /usr/local/bin/kubectl-vm https://github.com/aerokube/boot/releases/latest/download/kubectl-vm_darwin_amd64 # x86 version
+$ curl -o /usr/local/bin/kubectl-vm https://github.com/aerokube/boot/releases/latest/download/kubectl-vm_darwin_arm64 # ARM version
+$ chmod +x /usr/local/bin/kubectl-vm
+
+
+
+
Installing Boot plugin manually on Linux
+
+
$ curl -o /usr/local/bin/kubectl-vm https://github.com/aerokube/boot/releases/latest/download/kubectl-vm_linux_amd64
+$ chmod +x /usr/local/bin/kubectl-vm
+
+
+
+
Installing Boot plugin manually on Windows
+
+
> mkdir %USERPROFILE%\bin # Add %USERPROFILE%\bin to PATH
+> curl -o %USERPROFILE%\bin\kubectl-vm.exe https://github.com/aerokube/boot/releases/latest/download/kubectl-vm_windows_amd64.exe
+
+
+
+

To check that plugin was installed successfully:

+
+
+
+
$ kubectl vm --help
+
+
+
+

To create a virtual machine with plugin:

+
+
+
Creating a virtual machine with plugin
+
+
$ kubectl vm create -os ubuntu -n boot # Using latest operating system version
+$ kubectl vm create -os ubuntu:22.04 -n boot # Using exact operating system version
+
+
+
+

By default, Boot plugin will use your public key from ~/.ssh/id_rsa.pub, but you can explicitly provide a list of authorized keys:

+
+
+
Providing explicit authorized keys file to plugin
+
+
$ echo "ssh-rsa AAAAB3N... some-key" >> authorized_keys # Create a list of SSH authorized keys
+$ kubectl vm create -os ubuntu -auth-keys authorized_keys -n boot # Using latest operating system version
+
+
+
+
+
+

2.1.3. Deleting Virtual Machines

+
+

To delete a virtual machine use standard kubectl command:

+
+
+
Delete a virtual machine with kubectl
+
+
$ kubectl delete vm my-vm -n boot
+
+
+
+

To delete a virtual machine with kubectl plugin:

+
+
+
Delete a virtual machine with kubectl plugin
+
+
$ kubectl vm delete my-vm -n boot
+
+
+
+
+

2.1.4. Controlling Access

+
+

Boot supports two main SSH authentication technologies: authorized keys and user certificates. How they differ and work is described in Accessing Virtual Machines section. To add authorized public keys of allowed users - use authorizedKeys field in virtual machine specification:

+
+
+
Adding authorized keys
+
+
apiVersion: boot.aerokube.com/v1
+kind: VirtualMachine
+metadata:
+  name: my-vm
+  namespace: boot
+spec:
+  os: ubuntu
+  authorizedKeys: # A list of SSH public keys allowed to access this virtual machine
+    - ssh-rsa AAAAB3N... some-key
+    - ssh-rsa AAAAB3N... another-key
+
+
+
+

To allow users to access virtual machine with SSH certificates - you have to add certification authorities used to sign these user certificates to virtual machine trustedUserCAKeys field:

+
+
+
Adding trusted certification authorities
+
+
apiVersion: boot.aerokube.com/v1
+kind: VirtualMachine
+metadata:
+  name: my-vm
+  namespace: boot
+spec:
+  os: ubuntu
+  trustedUserCAKeys: # A list of trusted user certification authorities
+    - ssh-ed25519 AAAAC3N... certification-authority-1
+    - ssh-ed25519 AAAAC3N... certification-authority-2
+
+
+
+

How authorized keys and certificates differ is explained in detail in this section.

+
+
+
+

2.1.5. Adjusting Computing Resources

+
+ + + + + +
+ + +Boot is using exactly the same computing resources configuration YAML format as Kubernetes does. +
+
+
+

By default, virtual machine is consuming computing resources configured in #operating-systems-computing-resources[operating system settings]. You can easily override this behavior by adding resources explicitly:

+
+
+
Setting computing resources explicitly
+
+
apiVersion: boot.aerokube.com/v1
+kind: VirtualMachine
+metadata:
+  name: my-vm
+  namespace: boot
+spec:
+  os: ubuntu
+  resources: # Explicit computing resources
+    limits:
+      cpu: 1000m
+      memory: 512Mi
+    requests:
+      cpu: 1000m
+      memory: 512Mi
+
+
+
+

The same settings can be provided when creating virtual machine with kubectl plugin:

+
+
+
Providing explicit computing resource to a new virtual machine with a plugin
+
+
$ kubectl vm create -os ubuntu -cpu 1.0 -mem 512Mi -n boot
+$ kubectl vm create -os ubuntu -cpu '1.0/0.5' -mem '512Mi/256Mi' -n boot # Different requests and limits
+
+
+
+
+

2.1.6. Adding Volumes

+
+ + + + + +
+ + +Boot is using exactly the same volumes configuration YAML format as Kubernetes does. +
+
+
+

To share file between virtual machines or saving their state - use volumes. You can add a volume of any type supported by your Kubernetes cluster. To add a volume:

+
+
+
Setting computing resources explicitly
+
+
apiVersion: boot.aerokube.com/v1
+kind: VirtualMachine
+metadata:
+  name: my-vm
+  namespace: boot
+spec:
+  os: ubuntu
+  volumes: # We declare a volume
+  - name: my-volume
+    emptyDir: {}
+  volumeMounts: # We mount it to a path inside virtual machine
+  - name: my-volume
+    mountPath: /some/path
+
+
+
+
+

2.1.7. Adding Host Aliases

+
+

In some situations you may need to update /etc/hosts file inside virtual machine. To achieve this during virtual machine startup:

+
+
+
Updating /etc/hosts
+
+
apiVersion: boot.aerokube.com/v1
+kind: VirtualMachine
+metadata:
+  name: my-vm
+  namespace: boot
+spec:
+  os: ubuntu
+  hostAliases:
+    - ip: "127.0.0.1"
+      hostnames:
+      - "foo.local"
+      - "bar.local"
+    - ip: "10.1.2.3"
+      hostnames:
+      - "foo.remote"
+      - "bar.remote"
+
+
+
+
+
+

2.2. Accessing Virtual Machines

+
+

We already noted in virtual machines section that Boot virtual machines are accessed using SSH. This section describes different options allowing to configure secure access to virtual machines.

+
+
+

2.2.1. Overview

+
+

As you probably know SSH consists of two components: an SSH server (also known as sshd) and an SSH client (usually command is called just ssh). SSH server in the majority of cases is listening on standard TCP port 22. Boot is running in Kubernetes and thus every network port that you wish to access from the outside, should be explicitly configured as Kubernetes service. Our goal is to be able to run an unlimited number of virtual machines, so doing this for every machine is too much work. Also, we want to be able to globally protect all virtual machines from unauthorized access. SSH provides a standard way of solving this task called jump host. Jump host is an intermediary host that SSH client should successfully connect in order to access target host.

+
+
+
Jump Host
+

jump host

+
+
+

For security purposes Boot is by default using non-standard port 2222 for jump host. This is because in Unix family operating systems port numbers up to 1024 require root privileges for the process, and it is recommended to use non-privileged users when possible. When you install Boot with default settings jump host is configured automatically and should work out of the box. If you wish to have full control over security, existing jump host configuration options are described here.

+
+
+
+

2.2.2. Authentication Technologies

+
+

SSH supports two main authentication technologies: keys (simpler and more popular) and certificates (more complicated but more powerful). Both technologies rely on public key cryptography, which is itself based on mathematical problems that are easy to do in one direction and very hard to do in the reverse direction for modern computers. For example, RSA system is using computational complexity of finding factors of large prime numbers. Similarly, ECDSA and EdDSA systems are using computational complexity of finding discrete logarithm of an elliptic curve over finite fields.

+
+
+
User and Host Keys
+
+

In public key cryptography every side of communication (user and remote host) has a key. A key belonging to a user is called user key. This key is used to access remote hosts with SSH instead of providing a password. Similarly, a key belonging to a remote host (e.g. a Boot virtual machine) is called host key. The main goal of this key from the user point of view is verifying authenticity of the host, i.e. making sure you are accessing the host you expected to access.

+
+
+
Using private and public keys for encrypted data exchange
+

keypair encryption

+
+
+

Both user and host keys are divided into two parts: public key part (or just public key) and private key part (or just private key). Because every key contains two parts, it is often called a keypair. Public key part is storing exact parameters of the mathematical problem used by exact public cryptography algorithm, e.g. large prime numbers in RSA cryptography. Private key part is storing solution of the same mathematical problem, e.g. factors of the same large prime numbers. Because of that a public key part can only be used to encrypt some message that an owner of the private key part will be able to decrypt.

+
+
+
Using private and public keys for digital signatures
+

keypair signature

+
+
+

A private key part in addition to decrypting messages can be also used to sign any piece of information (called message). Signing (or creating a digital signature) means generating a set of bytes allowing to verify that a concrete message was created by the owner of the private key and that it was not modified since that time. Generated signature is sent simultaneously with the message and every recipient having a sender public key is able to verify that signature is valid. Because of these properties, a public key part can be freely sent over insecure communication channels. Contrarily, a private key part should be accessible to its owner only.

+
+
+

Going back to SSH, let’s take a look at a typical usage of the private and public keys. A pair of user private and public keys can be generated with one command:

+
+
+
Generating a key pair
+
+
$ ssh-keygen
+Generating public/private rsa key pair.
+Enter file in which to save the key (/home/myuser/.ssh/id_rsa): my_key
+Enter passphrase (empty for no passphrase):
+Enter same passphrase again:
+Your identification has been saved in my_key
+Your public key has been saved in my_key.pub
+The key fingerprint is:
+SHA256:gHexXzD2EuQFcXgsnJVTpkTKfDMSI5uFUK3J58/rjf8 my_key
+The key's randomart image is:
++---[RSA 3072]----+
+|      .o+=%O*oo  |
+|     .  .%*%=o   |
+|    . o.=oBoB.   |
+|     . o+..= o   |
+|        So.      |
+|          .      |
+|           o     |
+|            oo   |
+|           .+oo.E|
++----[SHA256]-----+
+
+
+
+

In this output private key was saved to my_key file in current directory and public key - to my_key.pub. These files look like this:

+
+
+
How keys look like
+
+
# Private key
+$ cat my_key
+-----BEGIN OPENSSH PRIVATE KEY-----
+b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn
+NhAAAAAwEAAQAAAYEAoi/l2u3snJTbHYgcOSSq2WHmUw9Wy10ldZgXDCgR7HahtxvWB0n0
+5XurIIeUmHpfLJoinJEK4ScI7uaagtwID7QoFHsok0zlkpSMHjJLbdYAAquotjmCzBuKtG
+3YMcPX0CojSCkDtFAliqUTDe+RfJ0aHVyUeqLJhCOKMEeG7UNALt0lzpUSDeVNYiBnlyd8
+B+iiBWtF7MGTiMTxk8fXbrO97XF8p1PjKhCppxjocilQIwhx5HB0rJjy5vDokpvrFc6kkc
+Mjff2FzLm1BbMH/WOfqNpszTRV3lK8PCAQgmWM/vBINRdY7Z6eOJVeDSvy+J+A88AjnfEq
+V0MHofSoYVrals26WGasFOCboJlTgsafFWYUxOi9BGPs5Ze1kqDZmbm6l3qWGC1TD0wyWw
+6Bg0XU0m5zOsmZPG6M8XbIAXWIErmxssTZHx1sUzfg5f5W7WdoWNoYqkajZY01+IB0PhCr
+BxRdio4B+pR4MVV0hgto8xtWr3x9qF+wa070SodfAAAFkI5E48yOROPMAAAAB3NzaC1yc2
+EAAAGBAKIv5drt7JyU2x2IHDkkqtlh5lMPVstdJXWYFwwoEex2obcb1gdJ9OV7qyCHlJh6
+XyyaIpyRCuEnCO7mmoLcCA+0KBR7KJNM5ZKUjB4yS23WAAKrqLY5gswbirRt2DHD19AqI0
+gpA7RQJYqlEw3vkXydGh1clHqiyYQjijBHhu1DQC7dJc6VEg3lTWIgZ5cnfAfoogVrRezB
+k4jE8ZPH126zve1xfKdT4yoQqacY6HIpUCMIceRwdKyY8ubw6JKb6xXOpJHDI339hcy5tQ
+WzB/1jn6jabM00Vd5SvDwgEIJljP7wSDUXWO2enjiVXg0r8vifgPPAI53xKldDB6H0qGFa
+2pbNulhmrBTgm6CZU4LGnxVmFMTovQRj7OWXtZKg2Zm5upd6lhgtUw9MMlsOgYNF1NJucz
+rJmTxujPF2yAF1iBK5sbLE2R8dbFM34OX+Vu1naFjaGKpGo2WNNfiAdD4QqwcUXYqOAfqU
+eDFVdIYLaPMbVq98fahfsGtO9EqHXwAAAAMBAAEAAAGAZlzastGlm7HrlXj5bytwVWEPsG
+6m9hVk9hI2wapsnZTGPj5oWBNaaJgkCpTnuVDKzui9XZnBhxdO8RFEhcD/qYGoJj0Q/97x
+qhDtWoWdy8XcHdNf2Rr1LYNYiMYnREl55V0jBYE1YFGRUC8dlpcUeNTizZNH9xrVGvwfVJ
+dgVlEyqiFTok29pl2J+JvBJcp64rb1w3vQFzyZtCGw4venRaxV/A27ghRU9JCtstPqqVrf
+xCypTWeYi/LAo/d6okWbGadc1y/RJZRob437+x2HuRg+bMStrOdBU3ry4OopHoHSF6UWHa
+/u9uhwbDASvXN0DgnfQe83GRTunzKDyK3zGUyRh6DpDP/btS1Y899CTE2QXIgOnSk6Ah/R
+zdY9Tw8yH02SvK4Ha4E+J1VUi3dWEt1izjv+qLbAgWJqn/i585vglp5NWE0BUGg56b6dme
+MbwJJ7uG1VRWNBwbzirVHmeElNo6KxvUvUYok/wNxCfKRIkpKrF+Z1cwSK8lGnVF5JAAAA
+wQCRweD9plo9rpk5vKggIHwGAsFVvgUBlfybJcCrza7fdGjw2fUnpdDDGN2nltLy3D+hn/
+jFeVgXBaa0TNzUSuLyKXflzuoHHimjoBGBSXCh/TLbsbebdXWr3LEorErqeBDj5NUiWdyx
+TEc0qXoL3k766ZtL67mYSM45kG76FKJxWpJ3JSB9NHG4FclQBGmZ1QphMEB4Nq1RiF5yH6
+ZqoguKWxmSfZ2LfLTTyR9k4TLWrIEzfMRkMCHF3fIRrTsygbkAAADBANSkkcgeiL9uBdyP
+C5W9zgLDPj4wCetdqlyP3Qoi1YvKBKwBNiW60UDQ1hda75roKmIr9rfFGZVYeYk0W/8JbS
+QXG9t8vtyoWghRciIYokqQ5LJpfO6fn7NOJe2t44pcQ1COux4lIlPukfMU762cXVEio8qv
+zAzlC09zbXtpxyQ9W/KT4PwI0o4AlYYANiSlPRjbXm8OtvBlD9Gp46Qp4Uzx1TBnWXEpKw
+E1dgbmAYuJ8NIiK7cF4TYlhRCFhJagnQAAAMEAw0Gpx6b3df776ZEDlL+9wOtiEpgNJzzI
+V/O9IzDXUIeJuduLnyPocuiPWm5bxUQD/m9ZHmZjmCPkRDszgTCc1wj32Yj2/ZqUidUkOd
+IlYreuzNFcfpsDj8tN3LMYcA5c+LrjYt/+pyu5lj5URKo/nMKy/2WZQW3jqZzuSt+YKTRC
+dxY2eKr+4J7p3UAJ6nU4YbTqfEvk6xZktrwfmJghW7Xsmjy2XTRU3UyzBM/K9REiuekry9
++jWazSoOMcZbErAAAAFXZhbmlhLXBvb2hAaTEwNzU5Nzk5OQECAwQF
+-----END OPENSSH PRIVATE KEY-----
+
+# Public key
+$ cat my_key.pub
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCiL+Xa7eyclNsdiBw5JKrZYeZTD1bLXSV1mBcMKBHsdqG3G9YHSfTle6sgh5SYel8smiKckQrhJwju5pqC3AgPtCgUeyiTTOWSlIweMktt1gACq6i2OYLMG4q0bdgxw9fQKiNIKQO0UCWKpRMN75F8nRodXJR6osmEI4owR4btQ0Au3SXOlRIN5U1iIGeXJ3wH6KIFa0XswZOIxPGTx9dus73tcXynU+MqEKmnGOhyKVAjCHHkcHSsmPLm8OiSm+sVzqSRwyN9/YXMubUFswf9Y5+o2mzNNFXeUrw8IBCCZYz+8Eg1F1jtnp44lV4NK/L4n4DzwCOd8SpXQweh9KhhWtqWzbpYZqwU4JugmVOCxp8VZhTE6L0EY+zll7WSoNmZubqXepYYLVMPTDJbDoGDRdTSbnM6yZk8bozxdsgBdYgSubGyxNkfHWxTN+Dl/lbtZ2hY2hiqRqNljTX4gHQ+EKsHFF2KjgH6lHgxVXSGC2jzG1avfH2oX7BrTvRKh18= my_key
+
+
+
+

Host keys have exactly the same format as user keys. Although you can generate these keys manually and point to them in SSH server configuration, in the majority of cases they are automatically generated during the first SSH server startup and usually reside in /etc/ssh/ssh_host_* files. To access a host in SSH you have to copy public keys of every authorized used to the list of authorized keys on this host. Exact location of the text configuration file(s) with a list of authorized user keys can be provided in SSH server settings. When working with virtual machines a typical workflow is to provide a list of authorized keys of a new virtual machine before virtual machine is created. Having this list, respective cloud platform automatically copies all authorized keys to correct configuration file inside virtual machine being started.

+
+
+

Boot works exactly the same as any other cloud platform and allows to provide a list of authorized public keys for every virtual machine. The only difference is that you have to indicate the same authorized public key in two locations:

+
+
+
    +
  1. +

    In jump host configuration. This is needed to allow authorized key owner to pass through jump host and to be able to connect (but not always access!) to the virtual machines created by Boot. A list of jump host authorized keys can be provided in values.yaml file when deploying Boot with Helm:

    +
    +
    Setting Boot jump host authorized keys in values.yaml
    +
    +
    boot:
+  jumphost:
+    authorizedKeys:
+      - ssh-rsa AAAAB3NzaC...rTvRKh18= my_key # See public key contents above
    +
    +
    +
    +

    How to do this is explained in detail in jump host documentation.

    +
    +
    2. +
  2. +

    In every created virtual machine properties. This allows to access concrete virtual machine. When creating virtual machines with kubectl and YAML respective definition looks like this:

    +
    +
    Adding authorized keys to new virtual machine YAML definition
    +
    +
    apiVersion: boot.aerokube.com/v1
+kind: VirtualMachine
+metadata:
+  name: my-vm
+  namespace: boot
+spec:
+  os: ubuntu
+  authorizedKeys: # A list of SSH public keys allowed to access this virtual machine
+    - ssh-rsa AAAAB3NzaC...rTvRKh18= my_key # See public key contents above
+    - ssh-rsa AAAAB3N... another-key # Another user public key contents
    +
    +
    +
    +

    If you are creating virtual machines with kubectl plugin - use -auth-keys flag:

    +
    +
    +
    Adding authorized keys to new virtual machine with kubectl plugin
    +
    +
    $ kubectl vm create -os ubuntu -auth-keys /path/to/authorized_keys.file -n boot
    +
    +
    +
    +

    When this flag is missing, plugin will try to use your public key from ~/.ssh/id_rsa.pub file.

    +
    +
    3. +
+
+
+
+
User and Host Certificates
+
+

Although user and host keys remain the most popular SSH authentication technology, they have several important drawbacks:

+
+
+
    +
  1. +

    Need to copy the same authorized user key to every virtual machine you need to access.

    +
    2. +
  2. +

    Need to explicitly trust every host key before accessing this host.

    +
    3. +
  3. +

    Keys do not store information about usernames allowed to authenticate with this key.

    +
    4. +
  4. +

    Keys have no duration settings, i.e. no start and expiration date, so they are rarely \ never rotated.

    +
    5. +
  5. +

    Keys have no granular permissions, i.e. they allow to use all features of SSH server that are not explicitly forbidden: interactive sessions, port forwarding, X11 forwarding, agent forwarding.

    +
    6. +
+
+
+

Recent SSH versions support another authentication technology called certificates. A certificate is a public user key with additional metadata (allowed usernames, duration settings, permissions and so on) signed by a certification authority (CA). A certification authority is any person or organization that you trust. This can be for example an information security department of your company or a specialized third-party organization.

+
+
+
Certificate is signed by a certification authority
+

certificate ca sign

+
+
+

Working with SSH certificates starts by creating a standard SSH key pair consisting of a private key and a public key. Then public key is sent to a certification authority, that adds a digital signature and required metadata to this public key. The resulting file is called an SSH certificate.

+
+
+
Certification authority should be trusted by user and host
+

certificate encryption

+
+
+

The main idea behind certificates is that instead of copying authorized keys of every user we have to trust all certificates signed by a certification authority that we trust. For example, we trust all SSH certificates created by our information security department. The only thing we need to provide in that case is certification authority certificate (CA certificate). Like any other certificate this certificate contains a public key and metadata: certification authority name, expiration information and other stuff. When we copy CA certificate to the list of trusted CA certificates a host, this host is then able to read public key from CA certificate and use it to check digital signature on every certificate that was created by this authority. When digital signature on user certificate is valid, then user SSH connection to the host is accepted.

+
+
+

Similarly to user certificate, every host can have a host certificate and users are able to check that this certificate was issued by a trusted certification authority. For connection encryption purposes certificates work exactly the same as authorized keys. To encrypt and send some data to a user, host during SSH connection receives certificate from user, extracts public key from this certificate and uses it for encryption. Only user having private key corresponding to certificate public key can decrypt the data.

+
+
+ + + + + +
+ + +
+
In the example below we create new certification authority keys. In real life your information security department or responsible third-party organization already has these keys. We demonstrate how to do this here just to show you the entire perspective.
+
    +
  1. +

    We use ed25519 encryption algorithm for certification authority because this is one of the most efficient algorithms for the moment.

    +
    2. +
+
+
+
+
+

Let’s now take a look at a typical usage of the user certificates. We assume that we already have a keypair consisting of my_key and my_key.pub files generated in previous section. Let’s start by creating a certification authority keys:

+
+
+
Creating certification authority
+
+
$ ssh-keygen -t ed25519 -f test_ca -C 'test-ca'
+Generating public/private ed25519 key pair.
+Enter passphrase (empty for no passphrase):
+Enter same passphrase again:
+Your identification has been saved in test_ca
+Your public key has been saved in test_ca.pub
+The key fingerprint is:
+SHA256:uGmsLCSBD3Zwu05jIDMPKu+xMap+eBQTk5Tqf0enhFg test-ca
+The key's randomart image is:
++--[ED25519 256]--+
+|  ..o            |
+| . *             |
+|. + +            |
+|O+ = E .         |
+|=Oo * o S        |
+|+.+B o = .       |
+|.+@ . B o        |
+| oo@ + o         |
+|=o=.+ .          |
++----[SHA256]-----+
+
+
+
+

Like with a regular SSH key pair, this will create two files:

+
+
+
Certification authority files
+
+
# Private key, keep it secret!
+$ cat test_ca
+-----BEGIN OPENSSH PRIVATE KEY-----
+b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW
+QyNTUxOQAAACDdXZr6c+pc0+IZskr+Hvgm8vU5HqSPQWfokHqaRLz1PwAAAJD4P2KI+D9i
+iAAAAAtzc2gtZWQyNTUxOQAAACDdXZr6c+pc0+IZskr+Hvgm8vU5HqSPQWfokHqaRLz1Pw
+AAAEAEVZJTuqanwlJTfVWJsX63gLv5W3ZoDXxQXLg+HHZnTd1dmvpz6lzT4hmySv4e+Cby
+9TkepI9BZ+iQeppEvPU/AAAAB3Rlc3QtY2EBAgMEBQY=
+-----END OPENSSH PRIVATE KEY-----
+
+# Public key, we will use it in Boot settings.
+$ cat test_ca.pub
+ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIN1dmvpz6lzT4hmySv4e+Cby9TkepI9BZ+iQeppEvPU/ test-ca
+
+
+
+

Having a certification authority, let’s now sign existing my_key.pub public key and create a user certificate:

+
+
+
Creating a user certificate
+
+
$ ssh-keygen -s test_ca -I 'my-test-cert' -z '0001' -n jump,root my_key.pub
+Signed user key my_key-cert.pub: id "my-test-cert" serial 1 for jump,root valid forever
+
+
+
+

In this example we use CA private key file (test_ca) to sign user public key (my_key.pub) giving a resulting certificate file an identifier my-test-cert (mainly used for logging purposes on the server), serial number 0001 and allowed users jump and root. We need to allow jump user to authenticate on the jump host and root to authenticate on the virtual machine. The resulting certificate is saved to my_key-cert.pub file and looks like this:

+
+
+
A typical user certificate
+
+
$ cat ~/my_key-cert.pub
+ssh-rsa-cert-v01@openssh.com AAAAHHNzaC1yc2EtY2VydC12MDFAb3BlbnNzaC5jb20AAAAgUmU82rtbl76TteeXo8oVJHhMR6m6//MG/KzG5O/3WGYAAAADAQABAAABgQCiL+Xa7eyclNsdiBw5JKrZYeZTD1bLXSV1mBcMKBHsdqG3G9YHSfTle6sgh5SYel8smiKckQrhJwju5pqC3AgPtCgUeyiTTOWSlIweMktt1gACq6i2OYLMG4q0bdgxw9fQKiNIKQO0UCWKpRMN75F8nRodXJR6osmEI4owR4btQ0Au3SXOlRIN5U1iIGeXJ3wH6KIFa0XswZOIxPGTx9dus73tcXynU+MqEKmnGOhyKVAjCHHkcHSsmPLm8OiSm+sVzqSRwyN9/YXMubUFswf9Y5+o2mzNNFXeUrw8IBCCZYz+8Eg1F1jtnp44lV4NK/L4n4DzwCOd8SpXQweh9KhhWtqWzbpYZqwU4JugmVOCxp8VZhTE6L0EY+zll7WSoNmZubqXepYYLVMPTDJbDoGDRdTSbnM6yZk8bozxdsgBdYgSubGyxNkfHWxTN+Dl/lbtZ2hY2hiqRqNljTX4gHQ+EKsHFF2KjgH6lHgxVXSGC2jzG1avfH2oX7BrTvRKh18AAAAAAAAAAQAAAAEAAAAMbXktdGVzdC1jZXJ0AAAAEAAAAARqdW1wAAAABHJvb3QAAAAAAAAAAP//////////AAAAAAAAAIIAAAAVcGVybWl0LVgxMS1mb3J3YXJkaW5nAAAAAAAAABdwZXJtaXQtYWdlbnQtZm9yd2FyZGluZwAAAAAAAAAWcGVybWl0LXBvcnQtZm9yd2FyZGluZwAAAAAAAAAKcGVybWl0LXB0eQAAAAAAAAAOcGVybWl0LXVzZXItcmMAAAAAAAAAAAAAADMAAAALc3NoLWVkMjU1MTkAAAAg3V2a+nPqXNPiGbJK/h74JvL1OR6kj0Fn6JB6mkS89T8AAABTAAAAC3NzaC1lZDI1NTE5AAAAQDoV3uwn0SBWrZcHtnzNuVVYPXe3orfxb3gpM6jhzszPaZ96G5Pu48lyQPN9tDiZE1PJJ7e/ovw07LQy4wqkSAo= some-comment-here
+
+
+
+

Now in order to work with created certificates in Boot - you need to add certification authority public key in two places:

+
+
+
    +
  1. +

    In jump host configuration. This is needed to allow SSH certificate owner to pass through jump host. A list of jump host trusted user certification authorities can be provided in values.yaml file when deploying Boot with Helm:

    +
    +
    Setting Boot jump host truster certification authorities in values.yaml
    +
    +
    boot:
+  jumphost:
+    trustedUserCAKeys:
+      - ssh-ed25519 AAAAC3NzaC1....eppEvPU/ test-ca # See CA public key contents above
    +
    +
    +
    +

    How to do this is explained in detail in jump host documentation.

    +
    +
    2. +
  2. +

    In every created virtual machine properties. This allows to access concrete virtual machine. When creating virtual machines with kubectl and YAML respective definition looks like this:

    +
    +
    Adding authorized keys to new virtual machine YAML definition
    +
    +
    apiVersion: boot.aerokube.com/v1
+kind: VirtualMachine
+metadata:
+  name: my-vm
+  namespace: boot
+spec:
+  os: ubuntu
+  trustedUserCAKeys: # A list of trusted certification authorities allowed to access this virtual machine
+    - ssh-ed25519 AAAAC3NzaC1....eppEvPU/ test-ca # See CA public key contents above
+    - ssh-ed25519 AAAAC3.... another-ca # Another CA public key contents
    +
    +
    +
    +

    If you are creating virtual machines with kubectl plugin - use `-user-ca-keys flag:

    +
    +
    +
    Adding trusted certification authorities to new virtual machine with kubectl plugin
    +
    +
    $ kubectl vm create -os ubuntu -user-ca-keys /path/to/truster_user_ca.file -n boot
    +
    +
    +
    3. +
+
+
+
+
+

2.2.3. Accessing a Virtual Machine

+
+

Having a configured Boot jump host and a virtual machine called my-vm, let’s now access it.

+
+
+
A typical virtual machine
+
+
$ kubectl get vm -n boot
+NAME    FQDN                              IP           OS             CPU    MEM   STATUS    UPTIME
+my-vm   my-vm.vm.boot.svc.cluster.local   172.17.0.4   ubuntu:22.04   250m   1Gi   Running   10s
+
+
+
+

A full SSH command to access some host through jump host is as follows:

+
+
+
Accessing a virtual machine through jump host
+
+
$ ssh -i /path/to/my_key -J jump@boot.example.com:2222 root@my-vm.vm.boot.svc.cluster.local
+
+
+
+

In this command we use -i flag to provide a private key file. Then -J is needed to use a jump host and jump@boot.example.com:2222 notation means to access jump host boot.example.com on port 2222 using jump user (these are Boot defaults). Finally, you provide your destination host root@my-vm.vm.boot.svc.cluster.local, i.e. you access host my-vm.vm.boot.svc.cluster.local inside Kubernetes with root user. This command will work for both authorized keys and user certificates. In case of user certificates SSH client will automatically try all certificates corresponding to private key file (e.g. my_key-cert.pub). You can certainly provide an exact certificate file as follows:

+
+
+
Providing an explicit SSH user certificate file
+
+
$ ssh -o CertificateFile=/path/to/my_key-cert.pub -i /path/to/my_key -J jump@boot.example.com:2222 root@my-vm.vm.boot.svc.cluster.local
+
+
+
+

These commands look complicated, right? But most of these values except virtual machine host rarely change, so let’s move them to SSH client configuration file (usually resides in ~/.ssh/config):

+
+
+
Moving connection options to SSH client configuration
+
+
$ cat ~/.ssh/config
+Host bootjump
+    IdentityFile /path/to/my_key # Private key location
+    HostName boot.example.com # Jump host name
+    Port 2222 # Just host port
+Host *.vm.boot.svc.cluster.local
+    IdentityFile /path/to/my_key # Private key location
+    ProxyJump jump@bootjump # Always use jump host declared above
+
+
+
+

Having this configuration file, SSH connection command can be simplified to:

+
+
+
Simplified SSH connection command
+
+
$ ssh root@my-vm.vm.boot.svc.cluster.local
+
+
+
+

But we can make this command even shorter. Virtual machine fully qualified domain name is my-vm.vm.boot.svc.cluster.local. In this name boot is namespace name and svc.cluster.local is standard Kubernetes DNS suffix. Kubernetes is able to automatically add this part of domain name, so you can add the following to SSH config:

+
+
+
How to support short domain name resolution
+
+
$ cat ~/.ssh/config
+Host *.vm
+    IdentityFile /path/to/my_key # Private key location
+    ProxyJump jump@bootjump
+
+
+
+

With this configuration section the following command will work:

+
+
+
Even shorter SSH connection command
+
+
$ ssh root@my-vm.vm
+
+
+
+
+

2.2.4. Trusting Boot Host Certificate

+
+

In previous section we already mentioned that both user and host can have SSH certificates. User certificates are used by SSH server to identify users. Host certificates allow user to be sure that connection is made to the host that was expected to access. By default, when you try to access a virtual machine for the first time you will see the following:

+
+
+
Accessing a virtual machine for the first time
+
+
$ ssh root@my-vm.vm
+The authenticity of host '[boot.example.com]:2222 ([XXX.XXX.XXX.XXX]:2222)' can't be established.
+ED25519 key fingerprint is SHA256:F/qgcVr/35EPaGtxX4Y1YHR5H8oD28sUKs0IRFiRWfI.
+This key is not known by any other names
+Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
+Warning: Permanently added '[boot.aerokube.local]:2222' (ED25519) to the list of known hosts.
+The authenticity of host 'my-vm.vm (<no hostip for proxy command>)' can't be established.
+ED25519 key fingerprint is SHA256:mGSVISH1obExpV3YceqKuzVrIzbnv7vwiUjc9CccS1k.
+This key is not known by any other names
+Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
+Warning: Permanently added 'my-vm.vm' (ED25519) to the list of known hosts.
+my-vm:~#
+
+
+
+

To stop seeing these warnings we need to add Boot virtual machines to the list of trusted hosts. During the installation Boot generates a certification authority that is used to create host certificates for all virtual machines. The pair of keys for this certification authority is stored in Kubernetes secret <boot-namespace>-vm-ssh-host-ca-keys, e.g. boot-vm-ssh-host-ca-keys when Boot is running in default boot namespace:

+
+
+
    +
  1. +

    Extract public key for certification authority from secret:

    +
    +
    +
    $ kubectl get secret boot-vm-ssh-host-ca-keys -o jsonpath='{.data.ca_key\.pub}' -n boot | base64 -D
+ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFHZc6j06RCf+8/J1IuS8SY1/hasLHlik6XRnO87+Zhu user@boot-generate-ssh-keys-4j2dw
    +
    +
    +
    2. +
  2. +

    Add this key to the list of SSH known hosts:

    +
    +
    +
    $ cat ~/.ssh/known_hosts
+# For fully qualified domain name
+@cert-authority *.vm.boot.svc.cluster.local ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFHZc6j06RCf+8/J1IuS8SY1/hasLHlik6XRnO87+Zhu user@boot-generate-ssh-keys-4j2dw
+
+# For short domain name
+@cert-authority *.vm ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFHZc6j06RCf+8/J1IuS8SY1/hasLHlik6XRnO87+Zhu user@boot-generate-ssh-keys-4j2dw
    +
    +
    +
    3. +
  3. +

    Now if you try to execute the same command - SSH will never ask you to confirm authenticity of virtual machines. You only need to confirm jump host (boot.example.com) authenticity once.

    +
    4. +
+
+
+
+
+
+
+

3. Configuration

+
+
+

3.1. Operating Systems

+
+ + + + + +
+ + +
+
    +
  1. +

    Boot virtual machines are always running in privileged mode, so we intentionally did not add some existing Kubernetes features like security context.

    +
    2. +
  2. +

    Default way of accessing Boot virtual machines is using [SSH] connection. An interactive SSH session never relies on server environment variables, so we did not add passing environment variables from operating system configuration. This simply makes no sense.

    +
    3. +
+
+
+
+
+

An operating system is the main Boot configuration unit. Every operating system object stores parameters used when you launch a virtual machine: container images, available computing resources, particular DNS configuration and so on. To list available operating systems:

+
+
+
Listing operating systems
+
+
$ kubectl get os -n boot
+NAME     REPOSITORY            VERSION(S)   CPU    MEM   AGE
+alpine   quay.io/boot/alpine   3.18         500m   1Gi   50m
+ubi      quay.io/boot/ubi      9.2          500m   1Gi   50m
+ubuntu   quay.io/boot/ubuntu   22.04        500m   1Gi   50m
+
+
+
+

A typical operating system definition looks like this:

+
+
+
Getting operating system definition in YAML format
+
+
$ kubectl get os ubuntu -n boot -o yaml
+apiVersion: boot.aerokube.com/v1
+kind: OperatingSystem
+metadata:
+  name: ubuntu
+  namespace: boot
+  # Other metadata fields...
+spec:
+  repository: quay.io/boot/ubuntu  (1)
+  resources:                       (2)
+    limits:
+      cpu: "0.5"
+      memory: 1Gi
+    requests:
+      cpu: "0.5"
+      memory: 1Gi
+  versions:                        (3)
+  - "22.04"
+status:
+  # Fields used for listing
+
+
+
+ + + + + + + + + + + + + +
1Repository with images for this operating system
2Available computing resources
3Available operating system versions
+
+
+

While such short configuration will work - more advanced settings exist:

+
+
+
More fields in operating system definition
+
+
$ kubectl get os ubuntu -n boot -o yaml
+apiVersion: boot.aerokube.com/v1
+kind: OperatingSystem
+metadata:
+  name: ubuntu
+  namespace: boot
+  # Other metadata fields...
+spec:
+  affinity:                            (1)
+    nodeAffinity:
+      requiredDuringSchedulingIgnoredDuringExecution:
+        nodeSelectorTerms:
+        - matchExpressions:
+          - key: node-label-1
+            operator: In
+            values:
+            - value-1
+            - value-2
+  dnsPolicy: None                      (2)
+  dnsConfig:                           (3)
+    nameservers:
+      - 192.0.2.1
+    searches:
+      - ns1.svc.cluster-domain.example
+      - my.dns.search.suffix
+    options:
+      # See Kubernetes documentation
+  nodeSelector:                        (4)
+      node-label-1: "label1-value"
+      node-label-2: "label2-value"
+  repository: quay.io/boot/ubuntu      (5)
+  resources:                           (6)
+    limits:
+      cpu: "0.5"
+      memory: 1Gi
+    requests:
+      cpu: "0.5"
+      memory: 1Gi
+  tolerations:                         (7)
+  - key: "key1"
+    operator: "Equal"
+    value: "value1"
+    effect: "NoSchedule"
+  versions:                            (8)
+  - "22.04"
+status:
+  # Fields used for listing
+
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
1Applied affinity configuration
2Applied DNS policy
3Applied DNS configuration
4Applied Kubernetes node selector
5Repository with images for this operating system - this is the only required field
6Available computing resources
7Applied Kubernetes tolerations
8Available operating system versions - limits allowed image tags that can be used for launching virtual machines
+
+
+

The next sections explain these settings in more detail.

+
+
+

To edit an operating system object:

+
+
+
Editing an operating system
+
+
$ kubectl edit os ubuntu -n boot # Do modifications in text editor, save and exit
+
+
+
+

Another possible option to edit an operating system definition that we recommend in the majority of cases is updating the same object with Helm. To do this - edit the following values.yaml section:

+
+
+
Updating operating systems with Helm
+
+
boot:
+  # More fields go here
+  os:
+    ubuntu: # Helm chart fields for every operating system object are exactly the same as described above
+      repository: quay.io/boot/ubuntu
+      versions:
+      - "22.04"
+    nodeSelector: # Let's add node selector
+      kubernetes.io/arch: "amd64"
+  # Other fields go here
+
+
+
+

Now simply reapply Helm chart with modifications in values.yaml file, and you are all set.

+
+
+

3.1.1. Repository and Versions

+
+

The most important settings in operating system definition are the name of image repository and available versions. For example, let’s assume that you have the following operating system definition:

+
+
+
Repository and versions in operating system definition
+
+
$ kubectl get os ubuntu -n boot -o yaml
+apiVersion: boot.aerokube.com/v1
+kind: OperatingSystem
+metadata:
+  name: ubuntu
+  namespace: boot
+  # Other metadata fields...
+spec:
+  repository: quay.io/boot/ubuntu
+  versions:
+  - "20.04"
+  - "22.04"
+status:
+  # Fields used for listing
+
+
+
+

Now you request to launch a virtual machine like this:

+
+
+
Requesting to launch an Ubuntu 20.04 virtual machine
+
+
apiVersion: boot.aerokube.com/v1
+kind: VirtualMachine
+metadata:
+  name: my-ubuntu-vm
+  namespace: boot
+spec:
+  os: ubuntu
+  version: "20.04"
+
+
+
+

Boot will then create a pod using quay.io/boot/ubuntu:20.04 container image. Similarly, when you change version to 22.04, respective container image will be quay.io/boot/ubuntu:20.04. Versions list is mainly needed to show recommended operating system versions to user. Nothing will prevent user from creating a virtual machine with version 18.04 is respective image (quay.io/boot/ubuntu:18.04) exists.

+
+
+
+

3.1.2. Computing Resources

+
+ + + + + +
+ + +
+
    +
  1. +

    Boot is using exactly the same computing resources configuration YAML format as Kubernetes does.

    +
    2. +
  2. +

    We recommend to set memory requests and limits to the same value because otherwise sooner or later processes inside a virtual machines can be killed because of lack of memory (out of memory, OOM).

    +
    3. +
+
+
+
+
+

To limit computing resources available to every virtual machine, you need to add resources section to respective operating system definition.

+
+
+
Assigning computing resources to operating system definition
+
+
boot:
+  # More fields go here
+  os:
+    ubuntu:
+      repository: quay.io/boot/ubuntu
+      resources: # Computing resources available to every virtual machine
+        limits:
+          cpu: "1.0"
+          memory: 1Gi
+        requests:
+          cpu: "0.5"
+          memory: 1Gi
+  # Other fields go here
+
+
+
+
+

3.1.3. Node Selector

+
+ + + + + +
+ + +Boot is using exactly the same node selector YAML format as Kubernetes does. +
+
+
+

Sometimes you may need to run virtual machines on particular Kubernetes nodes (i.e. hosts) only. Kubernetes allows to do this by specifying so-called node selectors. If you need to add the same node selector to an operating system definition:

+
+
+
Adding node selector to operating system definition
+
+
boot:
+  # More fields go here
+  os:
+    ubuntu:
+      repository: quay.io/boot/ubuntu
+      nodeSelector: # Node selector definition
+        node-label-1: "label1-value"
+  # Other fields go here
+
+
+
+
+

3.1.4. Affinity

+
+ + + + + +
+ + +Boot is using exactly the same affinity configuration YAML format as Kubernetes does. +
+
+
+

In addition to node selectors, you can also use all available node and pod affinity features available in Kubernetes. This allows you to have even more advanced pod scheduling settings like matching Kubernetes nodes against complex logical expressions, preventing some labeled pods to be running on the same node with another labeled pods and so on.

+
+
+
Adding affinity specification to operating system definition
+
+
boot:
+  # More fields go here
+  os:
+    ubuntu:
+      repository: quay.io/boot/ubuntu
+      affinity: # Affinity settings have exactly the same YAML fields as Kubernetes pod affinity settings
+        nodeAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            nodeSelectorTerms:
+            - matchExpressions:
+              - key: node-label-1
+                operator: In
+                values:
+                - value-1
+                - value-2
+  # Other fields go here
+
+
+
+
+

3.1.5. Networking

+
+

Some scenarios require flexible networking configuration. For example, you may need to override used DNS server. This can be easily done as follows:

+
+
+
Adding dns settings to operating system definition
+
+
boot:
+  # More fields go here
+  os:
+    ubuntu:
+      repository: quay.io/boot/ubuntu
+      dnsPolicy: None # DNS settings have exactly the same YAML fields (dnsPolicy and dnsConfig) as Kubernetes pod dns settings
+      dnsConfig:
+        nameservers:
+          - 192.0.2.1
+        searches:
+          - ns1.svc.cluster-domain.example
+          - my.dns.search.suffix
+        options:
+          # See Kubernetes documentation
+  # Other fields go here
+
+
+
+
+

3.1.6. Tolerations

+
+ + + + + +
+ + +Boot is using exactly the same tolerations configuration YAML format as Kubernetes does. +
+
+
+

In addition to node selector and affinity, Kubernetes has a concept of node taints. Taints allow nodes to repel some pods from being scheduled on them. If you wish to run browser pods on tainted nodes - you have to add tolerations, that is to say a number of conditions to match against tainted nodes.

+
+
+

To add the tolerations to operating system definition:

+
+
+
Adding tolerations to operating system definition
+
+
boot:
+  # More fields go here
+  os:
+    ubuntu:
+      repository: quay.io/boot/ubuntu
+      tolerations: # Tolerations settings have exactly the same YAML keys as Kubernetes pod tolerations settings
+      - key: "key1"
+        operator: "Equal"
+        value: "value1"
+        effect: "NoSchedule"
+  # Other fields go here
+
+
+
+
+
+

3.2. Jump Host Configuration

+
+

Jump host is the main entrypoint to Boot virtual machines. Jump host is reconfigured by updating values.yaml file and reapplying Helm chart.

+
+
+

3.2.1. User

+
+

Jump host processes are always run by an unprivileged user (default name is jump). This gives additional security. Boot users can only access jump host by SSH using the name of this user. To set it:

+
+
+
Updating jump host user settings
+
+
$ cat values.yaml
+boot:
+  jumphost:
+    user:
+      name: my-user # Custom login user
+      uid: 10000000 # User uid (you may need to change it in Openshift)
+      gid: 10000000 # User gid (you may need to change it in Openshift)
+
+
+
+
+

3.2.2. Network Port

+
+

To change network port being used to access jump host by SSH:

+
+
+
Updating jump host user settings
+
+
$ cat values.yaml
+boot:
+  jumphost:
+    port: 22222 # Setting custom network port
+
+
+
+
+

3.2.3. Allowed User Keys and Certificates

+
+

It is important to configure users allowed to access Boot cluster by SSH at all. This is done by adding authorized public SSH keys and trusted user CA certificates to jump host settings:

+
+
+
Updating jump host user authorized keys and trusted CA certificates
+
+
$ cat values.yaml
+boot:
+  jumphost:
+    authorizedKeys: # User authorized keys
+      - ssh-rsa AAAAB3N... some-key
+      - ssh-rsa AAAAB3N... another-key
+    trustedUserCAKeys: # A list of trusted user certification authorities
+      - ssh-ed25519 AAAAC3N... certification-authority-1
+      - ssh-ed25519 AAAAC3N... certification-authority-2
+
+
+
+
+

3.2.4. Computing Resources

+
+

If for some reason default jump host computing resources are not sufficient:

+
+
+
Updating jump host computing resources
+
+
$ cat values.yaml
+boot:
+  deployment:
+    containers:
+      jumphost:
+        resources: # Custom jump host computing resources
+          limits:
+            cpu: 2000m
+            memory: 1Gi
+          requests:
+            cpu: 200m
+            memory: 1Gi
+
+
+
+
+
+

3.3. License Key

+
+ + + + + +
+ + +
+
    +
  1. +

    According to license agreement you can use up to 1 CPU for free and for unlimited period of time. If you wish to have more CPUs - order a license key. This section describes how to list and install license keys. A limited duration trial license key with more CPUs can be generated on Boot website.

    +
    2. +
+
+
+
+
+

A typical license key is a text file with .key extension that looks like this:

+
+
+
+
$ cat license.key
+Ulh3SnN6S3JYTVg5UmIzaHNiWUpOaE1rcGltQzJxRVZVbGdHMVliNlZDbnNjVkc5b1M1eGNEbTRZYkN6c2RaTmtaaGs0cDExQWRlOTA2YVNxK3NNV2JORHd0NkFEUEZTNk16UXVCcWhQMVovajhhdWlJZDJzdW9yVEFRTFppSnp2NHloMkdZYXNVVlNhRk05Q2ZOUk4rd1JCNHlXRlFwRmNwbVRFWk9hdXRQWjJvVUM0TldGdXR2OUtiangrT0hkRmJNK2xtQUhCYVArWDhlUTJoNnFzRlExdHl2Zm11QmtVWUNhRHBSTEhzVTdLQXVEZFZKWlhUSU9PRjNjUlFIODhyYmZKTkZVWm1sNG5UZnJHM2RFRTJmYkMyakNwVndLWmJaMkgrVi9zeGRXd0dDblZMNFAyYXVyNjQ4cDhnb0xvRGdZMGlnRmM1WXFPODVGK0U4TlRPWWpyMGtPRThnY1lRcU1JT1JWZEkwQ0ZNVkk3SkFpbHI0UzhHcHduY2Vwcks3ZERtbnVLNmRIeGVnZHhqSGNIN1laZlR6U2prZ001S2R5Q1RCSlF0RXB2VjkvUlF5MUV3M3RIcCtTRWcvTjl3eUF5VE4xUFl4Q0xtU2t0QjFNblZVeURZby9sWXlCYlQrSGNlSUExdktTVThDQlZFaFNRZmdRS1BmbUxFblBuSmxhVHZmWXhnVUF3b3B3dmFwaHFmaExRNTVEM2d6RzA4ZDlsNTVGVGE3Vlo4b2Vpd2FabUFDWHZRZ3NlMTUzT29SdDV1M1VsNGNVTmFUOGUxbWgva2JKajJ1Mjk2cysvalBBa3JVRnNVdWlNZHA1a2Zrc2hTQlhybDZyVlFJYytDcWE3MUFBdWpyT1lPNm1JZ3BNZXAvYUI4cXhRR29uTUVzVGRrRlVKR289O2V5SnNhV05sYm5ObFpTSTZJa1JsWm1GMWJIUWlMQ0p3Y205a2RXTjBJam9pUW05dmRDSXNJbTFoZUZObGMzTnBiMjV6SWpveGZRPT0=
+
+
+
+

In Boot license keys (or just licenses) are stored in custom Kubernetes resource.

+
+
+

3.3.1. Listing License Keys

+
+ + + + + +
+ + +
+

Contrarily to other resources introduced by Boot, licenses are stored cluster-wide. Thus, you don’t need to provide namespace name in the following commands (-n boot is not needed).

+
+
+
+
+

To list available licenses:

+
+
+
Listing licenses (free license key output)
+
+
$ kubectl get licenses
+NAME   LICENSEE   CPUS   USAGE   EXPIRES   STATUS   NAMESPACE   AGE
+boot   Default    1      0%      Never     Ok       boot        2m42s
+
+
+
+

The output above is shown when the free license key is used. Columns meaning is as follows:

+
+
+
    +
  • +

    Name. License key object name.

    +
    • +
  • +

    Licensee. License key owner name. Usually equals to company name, e.g. Acme LLC. For free license key with 1 CPU equals to Default.

    +
    • +
  • +

    CPUs. Maximum number of CPUs available in this license key.

    +
    • +
  • +

    Expires. The number of days this license key expires in. Equals to Already when license key already expired and equals to Never if license key never expires.

    +
    • +
  • +

    Status. License key status. Can be one of: Ok - license key is active, Expired - license key has expired, Broken - invalid license key data was provided.

    +
    • +
  • +

    Namespace. Name of Kubernetes namespace where this license key is used.

    +
    • +
+
+
+

You may have multiple custom Kubernetes resources named license. In that case in order to work with Boot licenses - simply use fully qualified resource name:

+
+
+
Listing licenses (longer and fully qualified resource name)
+
+
$ kubectl get licenses.boot
+NAME   LICENSEE   CPUS   USAGE   EXPIRES   STATUS   NAMESPACE   AGE
+boot   Default    1      0%      Never     Ok       boot        2m42s
+
+$ kubectl get licenses.boot.aerokube.com
+NAME   LICENSEE   CPUS   USAGE   EXPIRES   STATUS   NAMESPACE   AGE
+boot   Default    1      0%      Never     Ok       boot        2m42s
+
+
+
+

To view a license key in YAML format:

+
+
+
Showing license in YAML format
+
+
$ kubectl get license boot -o yaml
+apiVersion: boot.aerokube.com/v1
+kind: License
+metadata:
+  name: boot                        (1)
+  # Other Kubernetes metadata
+spec:
+  data: YkV0dmNsaGxTak51Y2....      (2)
+status:
+  # Other keys and values
+
+
+
+ + + + + + + + + +
1License key name, should correspond to namespace name where this license key should be used
2License key contents
+
+
+
+

3.3.2. Updating a License Key

+
+

To update an existing license key - simply update data field in respective license object:

+
+
+
Updating license key with kubectl
+
+
$ kubectl edit license boot # Replace data field with your new license key in text editor, save and exit
+
+
+
+

When you update a license key - all changes are applied immediately. This usually also leads to graceful Boot pods restart (does not interrupt running virtual machines).

+
+
+
+

3.3.3. Multiple License Keys

+
+

Boot allows to share the same license key among several Kubernetes namespaces and in the majority of cases a single license key should be enough. However, in some cases you may want to use a separate Boot instance and a separate license key for some teams. To achieve this:

+
+
+
    +
  1. +

    Deploy two independent Boot clusters to namespace ns1 and ns2

    +
    2. +
  2. +

    Create two license objects with name field set to ns1 and ns2 and save them to file (e.g. license-keys.yaml):

    +
    +
    License keys to be created
    +
    +
    $ cat license-keys.yaml
+apiVersion: boot.aerokube.com/v1
+kind: License
+metadata:
+  name: ns1
+spec:
+  data: <license-key-1>
+---
+apiVersion: boot.aerokube.com/v1
+kind: License
+metadata:
+  name: ns2
+spec:
+  data: <license-key-2>
    +
    +
    +
    3. +
  3. +

    Apply resulting file:

    +
    +
    +
    $ kubectl apply -f license-keys.yaml
    +
    +
    +
    + + + + + +
    + + +
    +
      +
    • +

      If you try to create two license keys with the same data field value, then one of them will be considered as a duplicate and automatically deleted.

      +
      • +
    • +

      If you have two different license keys with the same namespace field, then Boot will always choose the most recently created one.

      +
      • +
    +
    +
    +
    +
    4. +
  4. +

    License keys will be applied automatically, and you will see the following in licenses list:

    +
    +
    Two license keys are applied
    +
    +
    $ kubectl get licenses
+NAME  LICENSEE   CPUS   USAGE   EXPIRES   STATUS   NAMESPACE   AGE
+ns1   Acme Inc.  10     0%      20d       Ok       ns1         2m42s
+ns2   Acme Inc.  20     0%      30d       Ok       ns2         2m42s
    +
    +
    +
    5. +
+
+
+
+

3.3.4. Deleting a License Key

+
+

To delete an existing license key - simply delete respective license object:

+
+
+
Deleting license key
+
+
$ kubectl delete license boot
+
+
+
+

When you delete the license key with name field set to some Boot namespace, Boot will automatically fall back to the free license key with 1 CPU included.

+
+
+
+

3.3.5. License Key Expiration

+
+

To check for expiring or expired license keys - just list them with kubectl:

+
+
+
Listing license keys to understand when they expire
+
+
$ kubectl get licenses
+NAME  LICENSEE   CPUS   USAGE   EXPIRES   STATUS   NAMESPACE   AGE
+ns1   Acme Inc.  10     0%      32d       Ok       ns1         2m42s
+ns2   Acme Inc.  20     0%      today     Ok       ns2         2m42s
+
+
+
+

You can see that Expires column is showing the number of days remaining for every license key. When a license key expires the same command output will be:

+
+
+
When one license key has expired
+
+
$ kubectl get licenses
+NAME  LICENSEE   CPUS   USAGE   EXPIRES   STATUS   NAMESPACE   AGE
+ns1   Acme Inc.  10     0%      32d       Ok       ns1         2m42s
+ns2   Acme Inc.  20     0%      Already   Expired  ns2         2m42s
+
+
+
+

For expired license key Expires column will be set to Already and license key status will be Expired.

+
+
+ + + + + +
+ + +You can also use Kubernetes API directly instead of kubectl to list license keys and find expiring or expired ones. +
+
+
+
+
+

3.4. Using Private Container Registry

+
+

By default, Boot images (aerokube/boot, aerokube/jumphost and so on) are downloaded from public container images registry. If in your environment due to security restrictions container images can only be downloaded from private registry (e.g. my-registry.example.com) you need to configure Boot to work with this registry. To do this:

+
+
+
    +
  1. +

    Configure Kubernetes authentication to your private registry:

    +
    +
    +
    $ kubectl create secret docker-registry my-registry.example.com --docker-server=my-registry.example.com --docker-username=some-user --docker-password=registry-password --docker-email=some-user@example.com -n boot
+$ kubectl patch serviceaccount boot -p '{"imagePullSecrets": [{"name": "my-registry.example.com"}]}' -n boot # Use correct service account name here
    +
    +
    +
    2. +
+
+
+
    +
  1. +

    Copy all desired virtual machine images to your registry:

    +
    +
    +
    quay.io/boot/ubuntu:22.04 => my-registry.example.com/boot/ubuntu:22.04
    +
    +
    +
    2. +
  2. +

    Update operating system definition in Helm chart:

    +
    +
    Operating system definition in Helm chart
    +
    +
    $ cat values.yaml
+boot:
+  namespaces:
+  os:
+    ubuntu:
+      repository: my-registry.example.com/boot/ubuntu
+      versions:
+      - "22.04"
    +
    +
    +
    3. +
  3. +

    Copy desired version of the Boot images to your registry:

    +
    +
    +
    aerokube/boot:1.0.0 => my-registry.example.com/aerokube/boot:1.0.0
+aerokube/jumphost:1.0.0 => my-registry.example.com/aerokube/jumphost:1.0.0
+aerokube/keygen:1.0.0 => my-registry.example.com/aerokube/keygen:1.0.0
+aerokube/reloader:1.0.0 => my-registry.example.com/aerokube/reloader:1.0.0
    +
    +
    +
    4. +
  4. +

    Use new Boot images from the previous step in Helm chart to start Boot:

    +
    +
    Custom registry for Boot images in Helm chart
    +
    +
    $ cat values.yaml
+boot:
+  deployment:
+    containers:
+      boot:
+        image:
+          repository: my-registry.example.com/aerokube/boot
+      jumphost:
+        image:
+          repository: my-registry.example.com/aerokube/jumphost
+      keygen:
+        image:
+          repository: my-registry.example.com/aerokube/keygen
+      reloader:
+        image:
+          repository: my-registry.example.com/aerokube/reloader
    +
    +
    +
    5. +
+
+
+
+

3.5. Using Multiple Namespaces

+
+

This section shows how to configure Boot to launch virtual machines in multiple namespaces. To do this:

+
+
+
    +
  1. +

    Create one or more namespaces where virtual machines will be running:

    +
    +
    Creating team namespaces for virtual machines
    +
    +
    $ kubectl create namespace team-alpha
+$ kubectl create namespace team-beta
    +
    +
    +
    +

    You can optionally limit computing resources available to every namespace by using Kubernetes resource quotas.

    +
    +
    2. +
  2. +

    Create a role for users in every namespace:

    +
    +
    Creating a namespace role
    +
    +
    $ cat team-alpha-roles.yaml
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: team-alpha-roles
+  namespace: team-alpha # Use correct namespace name here
+rules:
+- apiGroups:
+  - ""
+  resources:
+  - pods
+  verbs:
+  - get
+  - watch
+  - list
+  - create
+  - update
+  - patch
+  - delete
+- apiGroups:
+  - boot.aerokube.com
+  resources:
+  - operatingsystems
+  - operatingsystems/status
+  verbs:
+  - get
+  - watch
+  - list
+- apiGroups:
+  - boot.aerokube.com
+  resources:
+  - virtualmachines
+  - virtualmachines/status
+  verbs:
+  - get
+  - watch
+  - list
+  - create
+  - update
+  - patch
+  - delete
+
+$ kubectl apply -f team-alpha-roles.yaml # Create role
    +
    +
    +
    3. +
  3. +

    Grant the role from previous step to every user that should be able to create virtual machines:

    +
    +
    Granting roles to users
    +
    +
    $ kubectl create rolebinding my-user-roles --role=team-alpha-roles --user=my-user -n team-alpha # Add role to user (use correct login here)
    +
    +
    +
    4. +
  4. +

    Add namespaces to values.yaml file of Boot Helm chart:

    +
    +
    Adding namespaces to Boot
    +
    +
    $ cat values.yaml
+boot:
+  namespaces:
+    - team-alpha
+    - team-beta
    +
    +
    +
    5. +
  5. +

    Reapply Helm chart:

    +
    +
    Applying namespace settings to Boot
    +
    +
    $ helm upgrade --install -f values.yaml -n boot boot aerokube/boot
    +
    +
    +
    6. +
  6. +

    Now users should be able to create virtual machines in their namespaces:

    +
    +
    Creating a virtual machine from YAML definition
    +
    +
    $ cat vm.yaml
+apiVersion: boot.aerokube.com/v1
+kind: VirtualMachine
+metadata:
+  name: my-vm
+  namespace: team-alpha # Note namespace name here
+spec:
+  os: ubuntu
+  authorizedKeys:
+    - ssh-rsa AAAAB3N... some-key
+
+$ kubectl create -f ~/vm.yaml
+virtualmachine.boot.aerokube.com/my-vm created
    +
    +
    +
    +
    Creating a virtual machine with plugin
    +
    +
    $ kubectl vm create -os ubuntu -n team-beta # Like in any kubectl command, -n flag sets namespace name
    +
    +
    +
    7. +
+
+
+
+

3.6. Log Files

+
+

Although Boot should just work out of the box, sometimes you may need the log output. Every Boot component is outputting logs to standard output (stdout), so you can use well-known kubectl commands to see the log. Everything related to starting virtual machines is being output by boot container:

+
+
+
+
$ kubectl logs -lapp=boot -c boot -n boot
+
+
+
+

To follow the logs while running the tests add -f flag:

+
+
+
+
$ kubectl logs -f -lapp=boot -c boot -n boot
+
+
+
+

You can also take a look at reloader and jumphost logs as follows:

+
+
+
+
$ kubectl logs -f -lapp=boot -c reloader -n boot
+$ kubectl logs -f -lapp=boot -c reloader -n boot
+
+
+
+

If you are encountering issues with virtual machines then take a look at respective pod:

+
+
+
+
$ kubectl logs virtual-machine-01 -n boot
+
+
+
+

Here virtual-machine-01 is the name of the virtual machine.

+
+
+
+
+
+

4. License Agreement

+
+
+

Last updated August 1st, 2023. Replaces the prior version in its entirety.

+
+
+

This is a legal agreement. By downloading, installing, copying, saving on Customer’s computer, or otherwise using Aerokube software, support or products Customer becomes a party to this Agreement and Customer consents to be bound by all the terms and conditions set forth below.

+
+
+
    +
  1. +

    Parties

    +
    +
      +
    1. +

      "Aerokube", "Licensor" or "We" means Aerokube Software OÜ, having its principal place of business at Harju maakond, Tallinn, Kesklinna linnaosa, Karu tn 14-8, 10120, Estonia, registered in the Commercial Register of Estonia, registry code: 14653208.

      +
      2. +
    2. +

      "Customer", "Licensee" or "You" means the sole proprietor or legal entity specified in the Subscription Confirmation. For legal entities, "Customer" includes any entity which controls, is controlled by, or is under common control with Customer. For the purposes of this definition, "control" means one of the following:

      +
      +
        +
      1. +

        The power, directly or indirectly, to direct or manage such entity, whether by contract or otherwise.

        +
        2. +
      2. +

        Ownership of fifty percent (50%) or more of the outstanding shares or beneficial ownership of such entity.

        +
        3. +
      +
      +
      3. +
    +
    +
    2. +
  2. +

    Definitions

    +
    +
      +
    1. +

      "Agreement" means this License Agreement.

      +
      2. +
    2. +

      "Product" means Boot software identified by Licensor as a software developer tool. For the avoidance of doubt, the Product is not produced to the specifications of Customer nor customized through modification or personalization, is intended for mass distribution, and no software code will be provided to Customer. Detailed description of the product is provided on Licensor’s web site: https://aerokube.com/boot/.

      +
      3. +
    3. +

      "User" means any employee, independent contractor or other personnel obtaining access to the Product(s) from Customer.

      +
      4. +
    4. +

      "Number of Central Processing Units" means maximum number of central processing units (also herein referred to as CPUs) being used by the Product in parallel.

      +
      5. +
    5. +

      "License Key" means a unique key-code that enables a Licensee to use the Product by unlocking the fixed CPUs. Only Licensor and/or its representatives are permitted to produce License Keys for the Product.

      +
      6. +
    6. +

      "Subscription" means an arrangement for making use of the Product of periodic nature on a prepayment plan. For the purpose of clarity, Subscription includes the subscription term, Products provided to Customer, subscription fees, payment schedules and fixed number of License Keys.

      +
      7. +
    7. +

      "Product Evaluation" means using the Product without a valid License Key.

      +
      8. +
    8. +

      "Subscription Confirmation" means an email confirming Customer’s rights to access and use Products, including total Number of Concurrent Sessions.

      +
      9. +
    9. +

      "Product Installation" means a Product copy running on Customer’s computer device, hardware server or virtual machine.

      +
      10. +
    10. +

      "Product Version" means a release, update, or upgrade of a particular Product that is not identified by Licensor as being made for the purpose of fixing software bugs.

      +
      11. +
    11. +

      "Bug Fix Update" for a particular Product Version means a software update or release that is specifically identified by Licensor as a bug fix for that Product Version.

      +
      12. +
    12. +

      "E-mail Support" means a form of customer support provided by the Licensor. At the time of writing, the corresponding e-mail address is support@aerokube.com; should the address be changed, the new address will be referred to on the Licensor’s web site.

      +
      13. +
    13. +

      "Instant Messaging Support" means a form of customer support provided by the Licensor. At the time of writing, the corresponding address to support channel is https://t.me/aerokube_boot; should the address be changed, the new address will be referred to on the Licensor’s web site.

      +
      14. +
    14. +

      "Affiliate" means any entity belonging to the same group as the Licensor.

      +
      15. +
    +
    +
    3. +
  3. +

    How this Agreement Works

    +
    +
      +
    1. +

      Entire Agreement. This Agreement, including the Third-Party Software license terms, constitutes the entire agreement between the parties concerning its subject matter and supersedes any prior agreements between Customer and Licensor regarding Customer’s use of any Products. No purchase order, other ordering document or any handwritten or typewritten text which purports to modify or supplement the printed text of this Agreement or any schedule will add to or vary the terms of this Agreement unless signed by both Customer and Licensor.

      +
      2. +
    2. +

      Reservation of Rights. Aerokube reserves the right at any time to cease the support of the Product and to alter prices, features, specifications, capabilities, functions, terms of use, release dates, general availability or other characteristics of the Product.

      +
      3. +
    3. +

      Changes to this Agreement. We may update or modify this Agreement from time to time, including any referenced policies and other documents. If a revision meaningfully reduces Customer’s rights, we will use reasonable efforts to notify Customer. If we modify this Agreement, the modified version of the Agreement will be effective from the start of the next Subscription term. In this case, if Customer objects to the updated Agreement terms, as Customer’s exclusive remedy, Customer may cancel the Subscription. Customer may be required to click through the updated Agreement to show its acceptance. For the avoidance of doubt, each Subscription Confirmation is subject to the version of the Agreement in effect on the Subscription Confirmation date.

      +
      4. +
    4. +

      Opportunity to Review. Customer hereby declares that Customer has had sufficient opportunity to review this Agreement, understand the content of all of its clauses, negotiate its terms, and seek independent professional legal advice in that respect before entering into it. Consequently, any statutory "form contract" ("adhesion contract") regulations shall not be applicable to this Agreement.

      +
      5. +
    5. +

      Severability. If a particular term of this Agreement is not enforceable, the unenforceability of that term will not affect any other terms of this Agreement.

      +
      6. +
    6. +

      Headings. Headings and titles are for convenience only and do not affect the interpretation of this Agreement.

      +
      7. +
    7. +

      No Waiver. Our failure to enforce or exercise any part of this Agreement is not a waiver of that section.

      +
      8. +
    8. +

      Notice. Aerokube may deliver any notice to Customer via electronic mail to an email address provided by Customer, registered mail, personal delivery or renowned express courier (such as DHL, FedEx or UPS). Any such notice will be deemed to be effective:

      +
      +
        +
      1. +

        On the day the notice is sent to Customer via email.

        +
        2. +
      2. +

        Upon personal delivery.

        +
        3. +
      3. +

        One (1) day after deposit with an express courier or five (5) days after deposit in the mail, whichever occurs first.

        +
        4. +
      +
      +
      9. +
    9. +

      Governing Law. This Agreement will be governed by the laws of the Estonia, without reference to conflict of laws principles. Customer agrees that any litigation relating to this Agreement may only be brought in, and will be subject to the jurisdiction of, any competent court of the Estonia. The parties agree that the United Nations Convention on Contracts for the International Sale of Goods does not apply to this Agreement.

      +
      10. +
    10. +

      Exceptions or Modifications. For exceptions or modifications to this Agreement, please contact Aerokube at: support@aerokube.com In case the terms of this Agreement are in conflict with the terms of any agreement individually negotiated and agreed between Aerokube and Customer, the terms of the latter shall prevail.

      +
      11. +
    11. +

      Force Majeure. Except with respect to Customer’s payment obligations, neither party shall be liable to the other for any delay or failure to perform any obligation under this Agreement (except for a failure to pay fees) if the delay or failure is due to unforeseen events which occur after the signing of this Agreement and which are beyond the reasonable control of such party ("Force Majeure Event"), such as a strike, blockade, war, act of terrorism, riot, natural disaster, failure or diminishment of power or telecommunications or data networks or services, or refusal of a license by a government agency. In the event of a Force Majeure Event that prevents one part from substantially performing its obligations hereunder for a period of ten (10) days or more, either party may terminate this Agreement on five (5) days written notice.

      +
      12. +
    +
    +
    4. +
  4. +

    Grant of Rights

    +
    +
      +
    1. +

      The Product include code and libraries licensed to Licensor by third parties, including open source software.

      +
      2. +
    2. +

      The Product is provided basing on the CPUs. If Customer complies with the terms of this Agreement, Customer has the rights stipulated hereunder for each Subscription that Customer acquires. Customer’s rights acquired in relation to the Product are limited to those necessary to enable Customer and its Users to effectively operate the Product(s). All other rights remain reserved to Licensor.

      +
      3. +
    3. +

      Unless the Subscription has expired or this Agreement is terminated in accordance with respective section, and subject to the terms and conditions specified herein, Licensor grants Customer a non-exclusive and non-transferable right to use each Product covered by the Subscription as stipulated below.

      +
      4. +
    4. +

      Customer may:

      +
      +
        +
      1. +

        For each License Key included to Subscription have one Product Installation of any version covered by the Subscription on any operating system supported by the Product.

        +
        2. +
      2. +

        Do Product Evaluation on one Product Installation of any version on any operating system supported by the Product.

        +
        3. +
      3. +

        Make one backup copy of the Product solely for archival/security backup purposes.

        +
        4. +
      +
      +
      5. +
    5. +

      Customer may not:

      +
      +
        +
      1. +

        Allow the same Product Installation to be used concurrently by more than the Number of CPUs specified for used License Key in Subscription Confirmation.

        +
        2. +
      2. +

        Rent, lease, reproduce, modify, adapt, create derivative works of, distribute, sell, or transfer the Product.

        +
        3. +
      3. +

        Provide access to the Product or the right to use the Product to a third party.

        +
        4. +
      4. +

        Reverse engineer, decompile, disassemble, modify, translate, make any attempt to discover the source code of the Product.

        +
        5. +
      5. +

        Remove or obscure any proprietary or other notices contained in the Product.

        +
        6. +
      +
      +
      6. +
    6. +

      Customer acknowledges that no ownership right is conveyed to Customer under this Agreement, irrespective of the use of terms such as "purchase" or "sale". Licensor has and retains all rights, title and interest, including all intellectual property rights, in and to the Products and any and all related or underlying technology, and any modifications or derivative works thereof, including without limitation as they may incorporate Feedback (as defined below).

      +
      7. +
    7. +

      This Agreement applies whether Customer purchases a Subscription directly from Licensor or through resellers. If Customer purchases through a reseller, the Subscription details shall be as stated in the Subscription Confirmation issued by the reseller to Customer, and the reseller is responsible for the accuracy of any such Subscription Confirmation. Resellers are not authorized to make any promises or commitments on Licensor behalf, and Customer understands and agrees that Licensor is not bound by any obligations to Customer other than as specified in this Agreement.

      +
      8. +
    +
    +
    5. +
  5. +

    Access to Products

    +
    +
      +
    1. +

      All deliveries under this Agreement will be electronic. Customer and its Users must have an Internet connection in order to receive any deliveries. For the avoidance of doubt, Customer is responsible for downloading and installing the Products. Download instructions are made available on Licensor website at https://aerokube.com/boot/.

      +
      2. +
    2. +

      Customer enables full access to Product Installation by specifying a License Key from Subscription Confirmation.

      +
      3. +
    3. +

      Subject to the terms of this Agreement, Customer is granted a right to install and use the Product for evaluation purposes without charge for unlimited amount of time. The Product contains a feature that will automatically limit allowed Number of CPUs. Licensor reserves the right at any time to change that limit in new Product versions.

      +
      4. +
    +
    +
    6. +
  6. +

    Fees

    +
    +
      +
    1. +

      Customer shall pay its Subscription fees in accordance with Licensor Terms of Purchase or the reseller’s terms of purchase, whichever are applicable.

      +
      2. +
    2. +

      The Subscription fees shall be paid in full, and any levies, duties and/or taxes imposed by Customer’s jurisdiction (including, but not limited to, value added tax, sales tax and withholding tax), shall be borne solely by Customer.

      +
      3. +
    3. +

      Customer may not deduct any amounts from fees payable to Licensor or the reseller, unless otherwise specified in the applicable terms of purchase.

      +
      4. +
    +
    +
    7. +
  7. +

    Feedback

    +
    +
      +
    1. +

      Customer has no obligation to provide Licensor with ideas, suggestions, or proposals ("Feedback").

      +
      2. +
    2. +

      If Customer or Users submit Feedback to Licensor, then Customer grants Licensor a non-exclusive, worldwide, royalty-free license that is sub-licensable and transferable, to make, use, sell, have made, offer to sell, import, reproduce, publicly display, distribute, modify, or publicly perform the Feedback in any manner without any obligation, royalty or restriction based on intellectual property rights or otherwise.

      +
      3. +
    +
    +
    8. +
  8. +

    LIMITED WARRANTY

    +
    +

    ALL PRODUCTS ARE PROVIDED TO CUSTOMER ON AN "AS IS" AND "AS AVAILABLE" BASIS WITHOUT WARRANTIES. USE OF THE PRODUCTS IS AT YOUR OWN RISK. AEROKUBE MAKES NO WARRANTY AS TO THEIR USE OR PERFORMANCE. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, AEROKUBE, AND ITS SUPPLIERS (WHICH SHALL INCLUDE THE PROVIDERS OF THE THIRD PARTY SOFTWARE) AND RESELLERS, DISCLAIM ALL WARRANTIES AND CONDITIONS, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, AND NON-INFRINGEMENT, WITH REGARD TO THE PRODUCTS, AND THE PROVISION OF OR FAILURE TO PROVIDE SUPPORT SERVICES. THIS LIMITED WARRANTY GIVES CUSTOMER SPECIFIC LEGAL RIGHTS. CUSTOMER MAY HAVE OTHER RIGHTS, WHICH VARY FROM STATE/JURISDICTION TO STATE/JURISDICTION. AEROKUBE (AND ITS AFFILIATES, AGENTS, DIRECTORS AND EMPLOYEES) DOES NOT WARRANT:

    +
    +
    +
      +
    1. +

      THAT THE PRODUCTS ARE ACCURATE, RELIABLE OR CORRECT

      +
      2. +
    2. +

      THAT THE PRODUCTS WILL MEET YOUR REQUIREMENTS

      +
      3. +
    3. +

      THAT THE PRODUCTS WILL BE AVAILABLE AT ANY PARTICULAR TIME OR LOCATION, UNINTERRUPTED OR SECURE

      +
      4. +
    4. +

      THAT ANY DEFECTS OR ERRORS WILL BE CORRECTED

      +
      5. +
    5. +

      THAT THE PRODUCTS ARE FREE OF VIRUSES OR OTHER HARMFUL COMPONENTS

      +
      6. +
    +
    +
    +

    ANY CONTENT OR DATA DOWNLOADED OR OTHERWISE OBTAINED THROUGH THE USE OF THE PRODUCTS ARE DOWNLOADED AT YOUR OWN RISK AND YOU WILL BE SOLELY RESPONSIBLE FOR ANY DAMAGE TO YOUR PROPERTY OR LOSS OF DATA THAT RESULTS FROM SUCH DOWNLOAD. NO WARRANTY OR LIABILITY AT ALL IS GIVEN TO PRODUCTS UNDER EVALUATION.

    +
    +
    9. +
  9. +

    DISCLAIMER OF DAMAGES

    +
    +
      +
    1. +

      TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT WILL AEROKUBE (OR ITS AFFILIATES, AGENTS, DIRECTORS, OR EMPLOYEES), OR AEROKUBE LICENSORS, SUPPLIERS OR RESELLERS BE LIABLE TO CUSTOMER OR ANYONE ELSE FOR:

      +
      +
        +
      1. +

        ANY LOSS OF USE, DATA, GOODWILL, OR PROFITS, WHETHER OR NOT FORESEEABLE

        +
        2. +
      2. +

        ANY LOSS OR DAMAGES IN CONNECTION WITH TERMINATION OR SUSPENSION OF CUSTOMER’S ACCESS TO OUR PRODUCTS IN ACCORDANCE WITH THIS AGREEMENT

        +
        3. +
      3. +

        ANY SPECIAL, INCIDENTAL, INDIRECT, CONSEQUENTIAL, EXEMPLARY OR PUNITIVE DAMAGES WHATSOEVER (EVEN IF WE HAVE BEEN ADVISED OF THE POSSIBILITY OF THESE DAMAGES), INCLUDING THOSE:

        +
        +
          +
        1. +

          RESULTING FROM LOSS OF USE, DATA, OR PROFITS, WHETHER OR NOT FORESEEABLE

          +
          2. +
        2. +

          BASED ON ANY THEORY OF LIABILITY, INCLUDING BREACH OF CONTRACT OR WARRANTY, STRICT LIABILITY, NEGLIGENCE OR OTHER TORTIOUS ACTION

          +
          3. +
        3. +

          ARISING FROM ANY OTHER CLAIM ARISING OUT OF OR IN CONNECTION WITH CUSTOMER’S USE OF OR ACCESS TO THE PRODUCTS OR SUPPORT.

          +
          4. +
        +
        +
        4. +
      +
      +
      2. +
    2. +

      THE FOREGOING LIMITATION OF LIABILITY SHALL APPLY TO THE FULLEST EXTENT PERMITTED BY LAW IN THE APPLICABLE JURISDICTION.

      +
      3. +
    3. +

      THE TOTAL LIABILITY IN ANY MATTER ARISING OUT OF OR IN RELATION TO THIS AGREEMENT IS LIMITED TO ONE HUNDRED (100) US DOLLARS OR THE AGGREGATE AMOUNT PAID OR PAYABLE BY THE CUSTOMER FOR PRODUCTS DURING THE THREE-MONTH PERIOD PRECEDING THE EVENT GIVING RISE TO THE LIABILITY, WHICHEVER IS GREATER. THIS LIMITATION WILL APPLY EVEN IF WE OR YOU HAVE BEEN ADVISED OF THE POSSIBILITY OF LIABILITY EXCEEDING SUCH AMOUNT AND NOTWITHSTANDING ANY FAILURE OF THE ESSENTIAL PURPOSE OF ANY LIMITED REMEDY.

      +
      4. +
    +
    +
    10. +
  10. +

    Term and Termination

    +
    +
      +
    1. +

      The term of this Agreement will commence upon acceptance of this Agreement by Customer as set forth in the preamble above, and will continue for each Product through the end of the applicable subscription period specified in the respective Subscription Confirmation. This Agreement will automatically renew with respect to each Product for a successive subscription term, unless terminated as set forth herein.

      +
      2. +
    2. +

      Customer may terminate this Agreement at any time by cancelling its Product subscription. If such termination occurs during a then-current subscription period, this Agreement will continue to be effective until the end of that subscription period. Such termination does not relieve Customer of the obligation to pay any outstanding subscription fees owed to Licensor, and no credits or refunds will be issued to Customer for prepaid subscription fees (except as specified in the Licensor Terms of Purchase, if applicable).

      +
      3. +
    3. +

      Licensor may terminate this agreement if:

      +
      +
        +
      1. +

        Customer has materially breached this Agreement and fails to cure such breach within thirty (30) days of written notice thereof.

        +
        2. +
      2. +

        Customer fails to make the timely payment of subscription fees in accordance with "Fees" Section of this Agreement.

        +
        3. +
      3. +

        Licensor is required to do so by law (for example, where the provision of the Product to Customer is, or becomes, unlawful).

        +
        4. +
      4. +

        Licensor elects to discontinue providing the Product, in whole or in part.

        +
        5. +
      +
      +
      4. +
    4. +

      Licensor will make reasonable efforts to notify Customer via email as follows:

      +
      +
        +
      1. +

        Thirty (30) days prior to termination of the Agreement when required to terminate by law or because of discontinued Product. In such events Customer will be entitled to a refund of the unused portion of prepaid subscription fees, if applicable.

        +
        2. +
      2. +

        Three (3) days prior to termination of the Agreement in other cases. In such events Customer will not be entitled to any refund of the unused portion of prepaid subscription fees.

        +
        3. +
      +
      +
      5. +
    +
    +
    11. +
  11. +

    Temporary Suspension for Non-payment

    +
    +
      +
    1. +

      Licensor reserves the right to suspend or limit Customer’s access to Aerokube Products if Customer fails to pay subscription fees on time.

      +
      2. +
    2. +

      If Licensor suspends or limits Customer’s access to Aerokube Products for non-payment according, Customer must pay all past due amounts in order to restore full access to Aerokube Products.

      +
      3. +
    3. +

      Customer hereby agrees that Licensor is entitled to charge Customer for the time period during which Customer has access to Aerokube Products until Customer or Licensor terminates or suspends Customer’s subscription in accordance with this Agreement.

      +
      4. +
    +
    +
    12. +
  12. +

    Export Regulations

    +
    +

    Customer shall comply with all applicable laws and regulations with regards to economic sanctions, export controls, import regulations, and trade embargoes (all herein referred to as "Sanctions"), including those of the European Union and United States (specifically the Export Administration Regulations (EAR)). Customer declares that it is not a person targeted by Sanctions nor is it otherwise owned or controlled by or acting on behalf of any person targeted by Sanctions. Further, Customer warrants that it will not download or otherwise export or re-export the Product or any related technical data directly or indirectly to any person targeted by Sanctions or download or otherwise use the Product for any end-use prohibited or restricted by Sanctions.

    +
    +
    13. +
  13. +

    Customer Support

    +
    +
      +
    1. +

      Licensor provides Email Support as well as Instant Messaging Support. The response time will be reasonable, but no specific response time guarantees are given.

      +
      2. +
    2. +

      Customer may request additional paid support from Licensor which is subject of a supplementary individually negotiated Agreement between Customer and Licensor.

      +
      3. +
    3. +

      Any guarantees of support availability only apply to the latest version of Licensed Software available in Customer Subscription.

      +
      4. +
    +
    +
    14. +
  14. +

    Customer Data

    +
    +
      +
    1. +

      Use of Name and Logo. Customer agrees that Licensor may identify it as a customer of Aerokube and may refer to it by name, trade name and trademark, if applicable. Licensor may also briefly describe Customer’s business in Licensor marketing materials, on the Aerokube website and/or in public or legal documents. Customer hereby grants Licensor a worldwide, non-exclusive and royalty-free license to use Customer’s name and any of Customer’s trade names and trademarks solely pursuant to this marketing section. Notwithstanding anything to the contrary herein, Licensor acknowledges that in some cases Customer licenses and does not own marks or logos (for example, marks or logos of the Affiliates) and cannot permit Licensor to use such marks.

      +
      2. +
    2. +

      Gathering of Usage Statistics. Customer acknowledges and agrees that the Product may contain a feature that reports the usage statistics, diagnostics information and usage meta-information of the Product back to the Licensor. Customer may opt out of the gathering of usage statistics by turning off this feature in the Product settings.

      +
      3. +
    +
    +
    15. +
+
+
+
+
+

5. Pricing

+
+
+

Last updated August 1st, 2023. Replaces the prior version in its entirety.

+
+
+
    +
  1. +

    Boot price is based on the total number of CPUs you can use for your virtual machines. We control this by limiting total number of CPUs all running browser pods can consume in all namespaces.

    +
    2. +
  2. +

    When no license key is provided 1 (one) CPU maximum is allowed. If such limit is sufficient for you - you are allowed use Boot without license key for unlimited period of time.

    +
    3. +
  3. +

    If free limit is insufficient - you need a paid license. Such license can include any desired number of CPUs.

    +
    4. +
  4. +

    Every CPU has a fixed cost - $5 USD (five United States dollars). If you are an EU-based company - then we convert the price to euro (€).

    +
    +
    An example price calculation
    +
    +
    42 CPUs * $5/month = $210/month
    +
    +
    +
    5. +
  5. +

    For simplicity, we calculated monthly prices for some frequent cases:

    + + ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Table 4. Boot License Pricing
    Number of CPUsPrice per Month, USD

    1

    free

    5

    $25

    10

    $50

    15

    $75

    20

    $100

    25

    $125

    30

    $150

    40

    $200

    50

    $250

    75

    $375

    100

    $500

    150

    $750

    200

    $1000

    250

    $1250

    500

    $2500

    750

    $3750

    1000

    $5000

    +
    6. +
+
+
+
+
+ + + \ No newline at end of file