It is likely you have heard of virtual machines or used softwares such as VirtualBox or the like. While those emulate the entire hardware stack on your computer, they are known for being intensive in their usage of computation power. What is more, they reuse nothing of the host's hardware or software, which further increases the workload on your CPU. Of course, Docker exists as a lighter alternative for virtualization (provided you do not need X) but when all is said and done, you still need to haul around a Linux image.
This is where KVM comes in handy. Kernel-based Virtual Machine (KVM) is the Linux Kernel's level-zero virtual hypervisor. KVM can be used to create a virtual machine with a number of virtual CPUs from strach, and handle anything from registers to memory to interrupts.
Of course, this is for Linux only.
KVM is used in a number of programs. Notably, Firecracker, the runtime of AWS Lambda, also uses KVM. In our application, KVM is used in order to ensure isolation and security when running guest web assembly code on a server.
Calls to the KVM API are done by means of IOCTL (input/output control). For these examples, the C programming language will be used, but any language which can send IOCTL calls should do the trick.
First, we need to obtain a POSIX file descriptor to the /dev/kvm file in order to get started. For this, we use:
// Parameters: Read/Write/Execute flags
int kvm_fd = open("/dev/kvm", O_RDWR | O_CLOEXEC);Once a file descriptor is available for the /dev/kvm file, it is possible to start calling the KVM API using IOCTL. For instance here, we will verify that the KVM version is correct.
int version = ioctl(kvm, KVM_GET_API_VERSION, NULL);
if (version == -1)
err(1, "KVM_GET_API_VERSION");
printf("kvm version ok: %d\n", version);The various existing IOCTL calls are divided in three categories: system IOCTLs, vm IOCTLs, and vcpu IOCTLs based on what kind of file descriptors they require. System IOCTLs include API version checks, VM creation, etc. VM IOCTLs are mainly used to create VCPUs and handle memory, and vcpu IOCTLs are used to handle anything from registers to running code.
Creating a VM is a basic task that can be invoked with the KVM API. A VM serves as a compartiment to hold an undefined number of VCPUs which have to be manually created afterwards. To create a new VM, call the following system IOCTL:
//Parameters: VM ID (?)
int vmfd = ioctl(kvm_fd, KVM_CREATE_VM, (unsigned long)0);
if (vmfd == -1)
err(1, "KVM_CREATE_VM");Once a VM is created, you can run IOCTL calls using its file descriptor. Aside from creating VCPUs, VM ioctls are also useful for handling memory. Since x86 memory segmentation and handling is its own beast itself, we will handle it in another part.
Once a VM has been created, it can be used to fire up one or more VCPUs, which will then be able to run code. Do note, however, that you will need to allocate and handle memory appropriately in order to store your byte code somewhere where the VCPU can read it.
// Parameters: VCPU id
int vcpufd = ioctl(vmfd, KVM_CREATE_VCPU, (unsigned long)0);
if (vcpufd == -1)
err(1, "KVM_CREATE_VCPU");All in all, you now have a virtual machine and an associated virtual CPU with id 0. However... it is very much useless! Sadly, a CPU does not run on its own. We will now have an in-depth look at how to start the VCPU in the right mode.
First of all, we will have to decide where the guest code will be placed. Adresses in this will always be written in hexadecimal, and remember that adresses under Ox1000 cannot be used.
The memory of our VM will have to include some required elements: a MMU (memory mapping unit) table and a GDT (global descrptor table). We will have to build these two elements, and follow with the stack and the guest code.