Libvirt VM provisioning (step 1/4) #352
Conversation
* WIP: Provisioning an OpenShift Cluster on Libvirt * remove symlink * remove test user from ks files * remove yml file * edit inventry file * modify the host_vars * edit ks file path file * modify inventory and playbook
stopped VMs are silently ignored, works with Python3, add environment variables to configure script in regard to connection uri, VMs filtering and variables prefix in inventory.
…al network used. This makes it easier to avoid collisions with already existing networks. The playbook `provision.yml` runs through and 3 hosts are created but there is more to do in terms of storage and DNS/network setup to make it workable for OpenShift is my strong assumption.
|
|
||
| === Setup a local dnsmasq | ||
|
|
||
| Create a dummy network interface: |
There was a problem hiding this comment.
My opinion here is that these networking pieces should be handled as a prereq step. Then we could parameterize the specifics below and an inventory could be completely reproducible between machines (instead of wondering whether I had setup the networking pieces the same way between two machines).
| Start dnsmasq against this interface, defining our wildcard DNS domain *.apps.local | ||
|
|
||
| ------------------------------------------------------------------------ | ||
| sudo dnsmasq --interface=dummy0 --no-daemon --log-queries=extra \ |
There was a problem hiding this comment.
Same as above. Would like to see this done as part of a prereq task.
|
|
||
| === Create a separate network | ||
|
|
||
| Call `sudo virsh net-create --file libvirt-network-definition.xml` |
There was a problem hiding this comment.
Same as above. Would like to see this done as part of a prereq task.
| - make sure that `/tmp/authorized_keys` exists. FIXME: not sure yet for which use cases it is required, I just copy for now my own authorized keys. | ||
| - call `ansible-playbook -i ../../inventory/sample.libvirt.example.com.d/inventory libvirt/provision.yml`. | ||
| + | ||
| IMPORTANT: virt-install is only running synchronously because a virt-viewer UI is popping up. Close each virt-viewer once the corresponding installation has happened and not too long after. |
There was a problem hiding this comment.
Running this on Fedora 29 -- I didn't see any individual virt-viewer UI's opening (or that needed closing). I did see that the timeout didn't seem long enough to allow the VM's to spin up though, which required running the playbook multiple times.
|
|
||
| === provision-instances | ||
|
|
||
| - make sure `/dev/loopN` isn't mounted on `/var/www/html/installXXX`, and remove it from your `/etc/fstab` if you try multiple times with errors (something to FIXME). |
There was a problem hiding this comment.
I think this has more to do with the virt-install role from infra-ansible, so we may want to open up an issue there?
| reboot | ||
|
|
||
| # network | ||
| network --bootproto=dhcp --device=eth0 --noipv6 --activate --hostname=appnode.local |
There was a problem hiding this comment.
It seems like the only difference in all of these kickstart files is the --hostname. Could we make this a template so that all of the nodes could use a single file versus having to maintain this for multiple nodes?
| @@ -0,0 +1,14 @@ | |||
| <network ipv6='no'> | |||
There was a problem hiding this comment.
Connected to making the network setup part of a pre-req task, could we make this a template and then have the inventory provide things like bridge name, domain name, etc. ? Seems like there may/could be scenarios where not everyone would be able to use these defaults.
| ansible_host: localhost | ||
| ansible_user: root | ||
| ansible_connection: local | ||
| ansible_python_interpreter: /usr/bin/python3 |
There was a problem hiding this comment.
Should this work with python 2.X and 3.X ? Or is there a reason we want to force the use of python 3?
| @@ -0,0 +1,37 @@ | |||
| --- | |||
There was a problem hiding this comment.
I see that we're using a number of individual hostvars instead of having what the cluster "looks" like inside of all.yml. This looks like it works -- but seems very different from the other infra setups (aws, gcp, etc.). What I'd prefer to see is something that looks similar to:
libvirt_infrastructure:
libvirt_iso: /path/to/iso
libvirt_http_host: address of http host
libvirt_os_variant: rhel7
masters:
count: 1
cpu: 1
memory: 2048
name_prefix: <REPLACE WITH DESIRED NAME PREFIX FOR THE INSTANCE>
root_volume_size: 15
...
We could also look at using a "cluster template" similar to how AWS relies on a cloud formation template and GCP relies on a deployment template. I believe there's an equivalent Libvirt component to this, so perhaps a combination of the above and this could be an option?
Interested in seeing others thoughts here as well. I'd like to make sure we have a solid foundation here as I'm sure making changes here will require changes to the other steps in these PR's.
cc/ @oybed
|
@ericzolf Have you had a chance to look at the above comments? |
|
Sorry for the delay, I'll look in all comments (which I really appreciate) but I'm currently a bit under water. Be assured, you're not forgotten. |
What does this PR do?
It is the first step to get Libvirt as a working infrastructure for CASL. For the time being, it does only create the necessary VMs and instructions to setup DNS for wildcard resolution, but this is the basis for all the rest.
It is still work in progress but it would make it easier for others to help/test etc if it would be merged now, and it doesn't change anything (yet) outside of the Libvirt realm, so it shouldn't break anything. And it would make further merging also easier for me...
How should this be manually tested?
Follow the instructions from
docs/PROVISIONING_LIBVIRT.adoc(this is part of the test if they are understandable, if the VMs are created and DNS works as explained in the document, you're good to go.Is there a relevant Issue open for this?
Only discussion on GChat.
Other Relevant info, PRs, etc.
this requires the PR redhat-cop/infra-ansible#311 to be downloaded through galaxy for this PR to work.
Who would you like to review this?
cc: @redhat-cop/casl