Remove the virtualization code

[cbosdo]

What does this PR change?

Trash virtualization features.

GUI diff

The whole Virtualization tab in the systems details is gone!

Documentation

• Documentation issue was created: Link for SUSE Manager contributors, Link for community contributors.

• API documentation added: please review the Wiki page Writing Documentation for the API if you have any changes to API documentation.

• (OPTIONAL) Documentation PR

• DONE

Test coverage

ℹ️ If a major new functionality is added, it is strongly recommended that tests for the new functionality are added to the Cucumber test suite

• No tests: removing code and related tests will probably increase the code coverage in the end 😉

• DONE

Links

Issue(s): #
Port(s): # add downstream PR(s), if any

• DONE

Changelogs

Make sure the changelogs entries you are adding are compliant with https://github.com/uyuni-project/uyuni/wiki/Contributing#changelogs and https://github.com/uyuni-project/uyuni/wiki/Contributing#uyuni-projectuyuni-repository

If you don't need a changelog check, please mark this checkbox:

• No changelog needed

If you uncheck the checkbox after the PR is created, you will need to re-run changelog_test (see below)

Re-run a test

If you need to re-run a test, please mark the related checkbox, it will be unchecked automatically once it has re-run:

• Re-run test "changelog_test"
• Re-run test "backend_unittests_pgsql"
• Re-run test "java_pgsql_tests"
• Re-run test "schema_migration_test_pgsql"
• Re-run test "susemanager_unittests"
• Re-run test "javascript_lint"
• Re-run test "spacecmd_unittests"

Before you merge

Check How to branch and merge properly!

[github-actions]

👋 Hello! Thanks for contributing to our project.
Acceptance tests will take some time (aprox. 1h), please be patient ☕
You can see the progress at the end of this page and at https://github.com/uyuni-project/uyuni/pull/9276/checks
Once tests finish, if they fail, you can check 👀 the cucumber report. See the link at the output of the action.
You can also check the artifacts section, which contains the logs at https://github.com/uyuni-project/uyuni/pull/9276/checks.

If you are unsure the failing tests are related to your code, you can check the "reference jobs". These are jobs that run on a scheduled time with code from master. If they fail for the same reason as your build, it means the tests or the infrastructure are broken. If they do not fail, but yours do, it means it is related to your code.

Reference tests:

• https://github.com/uyuni-project/uyuni/actions/workflows/acceptance_tests_secondary_parallel.yml?query=event%3Aschedule

• https://github.com/uyuni-project/uyuni/actions/workflows/acceptance_tests_secondary.yml?query=event%3Aschedule

KNOWN ISSUES

Sometimes the build can fail when pulling new jar files from download.opensuse.org . This is a known limitation. Given this happens rarely, when it does, all you need to do is rerun the test. Sorry for the inconvenience.

For more tips on troubleshooting, see the troubleshooting guide.

Happy hacking!
⚠️ You should not merge if acceptance tests fail to pass. ⚠️

[cbosdo]

I think this is too much removal. We do not want to create VMs, but show the topology and the status is still needed. Let's discuss how far we want to go

Showing the topology is still possible in the Virtual systems list page.

[mcalmer]

I think this is too much removal. We do not want to create VMs, but show the topology and the status is still needed. Let's discuss how far we want to go

Showing the topology is still possible in the Virtual systems list page.

But who send the data from the libvirt host about the available VMs?
I think we need to add a VirtualHostGatherer for libvirt or is the salt engine still available?
@cbosdo if the VirtualHostGatherer plugin is needed, please open an issue for it.

[mcalmer]

I am not sure if this is a good idea. Would this change the Event History of systems which used such an action before?
I think changing the event history is critical. I would like to have the OK from @admd before doing it.

Is there maybe a way to drop these tables and classes but keep the event history?

[cbosdo]

Changing the history for actions that are supposed to not be used shouldn't be a problem. If that is a problem, it probably mean we need to keep the whole feature because someone uses it.

[admd]

It's a tricky one from legal perspective. Ideally, we should not remove any history from the systems, at least for a pre-defined time(6 months or so, different companies have different policies)

If we can somehow keep the history while removing the feature itself, that would be ideal. If that's not feasible and requires a lot more work, I would just go ahead with this change. However, would add a note in release notes that this will happen.

[cbosdo]

The problem is that we will have to keep a dozen of tables and the matching Java code, just to show empty content because no-one was using it.

[admd]

Ok if there is no easy way, we go ahead with your change. We will add a note in release notes when times comes.

[cbosdo]

I think this is too much removal. We do not want to create VMs, but show the topology and the status is still needed. Let's discuss how far we want to go

Showing the topology is still possible in the Virtual systems list page.

But who send the data from the libvirt host about the available VMs? I think we need to add a VirtualHostGatherer for libvirt or is the salt engine still available? @cbosdo if the VirtualHostGatherer plugin is needed, please open an issue for it.

There is a libvirt host gatherer that we could add. The engine was somewhat fragile and is now removed.

[admd]

I think this is too much removal. We do not want to create VMs, but show the topology and the status is still needed. Let's discuss how far we want to go

Showing the topology is still possible in the Virtual systems list page.

But who send the data from the libvirt host about the available VMs? I think we need to add a VirtualHostGatherer for libvirt or is the salt engine still available? @cbosdo if the VirtualHostGatherer plugin is needed, please open an issue for it.

There is a libvirt host gatherer that we could add. The engine was somewhat fragile and is now removed.

@cbosdo would the Virtual Host field on this page https://suma-long-43-srv.mgr.suse.de/rhn/manager/systems/list/virtual still be populated after the engine removal and we can see the relation between virtual host & virtual systems?

[cbosdo]

I think this is too much removal. We do not want to create VMs, but show the topology and the status is still needed. Let's discuss how far we want to go

Showing the topology is still possible in the Virtual systems list page.

But who send the data from the libvirt host about the available VMs? I think we need to add a VirtualHostGatherer for libvirt or is the salt engine still available? @cbosdo if the VirtualHostGatherer plugin is needed, please open an issue for it.

There is a libvirt host gatherer that we could add. The engine was somewhat fragile and is now removed.

@cbosdo would the Virtual Host field on this page https://suma-long-43-srv.mgr.suse.de/rhn/manager/systems/list/virtual still be populated after the engine removal and we can see the relation between virtual host & virtual systems?

Yes it will be populated: the hardware refresh and the virtual host gatherer are the ones involved on this IIRC.

[admd]

I think this is too much removal. We do not want to create VMs, but show the topology and the status is still

Yes it will be populated: the hardware refresh and the virtual host gatherer are the ones involved on this IIRC.

Good! We just need to make sure that we don't break that part.

[mcalmer]

I think this is too much removal. We do not want to create VMs, but show the topology and the status is still

Yes it will be populated: the hardware refresh and the virtual host gatherer are the ones involved on this IIRC.

Good! We just need to make sure that we don't break that part.

It will break, as the admin has to setup the libvirt module. Especially when an authentication is required.
But first I will create an issue for adding the libvirt module and implement a way to configure it

[mcalmer]

@admd I have created https://github.com/SUSE/spacewalk/issues/25437 and added it to the planning board

[cbosdo]

It will break, as the admin has to setup the libvirt module. Especially when an authentication is required. But first I will create an issue for adding the libvirt module and implement a way to configure it

I don't care if there is a migration path and the user have to take action.... otherwise we keep everything for ever! Either we want to remove the WHOLE feature or NOTHING!

[mcalmer]

ok, but we need to work on the replacement to keep the host <=> VM relation

[cbosdo]

ok, but we need to work on the replacement to keep the host <=> VM relation

Remember that the libvirt engine was only set up if the Virtualization Host Addon type was enabled on the host. The host / VM relation was there before... Everything else was there before

[rjmateus]

@mcalmer how was this information added and used? Was it part of the data that is sent to SCC?

We should not reduce the data quality we send to SCC. We should have a look and see if adding the connection to libvirt on "Virtual Host Managers" is enough (I think it should be, since this is also the case for other providers).
I tried to install the virtual host gatherer for libvirt and it returns an exception. Doesn't loo to complex but it will need some work in that regard (some fields should not be null by default, but instead a empty string)

[cbosdo]

Before looking at the libvirt virtual host gatherer, we need to check what happens with just the regular hardware update. For sure you will not know about the unregistered VM running on a host... but that has always been the case without the Virtualization Addon.

My point here is that we should not discuss keeping data that are gathered only using the virtualization addon as this thing is not supposed to be used

[rjmateus]

My point here is that we should not discuss keeping data that are gathered only using the virtualization addon as this thing is not supposed to be used

Agree. But we first need to check if is being used and what could be broken. Then plan on how to replace that data.

[mcalmer]

@cbosdo yes, it was working before you implemented the new virtualization feature. But we had this "Poller" in the past from the traditional stack and when I remember correctly, this was removed when you implemented the new way with the salt engine.
We need to check.

@rjmateus yes, this is used when sending data to SCC

[admd]

@cbosdo yes, it was working before you implemented the new virtualization feature. But we had this "Poller" in the past from the traditional stack and when I remember correctly, this was removed when you implemented the new way with the salt engine. We need to check.

@cbosdo do you remember anything around this?

@meaksh any thoughts on this one topic. We would like to remove this whole vritualization feature but would still like to show the relationship between Virtual host and it's guests. More discussion around this topic #9276 (comment)

[cbosdo]

@cbosdo yes, it was working before you implemented the new virtualization feature. But we had this "Poller" in the past from the traditional stack and when I remember correctly, this was removed when you implemented the new way with the salt engine. We need to check.

@cbosdo do you remember anything around this?

I don't remember but git does: 9ffc5bc2 the virtpoller beacon was only added on systems that had the virtualization entitlement... that one has been removed. Even before that the virtpoller beacon was running on all systems: 3be753a1. Mind that this happened 4 years ago... We could still add that beacon again with all the trouble it could be after 4 years of changes.

[admd]

I don't remember but git does: 9ffc5bc2 the virtpoller beacon was only added on systems that had the virtualization entitlement... that one has been removed. Even before that the virtpoller beacon was running on all systems: 3be753a1. Mind that this happened 4 years ago... We could still add that beacon again with all the trouble it could be after 4 years of changes.

Thank you Cedric for digging up. So we either need poller back in or virtual-host-gatherer-libvirt. Please do correct me if I am missing something.

I believe , we will be going this way https://github.com/SUSE/spacewalk/issues/25437 . We can proceed with merging the PR but we will need to get this https://github.com/SUSE/spacewalk/issues/25437 done before the 5.1 FCS. I wonder how much work this would require.

@rjmateus @mcalmer any final thoughts?

[rjmateus]

@admd nothing else from my side. I think we can/should move forward with this PR being merged, but we need to make sure we have a solution for the link between VM's and hostOS before 5.1 FCS as you mentioned.

[mcalmer]

The libvirt gatherer module was never tested in real environments.
Maybe it is working as bad as the virtpoller or even worth.
The gatherer must be configured which maybe people will not do or fill L3s if they do not understand.
Virtpoller can be automatically enabled and configured. We may not depend on a virtualization entitlement, maybe we simply check if libvirtd package is installed.
I would vote vor keeping this small component and stay away from the gatherer way.

[rjmateus]

Thank you @mcalmer
We can revive the poller component, and test it to check if it needs some adjustment, since it was removed a few years ago. As you said, if we use a virtual host gatherer users must configure it manually. With the poller we can configure it out of the box by checking the installed packages.
However, we must provide a way for users to disable it, in case someone doesn't want to collect the information.

[mcalmer]

Thank you @mcalmer We can revive the poller component, and test it to check if it needs some adjustment, since it was removed a few years ago. As you said, if we use a virtual host gatherer users must configure it manually. With the poller we can configure it out of the box by checking the installed packages. However, we must provide a way for users to disable it, in case someone doesn't want to collect the information.

Ohh, not "that" poller.
I was thinking about the salt engine which does the same with salt using events.
I think the old poller will not work anymore as it uses the traditional API which is gone.

[rjmateus]

@mcalmer I was thinking about this beacon, which looks to be firing salt events to the server: 9ffc5bc2#diff-e574bf1fbaa404654ba3a68642f6c2a4095f6e8db4f7283576031ed51a6b23d8L18

Did I get it wrong? I just look at the code in diagonal :\

[mcalmer]

@mcalmer I was thinking about this beacon, which looks to be firing salt events to the server: 9ffc5bc2#diff-e574bf1fbaa404654ba3a68642f6c2a4095f6e8db4f7283576031ed51a6b23d8L18

Did I get it wrong? I just look at the code in diagonal :\

Ok, we mean the same. This is the "new" poller. And I think this was still in use with the virtualization support.

The old was from spacewalk times and that cannot work anymore. When you said
"and test it to check if it needs some adjustment, since it was removed a few years ago" I was thinking you was referring to that old perl written ccomponent.

[rjmateus]

ah ok :)
That beacon looks like it got removed in that PR in 2020. I didn't find if that beacon is till in use or if it's a different implementation. For that Cedric can give more information.
From my side, I think the beacon is a good approach, we can install it automatically and get the data we need.

[cbosdo]

ah ok :) That beacon looks like it got removed in that PR in 2020. I didn't find if that beacon is till in use or if it's a different implementation. For that Cedric can give more information. From my side, I think the beacon is a good approach, we can install it automatically and get the data we need.

The beacon is gone... see the commit in 2020. But I think it's the easiest thing to get some mapping.

[meaksh]

Sorry to join late on the discussion. I would also vote for bringing back the virtpoller beacon that was removed when introducing the virtualization code that we are dropping now in this PR.

That should be enough to keep the mapping between VMs and virthost, particularly in the context of Virtual Host Managers.