Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pes_events_scanner: fix processing of RpmTransactionTasks #1296

Merged
merged 2 commits into from
Jan 13, 2025

Conversation

pirat89
Copy link
Member

@pirat89 pirat89 commented Sep 17, 2024

RpmTransactionTasks messages have higher priority than instructions based on PES data. Previously, if multiple such messages existed with duplicate instructions, this could lead to the crash of the actor - especially in case when an existing package has been asked to be removed several times. Ensure the occurance of each instruction is unique (list -> set).

jira: RHEL-50076

Note: just written on knees without testing. Unit tests should be updated too.

@pirat89 pirat89 added the bug Something isn't working label Sep 17, 2024
Copy link

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
If you want to request a review or rebuild a package in copr, you can use following commands as a comment:

  • review please @oamg/developers to notify leapp developers of the review request
  • /packit copr-build to submit a public copr build using packit

Packit will automatically schedule regression tests for this PR's build and latest upstream leapp build.
However, here are additional useful commands for packit:

  • /packit test to re-run manually the default tests
  • /packit retest-failed to re-run failed tests manually
  • /packit test oamg/leapp#42 to run tests with leapp builds for the leapp PR#42 (default is latest upstream - main - build)

Note that first time contributors cannot run tests automatically - they need to be started by a reviewer.

It is possible to schedule specific on-demand tests as well. Currently 2 test sets are supported, beaker-minimal and kernel-rt, both can be used to be run on all upgrade paths or just a couple of specific ones.
To launch on-demand tests with packit:

  • /packit test --labels kernel-rt to schedule kernel-rt tests set for all upgrade paths
  • /packit test --labels beaker-minimal-8.10to9.4,kernel-rt-8.10to9.4 to schedule kernel-rt and beaker-minimal test sets for 8.10->9.4 upgrade path

See other labels for particular jobs defined in the .packit.yaml file.

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please contact leapp-infra.

@pirat89
Copy link
Member Author

pirat89 commented Sep 18, 2024

/packit copr-build

@MichalHe
Copy link
Member

@pirat89 Although this fixes the crash, I believe that transaction configuration is handled in a weird manner altogether. If user says to_remove=['tomcat'] then the output of the pes_events_scanner actor will not contain tomcat in its to_remove fields -- see the failing unit test. Not sure whether this is intended (unlikely) but we (e.g. me) can fix this problem within this PR.

@pirat89
Copy link
Member Author

pirat89 commented Nov 2, 2024

@pirat89 Although this fixes the crash, I believe that transaction configuration is handled in a weird manner altogether. If user says to_remove=['tomcat'] then the output of the pes_events_scanner actor will not contain tomcat in its to_remove fields -- see the failing unit test. Not sure whether this is intended (unlikely) but we (e.g. me) can fix this problem within this PR.

@MichalHe you are right that this sounds weird. feel free to fix it properly in this PR - or close this one and open new one if you want. this has been really a draft to ilustrate what I found and initial possible fix of the problem. based on your finding, it seems there is something dirty still around.

@MichalHe
Copy link
Member

MichalHe commented Nov 3, 2024

@pirat89 I added a post-processing step that will add transaction configuration's fields to the scanner's output.

RpmTransactionTasks messages have higher priority than instructions
based on PES data. Previously, if multiple such messages existed
with duplicate instructions, this could lead to the crash of
the actor - especially in case when an existing package has been
asked to be removed several times. Ensure the occurance of each
instruction is unique (list -> set).

jira: RHEL-50076
@MichalHe MichalHe marked this pull request as ready for review January 9, 2025 16:05
@MichalHe
Copy link
Member

MichalHe commented Jan 9, 2025

/packit build

@MichalHe
Copy link
Member

MichalHe commented Jan 9, 2025

I have tested the patch on a setup where I introduced an actor that produces:

self.produce(RpmTransactionTasks(to_remove=['tomcat', 'tomcat']))
self.produce(RpmTransactionTasks(to_remove=['tomcat']))

Using current upstream build, the upgrade crashed with the following error:

2025-01-09 16:31:15.324 ERROR    PID: 45901 leapp.workflow.FactsCollection: Actor pes_events_scanner has crashed: Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/leapp/repository/actor_definition.py", line 75, in _do_run
    actor_instance.run(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/leapp/actors/__init__.py", line 316, in run
    self.process(*args)
  File "/etc/leapp/repos.d/system_upgrade/common/actors/peseventsscanner/actor.py", line 41, in process
    process()
  File "/etc/leapp/repos.d/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py", line 512, in process
    source_pkgs = apply_transaction_configuration(source_pkgs)
  File "/etc/leapp/repos.d/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py", line 482, in apply_transaction_configuration
    source_pkgs_with_conf_applied.remove(pkg_name_to_pkg_info_map[pkg.name])
KeyError: tomcat:rhel-8-for-x86_64-appstream-rpms

Using build based on this PR results with no errors and the upgrade continues normally, without raising the above error.

Hence, I believe this PR is ready for review.

Previously, pes_events_scanner used transaction configuration to
only modify the way it initializes event application. As a consequence,
if a user specified to_remove=['pkg'], then the information would
not make it to pes_events_scanner's output. Similar situation would
arise with to_install/to_keep. This patch adds a post-processing to
explicitly add transaction configuration to the result of applying PES
events.
@pirat89 pirat89 requested review from a team and MichalHe January 10, 2025 10:35
@pirat89
Copy link
Member Author

pirat89 commented Jan 10, 2025

giving +1 for @MichalHe changes. considering it good to go, but as original author of the PR, waiting for someone else to review it and merge it - even when my changes are minor in this PR.

@pirat89 pirat89 added this to the 8.10/9.6 milestone Jan 10, 2025
@MichalHe
Copy link
Member

The failing 9>10 tests are not related to this PR. Since two of us are OK with this PR, I am going to merge this.

Copy link
Member

@MichalHe MichalHe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks OK to me, hence we have agreement of 2 developers. e2e tests are passing, so lets merge this

@MichalHe MichalHe merged commit 4962708 into oamg:main Jan 13, 2025
20 of 23 checks passed
@MichalHe MichalHe added the changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant label Jan 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants