Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #33: rewrite processor initialization, fix task queue and synchronization issue on Linux #40

Merged
merged 5 commits into from
Oct 29, 2024

Conversation

rafa-be
Copy link
Collaborator

@rafa-be rafa-be commented Oct 25, 2024

This PR fixes a few issues that occurred on high nested task load (TestNestedTask.test_multiple_recursive_task):

  1. AsyncPriorityQueue now correctly removes tasks with a negative priority value;
  2. the worker VanillaHeartbeatManager now watches all processors (not only the active one), and better handle zombie and dead processes;
  3. prevents the processor to be suspended while still waiting on the pausing event (caused an unsupported re-entrant call on Linux);
  4. the processor manager now handles correctly balance/cancel and suspension commands on a processor that is not yet initialized.

@rafa-be
Copy link
Collaborator Author

rafa-be commented Oct 25, 2024

image

8 successful runs in a row 🎉

@rafa-be rafa-be requested a review from sharpener6 October 25, 2024 19:59
sharpener6
sharpener6 previously approved these changes Oct 29, 2024
@sharpener6
Copy link
Collaborator

Please bum up the version

@sharpener6 sharpener6 dismissed their stale review October 29, 2024 19:14

Please update the version number

@sharpener6 sharpener6 self-requested a review October 29, 2024 22:36
@sharpener6 sharpener6 merged commit a770072 into Citi:main Oct 29, 2024
5 checks passed
sharpener6 pushed a commit to sharpener6/scaler that referenced this pull request Dec 16, 2024
…ynchronization issue on Linux (Citi#40)

* Fixes a bug in the async priority queue when trying to remove a suspended task.

Signed-off-by: rafa-be <[email protected]>

* Fixes a worker agent crash when trying to profile a zombie process.

Signed-off-by: rafa-be <[email protected]>

* Fixes Citi#33: processors can be suspended during the initialization phase.

Signed-off-by: rafa-be <[email protected]>

* The worker's heart-beat manager watches all worker processes, not only the active one.

Signed-off-by: rafa-be <[email protected]>

* Task priorities are now positive numbers.

Signed-off-by: rafa-be <[email protected]>

---------

Signed-off-by: rafa-be <[email protected]>
sharpener6 pushed a commit to sharpener6/scaler that referenced this pull request Dec 20, 2024
…ynchronization issue on Linux (Citi#40)

* Fixes a bug in the async priority queue when trying to remove a suspended task.

Signed-off-by: rafa-be <[email protected]>

* Fixes a worker agent crash when trying to profile a zombie process.

Signed-off-by: rafa-be <[email protected]>

* Fixes Citi#33: processors can be suspended during the initialization phase.

Signed-off-by: rafa-be <[email protected]>

* The worker's heart-beat manager watches all worker processes, not only the active one.

Signed-off-by: rafa-be <[email protected]>

* Task priorities are now positive numbers.

Signed-off-by: rafa-be <[email protected]>

---------

Signed-off-by: rafa-be <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants