Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#4076 - Queue Monitoring - Schedulers Refactor - Warnings Metrics Counter #4186

Merged
merged 4 commits into from
Dec 27, 2024

Conversation

andrewsignori-aot
Copy link
Collaborator

@andrewsignori-aot andrewsignori-aot commented Dec 27, 2024

  • Collected a metric for jobs finalizing with some warnings to allow the creation of a Sysdig alert based on the counter.
  • Injected the service as a property to avoid passing the service to every single class inheriting from the BaseQueue.

New metric sample from metrics payload

queue_event_total_count {
  queueName="student-application-notifications",
  queueEvent="job-finalized-with-warnings",
  queueType="scheduler",
  app="queue-consumers"
} 2

image

Note: this change was a quick way to resolve the concerned raised and explained to the business in this comment.

@andrewsignori-aot andrewsignori-aot self-assigned this Dec 27, 2024
@andrewsignori-aot andrewsignori-aot changed the title #4076 schedulers refactor part 8 #4076 - Queue Monitoring - Schedulers Refactor - Warnings Metrics Counter Dec 27, 2024
@andrewsignori-aot andrewsignori-aot marked this pull request as ready for review December 27, 2024 21:18
@@ -62,6 +62,10 @@ export enum QueuesMetricsEvents {
* are not able to extend locks.
*/
LockExtensionFailed = "lock-extension-failed",
/**
* A job was finished with success but contains at leas one warning.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at least?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Copy link

Backend Unit Tests Coverage Report

Totals Coverage
Statements: 22.2% ( 3783 / 17038 )
Methods: 10.19% ( 219 / 2150 )
Lines: 25.65% ( 3278 / 12781 )
Branches: 13.57% ( 286 / 2107 )

Copy link

E2E Workflow Workers Coverage Report

Totals Coverage
Statements: 65.59% ( 589 / 898 )
Methods: 59.63% ( 65 / 109 )
Lines: 68.72% ( 468 / 681 )
Branches: 51.85% ( 56 / 108 )

Copy link
Collaborator

@dheepak-aot dheepak-aot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks for adding the metrics for warning(s). 👍

Copy link

E2E Queue Consumers Coverage Report

Totals Coverage
Statements: 86.14% ( 1249 / 1450 )
Methods: 82.42% ( 136 / 165 )
Lines: 88.51% ( 1032 / 1166 )
Branches: 68.07% ( 81 / 119 )

Copy link
Collaborator

@lewischen-aot lewischen-aot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 Thanks for making the improvement.

Copy link

E2E SIMS API Coverage Report

Totals Coverage
Statements: 67.4% ( 5922 / 8787 )
Methods: 65.09% ( 729 / 1120 )
Lines: 71.33% ( 4647 / 6515 )
Branches: 47.4% ( 546 / 1152 )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants