Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Grafana] Add a RayCluster variable to the Grafana Dashboard to enable filtering of different RayClusters #2685

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

win5923
Copy link
Contributor

@win5923 win5923 commented Dec 24, 2024

Why are these changes needed?

The current Grafana Dashboard aggregates metrics from all Ray clusters and summarizes them. Add a RayCluster variable for Ray core and Ray data dashboard to enable viewing metrics for individual RayClusters.

Ref: #2502 (comment)

Before:

  • Ray core dashboard:
    image
  • Ray data dashboard:
    image

After:

  • Ray core dashboard:
    image

image

  • Ray data dashboard:
    image

Related issue number

Closes #2502

Checks

  • I've made sure the tests are passing.
  • Testing Strategy
    • Unit tests
    • Manual tests
    • This PR is not tested :(

@win5923 win5923 force-pushed the grafana/ray_io_cluster branch from ad08687 to 0e7250a Compare December 24, 2024 14:08
@win5923
Copy link
Contributor Author

win5923 commented Dec 24, 2024

@kevin85421 PTAL

@win5923 win5923 force-pushed the grafana/ray_io_cluster branch from 0e7250a to a6b8183 Compare December 24, 2024 14:27
@win5923 win5923 changed the title [Metrics] Add a RayCluster variable to the Grafana Dashboard to nable filtering of different RayClusters [Metrics] Add a RayCluster variable to the Grafana Dashboard to enable filtering of different RayClusters Dec 24, 2024
@win5923 win5923 changed the title [Metrics] Add a RayCluster variable to the Grafana Dashboard to enable filtering of different RayClusters [Grafana] Add a RayCluster variable to the Grafana Dashboard to enable filtering of different RayClusters Dec 25, 2024
@kevin85421 kevin85421 self-assigned this Dec 25, 2024
@win5923 win5923 marked this pull request as draft December 26, 2024 02:51
@win5923
Copy link
Contributor Author

win5923 commented Dec 26, 2024

Ray places the JSON file in /tmp/ray/session_latest/metrics/grafana/dashboards/ inside the head pod. I will first modify the JSON file generated by Ray, and then update here.

image

@win5923 win5923 marked this pull request as ready for review December 26, 2024 05:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Multiple RayCluster CRs share the same Grafana
2 participants