Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HPC: Multiple Backends #85

Merged
merged 15 commits into from
Oct 12, 2024
Merged

HPC: Multiple Backends #85

merged 15 commits into from
Oct 12, 2024

Conversation

Schlevidon
Copy link
Collaborator

Summary

  • Reimplemented the load balancer making it possible to select different scheduling strategies. At the moment the two options are pure SLURM and HyperQueue.
  • Made a simple CLI for specifying the scheduler, port and job submission delay.
  • Prepared the code base for implementation of another communication layer between the load balancer and a job script in the future (i.e. network-based communication instead of the shared filesystem).
  • Performed some general cleanup and added docs.

Related issues

closes #80

@linusseelinger
Copy link
Member

Excellent work, thanks a lot @Schlevidon ! :)

I don't see any obvious issues. Merging now so we have this in time for the modeling week; we can merge potential fixes later on.

@linusseelinger linusseelinger merged commit 11fb3c9 into main Oct 12, 2024
26 checks passed
@Schlevidon Schlevidon deleted the hpc-multiple-backends branch November 19, 2024 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

HPC: Add multiple backend options (e.g. HyperQueue and pure SLURM)
2 participants