Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update NERSC Cori to NERSC Perlmutter in docs #609

Open
Andrew-S-Rosen opened this issue Jun 30, 2023 · 3 comments
Open

Update NERSC Cori to NERSC Perlmutter in docs #609

Andrew-S-Rosen opened this issue Jun 30, 2023 · 3 comments

Comments

@Andrew-S-Rosen
Copy link
Contributor

There is a sample configuration for Cori in the docs here. Cori is now retired. It'd probably be worth someone contributing an example for Perlmutter.

@guillaumeeb
Copy link
Member

Sure! Could you do that @arosen93?

@Andrew-S-Rosen
Copy link
Contributor Author

I'm still sorting it out myself, but once I get it locked in, I will absolutely do so!

@Andrew-S-Rosen
Copy link
Contributor Author

Andrew-S-Rosen commented Jul 29, 2023

@guillaumeeb: I've confirmed the following specs work for Perlmutter. I'm not too familiar with the configuration file YAML format, but it should hopefully be somewhat easy to go from the class kwargs to the configuration parameters!

from dask_jobqueue import SLURMCluster

n_workers = 1 # Number of Slurm jobs to launch in parallel
n_nodes_per_calc = 1 # Number of nodes to reserve for each Slurm job
n_cores_per_node = 48 # Number of CPU cores per node
mem_per_node = "512 GB" # Total memory per node
cluster_kwargs = {
    # Dask worker options
    "cores": n_cores_per_node, # total number of cores (per Slurm job) for Dask worker
    "memory": mem_per_node, # total memory (per Slurm job) for Dask worker
    # SLURM options
    "shebang": "#!/bin/bash",
    "account": "myaccount",
    "walltime": "00:10:00", # DD:HH:SS
    "job_mem": "0", # all memory on node
    "job_script_prologue": ["source ~/.bashrc"], # commands to run before calculation, including exports
    "job_directives_skip": ["-n", "--cpus-per-task"], # Slurm directives we can skip
    "job_extra_directives": [f"-N {n_nodes_per_calc}", "-q debug", "-C cpu"], # num. of nodes for calc (-N), queue (-q), and constraints (-c)
}
cluster = SLURMCluster(**cluster_kwargs)
cluster.scale(n_workers)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants