-
-
Notifications
You must be signed in to change notification settings - Fork 136
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update NERSC Cori to NERSC Perlmutter in docs #609
Comments
Sure! Could you do that @arosen93? |
I'm still sorting it out myself, but once I get it locked in, I will absolutely do so! |
@guillaumeeb: I've confirmed the following specs work for Perlmutter. I'm not too familiar with the configuration file YAML format, but it should hopefully be somewhat easy to go from the class kwargs to the configuration parameters! from dask_jobqueue import SLURMCluster
n_workers = 1 # Number of Slurm jobs to launch in parallel
n_nodes_per_calc = 1 # Number of nodes to reserve for each Slurm job
n_cores_per_node = 48 # Number of CPU cores per node
mem_per_node = "512 GB" # Total memory per node
cluster_kwargs = {
# Dask worker options
"cores": n_cores_per_node, # total number of cores (per Slurm job) for Dask worker
"memory": mem_per_node, # total memory (per Slurm job) for Dask worker
# SLURM options
"shebang": "#!/bin/bash",
"account": "myaccount",
"walltime": "00:10:00", # DD:HH:SS
"job_mem": "0", # all memory on node
"job_script_prologue": ["source ~/.bashrc"], # commands to run before calculation, including exports
"job_directives_skip": ["-n", "--cpus-per-task"], # Slurm directives we can skip
"job_extra_directives": [f"-N {n_nodes_per_calc}", "-q debug", "-C cpu"], # num. of nodes for calc (-N), queue (-q), and constraints (-c)
}
cluster = SLURMCluster(**cluster_kwargs)
cluster.scale(n_workers) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
There is a sample configuration for Cori in the docs here. Cori is now retired. It'd probably be worth someone contributing an example for Perlmutter.
The text was updated successfully, but these errors were encountered: