Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Openscapes] Put dask-gateway workers (and schedulers) on their own nodegroups #4519

Merged
merged 5 commits into from
Jul 30, 2024
Merged

[Openscapes] Put dask-gateway workers (and schedulers) on their own nodegroups #4519

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
997c894
Add a dask nodegroup per hub on openscales cluster
sgibson91 Jul 29, 2024
a2f72ca
Add nodeSelectors to put dask workers and schedulers on hub-specific …
sgibson91 Jul 29, 2024
71e3960
Tidy up jsonnet config
sgibson91 Jul 30, 2024
9c70217
Add missing backend key to dask-gateway config
sgibson91 Jul 30, 2024
5da9453
Merge branch 'main' into openscapes/dask-nodegroup-tagging
sgibson91 Jul 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions config/clusters/openscapes/prod.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,14 @@ basehub:
oauth_callback_url: "https://openscapes.2i2c.cloud/hub/oauth_callback"
CILogonOAuthenticator:
oauth_callback_url: "https://openscapes.2i2c.cloud/hub/oauth_callback"

dask-gateway:
gateway:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: prod
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: prod
12 changes: 12 additions & 0 deletions config/clusters/openscapes/staging.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,15 @@ basehub:
oauth_callback_url: "https://staging.openscapes.2i2c.cloud/hub/oauth_callback"
CILogonOAuthenticator:
oauth_callback_url: "https://staging.openscapes.2i2c.cloud/hub/oauth_callback"

dask-gateway:
gateway:
backend:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: staging
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: staging
11 changes: 11 additions & 0 deletions config/clusters/openscapes/workshop.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,3 +129,14 @@ basehub:
image: "{value}"
choices: {}
resource_allocation: *profile_options_resource_allocation

dask-gateway:
gateway:
scheduler:
extraPodConfig:
nodeSelector:
2i2c/hub-name: workshop
worker:
extraPodConfig:
nodeSelector:
2i2c/hub-name: workshop
24 changes: 22 additions & 2 deletions eksctl/openscapes.jsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,14 +84,31 @@ local daskNodes = [
// Node definitions for dask worker nodes. Config here is merged
// with our dask worker node definition, which uses spot instances.
// A `node.kubernetes.io/instance-type label is set to the name of the
// *first* item in instanceDistribution.instanceTypes, to match
// *first* item in instancesDistribution.instanceTypes, to match
// what we do with notebook nodes. Pods can request a particular
// kind of node with a nodeSelector
//
// A not yet fully established policy is being developed about using a single
// node pool, see https://github.com/2i2c-org/infrastructure/issues/2687.
//
{ instancesDistribution+: { instanceTypes: ["r5.4xlarge"] }},
{
namePrefix: "dask-staging",
labels+: { "2i2c/hub-name": "staging" },
tags+: { "2i2c:hub-name": "staging" },
instancesDistribution+: { instanceTypes: ["r5.4xlarge"] }
},
{
namePrefix: "dask-prod",
labels+: { "2i2c/hub-name": "prod" },
tags+: { "2i2c:hub-name": "prod" },
instancesDistribution+: { instanceTypes: ["r5.4xlarge"] }
},
{
namePrefix: "dask-workshop",
labels+: { "2i2c/hub-name": "workshop" },
tags+: { "2i2c:hub-name": "workshop" },
instancesDistribution+: { instanceTypes: ["r5.4xlarge"] }
},
];


Expand Down Expand Up @@ -181,6 +198,9 @@ local daskNodes = [
labels+: {
"k8s.dask.org/node-purpose": "worker"
},
tags+: {
"2i2c:node-purpose": "worker"
},
taints+: {
"k8s.dask.org_dedicated" : "worker:NoSchedule",
"k8s.dask.org/dedicated" : "worker:NoSchedule"
Expand Down