You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Using ensembleperturbation we setup many SCHISM run directories. Since this can be compute intensive for large ensembles we run the ensemble generation on compute nodes (no internet). In order to avoid the issue of downloading the data from compute node, I first setup a single run to cache the data and then setup the rest on the compute node.
At first I thought this solution should work, but then I realized PySCHISM still needs to have internet because the caching kicks in for data, but still metadata is fetched from the internet every time. See: Metadata
As a short term solution I can just copy paste the downloaded data from one setup directory to another, but since I'm using automation, around pyschism I was hoping to use the built-in caching capability in this case.
Using
ensembleperturbation
we setup many SCHISM run directories. Since this can be compute intensive for large ensembles we run the ensemble generation on compute nodes (no internet). In order to avoid the issue of downloading the data from compute node, I first setup a single run to cache the data and then setup the rest on the compute node.At first I thought this solution should work, but then I realized PySCHISM still needs to have internet because the caching kicks in for data, but still metadata is fetched from the internet every time. See:
Metadata
pyschism/pyschism/forcing/source_sink/nwm.py
Lines 490 to 498 in fc41b51
and
Data
pyschism/pyschism/forcing/source_sink/nwm.py
Lines 524 to 525 in fc41b51
The text was updated successfully, but these errors were encountered: