Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backport sensitivity analysis and parallel simulation workflow tools here #12

Open
12 tasks
achubaty opened this issue Apr 22, 2022 · 0 comments
Open
12 tasks
Assignees

Comments

@achubaty
Copy link
Contributor

achubaty commented Apr 22, 2022

(Eliot editing this list)
There is a broad framing for this class if issue, namely, analyses that emerge from multiple spades calls. Originally, this was framed within the experiment and experiment2 functions, but these were not reliable enough and they didn't try to cover all the classes of "meta" spades modeling.

Classes of problems:

  • Sensitivity Analyses (e.g., varying parameters and evaluating outputs as a result)
  • Replication (for stochastic models)
  • Validation (and importantly, inter-model comparisons of validation)
  • Consensus modeling (e.g., weighted averaging of multiple inputs, based on e.g., validation scores)
  • Ensemble modeling (e.g., range of modeling outcomes)
  • Multi-model comparisons (e.g., running many, visualizing each separately, but in organized structures)
  • Forecast horizon

Technical issues:

  • discuss/decide how best to incorporate specific sensitivity analyses from AB_NRV work;
  • consolidate all multi-simulation and parallel-related adjustments from WBI and NRV projects;
  • ensure compatibility for workstations with very high core counts;
  • implement support for simulation scheduling
  • use of simulation results databases

@eliotmcintire @CeresBarros I'd like to meet to discuss these changes/additions

@achubaty achubaty self-assigned this Apr 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant