Merge main into GH-191/auto-generate-environment.yml

README.md

@@ -16,70 +16,12 @@ flepimop-inference-main -c config_sample_2pop_inference.yml
 
 # flepiMoP
 
-Welcome to the Johns Hopkins University Infectious Disease Dynamics COVID-19 Working Group's `Flexible Epidemic Modeling Pipeline`(“FlepiMoP”, formerly the COVID Scenario Pipeline, “CSP”), a flexible modeling framework that projects epidemic trajectories and healthcare impacts under different suites of interventions in order to aid in scenario planning. The model is generic enough to be applied to different spatial scales given shapefiles, population data, and COVID-19 confirmed case data. There are multiple components to the pipeline, which may be characterized as follows: 1) epidemic seeding; 2) disease transmission and non-pharmaceutical intervention scenarios; 3) calculation of health outcomes (hospital and ICU admissions and bed use, ventilator use, and deaths); and 4) summarization of model outputs.
+Welcome to the Johns Hopkins University Infectious Disease Dynamics's `Flexible Epidemic Modeling Pipeline`. “FlepiMoP” provides a framework for quickly coding and simulating compartmental infectious disease models to project epidemic trajectories and their healthcare impacts, and to evaluate the impact of potential interventions. The package is a work-in-progress but is extensively documented https://iddynamics.gitbook.io/flepimop/, with instructions describing how to install the package, code up your model, run forward simulations, and infer model parameters from timeseries data. More details of the project are available on our dedicated website https://www.flepimop.org/. 
 
 We recommend that most new users use the code from the stable `main` branch. Please post questions to GitHub issues with the `question` tag. We are prioritizing direct support for individuals engaged in public health planning and emergency response.
 
-<!--
-For more information on getting started, please visit our [wiki](https://github.com/HopkinsIDD/COVID19_Minimal/wiki) at [HopkinsIDD/COVID19_Minimal](https://github.com/HopkinsIDD/COVID19_Minimal). We are trying to keep this page up-to-date for use with the `master` branch.
--->
-
-For more details on the methods and features of our model, visit our [preprint on medRxiv](https://www.medrxiv.org/content/10.1101/2020.06.11.20127894v1).
-
 This open-source project is licensed under GPL v3.0.
 
+Details on the methods and features of our model as of December 2023 were published in [Lemaitre JC, et al. flepiMoP: The evolution of a flexible infectious disease modeling pipeline during the COVID-19 pandemic. Epidemics. 2024;47:100753](https://www.sciencedirect.com/science/article/pii/S1755436524000148). Please cite this paper if you use flepiMoP in any of your own publications. 
 
-# Tools for using this repository
-## Docker
-
-A containerized environment is a packaged environment where all
-dependencies are bundled together. This means you're guaranteed to be
-using the same libraries and system configuration as everyone else and in
-any runtime environment. To learn more, [Docker
-Curriculum](https://docker-curriculum.com/) is a good starting point.
-
-### Starting environment
-
-A pre-built container can be pulled from Docker Hub via:
-```
-docker pull hopkinsidd/flepimop:latest-dev
-```
-
-To start the container:
-```
-docker run -it \
-  -v <dir1>\:/home/app/flepimop \
-  -v <dir2>:/home/app/drp \
-hopkinsidd/flepimop:latest
-```
-
-In this command we run the docker image hopkinsidd/flepimop. The -v command is used to allocate space from Docker and mount it at the given location. 
-This mounts the data folder <dir2> to a path called drp within the docker environment, and the flepimop folder <dir1> to flepimop. 
-
-You'll be dropped to the bash prompt where you can run the Python or
-R scripts (with dependencies already installed).
-
-### Building the container
-
-Run `docker build -f build/docker/Dockerfile .` if you ever need to rebuild the container after changing to the top directory of flepiMoP.
-
-Note that the container build supports amd64 CPU architecture only, other architectures are unconfirmed. If you are using M1 MAC etc., please use the build kit to build an image with specifying the platform/architecture.
-
 
-<!--
-# Tools for development
-## Profiling
-
-The Python SEIR simulation supports profiling as a command-line option with the
-`--profile` flag. To write output to a specific file, use the
-`--profile-output` command line option. If you're profiling, it's a good
-idea to run single-threaded (`-j 1`) to capture the statistics that would
-be lost within the child processes.
-
-Here's an example to run 10 simulations while profiling the simulation and
-outputting to `~/profile.output`.
-
-```
-$ ./simulate.py -n 10 --profile --profile-output $HOME/profile.output -j 1
-```
--->

batch/inference_job_launcher.py

@@ -806,7 +806,7 @@ def launch(self, job_name, config_filepath, seir_modifiers_scenarios, outcome_mo
                 slurm_job_id = stdout.decode().split(" ")[-1][:-1]
                 print(f">>> SUCCESS SCHEDULING JOB. Slurm job id is {slurm_job_id}")
 
-                postprod_command = f"""sbatch {export_str} --dependency=afterany:{slurm_job_id} --mem={12000}M --time={60} --job-name=post-{cur_job_name} --output=log_postprod_{self.run_id}_{cur_job_name}.txt {os.path.dirname(os.path.realpath(__file__))}/SLURM_postprocess_runner.run"""
+                postprod_command = f"""sbatch {export_str} --dependency=afterany:{slurm_job_id} --mem={12000}M --time={240} --job-name=post-{cur_job_name} --output=log_postprod_{self.run_id}_{cur_job_name}.txt {os.path.dirname(os.path.realpath(__file__))}/SLURM_postprocess_runner.run"""
                 print("post-processing command to be run >>>>>>>> ")
                 print(postprod_command)
                 print(" <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< ")

build/local_install.R

@@ -6,13 +6,14 @@ local({r <- getOption("repos")
 
 # Installs the custom-made packages in this repository
 
-library(devtools)
 
-install.packages(c("covidcast","data.table","vroom","dplyr"), quiet=TRUE)
+install.packages(c("data.table","vroom","dplyr"), quiet=TRUE) # removed "covidcast"
 # devtools::install_github("hrbrmstr/cdcfluview")
 
 # To run if operating in the container -----
 
+# library(devtools)
+
 initial.options <- commandArgs(trailingOnly = FALSE)
 file.arg.name <- "--file="
 script.name <- sub(file.arg.name, "", initial.options[grep(file.arg.name, initial.options)]) # get the name of this file, by looking for the option "--file" in the arguments that were used to start this R instance and getting the term that comes after

examples/test_cli.py

@@ -7,7 +7,7 @@
 # would be useful to also call the command directly
 
 def test_config_sample_2pop():
-  os.chdir(os.path.dirname(__file__) + "/tutorial_two_subpops")
+  os.chdir(os.path.dirname(__file__) + "/tutorials")
   runner = CliRunner()
   result = runner.invoke(simulate, ['-c', 'config_sample_2pop.yml'])
   print(result.output) # useful for debug
@@ -17,21 +17,10 @@ def test_config_sample_2pop():
   assert 'completed in' in result.output
 
 
-def test_sample_2pop_interventions_test():
-  os.chdir(os.path.dirname(__file__) + "/tutorial_two_subpops")
+def test_sample_2pop_modifiers():
+  os.chdir(os.path.dirname(__file__) + "/tutorials")
   runner = CliRunner()
-  result = runner.invoke(simulate, ['-c', 'config_sample_2pop_interventions_test.yml'])
-  print(result.output) # useful for debug
-  print(result.exit_code) # useful for debug
-  print(result.exception) # useful for debug
-  assert result.exit_code == 0
-  assert 'completed in' in result.output
-
-
-def test_simple_usa_statelevel():
-  os.chdir(os.path.dirname(__file__) + "/simple_usa_statelevel")
-  runner = CliRunner()
-  result = runner.invoke(simulate, ['-c', 'simple_usa_statelevel.yml', '-n', '1'])
+  result = runner.invoke(simulate, ['-c', 'config_sample_2pop_modifiers.yml'])
   print(result.output) # useful for debug
   print(result.exit_code) # useful for debug
   print(result.exception) # useful for debug