forked from NOAA-EMC/global-workflow
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
JCB-based obs+bias staging, Jedi class updates, and marine B-matrix r…
…efactoring (NOAA-EMC#2992) This PR is a companion to GDASApp PR NOAA-EMC/GDASApp#1312 (merged) and NOAA-EMC/jcb-gdas#31 (merged). This PR does three things: 1. It changes the observation and bias staging for the atmospheric analysis tasks to use JCB templates instead of reading the full JEDI input configuration dictionary in order to construct a list of files to stage. This is cleaner and places fewer constraints on how to initialize the analysis. 2. The ```Jedi``` constructor now takes as input a dictionary that is essentially subset of the ```task_config``` dictionary. This makes the code clearer and less opaque and makes debugging easier. Each dictionary is constructed from a YAML file with configuration parameters for each JEDI application that is run. 3. All JEDI applications and their input YAMLs are now initialized in the initialize job of the ```AtmAnalysis``` and ```AtmEnsAnalysis```. Before, in the ```atmensanl*``` jobs for example, the LETKF solver was initialized in the ```atmensanlinit```cjob, but the LETKF solver and FV3 increment converter were both initialized and executed in the ```atmensanlobs``` and ```atmensanlfv3inc``` jobs respectively. This makes more sense in terms of resource allocation. Addendum: I'm now rolling in the refactoring of the marine B-matrix task into this PR. That makes it also a companion of NOAA-EMC/GDASApp#1346 and NOAA-EMC/jcb-gdas#36. These new changes introduce the ```Jedi``` class and JCB into the marine B-matrix job. Partially resolvesNOAA-EMC/GDASApp#1296 --------- Co-authored-by: RussTreadon-NOAA <[email protected]> Co-authored-by: Rahul Mahajan <[email protected]>
- Loading branch information
1 parent
3209bea
commit 7ff942e
Showing
31 changed files
with
547 additions
and
585 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
atmanlvar: | ||
rundir: '{{ DATA }}' | ||
exe_src: '{{ EXECgfs }}/gdas.x' | ||
mpi_cmd: '{{ APRUN_ATMANLVAR }}' | ||
jedi_args: ['fv3jedi', 'variational'] | ||
jcb_base_yaml: '{{ PARMgfs }}/gdas/atm/jcb-base.yaml.j2' | ||
jcb_algo_yaml: '{{ JCB_ALGO_YAML_VAR }}' | ||
atmanlfv3inc: | ||
rundir: '{{ DATA }}' | ||
exe_src: '{{ EXECgfs }}/fv3jedi_fv3inc.x' | ||
mpi_cmd: '{{ APRUN_ATMANLFV3INC }}' | ||
jcb_base_yaml: '{{ PARMgfs }}/gdas/atm/jcb-base.yaml.j2' | ||
jcb_algo: fv3jedi_fv3inc_variational |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
atmensanlobs: | ||
rundir: '{{ DATA }}' | ||
exe_src: '{{ EXECgfs }}/gdas.x' | ||
mpi_cmd: '{{ APRUN_ATMENSANLOBS }}' | ||
jedi_args: ['fv3jedi', 'localensembleda'] | ||
jcb_base_yaml: '{{ PARMgfs }}/gdas/atm/jcb-base.yaml.j2' | ||
jcb_algo_yaml: '{{ JCB_ALGO_YAML_OBS }}' | ||
atmensanlsol: | ||
rundir: '{{ DATA }}' | ||
exe_src: '{{ EXECgfs }}/gdas.x' | ||
mpi_cmd: '{{ APRUN_ATMENSANLSOL }}' | ||
jedi_args: ['fv3jedi', 'localensembleda'] | ||
jcb_base_yaml: '{{ PARMgfs }}/gdas/atm/jcb-base.yaml.j2' | ||
jcb_algo_yaml: '{{ JCB_ALGO_YAML_SOL }}' | ||
atmensanlfv3inc: | ||
rundir: '{{ DATA }}' | ||
exe_src: '{{ EXECgfs }}/fv3jedi_fv3inc.x' | ||
mpi_cmd: '{{ APRUN_ATMENSANLFV3INC }}' | ||
jcb_base_yaml: '{{ PARMgfs }}/gdas/atm/jcb-base.yaml.j2' | ||
jcb_algo: fv3jedi_fv3inc_lgetkf | ||
atmensanlletkf: | ||
rundir: '{{ DATA }}' | ||
exe_src: '{{ EXECgfs }}/gdas.x' | ||
mpi_cmd: '{{ APRUN_ATMENSANLLETKF }}' | ||
jedi_args: ['fv3jedi', 'localensembleda'] | ||
jcb_base_yaml: '{{ PARMgfs }}/gdas/atm/jcb-base.yaml.j2' | ||
jcb_algo_yaml: '{{ JCB_ALGO_YAML_LETKF }}' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
gridgen: | ||
rundir: '{{ DATA }}' | ||
exe_src: '{{ EXECgfs }}/gdas_soca_gridgen.x' | ||
mpi_cmd: '{{ APRUN_MARINEBMAT }}' | ||
jcb_base_yaml: '{{ PARMgfs }}/gdas/soca/marine-jcb-base.yaml' | ||
jcb_algo: soca_gridgen | ||
soca_diagb: | ||
rundir: '{{ DATA }}' | ||
exe_src: '{{ EXECgfs }}/gdas_soca_diagb.x' | ||
mpi_cmd: '{{ APRUN_MARINEBMAT }}' | ||
jcb_base_yaml: '{{ PARMgfs }}/gdas/soca/marine-jcb-base.yaml' | ||
jcb_algo: soca_diagb | ||
soca_parameters_diffusion_vt: | ||
rundir: '{{ DATA }}' | ||
exe_src: '{{ EXECgfs }}/gdas_soca_error_covariance_toolbox.x' | ||
mpi_cmd: '{{ APRUN_MARINEBMAT }}' | ||
jcb_base_yaml: '{{ PARMgfs }}/gdas/soca/marine-jcb-base.yaml' | ||
jcb_algo: soca_parameters_diffusion_vt | ||
soca_setcorscales: | ||
rundir: '{{ DATA }}' | ||
exe_src: '{{ EXECgfs }}/gdas_soca_setcorscales.x' | ||
mpi_cmd: '{{ APRUN_MARINEBMAT }}' | ||
jcb_base_yaml: '{{ PARMgfs }}/gdas/soca/marine-jcb-base.yaml' | ||
jcb_algo: soca_setcorscales | ||
soca_parameters_diffusion_hz: | ||
rundir: '{{ DATA }}' | ||
exe_src: '{{ EXECgfs }}/gdas_soca_error_covariance_toolbox.x' | ||
mpi_cmd: '{{ APRUN_MARINEBMAT }}' | ||
jcb_base_yaml: '{{ PARMgfs }}/gdas/soca/marine-jcb-base.yaml' | ||
jcb_algo: soca_parameters_diffusion_hz | ||
soca_ensb: | ||
rundir: '{{ DATA }}' | ||
exe_src: '{{ EXECgfs }}/gdas_ens_handler.x' | ||
mpi_cmd: '{{ APRUN_MARINEBMAT }}' | ||
jcb_base_yaml: '{{ PARMgfs }}/gdas/soca/marine-jcb-base.yaml' | ||
jcb_algo: soca_ensb | ||
soca_ensweights: | ||
rundir: '{{ DATA }}' | ||
exe_src: '{{ EXECgfs }}/gdas_socahybridweights.x' | ||
mpi_cmd: '{{ APRUN_MARINEBMAT }}' | ||
jcb_base_yaml: '{{ PARMgfs }}/gdas/soca/marine-jcb-base.yaml' | ||
jcb_algo: soca_ensweights |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.