Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow for missing obs file w/o a JEDI crash #400

Draft
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

rtodling
Copy link
Contributor

@rtodling rtodling commented Aug 1, 2024

Description

This allows for ops-class (type) to be in yaml even when input file is missing ... JEDI will warn of missing file, but it won't crash - in OPS these would need to be set more restrictively.

See documentation for specifics on options for this:
https://jointcenterforsatellitedataassimilation-jedi-docs.readthedocs-hosted.com/en/latest/inside/jedi-components/ioda/file-formats.html#missing-file-action

Dependencies

None

Impact

None

Issue

This is part of what's needed to address issue #362 - in this case, wiring the obs-classes would be ok ... but r2d2 (or swell) must be less strict and allow for job to continue when obs file is not found - this is to be done and therefore this is still a DRAFT.

@rtodling rtodling changed the title the entry here allows for missing obs file w/o a crash allow for missing obs file w/o a JEDI crash Aug 1, 2024
@rtodling
Copy link
Contributor Author

The knob placed here will no fully work as they should. They only work when some filters are not present. This knob is supposed to create an zero-ob observer object corresponding to the missing instrument and create allow for things like satbias and satbias_cov (for radiance) to be propagated even in the absence of the instrument - which is what we want. Unfortunately, some of the filters don't work properly in the absence of observations (zero-ops) so the quest is now one of finding the offending filters and get them fixed; a non-trivial task.

@gmao-yzhu
Copy link
Contributor

gmao-yzhu commented Aug 15, 2024

The changes made are necessary. At the same time, in terms of the JEDI code workflow, when data are missing, the filters should all be skipped.

@gmao-jjin3
Copy link
Contributor

I made a few minor updates based on this PR. So far, my PR #415 works for missing convectional, ozone, and microwave radiance data. It crashes swell-3dfgat_atmos run when any of AIRS, IASI, CRIS-FSR, and AVHRR input files is missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants