Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update meta.filename on read & write to reflect actual on-disk filename #387

Open
schlafly opened this issue Sep 19, 2024 · 0 comments
Open

Comments

@schlafly
Copy link
Collaborator

schlafly commented Sep 19, 2024

In romancal we often use meta.filename as a default or a hint to determine what the output filename for a product should be.

When writing a file via roman_datamodels we make sure that the meta.filename of the output object matches where it was written out to:

with validate.nuke_validation(), _temporary_update_filename(self, Path(init).name):

However, when reading in a file, the meta.filename in the data model continues to match what is on disk.

That makes sense but can lead to confusion in the pipeline. The usual issue is that someone takes a file and copies it elsewhere with a new name. In the new file, meta.filename and the actual filename are different. Then when many romancal steps are run on the new file, meta.filename is used as the hint for the output filename, rather than the actual on-disk filename, leading to potentially confusingly-named products.

The proposal here is to update meta.filename when roman_datamodels reads a file, so that meta.filename always reflects the on-disk filename. Then romancal will do the right thing. This is also the approach that has been taken in the Webb pipeline. This presumably involves changes to rdm.datamodels.open(...).

This has the obviously downside that it is surprising if asdf.open(filename)['roman']['meta']['filename'] is different from rdm.open(filename).meta.filename. But it's also surprising if we have a different policy than Webb, and it would fix the immediate issue in romancal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant