Switch exposure pipeline to use ModelLibrary instead of a list of models

[braingram]

This PR updates the exposure pipeline to use a ModelLibrary instead of a list of models. This allows:

• removing the custom asn parsing in the pipeline
• the tweakreg (and other step) cal_step status to be correctly marked by stpipe when steps are skipped

Regtests: https://github.com/spacetelescope/RegressionTests/actions/runs/11861738024
show 2 expected failures:

FAILED romancal/regtest/test_wfi_image_pipeline.py::test_processing_pipeline_all_saturated - AssertionError: 
  Diff report for:
      result file: /runner/_work/_temp/pytest_basetemp/popen-gw2/test_processing_pipeline_all_s0/r0000101001001001001_0001_wfi01_ALL_SATURATED_cal.asdf
          model type: RampModel
      truth file: /runner/_work/_temp/pytest_basetemp/popen-gw2/test_processing_pipeline_all_s0/truth/r0000101001001001001_0001_wfi01_ALL_SATURATED_cal.asdf
          model type: RampModel
  
  {'dictionary_item_added': ["root['roman']['meta']['asn']", "root['roman']['meta']['exptype']", "root['roman']['meta']['group_id']"]}
assert False
 +  where False = <romancal.regtest.regtestdata.DiffResult object at 0x7f3c25b39310>.identical
FAILED romancal/regtest/test_wfi_image_pipeline.py::test_pipeline_suffix - AssertionError: 
  Diff report for:
      result file: /runner/_work/_temp/pytest_basetemp/popen-gw2/test_pipeline_suffix0/r0000101001001001001_0001_wfi01_star.asdf
          model type: ImageModel
      truth file: /runner/_work/_temp/pytest_basetemp/popen-gw2/test_pipeline_suffix0/truth/r0000101001001001001_0001_wfi01_star.asdf
          model type: ImageModel
  
  {'dictionary_item_added': ["root['roman']['meta']['asn']", "root['roman']['meta']['exptype']", "root['roman']['meta']['group_id']"],
   'values_changed': {"root['roman']['meta']['cal_step']['tweakreg']": {'new_value': 'SKIPPED',
                                                                        'old_value': 'INCOMPLETE'}}}
assert False

The first is a byproduct of using ModelLibrary. The output files for elp now contain asn information.
The second shows that meta.cal_step.tweakreg is being correctly marked as skipped for test_pipeline_suffix.

Tasks

• request a review from someone specific, to avoid making the maintainers review every PR
• add a build milestone, i.e. 24Q4_B15 (use the latest build if not sure)
• Does this PR change user-facing code / API? (if not, label with no-changelog-entry-needed)
  • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
  • update or add relevant tests
  • update relevant docstrings and / or docs/ page
  • start a regression test and include a link to the running job (click here for instructions)
    • Do truth files need to be updated ("okified")?
      • after the reviewer has approved these changes, run okify_regtests to update the truth files
• if a JIRA ticket exists, make sure it is resolved properly

news fragment change types...

• changes/<PR#>.general.rst: infrastructure or miscellaneous change
• changes/<PR#>.docs.rst
• changes/<PR#>.stpipe.rst
• changes/<PR#>.associations.rst
• changes/<PR#>.scripts.rst
• changes/<PR#>.mosaic_pipeline.rst
• changes/<PR#>.patch_match.rst

steps

• changes/<PR#>.dq_init.rst
• changes/<PR#>.saturation.rst
• changes/<PR#>.refpix.rst
• changes/<PR#>.linearity.rst
• changes/<PR#>.dark_current.rst
• changes/<PR#>.jump_detection.rst
• changes/<PR#>.ramp_fitting.rst
• changes/<PR#>.assign_wcs.rst
• changes/<PR#>.flatfield.rst
• changes/<PR#>.photom.rst
• changes/<PR#>.flux.rst
• changes/<PR#>.source_detection.rst
• changes/<PR#>.tweakreg.rst
• changes/<PR#>.skymatch.rst
• changes/<PR#>.outlier_detection.rst
• changes/<PR#>.resample.rst
• changes/<PR#>.source_catalog.rst

[codecov]

Codecov Report

Attention: Patch coverage is 2.94118% with 33 lines in your changes missing coverage. Please review.

Project coverage is 77.00%. Comparing base (281aa55) to head (b12c774).

Files with missing lines	Patch %	Lines
romancal/pipeline/exposure_pipeline.py	2.94%	33 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1525      +/-   ##
==========================================
+ Coverage   76.68%   77.00%   +0.31%     
==========================================
  Files         120      120              
  Lines        7832     7796      -36     
==========================================
- Hits         6006     6003       -3     
+ Misses       1826     1793      -33     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests

[braingram]

This assert isn't currently reached in the regtests due to the compare_against_truth failing:

FAILED romancal/regtest/test_wfi_image_pipeline.py::test_pipeline_suffix - AssertionError: 
  Diff report for:
      result file: /runner/_work/_temp/pytest_basetemp/popen-gw2/test_pipeline_suffix0/r0000101001001001001_0001_wfi01_star.asdf
          model type: ImageModel
      truth file: /runner/_work/_temp/pytest_basetemp/popen-gw2/test_pipeline_suffix0/truth/r0000101001001001001_0001_wfi01_star.asdf
          model type: ImageModel
  
  {'dictionary_item_added': ["root['roman']['meta']['asn']", "root['roman']['meta']['exptype']", "root['roman']['meta']['group_id']"],
   'values_changed': {"root['roman']['meta']['cal_step']['tweakreg']": {'new_value': 'SKIPPED',
                                                                        'old_value': 'INCOMPLETE'}}}

But when 'okified' this change to the assert will be needed.

[schlafly]

Thanks. This looks good.

You also deleted some orphaned fully saturated exposure handling here. I think following your investigation we probably do want to keep and start using that code again, rather than deleting it?
#1523
Maybe just to say that deleting create_fully_saturated_zeroed_image needn't be part of this PR.

@ddavis-stsci , this refactors some of the pipeline handling to use ModelLibrary more uniformly, so you may want to weigh in here.

[braingram]

Converted to draft to attempt to also address #1523 and some other elp cleanup.