ClimaCoupler interface

[juliasloan25]

Purpose

We want to refactor the ClimaCoupler to make a more modular interface.

Benefits

This will remove global-scope objects, making performance tracking faster and increasing usability of the interface. Code will be more generalized and therefore able to be used by different modules more easily.

Risks

We will develop an interface that will conflict with the current main stream of the model. To minimize this risk, we are developing this interface in parallel with development of the rest of the model. Eventually we will change over the rest of the code to align with this new interface design.

People and Personnel

• Lead: @juliasloan25
• Collaborators: @LenkaNovak
• Reviewers: @charleskawczynski @dennisYatunin @valeriabarra

Components

1. Modularize utilities used by the coupler driver
   a. Check unit test coverage
2. Move functions from the coupler driver to an external module
   a. Check unit test coverage
3. Optimize and generalize modules used by driver
   a. Re-evaluate scope of global variables and make them local if possible
   b. Add coupler-specific abstractions which are consistent with other CliMA model abstractions
   c. Optimize, generalize existing coupler functions

Module Conversion Task Breakdown

For each module:
PHASE 1 - moving and documentation

1. Copy file(s) to module in src/
2. Add module to src/ClimaCoupler.jl
3. Docstring for each function
   a. Add input/output types in docstring if not type annotated
   b. Add docstring info to docs/src, docs/make.jl
4. Add exports for externally-used functions
5. Flag potentially-removable parts of code

PHASE 2 - testing

1. Create test file in test/, add to test/runtests.jl
2. Unit test for each exported function in module
   a. Test simplest functions first
3. Integration tests as needed
4. Add any tests that use MPI in test/mpi_tests and in buildkite pipeline
   a. Add file to run_mpi_tests.jl

PHASE 3 - cleanup

1. Remove flagged parts of code
   a. Test after each one
2. Check all imports are necessary
3. Formatting
4. Read through comments

Results and deliverables

• One module for each utility used by the driver
  • Regridding (masker.jl, regridder.jl)
  • Conservation checker (conservation_checker.jl)
  • Input file reader (bcfile_reader.jl)
  • Callback manager + calendar (calendar_timer.jl)
  • General utilities (general_helper.jl, flux_calculator.jl)
  • Diagnostics/visualization/output file generator (viz_explorer.jl, output_dumper.jl)
  • Information exchange (push_pull.jl, flux_calculator.jl)
• Tests in ClimaCoupler/test/ for each new module
  • Move tests from coupler_utils/unit_tester.jl
• Comparison between new and previous implementations in buildkite
  • Update/improve ClimaCoupler CI

Quality expectations

• Unit tests for each module
• Documentation for each module

Performance metrics

Throughout development, we will track performance changes as we update modules. During the optimization stage (after translation to modules) flame graphs will be used to assess performance of the entire driver.

Task Breakdown And Tentative Due Dates

• Utility to module: Regridding (29 Nov) - J - Regridder interface #144
• Utility to module: General utilities (29 Nov) - J Regridder interface #144
• Utility to module: Conservation checker (6 Dec) - L Conservation checker interface #167
• Utility to module: Input file reader (6 Dec) - J Boundary Condition Reader interface #175 update modular driver to use BCReader module #204
• Utility to module: Callback/time manager (16 Dec) - J TimeManager interface module #209
• Utility to module: Diagnostics/visualization/output file generator (10 Jan) - L
  • online diagnostics Online Diagnostics Module #206
  • offline postprocessing Interface Postprocessor v2 #225
• Move tests from coupler_utils/unit_tester.jl to ClimaCoupler/test/ (22 Dec)
• Update driver to use new modules (22 Dec)
  • Move functions from driver to module
  • Use new modularized functions
  • Remove parameter overwriting

Next steps (partly addressed in issue #205 )

• Optimize, generalize existing coupler functions (6 Jan)
  • Discuss design
  • re-design exchange (punk! and pull) and surface_fluxes calls
  • Re-evaluate scope of global variables
  • Generalize time stepping (move from driver)

Proposed Delivery Date

10 January 2023

[tapios]

Looking good! Do you think you could add rough estimates of timelines, @juliasloan25?

[simonbyrne]

To follow up on our call this morning:

• work incrementally, moving items that are in examples or elsewhere to src, then importing those into the driver files.
• this list seems like a good breakdown, and order of priority.
• add docs and tests for each item as you go.
• don't worry about moving things to other repos for now:
  • it's easier to move things to dependencies at a later date, especially once they are documented and tested

[LenkaNovak]

Completed. 🎉

Next steps are to restructure experiments folder (#232), make the modular interface more performant (#205 and subsequent issues) and redesign the exchange (pull and push!) component (#211).