2024_10_09

[ChrisJohnNOAA]

Attendees:
Marco Alba
Jessy Barrette
Seth Champagne
Clifford Harms
Chris John
callum rollo
Shane St Savage
Micah Wengren

1) Action Item Review

#192

All completed.

Git LFS is cost prohibitive at this time. The current approach is good enough for now, but we're open to better ideas.

Official ERDDAP™ docker: discussions have started. The current plan is to have an officially supported docker image and repo. There will continue to also be an Axiom fork which contains experimental features. At this time the most notable of those is datasets.xml concatenation.
Chris' hope is that the new XInclude support will work for admins to build datasets.xml and other solutions like concatenation won't be needed. This will be re-evaluated with data.

2) Go over the priority projects and discuss any changes we'd like to make.

#158

Jessy: Improved S3 support is important to them, can we add this to the priority list.

The issue discussed was: #95

Seth volunteered to take a look. It's possibly an easy fix.
Micah requested to try to support everything using the S3 api. General agreement that is the ideal approach.

Chris stated Docusaurus seems to be the best approach for documentation given project requirements.
In particular it is free and supports building from git, serving on git pages, documentation versioning, and search of documentation contents.

3) 2.25 Review

The plan is to make a build soon (hopefully next week).

Changes (admin focused):

• New standard XML parser (that supports XInclude) (thanks Ayush)
• Zarr support (with tests, but not production tested)
• Parameter for admins to set the unusualActivityFailPercent (thanks Ayush)
• Core JVM metrics exported with Prometheus (no ERDDAP specific metrics yet)
• ERDDAP™ starts a load on startup instead of waiting for the first request

Technical Debt Improvements

• Most runtime resources now loaded through resources instead of hardcoded paths (thanks Seth)
• Easier adding of new dataset types (thanks Shane)
• Tests run automatically on each pull request (thanks Shane)

Bug Fixes:

• Weird interaction where making a WMS query on a dataset could cause incorrect results in other queries
• Email usernames for smtp login no longer need to be a complete email address
• EDDTableFromFiles can support queries derived outputs (Specifically requests with no data in the source files (only globals, jexl script, or variables) will now function.)
• Hardcoded CSP added to the search multiple ERDDAPs™ page

Performance Improvements

• EDDTableFromMultidimNcFiles can respond much faster to requests now if removeMVRows is set to false (may not be do-able for all datasets)

The things that still need to land and are mostly done:

• Parquet support, new dataset type and export option
• New translation approach

4) 2.26 Planning

Targeting mid-late January 2025

Improved Prometheus logging (ERDDAP™ metrics like those on the status page)
Ensure good automated test coverage
Official Docker support
Nightly (or on merge to main) WAR file (and maybe a nightly docker image)
Localization support (currently pending CF approval of NC changes) - this was bumped from 2.25

Marco is working on a service to help make it easy to manage a datasets.xml.
It's a web interface and service with an api to access the configuration and build the datasets.xml.
The idea is to make it public as soon as its stable, maybe a couple of months.

Jessy asked for localization plan if CF falls through? Chris said it's definitely something ERDDAP™ wants to support so we'll figure out a plan.

Action Items:

• (Chris, Shane supporting) Get the official ERDDAP™ Docker set up
• (Seth) Take a look at improving S3 support
• (Chris) Get the next meeting scheduled
• (Chris) Add improved S3 support to the Priority Projects