Bump tiledbsoma version on builder to 1.12.3

[ivirshup]

No description provided.

[ivirshup]

@pablo-gar @ebezzi is there any particular time we should do this, or can this happen whenever?

[pablo-gar]

The guidance here is that we should update the tiledbsoma pin of builder as soon possible after we perform an LTS data release, which just happened in July 1st.

This way there is good buffer period for testing the creation of data with 1.11 from now until the next LTS data release.

[ivirshup]

I couldn't remember why we said to bump to 1.11 when 1.12 is out. After a brief convo with @ebezzi we determined it's because:

• The last release of the client library was pinned to 1.11.4
• The storage representation of tiledbsoma isn't stable between minor versions. So we need to pin the builder always needs to be <= currently released client.

[ivirshup]

FWIW I would be up for making another minor release of census that basically just bumps the tiledbsoma version and implements the SPEC-0 dependency updates. In which case we could bump this task out a bit and go straight to 1.12 in the builder.

[johnkerl]

@ivirshup the storage version doesn't change until 1.13 / core 2.25

I had previously thought:

• storage-version update for current domain (what core calls the new shape support) would be in 2.24
• API for that in 2.25

But in fact the core storage version did not increment in 2.24 -- the storage-version update and the API support are both in core 2.25, which will be used in the upcoming tiledbsoma 1.13. (Although we won't fully release the new-shape feature at the TileDB-SOMA API level until 1.14.)

[ivirshup]

@johnkerl, just to be clear: you're saying that it should be safe for the builder to be on 1.12 even if the client is on 1.11?

If this is the case, how do we feel about going straight to 1.12 @pablo-gar @ebezzi?

[johnkerl]

@ivirshup correct :)

$ pwd
/Users/johnkerl/git/TileDB-Inc/TileDB

$ git diff -r 2.23.0 -r 2.24.0 format_spec/FORMAT_SPEC.md

$ git diff -r 2.24.0 -r 2.25.0 format_spec/FORMAT_SPEC.md
diff --git a/format_spec/FORMAT_SPEC.md b/format_spec/FORMAT_SPEC.md
index c4def7a9f..e7750a9b9 100644
--- a/format_spec/FORMAT_SPEC.md
+++ b/format_spec/FORMAT_SPEC.md
@@ -4,7 +4,7 @@ title: Format Specification

 **Notes:**

-* The current TileDB format version number is **21** (`uint32_t`).
+* The current TileDB format version number is **22** (`uint32_t`).
 * Data written by TileDB and referenced in this document is **little-endian**
   with the following exceptions:

@@ -36,6 +36,7 @@ title: Format Specification
 |19|2.16|[Vac files now use relative URIs](https://github.com/TileDB-Inc/TileDB/commit/ef3236a526b67c50138436a16f67ad274c2ca037)|
 |20|2.17|[Enumerations](https://github.com/TileDB-Inc/TileDB/commit/c0d7c6a50fdeffbcc7d8c9ba4a29230fe22baed6)|
 |21|2.19|[Tile metadata are now correctly calculated for nullable fixed size strings on dense arrays](https://github.com/TileDB-Inc/TileDB/commit/081bcc5f7ce4bee576f08b97de348236ac88d429)|
+|22|2.25|[Add array current domain](https://github.com/TileDB-Inc/TileDB/commit/9116d3c95a83d72545520acb9a7808fc63478963)|

[ivirshup]

Since the builder has a dependency on the client it might actually be a little difficult to get the builder running with a newer version of tiledbsoma than the client's pinned version. It might be easiest to make a quick release of the client if we go for 1.12 in the builder.

I'd be keen on this just because doing a full test of the builder takes a while and has manual steps and I'd prefer to only do this once.

[johnkerl]

All good! :)

[ivirshup]

Got the go ahead from @pablo-gar to just go straight to 1.12.3

[ivirshup]

Buuuut, because the latest cellxgene-census does a hard pin on 1.11 we will have to wait for that until we make a minor release.

I think we should still merge the 1.11.4 pr since using such an old version of tiledbsoma in the builder is giving CI errors:

https://github.com/chanzuckerberg/cellxgene-census/actions/runs/10291760694/job/28484756304