Error writing spatial metadata to MBTiles file

[spwoodcock]

First of all, thanks for this lib! It's ridiculously fast at downloading tiles - I'm impressed 😄

Failure writing metadata

• I'm not sure if there is something I am missing here, but I can't get the spatial metadata to write to the final .mbtiles file.

Command:

go run cmd/build/main.go -url-template 'https://services.arcgisonline.com/arcgis/rest/services/World_Imagery/MapServer/tile/{z}/{y}
/{x}' -dsn esri.mbtiles -zooms "0-10" -bounds "-10.0,-20.0,10.0,20.0"

(am I missing something in the command?)

Output:

2024/11/20 23:08:21 Expecting to fetch 9017 tiles
2024/11/20 23:08:21 Created mbtiles output
  77% |█████████████████████████████████████████████████████████████████████████████████████████████████████████████████                                  | (316 tile/s) [22s:6s]2024/11/20 23:08:45 Job queue closed
 100% |███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| (274 tile/s)2024/11/20 23:08:55 Finished making tile requests
2024/11/20 23:08:55 Processed 9017 tiles                                                                                                                                         
2024/11/20 23:08:55 Finished processing tiles
2024/11/20 23:08:55 Wrote tiles but failed to assign spatial metadata, Failed to add bounds metadata key, sql: database is closed

In can see the (presumed) gzipped images in the SQLite database table:

The error is here:

go-tilepacks/tilepack/mbtiles_outputter.go

Line 115 in fd9e5ed

return fmt.Errorf("Failed to add %s metadata key, %w", name, err)

Failing at the SQL command: INSERT OR REPLACE INTO metadata (name, value) VALUES(?, ?) because the database is closed already?

MBTiles database format

• On a side note, perhaps I also have a knowledge gap around the mbtiles spec, but I thought they were required to have database tables metadata and tiles with columns as such:

  

• However, the final mbtiles file has tables metadata, images, and map, with images having structure:

  

• And map having structure:

  

This is probably intended, so please forgive my ignorance if so 🙏
But either way, I can't verify due to the metadata issue above!

[iandees]

On the metadata:

There was a bug with how the metadata is written to the mbtiles file that is getting fixed in #33. See that ticket for more information about what the bug was.

On the database schema:

We use a database schema that stores the tile data in a separate table keyed by the hash of the data, allowing us to deduplicate data. The table required by the mbtiles spec is realized using a view that joins the data with its hash and the tile ID. On large (~more than a few thousand tiles) tilesets, the chance of duplication is pretty high, and it saves a lot of storage space to do this deduplication.

[iandees]

This should be resolved. Please reopen or file a new issue if you find something else wrong.

[spwoodcock]

Amazing work! Thanks so much for your help on this 🙏

The deduplication via a view is a very neat trick, thanks for explaining 😄

This got me wondering if I could use the same MBTiles file also as an "BigPlanet" db supported by OSMAnd, simply by adding a view for the required table structure