Skip to content

Latest commit

 

History

History
162 lines (123 loc) · 11.8 KB

README.md

File metadata and controls

162 lines (123 loc) · 11.8 KB
Raw

tilelive.js

Build Status Coverage Status

⚠️ tilelive and related mapbox-owned tilelive plugins are not actively maintained. Please open a new issue to check with the project maintainers before beginning work on new features.

Tilelive is designed for streaming map tiles from sources (like custom geographic data formats) to sinks (destinations, like file systems) by providing a consistent API. This repository enables the interaction between sources and sinks and is meant to be used in tandem with at least one Tilelive plugin. Tilelive plugins (modules) follow a consistent architecture (defined in API.md) and implement the logic for generating and reading map tiles from a source or putting map tiles to a destination, or both.

An example of a plugin that implements both reading (can be a source) and writing (can be a sink) is tilelive-s3.

An example use case for tilelive is creating vector tiles from a geojson file and putting them to Amazon S3. This can be accomplished by using tilelive-omnivore as the source and using tilelive-s3 as the sink. Tilelive omnivore performs special operations for generating map tiles (using mapnik), whereas tilelive-s3 is able to properly connect to Amazon S3 for putting tiles in their proper location. The Tilelive module performs all of the getting and putting within tilelive.copy.

Basic tilelive steps:

  1. Require tilelive in your script, var tilelive = require('@mapbox/tilelive')
  2. Register custom protocols via plugins, CustomTileSourcePlugin.registerProtocols(tilelive) or CustomTileSinkPlugin.registerProtocols(tilelive)
  3. Load protocols using tilelive.load, this creates read and write streams
  4. Copy from source to destination (the creating of tiles is left to the plugin) using tilelive.copy(source, sink, callback)
  5. Once tiles are copied the streams are closed

See Usage for more details on the tilelive module API.

Awesome tilelive modules

Ecosystem of tilelive

image

Usage

Tilelive doesn't ship with any implementing modules by default. To register a module as one tilelive recognizes:

require('[implementation]').registerProtocols(tilelive);

  • tilelive.list(source, callback): Lists all tilesets in a directory. source is a folder that is used by registered implementations to search for individual tilesets. callback receives an error object (or null) and a hash with keys being Tilestore IDs and values being Tilestore URIs. Example:

    {
    "world-light": "mbtiles:///path/to/file/world-light.mbtiles",
    "mapquest": "tilejson:///path/to/file/mapquest.tilejson"
}

  • tilelive.findID(source, id, callback): Looks for a particular tileset ID in a directory. callback receives an error object (or null) and the URI of the tileset.

  • tilelive.load(uri, callback): Loads the Tilestore object associated with the specified uri. callback receives an error object (or null) and the Tilestore object.

  • tilelive.info(uri, callback): Loads the Tilestore object associated with the specified uri and retrieves its metadata in a TileJSON compliant format. callback receives an error object (or null), the metadata hash and the Tilestore object.

  • tilelive.all(source, callback): Loads metadata in a TileJSON compliant format for all tilesets in the source directory. callback receives an error object (or null) and an array with TileJSON metadata about each tileset in that directory.

  • tilelive.verify(tilejson): Validates a TileJSON object and returns error objects for invalid entries.

Read/write streams

Tilelive provides an implementation of node object streams for copying tiles from one source to another.

// Copy all tiles and metadata from source A to source B.
var get = tilelive.createReadStream(sourceA);
var put = tilelive.createWriteStream(sourceB);
get.pipe(put);
put.on('finish', function() {
    console.log('done!');
});

See the tilelive-copy CLI and the streams tests for example usage of copy streams.

Parallel read streams

Tilelive can split a read operation into an arbitrary number of jobs. Pass a job parameter to options when using tilelive.createReadStream or tilelive.deserialize:

var readable = tilelive.createReadStream(src, { type: 'scanline', job: { total: 4, num: 1 } });

This instructs tilelive to only read tiles that would fall into job 1 of 4. A complete read would mean four calls each with a different num.

bin/tilelive-copy

tilelive can be used to copy data between tilestores. The CLI tool uses tilelive.auto() to register plugins by filename. For example, file.mbtiles will result in using the mbtiles: protocol and the @mapbox/mbtiles module.

# usage
tilelive-copy <src> <dst>

# example
tilelive-copy orig.mbtiles copy.mbtiles

Options:

  • --scheme=[scanline,pyramid,list] - Default: scanline.
  • --list=[filepath] - Filepath if scheme is list.
  • --concurrency=[number] - Control on the number of pending I/O operations with the underlying source during copy. Note: this is not CPU concurrency, which is handled by individual plugins typically by setting UV_THREADPOOL_SIZE=[number] as an environment variable.
  • --withoutprogress - Shows progress by default.
  • --timeout=[number] - Timeout after n ms of inactivity.
  • --slow=[number] - Warn on slow tiles.
  • --exit - Exit explicitly when copy is complete.
  • --bounds=[w,s,e,n] - as defined by the TileJSON specification
  • --minzoom=[number] - as defined by the TileJSON specification
  • --maxzoom=[number] - as defined by the TileJSON specification
  • --parts=[number] - total number of parts to copy (part splitting is used for processing in parallel, where specific parts only copy specific tiles from the tile pyramid)
  • --part=[number] - the specific part to copy
  • --retry=[number] - number of retry attempts

Tests

To run the tests

npm test