From d5bf83c58ed8929a3ec122703fea7c9562175fea Mon Sep 17 00:00:00 2001 From: Ian Dees Date: Wed, 19 Apr 2017 11:22:52 -0400 Subject: [PATCH 1/8] Add changelog line for tile pruner command --- CHANGELOG.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 813e4513..e9d9779a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,11 @@ CHANGELOG ========= +v1.7.0 +------ +* **New features:** + * Add a new command that 'prunes' the tiles of interest set based on various rules. See [#176](https://github.com/tilezen/tilequeue/pull/176), [#178](https://github.com/tilezen/tilequeue/pull/178), [#179](https://github.com/tilezen/tilequeue/pull/179), and [#180](https://github.com/tilezen/tilequeue/pull/180). + v1.6.0 ------ * **New features:** From 752ae48a948788d4e256c4a1e3a7b54d2af168ee Mon Sep 17 00:00:00 2001 From: Ian Dees Date: Wed, 19 Apr 2017 11:28:37 -0400 Subject: [PATCH 2/8] Add the other pruner PRs. --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e9d9779a..7b5b1efc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,7 @@ CHANGELOG v1.7.0 ------ * **New features:** - * Add a new command that 'prunes' the tiles of interest set based on various rules. See [#176](https://github.com/tilezen/tilequeue/pull/176), [#178](https://github.com/tilezen/tilequeue/pull/178), [#179](https://github.com/tilezen/tilequeue/pull/179), and [#180](https://github.com/tilezen/tilequeue/pull/180). + * Add a new command that 'prunes' the tiles of interest set based on various rules. See [#176](https://github.com/tilezen/tilequeue/pull/176), [#178](https://github.com/tilezen/tilequeue/pull/178), [#179](https://github.com/tilezen/tilequeue/pull/179), [#180](https://github.com/tilezen/tilequeue/pull/180), [#182](https://github.com/tilezen/tilequeue/pull/182), [#183](https://github.com/tilezen/tilequeue/pull/183), [#184](https://github.com/tilezen/tilequeue/pull/184), and [#189](https://github.com/tilezen/tilequeue/pull/189). v1.6.0 ------ From fd9c533747c3178e7e4eb9086079dbccffa89f92 Mon Sep 17 00:00:00 2001 From: Ian Dees Date: Wed, 19 Apr 2017 11:31:04 -0400 Subject: [PATCH 3/8] Add Rob's change --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7b5b1efc..7cd18b34 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,8 @@ v1.7.0 ------ * **New features:** * Add a new command that 'prunes' the tiles of interest set based on various rules. See [#176](https://github.com/tilezen/tilequeue/pull/176), [#178](https://github.com/tilezen/tilequeue/pull/178), [#179](https://github.com/tilezen/tilequeue/pull/179), [#180](https://github.com/tilezen/tilequeue/pull/180), [#182](https://github.com/tilezen/tilequeue/pull/182), [#183](https://github.com/tilezen/tilequeue/pull/183), [#184](https://github.com/tilezen/tilequeue/pull/184), and [#189](https://github.com/tilezen/tilequeue/pull/189). +* **Enhancements:** + * When enqueueing tiles of interest for seed rendering, enqueue at zoom level 15 instead of 16. See [#181](https://github.com/tilezen/tilequeue/pull/181). v1.6.0 ------ From 310ef3e290e624819cb0782b2dc19d10c1d75d21 Mon Sep 17 00:00:00 2001 From: Ian Dees Date: Wed, 19 Apr 2017 15:15:02 -0400 Subject: [PATCH 4/8] Version bump -> v1.7.0 --- VERSION | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VERSION b/VERSION index ce6a70b9..bd8bf882 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.6.0 \ No newline at end of file +1.7.0 From 646f27b566eab719fb355ced551c3e516c97eb58 Mon Sep 17 00:00:00 2001 From: Ian Dees Date: Wed, 19 Apr 2017 15:16:13 -0400 Subject: [PATCH 5/8] Update CHANGELOG.md --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7cd18b34..f2643518 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,7 @@ CHANGELOG v1.7.0 ------ * **New features:** - * Add a new command that 'prunes' the tiles of interest set based on various rules. See [#176](https://github.com/tilezen/tilequeue/pull/176), [#178](https://github.com/tilezen/tilequeue/pull/178), [#179](https://github.com/tilezen/tilequeue/pull/179), [#180](https://github.com/tilezen/tilequeue/pull/180), [#182](https://github.com/tilezen/tilequeue/pull/182), [#183](https://github.com/tilezen/tilequeue/pull/183), [#184](https://github.com/tilezen/tilequeue/pull/184), and [#189](https://github.com/tilezen/tilequeue/pull/189). + * Add new command that 'gardens' the tiles of interest set to add and remove tiles based on various rules. See [#176](https://github.com/tilezen/tilequeue/pull/176), [#178](https://github.com/tilezen/tilequeue/pull/178), [#179](https://github.com/tilezen/tilequeue/pull/179), [#180](https://github.com/tilezen/tilequeue/pull/180), [#182](https://github.com/tilezen/tilequeue/pull/182), [#183](https://github.com/tilezen/tilequeue/pull/183), [#184](https://github.com/tilezen/tilequeue/pull/184), and [#189](https://github.com/tilezen/tilequeue/pull/189). * **Enhancements:** * When enqueueing tiles of interest for seed rendering, enqueue at zoom level 15 instead of 16. See [#181](https://github.com/tilezen/tilequeue/pull/181). From ba6cbc34e0e4431c9d9072be4c1186153a8e3819 Mon Sep 17 00:00:00 2001 From: Ian Dees Date: Wed, 19 Apr 2017 15:50:06 -0400 Subject: [PATCH 6/8] Add a mention of statsd. --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index f2643518..2eea3f9b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,7 @@ v1.7.0 * Add new command that 'gardens' the tiles of interest set to add and remove tiles based on various rules. See [#176](https://github.com/tilezen/tilequeue/pull/176), [#178](https://github.com/tilezen/tilequeue/pull/178), [#179](https://github.com/tilezen/tilequeue/pull/179), [#180](https://github.com/tilezen/tilequeue/pull/180), [#182](https://github.com/tilezen/tilequeue/pull/182), [#183](https://github.com/tilezen/tilequeue/pull/183), [#184](https://github.com/tilezen/tilequeue/pull/184), and [#189](https://github.com/tilezen/tilequeue/pull/189). * **Enhancements:** * When enqueueing tiles of interest for seed rendering, enqueue at zoom level 15 instead of 16. See [#181](https://github.com/tilezen/tilequeue/pull/181). + * Add optional support for `statsd` in command.py. See [#185](https://github.com/tilezen/tilequeue/pull/185). v1.6.0 ------ From 3a51b853e7f7f86610736fee3a75319c0125da03 Mon Sep 17 00:00:00 2001 From: Ian Dees Date: Wed, 19 Apr 2017 16:24:31 -0400 Subject: [PATCH 7/8] Add a few samples for the pruner/gardener config --- config.yaml.sample | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/config.yaml.sample b/config.yaml.sample index dd6ff921..8007e6ba 100644 --- a/config.yaml.sample +++ b/config.yaml.sample @@ -173,3 +173,46 @@ metatile: # addition to the metatile. this can be useful when gracefully "cutting over" # to a metatile-based system from an individual format based one. store_metatile_and_originals: false + +# Configuration for the tiles of interest prune/garden command. +toi-prune: + # Connection and query configuration for a RedShift database containing + # request information for tiles. + redshift: + database-uri: postgresql://user:password@localhost:5439/database + # The number of days of history to query for. + days: 30 + # The maximum zoom level to query redshift. + max-zoom: 16 + cutoff: + # The minimum number of requests per tile for a tile to be included in + # the tiles of interest + min-requests: 16 + # The maximum number of tiles to include from the RedShift-based calculation + max-tiles: 100000 + s3: + # Connection information for deleting tiles from S3 when the tile is no longer + # in the tiles of interest. + bucket: mapzen-tiles-dev-us-east + date-prefix: 20170322 + path: osm + layer: all + format: zip + always-include: + # Sets of tiles to always include in the tiles of interest + world: + bbox: -180.0,-85.06,180.0,85.06 + min_zoom: 0 + max_zoom: 10 + # conus: + # bbox: -124.8,24.8,-66.1,49.3 + # min_zoom: 11 + # max_zoom: 14 + # sf_metro: + # bbox: -122.524338,37.571794,-122.345123,37.821175 + # min_zoom: 0 + # max_zoom: 15 + # ny_metro: + # bbox: -74.09111,40.652513,-73.831558,40.911956 + # min_zoom: 0 + # max_zoom: 15 From 67691da26d53d4efcf159978721e187bd28da0cd Mon Sep 17 00:00:00 2001 From: Ian Dees Date: Wed, 19 Apr 2017 16:28:34 -0400 Subject: [PATCH 8/8] Add examples of other immortal tile section options --- config.yaml.sample | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/config.yaml.sample b/config.yaml.sample index 8007e6ba..08da8e11 100644 --- a/config.yaml.sample +++ b/config.yaml.sample @@ -199,20 +199,18 @@ toi-prune: layer: all format: zip always-include: - # Sets of tiles to always include in the tiles of interest + # Sets of tiles to always include in the tiles of interest. + # For more information about options here, see the code: + # https://github.com/tilezen/tilequeue/blob/1ca908e/tilequeue/command.py#L1046-L1061 world: bbox: -180.0,-85.06,180.0,85.06 min_zoom: 0 max_zoom: 10 - # conus: - # bbox: -124.8,24.8,-66.1,49.3 - # min_zoom: 11 - # max_zoom: 14 # sf_metro: # bbox: -122.524338,37.571794,-122.345123,37.821175 # min_zoom: 0 # max_zoom: 15 - # ny_metro: - # bbox: -74.09111,40.652513,-73.831558,40.911956 - # min_zoom: 0 - # max_zoom: 15 + # tests: + # tiles: 0/0/0,1/1/0 + # tests-file: + # file: list-of-tile-coordinates.txt