Wykres commitów

21 Commity (4e4337a87d0054f3994344aa5e9532deb63746e6)

Autor SHA1 Wiadomość Data
bbilger 264b3515c4 add tile-copy utility
to copy tiles from one archive into another
e.g. mbtiles to files, pmtiles to mbtiles, ...
2024-01-23 19:19:23 +01:00
Björn Bilger c480b35f1c
Add support for "files"-archive (#761)
* Add support for "files"-archive

i.e. write individual pbf-files to disk in the format <base>/z/x/y.pbf

in order to use that format it must be passed as "--ouput=/path/to/tiles?format=files"

Fixes #536

* default to files format

...if no explict format query param given, path ends with a slash, or no extension given

* output metadata.json in files-archive

and refactor TileArchiveMetadata
1. put zoom into center (CoordinateXY->Coordinate) - in sync with mbtiles-format
2. add (De-)Serializer for Coordinate+Envelop => avoid duplication and cleaner
3. change the json and proto output for TileArchiveMetadata to be (more) in sync with mbtiles-format

* add support for custom tile scheme in files-archive

{z}/{x}/{y}.pbf is the default and can be configured as needed - e.g.:
- different order: {x}/{y}/{z}.pbf
- with intermediate dirs: {x}/a/{y}/b/{z}.pbf
- with different extension: {z}/{y}/{y}.pbf.gz

instead of {x} and {y}, {xs} and {xy} can be used which breaks up
x and y into 2 directories each and ensures that each directory has <1000 children

* fix issues with multiple writers

1. call finish archive only once after all writers are finished
   ...and not every time a writer finishes
2. log "zoom-progress" for the first tile write only
   (Finished z11 ... now starting z12)
3. remove file/dir-size progress logger bottleneck for files archive
   => each archive now reports the bytes written, which also fixes
   the issues of stream-archives reporting the size incorrectly
4. introduce printStats-hook on archive-level

* add async file write support to files archive

...allow to use virtual threads ExecturService (bound only!) for tile writing

also add some benchmark for writing tiles to disk: fixed, bound virtual, async, unbound virtual

* Revert "add async file write support to files archive"

This reverts commit b8cfa56977d98520aa8b62252c3a2726d440afe0.

* few improvements

- extract TileSchemeEncoding
- use Counter.MultithreadCounter rather than LongAdder to count bytes written
- add some JavaDoc

* simplify files archive usage

1. allow to pass tile scheme directly via output: --output=tiles/{x}/{y}/{z}.pbf
2. auto-encode { (%7B) and } (%7D) => no need to encode it the URI on CLI

* few more adjustments according to PR feeback

1. use WriteableTileArchive#bytesWritten in summmary as well
2. call WriteableTileArchive#init in a safer manner

..and a few more adjustments

* more PR feedback
2024-01-02 20:37:49 -05:00
Michael Barry cbb092a1f7
Compute layer attr stats from actual vector tile features (#752) 2023-12-14 19:26:27 -05:00
Michael Barry 1b53493ac7
Upgrade omt to pull in tile size optimizations (#672) 2023-09-25 06:04:53 -04:00
Michael Barry 1f23b55274
Tile stats (#656) 2023-09-21 21:44:09 -04:00
Björn Bilger db796e1720
Add support for simple file output streams: CSV, JSON, protobuf (#639) 2023-08-23 20:24:27 -04:00
Michael Barry 74db638dbc
Expose pmtiles writer (#520) 2023-03-18 14:38:04 -04:00
Michael Barry c5d6daf671
metadataTable alias (#517) 2023-03-15 13:15:57 -04:00
Michael Barry 9a704e773e
Convert get all tile coords to iterator (#463) 2023-02-05 14:16:05 -05:00
Brandon Liu 9ce9d1f5b2
FeatureGroup can emit in either TMS or Hilbert order. [#98] (#455) 2023-01-26 21:43:07 -05:00
Brandon Liu f772180fb7
refactor VectorLayers from mbtiles into util.LayerStats [#98] (#456) 2023-01-25 21:15:43 -05:00
Brandon Liu 09fd4ba2ba
Tile archive refactor (#443) 2023-01-17 07:05:45 -05:00
Michael Barry c6ad30cc9a
OSM QA Tiles Example Profile (#278) 2022-07-14 05:26:53 -04:00
Michael Barry b0f634bcaf
Use fnv1a64 for tile hash (#253) 2022-06-03 20:44:49 -04:00
Björn Bilger 97642fc096
Improve DB Indexes and Fix Deferred Index Logging in Compact DB Mode (#245) 2022-06-01 21:29:59 -04:00
Björn Bilger f5206b3a73
add compact DB mode (--compact-db) to de-duplicate mbtiles output (#219) 2022-05-24 17:46:56 -04:00
Michael Barry 5341d4d712
Fix sonar warnings (#181) 2022-04-23 05:58:49 -04:00
Michael Barry cce51668f2
Migrate to eclipse formatter to support multiple IDEs (#122) 2022-03-08 21:08:03 -05:00
Mike Barry e7e5cd88c4 fix warnings 2022-02-23 20:45:56 -05:00
Michael Barry 8d90470d55
Fix unit tests in other locales (#64) 2022-01-27 20:23:24 -05:00
Michael Barry 9e9cf7c413
Change name to Planetiler (#40)
* change name from flatmap to planetiler
* bump version to 0.2-SNAPSHOT
2021-12-23 05:42:24 -05:00