mirror
/
planetiler
kopia lustrzana https://github.com/onthegomap/planetiler
1
0
Forkuj 0
Kod Zgłoszenia Projekty Wydania Wiki Aktywność
875 Commity
86 Gałęzie
10 Tagi
57 MiB
Wykres commitów

46 Commity (main)

Autor SHA1 Wiadomość Data
Michael Barry ef1702ec73
ignore duplicate relation members (#876) 2024-05-02 06:21:01 -04:00
Michael Barry fa7bffb04f
Add whole-tile postprocess hook (#802) 2024-01-23 07:08:18 -05:00
Björn Bilger c480b35f1c
Add support for "files"-archive (#761) 
* Add support for "files"-archive

i.e. write individual pbf-files to disk in the format <base>/z/x/y.pbf

in order to use that format it must be passed as "--ouput=/path/to/tiles?format=files"

Fixes #536

* default to files format

...if no explict format query param given, path ends with a slash, or no extension given

* output metadata.json in files-archive

and refactor TileArchiveMetadata
1. put zoom into center (CoordinateXY->Coordinate) - in sync with mbtiles-format
2. add (De-)Serializer for Coordinate+Envelop => avoid duplication and cleaner
3. change the json and proto output for TileArchiveMetadata to be (more) in sync with mbtiles-format

* add support for custom tile scheme in files-archive

{z}/{x}/{y}.pbf is the default and can be configured as needed - e.g.:
- different order: {x}/{y}/{z}.pbf
- with intermediate dirs: {x}/a/{y}/b/{z}.pbf
- with different extension: {z}/{y}/{y}.pbf.gz

instead of {x} and {y}, {xs} and {xy} can be used which breaks up
x and y into 2 directories each and ensures that each directory has <1000 children

* fix issues with multiple writers

1. call finish archive only once after all writers are finished
   ...and not every time a writer finishes
2. log "zoom-progress" for the first tile write only
   (Finished z11 ... now starting z12)
3. remove file/dir-size progress logger bottleneck for files archive
   => each archive now reports the bytes written, which also fixes
   the issues of stream-archives reporting the size incorrectly
4. introduce printStats-hook on archive-level

* add async file write support to files archive

...allow to use virtual threads ExecturService (bound only!) for tile writing

also add some benchmark for writing tiles to disk: fixed, bound virtual, async, unbound virtual

* Revert "add async file write support to files archive"

This reverts commit b8cfa56977d98520aa8b62252c3a2726d440afe0.

* few improvements

- extract TileSchemeEncoding
- use Counter.MultithreadCounter rather than LongAdder to count bytes written
- add some JavaDoc

* simplify files archive usage

1. allow to pass tile scheme directly via output: --output=tiles/{x}/{y}/{z}.pbf
2. auto-encode { (%7B) and } (%7D) => no need to encode it the URI on CLI

* few more adjustments according to PR feeback

1. use WriteableTileArchive#bytesWritten in summmary as well
2. call WriteableTileArchive#init in a safer manner

..and a few more adjustments

* more PR feedback
 2024-01-02 20:37:49 -05:00
Michael Barry 1df1bf04e4
Add `setAttrWithMinSize` to feature API (#725) 2023-11-20 06:15:52 -05:00
Michael Barry ec6430dc49
Min polygon area centroid (#720) 2023-11-13 07:16:55 -05:00
Michael Barry 1be2fca45f
Use some new Java 21 features (#695) 2023-10-27 20:29:26 -04:00
Michael Barry e7e18d5f17
Sort merged vector tile features by hilbert order (#673) 2023-09-26 20:56:04 -04:00
Michael Barry 2f86ea12ae
Utilities to reduce tile size (#669) 2023-09-24 08:10:47 -04:00
Michael Barry 1f23b55274
Tile stats (#656) 2023-09-21 21:44:09 -04:00
Björn Bilger db796e1720
Add support for simple file output streams: CSV, JSON, protobuf (#639) 2023-08-23 20:24:27 -04:00
Michael Barry 8d0e06c667
Fix invalid polygons before snapping to tile coordinates (#566) 2023-04-27 08:19:31 -04:00
Michael Barry 509795401e
Fix geometry errors (#526) 2023-03-20 16:41:18 -04:00
Michael Barry 74db638dbc
Expose pmtiles writer (#520) 2023-03-18 14:38:04 -04:00
Michael Barry f4d07ea141
Detect certain kinds of invalid polygons when slicing into tiles (#499) 2023-02-24 13:14:50 -05:00
Michael Barry 9a704e773e
Convert get all tile coords to iterator (#463) 2023-02-05 14:16:05 -05:00
Brandon Liu 9ce9d1f5b2
FeatureGroup can emit in either TMS or Hilbert order. [#98] (#455) 2023-01-26 21:43:07 -05:00
Erik Price a0f8c67c78
Support unzipping GeoPackage sources at runtime (#430) 2023-01-25 20:56:30 -05:00
Brandon Liu 09fd4ba2ba
Tile archive refactor (#443) 2023-01-17 07:05:45 -05:00
Erik Price ef24e91f0b
Add basic support for reading GeoPackage files. (#413) 2023-01-02 12:19:05 -05:00
Michael Barry aea309e094
add layer to shapefile (#429) 2023-01-02 11:41:12 -05:00
Michael Barry dad2073f61
Add planetiler-specific metadata (#428) 2023-01-02 11:26:00 -05:00
Erik Price 184f950051
Extract ZIP file walker out of ShapefileReader. (#423) 2023-01-01 17:29:00 -05:00
Erik Price ea8589bd6f
Add shapefile directory source (#406) 2022-12-15 14:19:22 -05:00
Michael Barry 0eb148ee3c
Expose arguments via config (#363) 2022-10-04 19:57:59 -04:00
Michael Barry 621a8ed759
Support zoom levels up to 15 (#303) 2022-07-26 07:51:31 -04:00
farfromrefuge 7818634774
feat: `--polygon` argument to constrain mbtiles to a poly shape (#280) 2022-07-22 06:48:04 -04:00
Michael Barry d1d68cf753
Memory-mapped lazy reads by default (#300) 2022-07-20 08:06:08 -04:00
farfromrefuge 76c7880d6b
feat: added `maxzoomForRendering` and `minzoomForRendering` (#272) 2022-06-20 06:31:50 -04:00
Michael Barry bed2220e83
Cheap isFill check and add `--skip-filled-tiles` option (#234) 2022-06-03 21:04:17 -04:00
Michael Barry 4ba36655d4
Fall back to parsing bounds from all nodes (#251) 2022-06-03 05:25:17 -04:00
Michael Barry 99c0f8ae9c
Add `--feature-read-threads` option to read features in parallel when writing tiles (#225) 2022-05-24 18:46:52 -04:00
Björn Bilger f5206b3a73
add compact DB mode (--compact-db) to de-duplicate mbtiles output (#219) 2022-05-24 17:46:56 -04:00
Michael Barry 726e6d0107
Parallel temp feature writes using `--write-threads` argument (#213) 2022-05-05 22:02:18 -04:00
Michael Barry 29a50f59e4
fix sonar warnings in tests (#191) 2022-04-23 06:36:24 -04:00
Michael Barry e20b41c88d
Offload multipolygon storage to disk (reduce memory usage by 10-15gb) (#141) 2022-03-22 20:34:54 -04:00
Michael Barry bf081692ce
Node location cache: off-heap storage and “array” implementation that supports parallel inserts (#131) 
* Add --nodemap-type=array option for 2-3x faster osm pass 1 imports
* Add --nodemap-storage=direct option to experiment with direct (off-heap) memory usage
* Extract ResourceUsage and OsmPhaser utilities
 2022-03-19 05:46:03 -04:00
Michael Barry cce51668f2
Migrate to eclipse formatter to support multiple IDEs (#122) 2022-03-08 21:08:03 -05:00
Michael Barry f051178b46
Test for freeing input file after read (#107) 2022-03-03 08:52:45 -05:00
Michael Barry 20c7a05caa
Warn when running with insufficient memory or disk space (#73) 2022-03-03 07:25:24 -05:00
Michael Barry 2f05f942b2
Custom PBF parsing (#82) 
Pull in pbf parsing to give more control over threading model and performance.
 2022-02-28 20:52:30 -05:00
Michael Barry e092586dfc
Gracefully handle exceptions from profile (#65) 2022-02-04 06:26:26 -05:00
Michael Barry 6f2dbc4f4b
Add profile hooks for preprocessing OSM nodes and ways (#56) 2022-01-16 10:00:57 -05:00
Mike Barry b6ba928be1 Revert "allow using label grid on lines and polygons too (#51)" 
This reverts commit 1c622ffdfb.
 2022-01-10 06:41:15 -05:00
Michael Barry 1c622ffdfb
allow using label grid on lines and polygons too (#51) 2022-01-09 12:02:46 -05:00
Michael Barry bf99918198
Treat boundary and land_area relation types as multipolygons (#50) 2022-01-07 10:01:22 -05:00
Michael Barry 9e9cf7c413
Change name to Planetiler (#40) 
* change name from flatmap to planetiler
* bump version to 0.2-SNAPSHOT
 2021-12-23 05:42:24 -05:00