VectorTiles generated via the [openmaptiles-tools](https://github.com/openmaptiles/openmaptiles-tools) toolchain and using the OpenMaptiles-SQL-Layers as the query-source result in tiles which can contain duplicated keys.
This is because PostGIS `ST_AsMVT` is used to generate the Tiles and due to the way they [implemented parallelization](https://trac.osgeo.org/postgis/ticket/4310).
Although it is not prohibited by the Mapbox-Vector-Tile-Specification MapBox as well as MapLibre depending on version either throw a lot of warnings or even crash when encountering tiles with duplicated keys.
https://github.com/mapbox/vector-tile/issues/55https://github.com/maplibre/maplibre-native/issues/795
After investigating I identified the boundary layer in zoom-leves 1 - 4 as the only layers containing duplicated keys.
This PR addresses two main issues introduced by the new ID-Based Merged-LineString Updates
1. Partial Indexes can only be accessed when the query matches more or less exactly and the query-planner will fail to use indexes when targeted via the join-condition and not the where-condition
- `osm_transportation_merge_linestring_gen_z9`
- `osm_transportation_name_network`
- `osm_shipway_linestring`
- `osm_aerialway_linestring`
- `osm_waterway_linestring`
2. When intersecting updated Source-LineStrings with the existing Merged-LineStrings we join the Source-IDs of each existing Merged-LineString. This bloats the table unnecessarily and slows down bigger updates considerably.
- This is addressed by aggregating the Source-IDs of each existing Merged-LineString into an array and concatinating these arrays when grouping them. Afterwards we add the IDs of updated SourceLineStrings and deduplicate the result before adding it to the Source-IDs-Table.
LEAKPROOF requires the executing user to have superuser privileges when creating the functions which would be the only statement required to be executed as a superuser.
Fixes a typo analyzing the `name_changes` table during execution of `transportation_name.refresh_shipway_linestring` and `transportation_name.refresh_aerialway_linestring`
Paint shop is missing from the schema, so shops of this type do not appear on the map beside other POIs
---------
Co-authored-by: Tomas Pohanka <TomPohys@gmail.com>
* Chore(quickstart.sh):
Handle spaces in fonts name when calculating their md5 digest
* Chore(Makefile):
Change docker pull to docker-compose pull to read image from compose file
Some parts of quickstart.sh reference `${area}.mbtiles`, but the file that gets created with the default `.env` is `tiles.mbtiles`. We should either reference `$MBTILES_FILE` instead, or override the filename with `${area}.mbtiles` (which is the approach taken by this PR).
Based on analysis from planet builds for this layer as well as running updates via the integrity script as discussed in #1516, these three indexes are not used and can be removed.
Multi-Column Primary-Keys (Unique-Constraints) must not contain NULL values in postgres up until version 15.
This PR coalesces NULL to empty strings for the primary columns during `transportation_route_member_coalesced` updates and import.
Additionally adds the missing `concurrency_index` to UPSERT statement of `transportation_route_member_coalesced`
This PR adds support for `boundary=aboriginal_lands` by adding it to the `park` layer. While these lands are certainly not "parks", they have similar treatment from a technology perspective and can benefit from the existing processing chain established in that layer. I set all of these objects to `class=aboriginal_lands` in the tiles, including at the lowest zoom in order to separate it from the protected area merging implemented in #1160. In order to distinguish these from general parks, I expose the `class` attribute for these objects at z4 and also ensure that the z4 generalization is performed separately for protected areas versus aboriginal lands.
This unblocks #ZeLonewolf/openstreetmap-americana#105, which describes why having indigenous land boundaries is an important general feature on the map. This is also consistent with my suggested implementation in https://github.com/openmaptiles/openmaptiles/issues/1296#issuecomment-967749403.
Improved update performance of water_name layer
- Implemented diff updates for update_water_lakeline.sql and update_water_point.sql
- Unified update_water_lakeline.sql and update_water_point.sql to update_water_name.sql
- Refactored IDs to be unique in water_name_marine.osm_ids
- Restricted updates to INSERT and UPDATE operations during water_name_marine.refresh
- Added analyze statements before update queries during water_name_marine.refresh
Improved update performance of transportation_name layer
- Refactored LineString-merging and diff updates in update_transportation_name.sql
- Refactored transportation_route_member_coalesced materialized view to table
- Added analyze statements before update queries during transportation_name.refresh_network and update_osm_route_member
Improved update performance of poi layer
- Refactored update_poi_point.sql to partial diff update
- Refactored IDs to be unique in poi_polygon.osm_ids
- Restricted updates to INSERT and UPDATE operations during poi_polygon.refresh
- Added analyze statements before update queries during poi_polygon.refresh
Improved update performance of place layer
- Refactored IDs to be unique in the following tables
- place_city.osm_ids
- place_continent_point.osm_ids
- place_country.osm_ids
- place_island_point.osm_ids
- place_island_polygon.osm_ids
- place_state.osm_ids
- Added analyze statements before update queries and restricted updates to INSERT and UPDATE operations during execution of the following functions
- place_city.refresh
- place_continent_point.refresh
- place_country.refresh
- place_island_point.refresh
- place_island_polygon.refresh
- place_state.refresh
Improved update performance of mountain_peak layer
- Refactored IDs to be unique in mountain_linestring.osm_ids and mountain_peak_point.osm_ids
- Restricted updates to INSERT and UPDATE operations during mountain_linestring.refresh and mountain_peak_point.refresh
- Added analyze statements before update queries during mountain_linestring.refresh and mountain_peak_point.refresh
Improved update performance of housenumber layer
- Refactored IDs to be unique in housenumber.osm_ids
- Restricted updates to INSERT and UPDATE operations during housenumber.refresh
- Added analyze statements before update queries during housenumber.refresh
Improved update performance of aerodrome_label layer
- Refactored IDs to be unique in aerodrome_label.osm_ids
- Restricted updates to INSERT and UPDATE operations during aerodrome_label.refresh
- Added analyze statements before update queries during aerodrome_label.refresh
See #1433 for more detailed discussion.
In some areas of the world, ship transportation is the main means of long-distance travel. For example, in Alaska, the [Alaska Marine Highway System](https://dot.alaska.gov/amhs/route.shtml) is the principal means of transportation through the Alaska Panhandle and down the Aleutian island chain. The AMHS carries the same importance that the Interstate Highway System has on land, and these ferry routes are exceptionally long distance.
This PR now removes bridge/tunnel/ford attributes from the `transportation` layer from sections of bridge/tunnel etc that are too small to be meaningfully visible at a particular zoom level. This allows those segments to be generalized into geometries along with the surrounding sections of road, thereby dropping lots of useless little pieces of geometry in lower-zoom tiles.
This PR changes the three-column unique index to an on-insert trigger. This should fix the issues we're having with CI failures and still achieve the behavior of coalescing duplicate routes.
I moved the concurrency_index calculation into an intermediate materialized view to separate the de-duplication capability from both DENSE_RANK() and from imposm updates.
This PR sets `surface=paved` for `highway=track` + `tracktype=grade1` if no other value of `surface` is set. Roads tagged in this way are [generally paved](https://wiki.openstreetmap.org/wiki/Key:tracktype) and therefore encoding them as a paved road is appropriate.
# Problem description
#1128 introduced route relation concurrency information in OpenMapTiles via the `route_X` attributes. The original implementation assumed that there would be a single route relation for each `network` and `ref` pair. However, it is increasingly common practice to tag a separate route relation for each direction of a route in order to provide awareness to routers and other data consumers of the directionality of a route. This standard and growing practice is described on the [OSM wiki page on route directions](https://wiki.openstreetmap.org/wiki/Route_directions). Thus, the naïve implementation of #1128 caused duplicate entries to be added as `route_X` attributes in the case where separate route relations were used for directional routes.
# Solution description
This PR adds grouping when computing route concurrency information, such that duplicate entries are coalesced in a predictable way. Since this grouping is done only within a route membership join of a single member way, the computational complexity should be trivial.
This PR adapts the code used in #1457 to allow POIs to render up to z10 for very large features. I set the threshold at a very conservative value of 10% of a tile. Thus, it will render if an area POI covers at least 10% of a tile at a given zoom. Only for 'university' and 'college'.
Additionally, this consolidates a double UNION ALL on the same table with a WHERE/CASE combination, which is simpler.
This PR adds eight missing languages, all of which are among the top 25 languages spoken in the United States, as discussed in ZeLonewolf/openstreetmap-americana#586, and have at least 10,000 `name:xx` usages in the database. Additionally, most of these languages are the primary or national language in the countries where they are most commonly spoken.
This PR introduces simple filtering of duplicate housenumbers.
Simple means that filtering is done withing the tile.
Duplicates are defined as same housenumber, street, block_number[1].
Duplicates are usually caused by POIs. People like to add addresses to them. Most POIs have names so to prioritize addresses we pick features without names first.
Formula is: `row_number() OVER(PARTITION BY concat(street, block_number, housenumber) ORDER BY has_name ASC) == 1`
This PR reduces the minimum lake line zoom to z3, allowing the largest lakes to show a renderable label. The lakeline function is restricted by the LineLabel() function, which limits lakelines to only those that are big enough to render a text label. Therefore, further restricting lakelines by zoom is not necessary.
This PR follows #1465 to implement pseudo-route relations for Republic of Ireland road routes as described in the [OSM wiki](https://wiki.openstreetmap.org/wiki/Ireland/Roads). Irish road routes work in the same way as UK road routes in that signage is derived from highway classification on a 1:1 basis. The scheme described in the OSM wiki is confirmed via [overpass query](https://overpass-turbo.eu/s/1por).
This feature follows the implementation currently used for UK road routes, in which a 10m Natural Earth polygon is used to determine which roads are located within Ireland to apply this processing.
This PR implements 3 classes of routes:
* M-roads, which are `highway=motorway` and signed with blue/white lettering.
* N-roads ("national roads"), which are `highway=trunk` or `highway=primary` and are signed with green/yellow lettering.
* R- and L-roads ("regional and local roads"), which are `highway=secondary`, `highway=tertiary` or `highway=unclassified`, and signed with white/black lettering
Currently, river areas (`natural=water` + `water=river`) are rendered in the tiles as `class=river`. However, canal, stream, ditch, and drain water areas are rendered as `class=lake`. Since these types of objects are all flowing water, they should be grouped together into `class=river`, which allows for styles that render flowing water differently to do so on a consistent basis.
This PR adds an additional pseudo-network for UK non-primary [routes](https://en.wikipedia.org/wiki/Great_Britain_road_numbering_scheme), which are signed with black on white signage with "A" and "B" prefixes. This scheme is described on the [OSM wiki](https://wiki.openstreetmap.org/wiki/Roads_in_the_United_Kingdom). Adding this third psuedo network is needed for generating shields for UK white/black signage for non-primary routes.
Additionally, this fixes a minor bug under which single-digit refs would have been included. Single-digit refs don't exist in the UK, because the numbering scheme is always prefixed by a letter, e.g. "A4".
I also included documentation updates and fixed the typo where a view was named "bg" instead of "gb".
* At zooms 9-13, water labels will now be rendered for water features that are at least 0.25 tiles in size. This means that water features that are the same size as a 128x128 pixel square will get a point label.
* Labels for large water features are now shown at the lowest zoom at which it's appropriate to show a label.
* Bay and strait names (`natural=bay` and `natural=strait`) are included.
* Large seas modeled as areas (e.g. Caspian Sea) render as `class=sea` rather than `class=lake`
This PR addresses that by:
* rendering oneway arrows on more than `subclass=cycleway`
* less opacity for path/footway casing
* adding the thin black casing to bridges
* adding `subclass=steps` & `subclass=bridleway` rendering
* render `bicycle=designated` as `subclass=cycleway` and `horse=designated` as `subclass=bridleway` (bike taking priority)
* improved tunnel rendering with double casing & adding main line
* made casing min zoom consistent across subclasses
All of these changes were made to mimic OSM Carto.
This PR proposes to add a new class, "pond" to the water layer, which allows for a style to de-emphasize minor bodies of water (tagged `water=pond,basin,wastewater`) when compared to more significant objects tagged reservoir/lake. This PR retains the default behavior of classifying a water body as a "lake" if more specific tagging is not added.
This PR defines the set of routes in Canada which are equivalent in national importance to the US Interstate Highway System, and are therefore appropriate to render at zoom 4. This creates a sane, connected highway network at this zoom level across the USA and Canada.
This adds two additional network types for Canadian highways, ca-provincial for provincial-level roads, and ca-provincial-arterial for "highest importance" roads that are not part of the Trans-Canada highway but should be regarded as equivalent for low-zoom rendering purposes.
Additionally, this extracts out the country-specific network checks to a separate function in order to define "equivalent top-level networks" by country, providing a place to add additional national definitions as they're added by contributors.
Brian Sperlongano