Improved update performance of transportation_name layer
- Refactored LineString-merging and diff updates in update_transportation_name.sql
- Refactored transportation_route_member_coalesced materialized view to table
- Added analyze statements before update queries during transportation_name.refresh_network and update_osm_route_member
In some areas of the world, ship transportation is the main means of long-distance travel. For example, in Alaska, the [Alaska Marine Highway System](https://dot.alaska.gov/amhs/route.shtml) is the principal means of transportation through the Alaska Panhandle and down the Aleutian island chain. The AMHS carries the same importance that the Interstate Highway System has on land, and these ferry routes are exceptionally long distance.
This PR changes the three-column unique index to an on-insert trigger. This should fix the issues we're having with CI failures and still achieve the behavior of coalescing duplicate routes.
I moved the concurrency_index calculation into an intermediate materialized view to separate the de-duplication capability from both DENSE_RANK() and from imposm updates.
This PR follows #1465 to implement pseudo-route relations for Republic of Ireland road routes as described in the [OSM wiki](https://wiki.openstreetmap.org/wiki/Ireland/Roads). Irish road routes work in the same way as UK road routes in that signage is derived from highway classification on a 1:1 basis. The scheme described in the OSM wiki is confirmed via [overpass query](https://overpass-turbo.eu/s/1por).
This feature follows the implementation currently used for UK road routes, in which a 10m Natural Earth polygon is used to determine which roads are located within Ireland to apply this processing.
This PR implements 3 classes of routes:
* M-roads, which are `highway=motorway` and signed with blue/white lettering.
* N-roads ("national roads"), which are `highway=trunk` or `highway=primary` and are signed with green/yellow lettering.
* R- and L-roads ("regional and local roads"), which are `highway=secondary`, `highway=tertiary` or `highway=unclassified`, and signed with white/black lettering
This PR adds an additional pseudo-network for UK non-primary [routes](https://en.wikipedia.org/wiki/Great_Britain_road_numbering_scheme), which are signed with black on white signage with "A" and "B" prefixes. This scheme is described on the [OSM wiki](https://wiki.openstreetmap.org/wiki/Roads_in_the_United_Kingdom). Adding this third psuedo network is needed for generating shields for UK white/black signage for non-primary routes.
Additionally, this fixes a minor bug under which single-digit refs would have been included. Single-digit refs don't exist in the UK, because the numbering scheme is always prefixed by a letter, e.g. "A4".
I also included documentation updates and fixed the typo where a view was named "bg" instead of "gb".
Closes#271
This PR adds track and path rendering at lower zooms than currently provided, and also achieves near-parity with openstreetmap-carto on track and path rendering. A previously-abandoned attempt, with significant discussion, was #1169.
This PR updates the `transportation` layer creation scripts to simplify the SQL and remove unneeded sub-selects, checks and conditionals, fix indexes, and improve inline documentation.
Currently, there are two sequences of materialized view creations. There is one from zoom 11 through 9, and a second one for zoom 8 through 4. This PR removes that break in the sequence of transportation table materialized view creations, in favor of one in which high-zoom views are created first, and then each lower zoom is created from the zoom above.
Instead, the current generalized zoom 8 transportation table is built directly from `osm_transportation_linestring` rather than being built from the zoom 9 transportation table. This means that when building the zoom 8 table, it must scan the entire transportation network rather than just selecting from the pre-filtered zoom 9 table.
This PR removes an unneeded sub-select in the build of the zoom 8 table, which appears to be a leftover from an old version of the SQL that did some sort of merge. Once this PR is implemented all zooms from 11 through 4 will be linked via a progressive series of materialized views.
Lastly, this adds in missing materialized view refreshes for zooms 9-11, which appear to have been entirely missing, and as far as I can tell aren't getting updated in the current version of this code.
In addition, the following optimizations were added as part of this commit:
1. Updates the `osm_highway_linestring_highway_partial_idx` partial index to match the `SELECT..WHERE` clause actually present in `transportation/update_route_member.sql`, which is where it appears to be actually used, and update inline documentation to reflect this.
2. Collapses unnecessary sub-select block in `osm_transportation_merge_linestring_gen_z11`, and removes unnecessary ST_IsValid() call, which already provided in `mapping.yaml`, and update inline documentation to reflect these assumptions.
3. Updates `WHERE` blocks to remove unnecesary checks and further document assumptions. The `highway=construction` check is unnecessary in all cases, because it is sufficient to check the `construction` key alone. If `construction=*` is set, then `highway=construction` is implied.
4. Two indexes were added to `layers/transportation/update_route_member.sql` to improve route population performance.
In testing locally, I'm seeing performance improvements around 10% in the generation of the `transportation` layer, based on modifying `openmaptiles.yaml` to generate only the transportation layer and then repeatedly running `time make import-sql`, however, this timing might be impacted by docker, so I would ask for confirmation of acceptable performance.
In addition, this PR shortens the length of the transportation update SQL file by 30 lines, which makes it easier for contributors to work with.
PR #1168 removed several `WHERE` clauses in the `transportation_merge_*` table series. With those removed, it appears that `osm_transportation_merge_linestring_gen_z8` and `osm_transportation_merge_linestring` are nearly identical, with the former simply adding an `ST_Simplify()` operation.
A grep of the codebase indicates that the _only_ use for `osm_transportation_merge_linestring` is to hold the intermediate result of an `ST_Dump(geometry))` before it is fed into `ST_Simplify()`. Therefore, it appears that we're holding an entire zoom 8 copy of the transportation layer (all motorway/trunk/primary roads) in a materialized view for absolutely no reason at all.
This PR removes the `osm_transportation_merge_linestring` intermediate table and changes the definition of `osm_transportation_merge_linestring_gen_z8` to perform the `ST_Dump()` and `ST_Simplify()` transformations directly from the `osm_highway_linestring` table in a single operation.
This PR adds a layer for `highway=motorway_junction` features.
This implementation of highway exits in the transportation_name layer add to the existing layer table structure, and renames the internal column name from "construction" (which was already overloaded with non-construction usages) to "subclass", which will be less confusing to future developers. The string 'junction' is used as the universal sub-class for highway exits.
A new documentation PR has been opened at openmaptiles/www.openmaptiles.org#69 to reflect these changes in the documentation.
To avoid discontinuous transportation lines between zooms 9 and 11.
- Originally limit geometry by length for z9 - z11 (`ST_Length(geometry) > ZRes(11)`)
- highway z9 to z11 was generalized during import-osm
- now just create a filtered and generalized z11 table
- then merge segments in the same way as from (full-featured) osm_highway_linestring and used this merged z11 for mat.view z10 and z9
Close#1107
Seems like etl and mapping diagrams have been neglected
for a long time. Now it regenerates the files and places
them in the source dir.
This PR also fixes two broken files:
* layers/aerodrome_label/mapping_diagram.png
* layers/housenumber/mapping_diagram.png
They were generated using the newest tools version with the fix
https://github.com/openmaptiles/openmaptiles-tools/pull/65
Tomas Pohanka
