Wykres commitów

590 Commity (9e4be3e3b03c26fcc1e63ee4976f44a5d1f8dba1)

Autor SHA1 Wiadomość Data
Brian Sperlongano 9e4be3e3b0
Remove unused network type params (#1155)
Changes in #1143 removed the need for `name` and `ref` parameters in the network-to-network_type conversion function.  This PR removes this dead code.

Verified by running ./quickstart rhode-island which completed successfully.
2021-07-26 16:37:12 +02:00
Brian Sperlongano 951aa907b2
Fix SQL failures in transportation_name update code (#1154)
This PR fixed bugs introduced in #1147 and #1119 which broke the update triggers in `transportation_name_update`.  I noticed this issue while I was working a different PR and tried to update a table in the `transportation` layer.

**Test**

The following code currently fails because of cascading update failures, but will complete successfully after this PR is merged.
```
UPDATE osm_highway_linestring hl
    SET network = rm.network_type
    FROM osm_route_member rm
    WHERE hl.osm_id=rm.member;
```
2021-07-26 15:22:36 +02:00
Brian Sperlongano b011b27e52
Implement concurrent highway routes (#1152)
Fixes #1128

This PR adds 6 new columns to the `transportation_name` column, named `route_1` through `route_6`.  These columns contain route information for a section of roadway.  The value is stored in the form **network=ref**.  For example, Interstate 95 in the United States would be `US:I=95`.  Thus, each `route_N` value contains enough information to render a highway shield.  Since a section of road can be a part of more than one route, the `route_2`, `route_3`, etc, will contain the 2nd, 3rd, etc., concurrent routes.

The technical approach was to extend the change in #1135, which added ordered concurrency indexes to the `osm_route_member` table by joining up to the the first six entries to the `osm_transportation_name_network` table.  In addition, that PR provided a ranking system for concurrent highways, ordering first by `network_type` (for example, `us-interstate`, `us-state`, etc), then alphabetically by network name, and then by ref in ascending order.  This ordering of concurrent route memberships is now exposed in this PR in the sequential `route_N` values, meaning that rendered concurrent highway shields will be reasonably sorted.

The renderings below were generated using this branch of OpenMapTiles, as well as a separate branch of openstreetmap-americana:
https://github.com/ZeLonewolf/openstreetmap-americana/tree/openmaptilers-new-features-test

The rendering approach is to use the [formatted expressions](https://maplibre.org/maplibre-gl-js-docs/style-spec/expressions/) feature in mapLibre to insert images into a string of text.  Blank shields are added to the sprite sheet for all possible route networks.  Next, a [styleimagemissing](https://maplibre.org/maplibre-gl-js-docs/api/map/) callback is registered.  As each shield ID is requested, the callback retrieves the sprite shield blank associated with the route's network, draw the `ref` text on the shield, and insert the complete shield back into the map.

Of note, this approach currently results in shields which are rotated about the road rather than being viewport aligned.  This issue is currently documented as maplibre/maplibre-gl-js#188.  A separate repository (https://github.com/ZeLonewolf/maplibre-shield-rotation-sample) has been created as a test case to fix this rotation issue.

Adding route concurrency information to OpenMapTiles would be a major step forward in achieving comprehensive highway shield renderings in a vector map!

**Renderings**:

![routes_1](https://user-images.githubusercontent.com/3254090/126054350-fa7475a7-1b60-4989-bbc2-107678e6c73b.png)
![routes_2](https://user-images.githubusercontent.com/3254090/126054351-fe73bc70-d75f-4ab5-8365-0ee3c3d3eab0.png)
![routes_3](https://user-images.githubusercontent.com/3254090/126054353-a1e74c8f-df21-423c-a300-b7f1a7c9231c.png)
![routes_4](https://user-images.githubusercontent.com/3254090/126054355-6b5dcc83-c611-42b3-bb67-d4f26d789744.png)
2021-07-26 14:28:36 +02:00
Brian Sperlongano 3c15679555
Update brunnel aggregation to avoid splitting highways (#1141)
Fixes #1131

This change does the following:
1. Excludes roads from `transportation_name` that don't have a `name` or a `ref`
2. Updates the road name merging logic to exclude changes in `brunnel` status.  This will ensure that minor bridges don't disrupt the continuity of named roads as the map zooms out.
3. The `brunnel` tag will now only be set when a bridge or tunnel is distinctly named.  Distinctly named is defined as "has a different name from the road on either side".

This example shows an unnamed interstate highway rendered as a continuous feature at low zoom.  This road has many small bridges along its length:

![image](https://user-images.githubusercontent.com/3254090/124370289-b30faa80-dc43-11eb-80d6-034c18ce99ad.png)


This example shows a named bridge rendered with `brunnel` tag set:

![image](https://user-images.githubusercontent.com/3254090/124370298-d0dd0f80-dc43-11eb-8a78-183420a6bd62.png)
2021-07-13 14:44:59 +02:00
Brian Sperlongano 197ea39ae3
Remove unused osm_id column (#1147)
Fixes #1146

This PR removes the always-null osm_id column from the `transportation_name_linestring` table and the series of generalized tables that derive from it.

Demonstration of `transportation_name` objects behaving normally after the column has been removed:

![image](https://user-images.githubusercontent.com/3254090/124684944-4512ef80-de9e-11eb-998c-b66bc23be09e.png)
2021-07-13 13:39:28 +02:00
Frédéric Rodrigo a851f2c9e9
Review index on osm_waterway_linestring on osm_important_waterway_linestring_gen_z* (#1130)
Review index waterway.

* Remove not used index
* Fit index `osm_waterway_linestring_waterway_partial_idx` to query
2021-07-09 09:35:46 +02:00
Brian Sperlongano 4eb240466e
Simplify Trans-Canada Highway logic (#1143)
Fixes #1142

Screenshot from Alberta showing TCH still shown in `transportation_name` layer:
![image](https://user-images.githubusercontent.com/3254090/124532399-77f5ae80-ddde-11eb-8929-36970ebb8386.png)

Screenshot from Ontario showing TCH still shown in `transportation_name` layer:
![image](https://user-images.githubusercontent.com/3254090/124532512-b68b6900-ddde-11eb-9b6a-636c872375c1.png)
2021-07-06 08:43:26 +02:00
Brian Sperlongano c8377d38ac
include route relation ref in existence check (#1139)
Fixes #1138
Unblocks #1131
Unblocks #1128 

Below is an example of a way that is part of a route relation (in this case, RI-3), but does not have a ref set on the way itself:
![image](https://user-images.githubusercontent.com/3254090/124366616-1509e900-dc1f-11eb-87ab-aadd4f21287d.png)

This screenshot demonstrates that the `transportation_name` object now contains the ref from the route relation:
![image](https://user-images.githubusercontent.com/3254090/124366563-b5abd900-dc1e-11eb-9fd9-96315ddf7682.png)
2021-07-05 11:21:35 +02:00
Brian Sperlongano 1b0b1fd121
Compute separate concurrency_index on osm_route_member (#1135)
Closes #1134 

Adds a `concurrency_index` column to `osm_route_member` as described in #1134.  For example, listing concurrencies on a [segment of the Washington Bridge](https://www.openstreetmap.org/way/43080535) in Providence, Rhode Island, USA:

```
openmaptiles=# select osm_id, network, ref, network_type, name, concurrency_index from osm_route_member where member=43080535 order by concurrency_index asc;
  osm_id  | network | ref | network_type  |     name      | concurrency_index
----------+---------+-----+---------------+---------------+-------------------
 -1694950 | US:I    | 195 | us-interstate | I 195 (RI/MA) |                 1
 -2308410 | US:US   | 6   | us-highway    | US 6 (RI)     |                 2
 -1347856 | US:US   | 1A  | us-highway    | US 1A (RI)    |                 3
 -2309143 | US:US   | 44  | us-highway    | US 44 (RI)    |                 4
(4 rows)
```
2021-07-02 15:00:25 +02:00
Brian Sperlongano 3f70b878e2
nulling subclass when not present (#1132)
PR #1119 (adding support for `highway=motorway_junction`) missed a few cases where NULLIF functions were needed in order to suppress empty-string `subclass` tags from appearing in vector tiles, resulting in unnecessary  `subclass` entries on `transportation_name` objects where it was not needed, for example:
![image](https://user-images.githubusercontent.com/3254090/123889414-7a11c600-d923-11eb-938c-f278b9bf1ffa.png)

This PR adds the missing NULLIF function so that those empty tags are suppressed:
![image](https://user-images.githubusercontent.com/3254090/123889540-b6452680-d923-11eb-8f0b-5c8c03640059.png)
2021-06-30 07:06:22 +02:00
Frédéric Rodrigo a0847b85f1
Update osm_poi_point only on change (#1129)
After each diff import the table `osm_poi_point` is fully rewritten due to a full update of the field tags. It is now good to do an a update, event if the content does not change, postgres delete and reinsert internally the record. Resulting in more write and internal table size raising.

Note: not directly linked, but there is a problem in this case, the autocaccum is not sufficient to keep this table size moderate, but grow indefinitely.
2021-06-29 15:05:45 +02:00
Frédéric Rodrigo 3b8248af63
Switch from amenity=embassy to office=diplomatic #1084 (#1085)
Switch from amenity=embassy to office=diplomatic as was about here #1084.
2021-06-28 16:58:21 +02:00
Brian Sperlongano 79c2ec929d
Add new layer to serve highway=motorway_junction nodes (#1119)
This PR adds a layer for `highway=motorway_junction` features.

This implementation of highway exits in the transportation_name layer add to the existing layer table structure, and renames the internal column name from "construction" (which was already overloaded with non-construction usages) to "subclass", which will be less confusing to future developers.  The string 'junction' is used as the universal sub-class for highway exits.

A new documentation PR has been opened at openmaptiles/www.openmaptiles.org#69 to reflect these changes in the documentation.
2021-06-28 14:35:27 +02:00
Tomas Pohanka d0ebdde458
Fix transportation road segments disconnection (#1109)
To avoid discontinuous transportation lines between zooms 9 and 11. 
 - Originally limit geometry by length for z9 - z11 (`ST_Length(geometry) > ZRes(11)`)
 - highway z9 to z11 was generalized during import-osm
 - now just create a filtered and generalized z11 table
 - then merge segments in the same way as from (full-featured) osm_highway_linestring and used this merged z11 for mat.view z10 and z9

Close #1107
2021-04-30 15:39:56 +02:00
Eva Jelinkova f4a1a2537a
Water - add ogc_fid for NE sources, edit etldoc (#1098)
1) Adding ogc_fid column into materialized views of NE lakes (for having option to filter or compare)
2) Edit bug in etldoc

* add ogc_fid for NE sources, edit etldoc

* etl_diagram update
2021-04-12 12:09:45 +02:00
Tomas Pohanka fadc4a3ba6
Explicit mention for using `iso_a2` attribute. (#1073)
Resolve #939
2021-03-16 11:21:25 +01:00
Tomas Pohanka bf9e6a19cd
osm_route_member bugfix (#1088)
`!=` is not handling `NULL` values.

Then all networks are `road` and not e.g. `us-interstate` or `ca-transcanada`.
2021-03-16 10:55:26 +01:00
Frédéric Rodrigo 50d569dfee
Remove useless indices on osm_transportation_merge_linestring_genX (#1079)
On creation of materialized views `osm_transportation_merge_linestring_gen_zX`, an index is created each time the new view and not used else where than the next generalization step.
It does not worth it to to a scan on the table to create index that will be used only once, while the next generalisation step can also be done in one full scan.
So, removing these indices.
2021-03-05 09:42:27 +01:00
Jeroen Hoek 295688eb80
Add place=quarter to city subdivisions (#1065)
This adds `place=quarter` which was introduced in 2011 as a subdivision
of villages/towns/cities as a value that sits between `place=suburb` and
`place=neighbourhood`. It has by now been used over 40,000 times and is
rendered in the OSM-Carto.
2021-01-18 14:47:25 +01:00
Adam Laža 461043960e
Bugfix landcover osm (#1061)
* Release 3.12 prerequisite

* Add missing osm_id column to osm_landcover_polygon

Co-authored-by: Tomáš <tompohys@gmail.com>
2021-01-08 15:44:11 +01:00
Sergii Golubev 2aa9630b41
add forgotten 'garages' landuse to the docs (#1059)
In https://github.com/openmaptiles/openmaptiles/pull/720 I've forgotten to add the `garages` class to the docs. This PR is fixing that issue,
2020-12-29 14:26:14 +02:00
Adam Laža 5772e61244
Bugfix in osm_landcover_gen_z10. (#1054)
Add missing `NOT` in WHERE condition.
It causes there are duplicated features of subclass `wood` and `forest` and features of other subclasses are missing.
2020-12-14 12:47:45 +02:00
Adam Laža 512b3435ad
Bugfix update_aerodrome_label_point() (#1051)
Bug introduced in #944.
Missing exclamation mark in not equal operator caused that tags were not updated thus there were missing `name_int`, `name:latin` and `name:nonlatin`.
2020-12-08 17:27:12 +02:00
Adam Laža 77f5d76e37
Refactor schema, one _gen_z view per zoom. (#1045)
* Refactor layer aeroway.

* Refactor layer boundary.

* Refactor layer landcover.

* Refactor layer landuse.

* Refactor layer park.

* Refactor layer transportation.

* Refactor layer water.

* Refactor layer waterway.

* Re-generate water* layers etl_diagrams.

* Regenerate etl_diagrams for waterway.

* Cast NULL to text.
2020-11-26 13:54:00 +01:00
Tomas Pohanka 0c6fe2d9ba
Building aggregation fix (#1044)
This PR solving the speed and memory issue with buildings aggregation on zoom level 13. The problem was to create ST_ClusterDBScan which failed after try to cluster a larger area. Now it will make cluster only in "small" polygon using 'country_osm_grid` which covers the world's lands. 

This PR solving #1022 and #974
2020-11-24 12:27:06 +02:00
Tomas Pohanka 24b9328a5f
Remove unused generalization from waterway (#1043)
During the import-osm step, there are three generalized tables for the waterway that are not used for subsequent processing.
Thanks @lazaa32 for the notice.
2020-11-24 11:30:00 +02:00
Adam Laža da689f9e42
Fix landcover generalization. (#1042)
This PR brings few modification which make it possible to generalize landcover on larger areas. Tested on Europe.

- Use ST_SnapToGrid to avoid ERROR:  GEOSUnaryUnion: TopologyException: found non-noded intersection between LINESTRING.
- Cluster only landcover subclasses: wood and forest.
- Use less generalized tables as a starting point for next step generalization.
2020-11-24 10:33:06 +02:00
Tomas Pohanka 3ef364552e
add state name to national borders (#1039)
This PR adds state name to national borders (`admin_level=2`)

From zoom level 3 there is an added `adm0_l` and `adm0_r` attribute for national borders.

There is 3 main steps: 
1) union border lines from OSM and keep separate LineString between intersects
2) create polygons from step 1), create a point in each state polygon, add to this point a state abbreviation from NaturalEarth Data
3) create a short parallel line on the left and right of a small part of the border in 70m distance, and add information about which state overlap this short line.
2020-11-12 12:14:13 +02:00
Eva Jelinkova 0aa6648bd9
adding brunnel into z<9 (#1038)
Following https://github.com/openmaptiles/openmaptiles/pull/1027, solving https://github.com/openmaptiles/openmaptiles/issues/999.

Adding brunnel attributes to merge tables of `transportation` layer, now it is possible to style/filter tunnels, bridges and fords in all zoom levels. Tested on Switzerland and Europe. Changes in generation speed and filesize minimal.
2020-11-11 15:37:09 +02:00
Tomas Pohanka 4a1b0afa26
reworked landcover layer (#1035)
Instead of using [Douglas-Peucker algorithm](https://postgis.net/docs/ST_SimplifyPreserveTopology.html) which is using for generalized tables [imposm](https://imposm.org/docs/imposm3/latest/mapping.html#generalized-tables) it is used [Visvalingam-Whyatt algorithm](https://postgis.net/docs/ST_SimplifyVW.html).

Solution:
remove imposm generalized tables (during `import-osm`) and create generalization in `import-sql` step for zooms 7-14. Zooms 0-6 are from Natural Earth data.

Upper zoom levels (7 and 8) are ok to merge with no big impact on creation speed. In Canada from z9 it took too long. Between zoom levels 10 - 13 there can be union polygons with less than 300 edge points (empirical number based on a test on Canada forest, can be discussed). Polygons with more than 300 edge points are just simplified. Zoom 14 is from the original dataset `osm_landcover_polygon`.

There is also removed `osm_id` which is never used and not passed into vector tiles.
2020-11-11 15:11:48 +02:00
Adam Laža 0dc8c3256c
Rename generic layer.sql to actual_layer_name .sql (#1034)
Following layers had generic `layer.sql` name.

- `aerodrome_label`
- `aeroway`
- `housenumber`
- `mountain_peak`
- `park`
- `place`
- `poi`
- `transportation`
- `transportation_name`
- `water_name`

This PR renames `layer.sql` to `aerodrome_label.sql`, `aeroway.sql`...
2020-11-09 11:43:35 +02:00
Eva Jelinkova a36db48a3e
Revert "#766 remove housenumbers inside polygon and same number (#983)" (#1033)
This reverts commit 1685eaccbd.
2020-11-04 21:53:15 +01:00
Eva Jelinkova 8ba2e18dea
Revert "Housenumbers - diff update, multipolygons (#990)" (#1032)
This reverts commit f7d7a0ced3.
2020-11-04 21:07:09 +01:00
Tomas Pohanka 72af34612a
Add gates to airports (#1029) 2020-10-19 12:24:16 +03:00
Eva Jelinkova 341c4df171
Brunnel changes in transportation and transportation_name layers (#1027)
https://github.com/openmaptiles/openmaptiles/issues/999

- Point 1 solved only partially - there is `brunnel` value added into zooms 9, 10, 11. For zooms 8 and smaller it would need change in osm_transportation_merge_ tables - I am going to try it but maybe its SQL costs will be to high.

- Point 2 solved.
2020-10-14 15:26:43 +03:00
Tomas Pohanka b7429ce6f5
union ocean regular squares into complex polygon (#1021)
The ocean layer (table osm_ocean_polygon insert by `make import-data`) is made from squares that have a 10m buffer. 

This creates extra geometry in vector tiles as:

Union of full ocean squares should decrease the size of the water layer.

there are 8042 polygons (squares, 5 points, more then 100km<sup>2</sup>), which was reduced to 22 polygons - 1184kB vs 40kB respectively. 

In mbtiles should be the size reduction even more significant, due to the creation of more polygons based on a 10m buffer of each square.
2020-10-14 14:56:04 +03:00
Tomas Pohanka 72165933be
add ST_IsValid where ST_PointOnSurface (#1025)
Same as #1015 for another ST_PointOnSurface
2020-10-13 16:28:20 +03:00
Eva Jelinkova 7a7e9326b9
adding ST_IsValid to housnumber geometry (#1015)
Adding check of geometry in layers/housenumber/housenumber_centroid.sql.
2020-10-12 15:05:20 +03:00
Eva Jelinkova 3d21523fbe
aerialway class for the transportation layer (#1011)
Co-authored-by: Tomas Pohanka <TomPohys@gmail.com>
2020-10-12 12:44:33 +02:00
Frédéric Rodrigo 36b7533d3b
Add timers to trigger function (#1006)
Add timer to all trigger refresh functions to help point where the time is spend on update.
2020-10-08 16:57:16 +03:00
Frédéric Rodrigo 883a997dcf
Remove large unused index osm_highway_linestring_highway_idx (#1002)
The index `osm_highway_linestring_highway_idx` is not used. There is also a used partial index `osm_highway_linestring_highway_partial_idx`.

I cross check the usage of the index from postgres query stats and all SQL query code where the table `osm_highway_linestring` is refered.

This index is relatively big.
2020-09-30 11:30:18 +03:00
Frédéric Rodrigo a3ce04fe23
Add landuse=salt_pond as water (#1000)
Add salt pond as water.

Most of salt pond are already tagged with natural=water even if it's not required. So the load added by this tag is very light.
2020-09-29 07:53:35 +03:00
Frédéric Rodrigo c8f919e9d6
Better update of osm_important_waterway_linestring, use frist and last version of osm object (#997)
Current implementation of osm_important_waterway_linestring have two bugs:
* The distinct on is_old keep the oldest version of the old object, and the oldest version of the new object, but need the last version of the new object.
* Delete the old version of the object and rebuild the using the new version of the object. But we need to remove matchings the old and the new version of the object, then rebuild the two.

Then only delete and update using first and last version of the object, intermediate versions are ignored.

Similar implementation of what is done in #996.
2020-09-28 17:07:37 +03:00
Frédéric Rodrigo 8bb77b67a1
Diff update osm transportation name network linestring (#996)
Replacing materialized view by a tables with update from trigger on change only.

Differential update of `osm_transportation_name_linestring`, `osm_transportation_name_linestring` and `osm_transportation_name_linestring_genX`).

At the end of the transaction the dependent rows are updated.

The goal is to update more quickly the content of derivated tables by just updating the changing content. It replaces the update of materialized view because their need a full recompute (with lock issue).

It is the last part of the replacement of materialized view for the transportation layer.

It addresses #814 and a part of #809.
2020-09-28 15:56:10 +03:00
Frédéric Rodrigo 47cdfc2c8b
Move where condition from osm_transportation_name_linestring to materialized view osm_transportation_name_network (fix) (#993)
On PR #991 the differential update was not updated like the main code.

Make similar change to the update code.
2020-09-23 11:10:59 +03:00
Eva Jelinkova f7d7a0ced3
Housenumbers - diff update, multipolygons (#990)
Following https://github.com/openmaptiles/openmaptiles/pull/983 - removing points also on multipolygons and checking only new geometry when updated.
2020-09-21 14:47:55 +03:00
zstadler 45f5b53c9c
Stitch `transportation_name` by all language tags (#989)
When OSM roads in the `transportation_name` layer are stitched together, their grouping does not consider all `name:*` tags.
As a result, roads with different `name:*` tags may be stitched together.

The `waterway` layer performs the grouping properly, for the same purpose:
1685eaccbd/layers/waterway/update_important_waterway.sql (L34)

Co-authored-by: Frédéric Rodrigo <fred.rodrigo@gmail.com>
2020-09-21 12:38:13 +03:00
Frédéric Rodrigo 5c640daf4f
Move where condition from osm_transportation_name_linestring to materialized view osm_transportation_name_network (#991)
Reduce the size of the materialized view osm_transportation_name_network by moving the weare clause.
osm_transportation_name_network is only used for osm_transportation_name_linestring.

Co-authored-by: zstadler <zeev.stadler@gmail.com>
2020-09-21 10:02:20 +03:00
Frédéric Rodrigo 550b1f8f90
Replace materialized view osm_transportation_name_network by a table with diff update (#987)
Replacing materialized view by a tables with update from trigger on change only.

The osm_id of object changing on table osm_route_member and osm_highway_linestring are logged.
At the end of the transaction the dependent row of osm_transportation_name_network are updated.

The goal is to update more quickly the content of derivated tables by just updating the changing content. It replaces the update of materialized view because their need a full recompute (with lock issue).

Note, it is only a part of the materialized view of the transportation layer, I am also working on the other parts, other parts are in #892 (omaterialized view on sm_transportation_name_linestring and osm_transportation_name_linestring_genX).

It addresses #814 and a part of #809.
2020-09-18 16:49:55 +03:00
Taro Matsuzawa aka. btm 1685eaccbd
#766 remove housenumbers inside polygon and same number (#983) 2020-09-16 17:06:48 +02:00