This PR creates larger polygons of `residential` land-use areas for upper zoom levels (especially for zoom 6 - 10).
Clusters are created with the same logic as for [`building on z13`](b14da2eafb/layers/building/update_building.sql (L34-L49)).
The generalization of land-use prolongs the `import-sql` step of the `landuse` layer. But it is still faster than the `building z13` generalization which with the parallel importing keeps the whole `import-sql` time untouched.
This PR updating Natural Earth dataset from v4.1 to 5.1.2. It is in cooperation with https://github.com/openmaptiles/openmaptiles-tools/pull/414, where is `import-data` docker image updated.
PR removes manual fixes for lake merging (already fixed in version 5.1.2)
Switched comparison from `gn_ascii` (removed from NE5) to `name_en`.
Could be merged after https://github.com/openmaptiles/openmaptiles-tools/pull/413. After OMT-T is merged, it will use `latest` image (until the release of OMT-T v7)
Guided busways (or bus guideways in OSM terminology) are special roadways along which bus operators can travel smoothly at high speeds without steering. These are tagged highway=bus_guideway on OpenStreetMap.
Currently, highway=busway is surfaced in OpenMapTiles as the busway class in the transportation layer, but highway=bus_guideway is not present. These two types of roadways serve generally similar purposes, so it would make sense to have guided busways at the same zoom levels.
This PR adds support for `capital=6` in the place layer, which is used for tagging county seats in the US. A county seat is the capital of a county. In addition, I reorganized the code a bit to be simpler.
As noted in ZeLonewolf/openstreetmap-americana#384, we would like to render county seats along with state and national capitals, as this is typical styling on American-style maps.
Update with also with `capital=3` and `capital=5`
closes#1370
This PR adds a parcel locker to post class.
Similarly to ATMs from PR #1375 if a name is not present we try to use `brand` or `operator`. Here we also try to add `ref` to the brand or operator since it can disambiguate parcel lockers that are next to each other (which happens).
This PR adding OSM ID to the lakes
OSM lakes are used from zoom 6
From zoom 0 to zoom 5 are used Natural Earth lakes.
- There is a new joining mat. view (`match_osm_ne_id`) contains which Natural Earth ID should convert to OSM ID. This logic is used to keep a consistent ID between switching between NE (zooms 0 - 5) and OSM (zooms 6 - 12). For smaller areas (not the whole planet), where are not available OSM lakes, the NE lakes keep their NE ID.
- There are also switch the final views to the mat. views - this should slightly increase performance (get rid of multipolygons).
- FIX typo for zooms 0 - 3 (`lakes` instead of `lake`).
Fixes#1374
This PR fixes `update_osm_route_member()` function. Before this the column `network_type` became empty after daily-update.
This PR adds `network_type` into upsert together with `rank` and `concurrency_index` columns.
This bug had occurred once already and had been fixed in https://github.com/openmaptiles/openmaptiles/pull/1239 but then some columns were added and these new columns were not added into the upsert.
Replace the `REFRESH MATERIALIZED` on `osm_important_waterway_linestring*` by differential update.
The way of doing this is the same as other differential updates.
Replace the `REFRESH MATERIALIZED` on `osm_transportation_merge_linestring_gen_z*` by differential update.
The way of doing this is the same as other differential updates. But in this case there two `GROUP BY` level. So all the the process have to be done twice.
Since PR #922 the contents of `.env` are included in `Makefile` in order for `make` to by aligned with the `docker-compose` settings.
```
# Make all .env variables available for make targets
include .env
```
The down-side of employing the `include` mechanism is that the settings in `.env` now take higher precedence than the shell environment variables. As a result, controlling the OpenMapTiles flow because more difficult. For example, tests for `DIFF_MODE=true` had to modify the contents of `.env` in order to work:
b0e7f7884c/.github/workflows/integrity.yml (L45-L47) and
b0e7f7884c/Makefile (L629-L630)
Users were also faced with similar difficulties.
This PR restores the ability to control `make` and `quickstart.sh` using environment variable while keeping the use of the `.env` at a lower priority.
The result is restoring the ability to easily adjust the OpenMapTiles flow using environment variables, such as:
```
PGPORT=54321 DIFF_MODE=true ./quickstart.sh monaco
```
#### Notes:
1. This PR depends on #1363
2. This PR includes some clean-up of `Makefile`
This PR removes an unnecessary check in water.yml.
generated SQL before:
```
SELECT CASE
WHEN waterway='riverbank' THEN 'river'
WHEN "waterway" = 'dock' THEN 'dock'
WHEN "water" = 'river'
OR "waterway" = 'riverbank'
THEN 'river'
WHEN "waterway" = '' THEN 'lake'
ELSE 'lake'
END;
```
generated SQL after:
```
SELECT CASE
WHEN waterway='riverbank' THEN 'river'
WHEN "waterway" = 'dock' THEN 'dock'
WHEN "water" = 'river'
OR "waterway" = 'riverbank'
THEN 'river'
ELSE 'lake'
END;
```
The `waterway=''` test is purely redundant because it returns the same value as the fall-through ELSE case: `lake`.
Make targets `start-tileserver`, `start-maputnik`, and `update-osm` initialize on-going processes rather than perform a one-time task. It is more appropriate to execute them using `docker-compose up -d`. With the current `docker-compose run` execution, the user needs to run `make` as a background process. This PR also allows the use of `docker-compose logs` rather than the need to handle `stdout` and `stderr` redirection and avoids the dependency of `stdin` handling for `docker-compose run`
Using `docker-compose` rather than `docker`
Use appropriate` start-*` and `stop-*` make target names
Resolves#1352
Before this, only `aerialway=cable_car` and `aerialway=gondola` were included in the tiles. That misses some widely-used tags, notably `aerialway=chair_lift`, `aerialway=drag_lift`, and `aerialway=platter`. The usage of all the tags added in this commit, by fraction of all aerialway way keys (according to TagInfo), is:
| Tag | Usage |
|------------------------|-------:|
| `aerialway=chair_lift` | 25.51% |
| `aerialway=drag_lift` | 12.93% |
| `aerialway=platter` | 11.35% |
| `aerialway=t-bar` | 7.93% |
| `aerialway=j-bar` | 0.63% |
| `aerialway=mixed_lift` | 0.24% |
Some of these tags were also added in PR #620 back in May 2019, but were removed again before it was merged.
Fixes#1342.
Updates on `transporation_name` layer take much more time than before.
In 3.13 there had been introduced highway concurrency into `transportation` and `transportation_name` so I expected the update process will take more time but not that much. Because of this it's impossible to use updates on larger areas because the process takes too long.
The issue is caused by `IS NOT DISTINCT FROM` operator over `tags` (hstore) columns. I replaced it with `=` operator in combination with `coalesce()` function which returns the same results but in shorter time.
When docker compose returns a version with v prefix,
the version check would incorrectly treat it as version component
of 000, leading to always failing the docker-compose version check
even if the version is recent enough.
This happens at least with docker-compose 2.2.1 where
docker-compose version --short
returns v2.2.1 with the leading v.
This PR fixes a bug that causes that `track` lines disappear at z13. This bug was introduced in https://github.com/openmaptiles/openmaptiles/pull/1190, which adds rendering of paths and tracks at z12 and z13.
Before this PR:
z12: lines with `route_rank = 1` are added (no matter what `highway`).
z13: lines with `route_rank BETWEEN 1 AND 2` and `highway = 'path'` are added.
-> tracks with `route_rank=1` are added at z12 but not at z13.
After this PR
z12: lines with `route_rank = 1` and `highway IN ('path', 'track')` are added.
z13: lines with `route_rank BETWEEN 1 AND 2` and `highway IN ('path', 'track')` are added .
-> only tracks and paths are added at z12 and z13 (which was IMHO the goal of https://github.com/openmaptiles/openmaptiles/pull/1190)
* Add only the most important paths and tracks (route_rank=1) to z12 and more important ones (route_rank between 1-2 or sac_scale or has name) to z13.
Remove `FILTER_MAPNIK_OUTPUT` variables from docker-compose which stop raising warnings during start-up dockers.
This functionality was removed in https://github.com/openmaptiles/openmaptiles-tools/pull/349.
The current version of the mapnik does not need `FILTER_MAPNIK_OUTPUT`.
PR move `get_basic_names(tags, geometry)` function from `transportation_name_tags` function and adding it after `transportation_name_tags` resolve street abbrevation.
PR runs through `test-sql` test without error. Adding to `osm_transportation_name_network` (table and for update) was done in consequence of `test-sql` errors ([update test 500](a747d98550/tests/test-post-update.sql (L53))). Originally `get_basic_names` was just in `osm_transportation_name_linestring` table.
FIX#1324
Remove `STRICT` statement from `transportation_name_tags` function.
Concurrently with remove `STRICT` from `get_basic_names` and `get_latin_name` in OMT-T
FIX#1319
* Add info about using MID_ZOOM to optimize tile generation
* Remove params for generation that are now automatically detected
Co-authored-by: Tomas Pohanka <TomPohys@gmail.com>
Fixes#1148
This PR adds expressway tagging to the `transportation` layer, by setting `expressway=1` for non-motorway roads tagged `expressway=yes`, and omitting the tag otherwise. Additionally, I've added a few unit tests to verify that the expressway tagging is being imported and updated into the intermediate tables.
Here is an example of expressway tagging on US-1 in Rhode Island, USA:
I discovered this bug while investigating issues with the updates process related to #1190#1292, and #814.
The `transportation_name` layer produces slightly different `tags` hstore values in the `osm_transportation_name_linestring` table during the initial import versus when running an update. As currently written, the import code produces null-value keys in the `tags` column, while the update code suppresses them. This PR removes that difference and makes the import code use same method that is currently used in the update code.
With a test case I've written, the import code produces a tags hstore that looks like this:
`"name"=>"OpenMapTiles Secondary 2", "name:de"=>NULL, "name:en"=>NULL, "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"`
...while the update code produces a tags hstore that looks like this:
`"name"=>"OpenMapTiles Secondary 2", "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"`
Note the missing NULL values.
This bug causes a small amount of space wastage after an update is run, because the update matching code detects the `tags` value as different, resulting in a duplicate copy of the tags value if that row is updated. This causes duplicate objects and breaks GROUP BY clauses that expect to group same-tagged features together. I've tested this by inspection of a generated mbtiles, database spot checks, and the unit test code included in this PR.
This PR implements non-blocking updates in the park layer.
The approach was to use the `REFRESH MATERIALIZED VIEW CONCURRENTLY` feature in postgres. In order to achieve this, a unique index was added on the z4 dissolved park area table. The `ST_Union` / `ST_Dump` sequence was changed to an explicit cluster DB scan (an equivalent operation) so that a unique osm ID could be generated from each dissolved polygon.
Below is a screen shot from Idaho, USA showing that the dissolved z4 still works as expected.
Closes#271
This PR adds track and path rendering at lower zooms than currently provided, and also achieves near-parity with openstreetmap-carto on track and path rendering. A previously-abandoned attempt, with significant discussion, was #1169.
This PR adds unit tests for the transportation layer, and fixes several bugs in the unit testing code. It verifies the correct zoom association to the various highway classes, and verifies that updates to highway attributes are populated.
Additionally, the documentation for unit tests in the CONTRIB guide is updated to note the specific make commands for developers to run.
This PR is solving https://github.com/openmaptiles/openmaptiles/issues/406, partially also https://github.com/openmaptiles/openmaptiles/issues/320.
It is changing source of data for zoomlevel 6 - 8 in `waterway` layer to OSM (relation `waterway = river`), this way it corresponds with `water` layer.
Testing on Europe, z0 - z9 tiles increase size from 20.7 MB to 21.4 MB. We can discuss the level of generalization, but I think this size change is not critical.
Tomas Pohanka
