32 Commity (master)

Autor	SHA1	Wiadomość	Data
Tomas Pohanka	e079e82cb1	Better generalization for `landuse=residential`. (#1414) ... This PR creates larger polygons of `residential` land-use areas for upper zoom levels (especially for zoom 6 - 10). Clusters are created with the same logic as for [`building on z13`](b14da2eafb/layers/building/update_building.sql (L34-L49)). The generalization of land-use prolongs the `import-sql` step of the `landuse` layer. But it is still faster than the `building z13` generalization which with the parallel importing keeps the whole `import-sql` time untouched.	2022-09-02 10:01:41 +02:00
Brian Sperlongano	d186856ac5	Add support for cemetery tagged as grave_yard (#1175) ... Fixes #1057 This PR adds `amenity=grave_yard` to the `landuse` layer. A unification function was implemented which encodes all `class=grave_yard` as `class=cemetery` in the tiles, which adds these features for existing users of `class=cemetery` with no change. The unification function can serve as a basis for any other tags that we might want to unify in the `landuse` layer. Tile rendering for an `amenity=grave_yard`. ([Location](https://www.openstreetmap.org/way/857383420))  Tile rendering for a `landuse=cemetery`. ([Location](https://www.openstreetmap.org/way/385779531)) 	2021-08-18 09:09:51 +02:00
Adam Laža	77f5d76e37	Refactor schema, one _gen_z view per zoom. (#1045) ... * Refactor layer aeroway. * Refactor layer boundary. * Refactor layer landcover. * Refactor layer landuse. * Refactor layer park. * Refactor layer transportation. * Refactor layer water. * Refactor layer waterway. * Re-generate water* layers etl_diagrams. * Regenerate etl_diagrams for waterway. * Cast NULL to text.	2020-11-26 13:54:00 +01:00
Frédéric Rodrigo	9bb17792a6	Remove alignment of AS in SQL and few others (#932) ... * Remove alignment of AS in SQL * Remove alignment of CREATE TABLE in SQL	2020-07-22 13:48:25 +02:00
Yuri Astrakhan	2b95d1cffa	Fix & optimize incorrect function declarations (#918) ... * All functions that access database must be declared as `STABLE`, not `IMMUTABLE` -- because database can change at any moment, e.g. during an update * there are a few functions that could be made `STRICT` -- passing `NULL` as a parameter will always result in a `NULL`, but for some reason that causes a significant decrease in perf. * tagged one function as parallel safe NOTE: somehow `ST_AsMVT()` method of tile generation is showing 70-90% slowdown with this patch. I am not sure of why this is happening. If the reason is the `IMMUTABLE` -> `STABLE` change, we may have to dig deeper into PG optimization	2020-06-17 12:15:26 -04:00
Yuri Astrakhan	6457419e0d	NOOP: Format all layer's SQL code (#917) ... I would like to reformat all of our SQL to have a concise coding style. This makes it far easier to understand the code for a casual contributor, and lets us spot errors more easily. Most importantly, it makes it much easier to grep (search) the code because it is more likely to be in the same syntax Some key changes: * SQL keywords are always UPPERCASE, e.g. `SELECT WHEN AS END ...` * types, variables, aliases, and field names (identifiers) are always lower case * `LANGUAGE 'plpgsql'` is now `LANGUAGE plpgsql` (no quotes) * a few minor spacing/semicolon cleanups P.S. Per @TomPohys request, `TABLE` is spelled using upper case despite being a type for consistency with PG Docs. Same for `LANGUAGE SQL` vs `LANGUAGE plpgsql`.	2020-06-08 12:19:55 -04:00
Ludovic Delauné	924ccd6015	remove useless sql selection (#752)	2020-03-13 15:14:30 -04:00
Jorge Sanz	ace759590e	Parallel capability to layer functions (#728) ... This PR allows queries to be parallelized on recent versions of Postgres. The `PARALLEL SAFE` modifier has been added to the layer functions and a PLPGSQL function to convert strings into number has been replaced. `PARALLEL SAFE` is a modifier for `CREATE FUNCTION` available since Postgres 9.6, so this change does not break current OpenMapTiles supported database version. More details about this topic [here](https://www.postgresql.org/docs/current/parallel-safety.html) and at the reference documentation for [`CREATE FUNCTION`](https://www.postgresql.org/docs/current/sql-createfunction.html). ### Testing procedure The procedure to test this was: * Imported `spain.pbf` in a clean environment * Dumped the OpenMapTiles database from the Postgres Docker image * Created a clean Postgres 12 database using the default Docker image * Installed `postgis` 3 from the default Debian package and `osml10n` 2.5.8 from the repository (`make`, etc.) * Restored the dump * Lowered the postgres planner parameters for triggering parallel plans: ```sql set parallel_setup_cost = 5; set parallel_tuple_cost = 0.005; ``` * Manually added the `PARALLEL SAFE` modifier to each function involved in layer queries (not on updates or inserting functions). * For each layer, run a testing query to confirm parallel workers were created, something like this: ```sql explain analyze select * from layer_aerodrome_label(tilebbox(8,128,95),10,null) union all select * from layer_aerodrome_label(tilebbox(8,128,97),10,null); ``` * After all the layers were processed and confirmed to start parallel executions, a more complete example was run. This example just retrieves the geometries for all the layers from the same tile but without using any MVT related function. <details><summary>Testing query</summary> ```sql -- Using the function layer_landuse explain analyze select geometry from layer_water(tilebbox(14,8020,6178),14) union all select geometry from layer_waterway(tilebbox(14,8020,6178),14) union all select geometry from layer_landcover(tilebbox(14,8020,6178),14) union all select geometry from layer_landuse(tilebbox(14,8020,6178),14) union all select geometry from layer_mountain_peak(tilebbox(14,8020,6178),14) union all select geometry from layer_park(tilebbox(14,8020,6178),14) union all select geometry from layer_boundary(tilebbox(14,8020,6178),14) union all select geometry from layer_aeroway(tilebbox(14,8020,6178),14) union all select geometry from layer_transportation(tilebbox(14,8020,6178),14) union all select geometry from layer_building(tilebbox(14,8020,6178),14) union all select geometry from layer_water_name(tilebbox(14,8020,6178),14) union all select geometry from layer_transportation_name(tilebbox(14,8020,6178),14) union all select geometry from layer_place(tilebbox(14,8020,6178),14) union all select geometry from layer_housenumber(tilebbox(14,8020,6178),14) union all select geometry from layer_poi(tilebbox(14,8020,6178),14) union all select geometry from layer_aerodrome_label(tilebbox(14,8020,6178),14); ``` </details> You can inspect the execution plan and results on [this page](https://explain.dalibo.com/plan/3z). Also [attaching](https://github.com/openmaptiles/openmaptiles/files/3951822/explain-tile-simple.tar.gz) the query and JSON output for future reference. The website gives a ton of details, but you may want to search for nodes mentioning `workers` or `parallel` like in this area referring to `osm_border` or `osm_aeroway_linestring` entities  ### Next steps Since the execution plan is not showing a parallel append at the top level, meaning it's not running each layer individually, I want to continue experimenting with parameters and queries to see if it's possible to even parallelize more the request. I will post my finding here, even no change in the code should happen. cc. @nyurik Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>	2020-01-31 19:36:02 -05:00
Eva J	b62619c00c	etl_diagram update	2019-11-27 12:50:24 +01:00
Eva J	60b4a4cebc	removing ne_10m source	2019-11-27 11:51:15 +01:00
Eva J	fcee8571fb	adding waterway=dam	2019-11-20 18:05:07 +01:00
Eva J	c320b92a4b	place=suburb and place=neighbourhood added	2019-04-11 11:57:07 +02:00
nlehuby	940f70a415	Handle theme_park as a landuse	2018-01-12 15:40:54 +01:00
stirringhalo	90690d2a39	Switch to ZRes (#214)	2017-04-25 18:03:29 -04:00
stirringhalo	bab5c3d4b6	Switch from geom to geometry	2016-12-20 22:32:34 -05:00
ImreSamu	f5ac3def5e	etldoc fixes	2016-12-04 02:52:52 +01:00
Lukas Martinelli	5ec4f0766b	Add zoom level range slots to landcover diagram	2016-12-01 08:29:34 +00:00
Lukas Martinelli	c1b1913c7c	Fix wrong etl docs	2016-11-30 18:38:28 +00:00
Lukas Martinelli	1ef69a9fd8	Show residential from OSM on z9	2016-11-30 14:05:15 +00:00
Lukas Martinelli	7ca0fc6646	Remove subclass for landuse	2016-11-30 09:32:14 +00:00
Lukas Martinelli	9ba6a0bb0f	Fix missing column issues with landcover and landuse	2016-11-30 09:06:40 +00:00
Lukas Martinelli	6c4ac01dfb	More intermediate zoom levels for residential	2016-11-29 14:00:32 +00:00
Lukas Martinelli	327bdc7313	Remove mappings from landcover and landuse	2016-11-28 19:16:00 +00:00
stirringhalo	4f795b6287	Quiet down 'has type unknown'	2016-11-15 23:37:41 -05:00
ImreSamu	5580f42458	add etldoc to layer_landuse	2016-11-10 02:12:39 +01:00
lukasmartinelli	2d55f4fcf0	Map leisure=park into landuse	2016-10-30 18:41:33 +01:00
lukasmartinelli	81876adcf4	Remove strictness from class function in landuse	2016-10-28 12:01:22 +02:00
lukasmartinelli	9c50026f16	Remove osm_id in vector tiles coz of huge size savings	2016-10-25 19:15:51 +02:00
lukasmartinelli	7ec0db640c	Forgot bbox constraint	2016-10-25 12:03:24 +02:00
lukasmartinelli	030e1e3204	Add national parks	2016-10-24 21:49:03 +02:00
lukasmartinelli	bdea298990	Introduce landuse classes	2016-10-24 14:28:09 +02:00
lukasmartinelli	79d04f3792	Integrate urban into landuse	2016-10-24 12:02:33 +02:00