openmaptiles/layers/place/update_continent_point.sql

88 wiersze
2.5 KiB
MySQL

DROP TRIGGER IF EXISTS trigger_flag ON osm_continent_point;
Make more simple incremental update (#944) Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) en ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. I try many code layout before done it in this way with the goal to keep the code for initial pass and for update. It should have low impact on initial data load. Better performance for row update can be achieve with BEFORE UPDATE, but require to duplicate the logic. It is not based on the already merged https://github.com/openmaptiles/openmaptiles/pull/896 because calling and update within a function for each updated row was not efficient for larger table (like housenumber). It addresses #814. * Remake update_peak_point use incremental update #814 * Make update_aerodrome_label_point use incremental update #814 * Make housenumber_centroid use incremental update #814 * Make update_continent_point use incremental update #814 * Make update_island_point use incremental update #814 * Make update_island_polygon use incremental update #814 * Remove dead code in update_state_point.sql * Make update_state_point use incremental update #814 * Remove dead code in update_country_point.sql * Make update_country_point use incremental update #814 * Make osm_poi_polygon use incremental update #814 Thanks @frodrigo
2020-08-28 09:03:27 +00:00
DROP TRIGGER IF EXISTS trigger_store ON osm_continent_point;
DROP TRIGGER IF EXISTS trigger_refresh ON place_continent_point.updates;
Make more simple incremental update (#944) Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) en ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. I try many code layout before done it in this way with the goal to keep the code for initial pass and for update. It should have low impact on initial data load. Better performance for row update can be achieve with BEFORE UPDATE, but require to duplicate the logic. It is not based on the already merged https://github.com/openmaptiles/openmaptiles/pull/896 because calling and update within a function for each updated row was not efficient for larger table (like housenumber). It addresses #814. * Remake update_peak_point use incremental update #814 * Make update_aerodrome_label_point use incremental update #814 * Make housenumber_centroid use incremental update #814 * Make update_continent_point use incremental update #814 * Make update_island_point use incremental update #814 * Make update_island_polygon use incremental update #814 * Remove dead code in update_state_point.sql * Make update_state_point use incremental update #814 * Remove dead code in update_country_point.sql * Make update_country_point use incremental update #814 * Make osm_poi_polygon use incremental update #814 Thanks @frodrigo
2020-08-28 09:03:27 +00:00
CREATE SCHEMA IF NOT EXISTS place_continent_point;
CREATE TABLE IF NOT EXISTS place_continent_point.osm_ids
(
osm_id bigint PRIMARY KEY
Make more simple incremental update (#944) Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) en ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. I try many code layout before done it in this way with the goal to keep the code for initial pass and for update. It should have low impact on initial data load. Better performance for row update can be achieve with BEFORE UPDATE, but require to duplicate the logic. It is not based on the already merged https://github.com/openmaptiles/openmaptiles/pull/896 because calling and update within a function for each updated row was not efficient for larger table (like housenumber). It addresses #814. * Remake update_peak_point use incremental update #814 * Make update_aerodrome_label_point use incremental update #814 * Make housenumber_centroid use incremental update #814 * Make update_continent_point use incremental update #814 * Make update_island_point use incremental update #814 * Make update_island_polygon use incremental update #814 * Remove dead code in update_state_point.sql * Make update_state_point use incremental update #814 * Remove dead code in update_country_point.sql * Make update_country_point use incremental update #814 * Make osm_poi_polygon use incremental update #814 Thanks @frodrigo
2020-08-28 09:03:27 +00:00
);
-- etldoc: osm_continent_point -> osm_continent_point
Make more simple incremental update (#944) Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) en ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. I try many code layout before done it in this way with the goal to keep the code for initial pass and for update. It should have low impact on initial data load. Better performance for row update can be achieve with BEFORE UPDATE, but require to duplicate the logic. It is not based on the already merged https://github.com/openmaptiles/openmaptiles/pull/896 because calling and update within a function for each updated row was not efficient for larger table (like housenumber). It addresses #814. * Remake update_peak_point use incremental update #814 * Make update_aerodrome_label_point use incremental update #814 * Make housenumber_centroid use incremental update #814 * Make update_continent_point use incremental update #814 * Make update_island_point use incremental update #814 * Make update_island_polygon use incremental update #814 * Remove dead code in update_state_point.sql * Make update_state_point use incremental update #814 * Remove dead code in update_country_point.sql * Make update_country_point use incremental update #814 * Make osm_poi_polygon use incremental update #814 Thanks @frodrigo
2020-08-28 09:03:27 +00:00
CREATE OR REPLACE FUNCTION update_osm_continent_point(full_update boolean) RETURNS void AS
$$
UPDATE osm_continent_point
SET tags = update_tags(tags, geometry)
Make more simple incremental update (#944) Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) en ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. I try many code layout before done it in this way with the goal to keep the code for initial pass and for update. It should have low impact on initial data load. Better performance for row update can be achieve with BEFORE UPDATE, but require to duplicate the logic. It is not based on the already merged https://github.com/openmaptiles/openmaptiles/pull/896 because calling and update within a function for each updated row was not efficient for larger table (like housenumber). It addresses #814. * Remake update_peak_point use incremental update #814 * Make update_aerodrome_label_point use incremental update #814 * Make housenumber_centroid use incremental update #814 * Make update_continent_point use incremental update #814 * Make update_island_point use incremental update #814 * Make update_island_polygon use incremental update #814 * Remove dead code in update_state_point.sql * Make update_state_point use incremental update #814 * Remove dead code in update_country_point.sql * Make update_country_point use incremental update #814 * Make osm_poi_polygon use incremental update #814 Thanks @frodrigo
2020-08-28 09:03:27 +00:00
WHERE (full_update OR osm_id IN (SELECT osm_id FROM place_continent_point.osm_ids))
AND COALESCE(tags->'name:latin', tags->'name:nonlatin', tags->'name_int') IS NULL
AND tags != update_tags(tags, geometry);
$$ LANGUAGE SQL;
Make more simple incremental update (#944) Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) en ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. I try many code layout before done it in this way with the goal to keep the code for initial pass and for update. It should have low impact on initial data load. Better performance for row update can be achieve with BEFORE UPDATE, but require to duplicate the logic. It is not based on the already merged https://github.com/openmaptiles/openmaptiles/pull/896 because calling and update within a function for each updated row was not efficient for larger table (like housenumber). It addresses #814. * Remake update_peak_point use incremental update #814 * Make update_aerodrome_label_point use incremental update #814 * Make housenumber_centroid use incremental update #814 * Make update_continent_point use incremental update #814 * Make update_island_point use incremental update #814 * Make update_island_polygon use incremental update #814 * Remove dead code in update_state_point.sql * Make update_state_point use incremental update #814 * Remove dead code in update_country_point.sql * Make update_country_point use incremental update #814 * Make osm_poi_polygon use incremental update #814 Thanks @frodrigo
2020-08-28 09:03:27 +00:00
SELECT update_osm_continent_point(true);
-- Handle updates
Make more simple incremental update (#944) Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) en ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. I try many code layout before done it in this way with the goal to keep the code for initial pass and for update. It should have low impact on initial data load. Better performance for row update can be achieve with BEFORE UPDATE, but require to duplicate the logic. It is not based on the already merged https://github.com/openmaptiles/openmaptiles/pull/896 because calling and update within a function for each updated row was not efficient for larger table (like housenumber). It addresses #814. * Remake update_peak_point use incremental update #814 * Make update_aerodrome_label_point use incremental update #814 * Make housenumber_centroid use incremental update #814 * Make update_continent_point use incremental update #814 * Make update_island_point use incremental update #814 * Make update_island_polygon use incremental update #814 * Remove dead code in update_state_point.sql * Make update_state_point use incremental update #814 * Remove dead code in update_country_point.sql * Make update_country_point use incremental update #814 * Make osm_poi_polygon use incremental update #814 Thanks @frodrigo
2020-08-28 09:03:27 +00:00
CREATE OR REPLACE FUNCTION place_continent_point.store() RETURNS trigger AS
$$
BEGIN
INSERT INTO place_continent_point.osm_ids VALUES (NEW.osm_id) ON CONFLICT (osm_id) DO NOTHING;
Make more simple incremental update (#944) Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) en ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. I try many code layout before done it in this way with the goal to keep the code for initial pass and for update. It should have low impact on initial data load. Better performance for row update can be achieve with BEFORE UPDATE, but require to duplicate the logic. It is not based on the already merged https://github.com/openmaptiles/openmaptiles/pull/896 because calling and update within a function for each updated row was not efficient for larger table (like housenumber). It addresses #814. * Remake update_peak_point use incremental update #814 * Make update_aerodrome_label_point use incremental update #814 * Make housenumber_centroid use incremental update #814 * Make update_continent_point use incremental update #814 * Make update_island_point use incremental update #814 * Make update_island_polygon use incremental update #814 * Remove dead code in update_state_point.sql * Make update_state_point use incremental update #814 * Remove dead code in update_country_point.sql * Make update_country_point use incremental update #814 * Make osm_poi_polygon use incremental update #814 Thanks @frodrigo
2020-08-28 09:03:27 +00:00
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
CREATE TABLE IF NOT EXISTS place_continent_point.updates
(
id serial PRIMARY KEY,
t text,
UNIQUE (t)
);
CREATE OR REPLACE FUNCTION place_continent_point.flag() RETURNS trigger AS
$$
BEGIN
INSERT INTO place_continent_point.updates(t) VALUES ('y') ON CONFLICT(t) DO NOTHING;
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION place_continent_point.refresh() RETURNS trigger AS
$$
DECLARE
t TIMESTAMP WITH TIME ZONE := clock_timestamp();
BEGIN
RAISE LOG 'Refresh place_continent_point';
-- Analyze tracking and source tables before performing update
ANALYZE place_continent_point.osm_ids;
ANALYZE osm_continent_point;
Make more simple incremental update (#944) Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) en ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. I try many code layout before done it in this way with the goal to keep the code for initial pass and for update. It should have low impact on initial data load. Better performance for row update can be achieve with BEFORE UPDATE, but require to duplicate the logic. It is not based on the already merged https://github.com/openmaptiles/openmaptiles/pull/896 because calling and update within a function for each updated row was not efficient for larger table (like housenumber). It addresses #814. * Remake update_peak_point use incremental update #814 * Make update_aerodrome_label_point use incremental update #814 * Make housenumber_centroid use incremental update #814 * Make update_continent_point use incremental update #814 * Make update_island_point use incremental update #814 * Make update_island_polygon use incremental update #814 * Remove dead code in update_state_point.sql * Make update_state_point use incremental update #814 * Remove dead code in update_country_point.sql * Make update_country_point use incremental update #814 * Make osm_poi_polygon use incremental update #814 Thanks @frodrigo
2020-08-28 09:03:27 +00:00
PERFORM update_osm_continent_point(false);
-- noinspection SqlWithoutWhere
DELETE FROM place_continent_point.osm_ids;
-- noinspection SqlWithoutWhere
DELETE FROM place_continent_point.updates;
RAISE LOG 'Refresh place_continent_point done in %', age(clock_timestamp(), t);
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
Make more simple incremental update (#944) Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) en ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. I try many code layout before done it in this way with the goal to keep the code for initial pass and for update. It should have low impact on initial data load. Better performance for row update can be achieve with BEFORE UPDATE, but require to duplicate the logic. It is not based on the already merged https://github.com/openmaptiles/openmaptiles/pull/896 because calling and update within a function for each updated row was not efficient for larger table (like housenumber). It addresses #814. * Remake update_peak_point use incremental update #814 * Make update_aerodrome_label_point use incremental update #814 * Make housenumber_centroid use incremental update #814 * Make update_continent_point use incremental update #814 * Make update_island_point use incremental update #814 * Make update_island_polygon use incremental update #814 * Remove dead code in update_state_point.sql * Make update_state_point use incremental update #814 * Remove dead code in update_country_point.sql * Make update_country_point use incremental update #814 * Make osm_poi_polygon use incremental update #814 Thanks @frodrigo
2020-08-28 09:03:27 +00:00
CREATE TRIGGER trigger_store
AFTER INSERT OR UPDATE
Make more simple incremental update (#944) Replacing update on the whole table with an update only on changed rows. The goal is to update more quickly by just updating the changing content. The update now focus on osm_id of changed rows, it use index. Add a where clause tags != update_tags(tags, geometry) en ensure only update when changed. It requires one more trigger and a table to store changed osm_id. The UPDATE is keep in a function to be reusable for initial setup and trigger update. I try many code layout before done it in this way with the goal to keep the code for initial pass and for update. It should have low impact on initial data load. Better performance for row update can be achieve with BEFORE UPDATE, but require to duplicate the logic. It is not based on the already merged https://github.com/openmaptiles/openmaptiles/pull/896 because calling and update within a function for each updated row was not efficient for larger table (like housenumber). It addresses #814. * Remake update_peak_point use incremental update #814 * Make update_aerodrome_label_point use incremental update #814 * Make housenumber_centroid use incremental update #814 * Make update_continent_point use incremental update #814 * Make update_island_point use incremental update #814 * Make update_island_polygon use incremental update #814 * Remove dead code in update_state_point.sql * Make update_state_point use incremental update #814 * Remove dead code in update_country_point.sql * Make update_country_point use incremental update #814 * Make osm_poi_polygon use incremental update #814 Thanks @frodrigo
2020-08-28 09:03:27 +00:00
ON osm_continent_point
FOR EACH ROW
EXECUTE PROCEDURE place_continent_point.store();
CREATE TRIGGER trigger_flag
AFTER INSERT OR UPDATE
ON osm_continent_point
FOR EACH STATEMENT
EXECUTE PROCEDURE place_continent_point.flag();
CREATE CONSTRAINT TRIGGER trigger_refresh
AFTER INSERT
ON place_continent_point.updates
INITIALLY DEFERRED
FOR EACH ROW
EXECUTE PROCEDURE place_continent_point.refresh();