openmaptiles/tests/test-post-import.sql

230 wiersze
9.7 KiB
MySQL

Framework for SQL-based unit tests for import and updates (#1249) This PR adds the ability to create SQL tests that ensure that OSM data is properly imported and updated in the OpenMapTiles data schema. The tests work by injecting test OSM data and updates into the database and checking to ensure that the data is properly loaded into the database using standard SQL statements. With this framework in place, developers can now write small tests to inject known data into the database and ensure that imports and updates are working correctly. In addition to the framework, basic tests are provided for four layers. These initial tests are in no way comprehensive, but they provide a structure and framework for developers to add their own tests or expand the existing ones to cover more cases. Usage: `make clean && make sql-test` ## How it works The SQL tests consist of the following parts: 1. **Test import data**, located in `tests/import`. This test data is in the [OSM XML](https://wiki.openstreetmap.org/wiki/OSM_XML) format and contains the data that should be initially injected into the database. The files are numbered in order to ensure that each test data file contains OSM id numbers that are distinct from the other files. For example, the file starting with `100` will use node ids from 100000-199999, way ids from 1000-1999, and relation ids from 100-199. 1. **Test update data**, located in `tests/update`. This test data is in the [osmChange XML](https://wiki.openstreetmap.org/wiki/OsmChange) format, and contains the data that will be used to update the test import data (in order to verify that the update process is working correctly. These files are also numbered using the same scheme as the test import data. 1. **Import SQL test script**, located at `tests/test-post-import.sql`. This script is executed after the test import data has been injected, and runs SQL-based checks to ensure that the import data was properly imported. If there are failures in the tests, an entry will be added to the table `omt_test_failures`, with one record per error that occurs during the import process. A test failure will also fail the build. To inspect the test failure messages, run `make psql` and issue the comment `SELECT * FROM omt_test_failures`. 1. **Update SQL test script**, located at `tests/test-post-update.sql`. This script performs the same function as the import test script, except that it occurs after the test update data has been applied to the database. Note that script will only run if the import script passes all tests.
2021-10-28 11:38:46 +00:00
-- Store test results
DROP TABLE IF EXISTS omt_test_failures;
CREATE TABLE omt_test_failures(
test_id integer,
test_type varchar(6),
error_message text
);
-- Checks to ensure that test data was imported correctly
DO $$
DECLARE
cnt integer;
BEGIN
-- Test 100
SELECT COUNT(*) INTO cnt FROM osm_park_polygon;
IF cnt <> 3 THEN
INSERT INTO omt_test_failures VALUES(100, 'import', 'osm_park_polygon expected 3, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_park_polygon_gen_z5;
IF cnt <> 3 THEN
INSERT INTO omt_test_failures VALUES(100, 'import', 'osm_park_polygon_gen_z5 expected 3, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_park_polygon_gen_z5 WHERE leisure='nature_reserve';
IF cnt <> 1 THEN
INSERT INTO omt_test_failures VALUES(100, 'import', 'osm_park_polygon_gen_z5 nature_reserve expected 1, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_park_polygon_gen_z5 WHERE boundary='protected_area';
IF cnt <> 1 THEN
INSERT INTO omt_test_failures VALUES(100, 'import', 'osm_park_polygon_gen_z5 protected_area expected 1, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_park_polygon_gen_z5 WHERE boundary='national_park';
IF cnt <> 1 THEN
INSERT INTO omt_test_failures VALUES(100, 'import', 'osm_park_polygon_gen_z5 national_park expected 1, got ' || cnt);
END IF;
-- Test 200
SELECT COUNT(*) INTO cnt FROM osm_aerodrome_label_point;
IF cnt <> 3 THEN
INSERT INTO omt_test_failures VALUES(200, 'import', 'osm_aerodrome_label expected 3, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_aerodrome_label_point WHERE ele='123';
Framework for SQL-based unit tests for import and updates (#1249) This PR adds the ability to create SQL tests that ensure that OSM data is properly imported and updated in the OpenMapTiles data schema. The tests work by injecting test OSM data and updates into the database and checking to ensure that the data is properly loaded into the database using standard SQL statements. With this framework in place, developers can now write small tests to inject known data into the database and ensure that imports and updates are working correctly. In addition to the framework, basic tests are provided for four layers. These initial tests are in no way comprehensive, but they provide a structure and framework for developers to add their own tests or expand the existing ones to cover more cases. Usage: `make clean && make sql-test` ## How it works The SQL tests consist of the following parts: 1. **Test import data**, located in `tests/import`. This test data is in the [OSM XML](https://wiki.openstreetmap.org/wiki/OSM_XML) format and contains the data that should be initially injected into the database. The files are numbered in order to ensure that each test data file contains OSM id numbers that are distinct from the other files. For example, the file starting with `100` will use node ids from 100000-199999, way ids from 1000-1999, and relation ids from 100-199. 1. **Test update data**, located in `tests/update`. This test data is in the [osmChange XML](https://wiki.openstreetmap.org/wiki/OsmChange) format, and contains the data that will be used to update the test import data (in order to verify that the update process is working correctly. These files are also numbered using the same scheme as the test import data. 1. **Import SQL test script**, located at `tests/test-post-import.sql`. This script is executed after the test import data has been injected, and runs SQL-based checks to ensure that the import data was properly imported. If there are failures in the tests, an entry will be added to the table `omt_test_failures`, with one record per error that occurs during the import process. A test failure will also fail the build. To inspect the test failure messages, run `make psql` and issue the comment `SELECT * FROM omt_test_failures`. 1. **Update SQL test script**, located at `tests/test-post-update.sql`. This script performs the same function as the import test script, except that it occurs after the test update data has been applied to the database. Note that script will only run if the import script passes all tests.
2021-10-28 11:38:46 +00:00
IF cnt <> 1 THEN
INSERT INTO omt_test_failures VALUES(200, 'import', 'osm_aerodrome_label ele=123 expected 1, got ' || cnt);
END IF;
-- Test 300
SELECT COUNT(*) INTO cnt FROM osm_landcover_polygon WHERE mapping_key='natural' AND subclass='wood';
IF cnt <> 1 THEN
INSERT INTO omt_test_failures VALUES(300, 'import', 'osm_landcover_polygon natural=wood expected 1, got ' || cnt);
END IF;
-- Test 400
SELECT COUNT(DISTINCT relation_id) INTO cnt FROM osm_border_linestring WHERE admin_level=8;
IF cnt <> 1 THEN
INSERT INTO omt_test_failures VALUES(400, 'import', 'osm_border_linestring city count expected 1, got ' || cnt);
Framework for SQL-based unit tests for import and updates (#1249) This PR adds the ability to create SQL tests that ensure that OSM data is properly imported and updated in the OpenMapTiles data schema. The tests work by injecting test OSM data and updates into the database and checking to ensure that the data is properly loaded into the database using standard SQL statements. With this framework in place, developers can now write small tests to inject known data into the database and ensure that imports and updates are working correctly. In addition to the framework, basic tests are provided for four layers. These initial tests are in no way comprehensive, but they provide a structure and framework for developers to add their own tests or expand the existing ones to cover more cases. Usage: `make clean && make sql-test` ## How it works The SQL tests consist of the following parts: 1. **Test import data**, located in `tests/import`. This test data is in the [OSM XML](https://wiki.openstreetmap.org/wiki/OSM_XML) format and contains the data that should be initially injected into the database. The files are numbered in order to ensure that each test data file contains OSM id numbers that are distinct from the other files. For example, the file starting with `100` will use node ids from 100000-199999, way ids from 1000-1999, and relation ids from 100-199. 1. **Test update data**, located in `tests/update`. This test data is in the [osmChange XML](https://wiki.openstreetmap.org/wiki/OsmChange) format, and contains the data that will be used to update the test import data (in order to verify that the update process is working correctly. These files are also numbered using the same scheme as the test import data. 1. **Import SQL test script**, located at `tests/test-post-import.sql`. This script is executed after the test import data has been injected, and runs SQL-based checks to ensure that the import data was properly imported. If there are failures in the tests, an entry will be added to the table `omt_test_failures`, with one record per error that occurs during the import process. A test failure will also fail the build. To inspect the test failure messages, run `make psql` and issue the comment `SELECT * FROM omt_test_failures`. 1. **Update SQL test script**, located at `tests/test-post-update.sql`. This script performs the same function as the import test script, except that it occurs after the test update data has been applied to the database. Note that script will only run if the import script passes all tests.
2021-10-28 11:38:46 +00:00
END IF;
SELECT COUNT(DISTINCT relation_id) INTO cnt FROM osm_border_linestring WHERE admin_level=2;
IF cnt <> 1 THEN
INSERT INTO omt_test_failures VALUES(400, 'import', 'osm_border_linestring country count expected 1, got ' || cnt);
END IF;
-- Test 500
-- Verify that road classifications show up at the right zoom level
SELECT COUNT(*) INTO cnt FROM osm_transportation_merge_linestring_gen_z4 WHERE osm_national_network(network);
BUGFIX: Fix name-based way fragmentation in transportation_name (#1295) I discovered this bug while investigating issues with the updates process related to #1190 #1292, and #814. The `transportation_name` layer produces slightly different `tags` hstore values in the `osm_transportation_name_linestring` table during the initial import versus when running an update. As currently written, the import code produces null-value keys in the `tags` column, while the update code suppresses them. This PR removes that difference and makes the import code use same method that is currently used in the update code. With a test case I've written, the import code produces a tags hstore that looks like this: `"name"=>"OpenMapTiles Secondary 2", "name:de"=>NULL, "name:en"=>NULL, "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"` ...while the update code produces a tags hstore that looks like this: `"name"=>"OpenMapTiles Secondary 2", "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"` Note the missing NULL values. This bug causes a small amount of space wastage after an update is run, because the update matching code detects the `tags` value as different, resulting in a duplicate copy of the tags value if that row is updated. This causes duplicate objects and breaks GROUP BY clauses that expect to group same-tagged features together. I've tested this by inspection of a generated mbtiles, database spot checks, and the unit test code included in this PR.
2021-11-25 09:45:11 +00:00
IF cnt < 1 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring z4 national network count expected >=1, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_merge_linestring_gen_z4 WHERE NOT osm_national_network(network);
IF cnt <> 0 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring z4 not national network count expected 0, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_merge_linestring_gen_z5 WHERE highway='motorway';
BUGFIX: Fix name-based way fragmentation in transportation_name (#1295) I discovered this bug while investigating issues with the updates process related to #1190 #1292, and #814. The `transportation_name` layer produces slightly different `tags` hstore values in the `osm_transportation_name_linestring` table during the initial import versus when running an update. As currently written, the import code produces null-value keys in the `tags` column, while the update code suppresses them. This PR removes that difference and makes the import code use same method that is currently used in the update code. With a test case I've written, the import code produces a tags hstore that looks like this: `"name"=>"OpenMapTiles Secondary 2", "name:de"=>NULL, "name:en"=>NULL, "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"` ...while the update code produces a tags hstore that looks like this: `"name"=>"OpenMapTiles Secondary 2", "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"` Note the missing NULL values. This bug causes a small amount of space wastage after an update is run, because the update matching code detects the `tags` value as different, resulting in a duplicate copy of the tags value if that row is updated. This causes duplicate objects and breaks GROUP BY clauses that expect to group same-tagged features together. I've tested this by inspection of a generated mbtiles, database spot checks, and the unit test code included in this PR.
2021-11-25 09:45:11 +00:00
IF cnt < 1 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring z5 motorway count expected >=1, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_merge_linestring_gen_z5 WHERE highway='trunk' AND osm_national_network(network);
IF cnt < 1 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring z5 trunk and national network count expected >=1, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_merge_linestring_gen_z6 WHERE highway='primary';
IF cnt <> 0 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring z6 primary count expected 0, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_merge_linestring_gen_z7 WHERE highway='primary';
BUGFIX: Fix name-based way fragmentation in transportation_name (#1295) I discovered this bug while investigating issues with the updates process related to #1190 #1292, and #814. The `transportation_name` layer produces slightly different `tags` hstore values in the `osm_transportation_name_linestring` table during the initial import versus when running an update. As currently written, the import code produces null-value keys in the `tags` column, while the update code suppresses them. This PR removes that difference and makes the import code use same method that is currently used in the update code. With a test case I've written, the import code produces a tags hstore that looks like this: `"name"=>"OpenMapTiles Secondary 2", "name:de"=>NULL, "name:en"=>NULL, "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"` ...while the update code produces a tags hstore that looks like this: `"name"=>"OpenMapTiles Secondary 2", "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"` Note the missing NULL values. This bug causes a small amount of space wastage after an update is run, because the update matching code detects the `tags` value as different, resulting in a duplicate copy of the tags value if that row is updated. This causes duplicate objects and breaks GROUP BY clauses that expect to group same-tagged features together. I've tested this by inspection of a generated mbtiles, database spot checks, and the unit test code included in this PR.
2021-11-25 09:45:11 +00:00
IF cnt < 1 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring z7 primary count expected >=1, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_merge_linestring_gen_z8 WHERE highway='secondary';
IF cnt <> 0 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring z8 secondary count expected 0, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_merge_linestring_gen_z9 WHERE highway='secondary';
BUGFIX: Fix name-based way fragmentation in transportation_name (#1295) I discovered this bug while investigating issues with the updates process related to #1190 #1292, and #814. The `transportation_name` layer produces slightly different `tags` hstore values in the `osm_transportation_name_linestring` table during the initial import versus when running an update. As currently written, the import code produces null-value keys in the `tags` column, while the update code suppresses them. This PR removes that difference and makes the import code use same method that is currently used in the update code. With a test case I've written, the import code produces a tags hstore that looks like this: `"name"=>"OpenMapTiles Secondary 2", "name:de"=>NULL, "name:en"=>NULL, "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"` ...while the update code produces a tags hstore that looks like this: `"name"=>"OpenMapTiles Secondary 2", "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"` Note the missing NULL values. This bug causes a small amount of space wastage after an update is run, because the update matching code detects the `tags` value as different, resulting in a duplicate copy of the tags value if that row is updated. This causes duplicate objects and breaks GROUP BY clauses that expect to group same-tagged features together. I've tested this by inspection of a generated mbtiles, database spot checks, and the unit test code included in this PR.
2021-11-25 09:45:11 +00:00
IF cnt < 1 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring z9 secondary count expected >=1, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_merge_linestring_gen_z10 WHERE highway='tertiary';
IF cnt <> 0 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring z10 tertiary count expected 0, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_merge_linestring_gen_z11 WHERE highway='tertiary';
BUGFIX: Fix name-based way fragmentation in transportation_name (#1295) I discovered this bug while investigating issues with the updates process related to #1190 #1292, and #814. The `transportation_name` layer produces slightly different `tags` hstore values in the `osm_transportation_name_linestring` table during the initial import versus when running an update. As currently written, the import code produces null-value keys in the `tags` column, while the update code suppresses them. This PR removes that difference and makes the import code use same method that is currently used in the update code. With a test case I've written, the import code produces a tags hstore that looks like this: `"name"=>"OpenMapTiles Secondary 2", "name:de"=>NULL, "name:en"=>NULL, "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"` ...while the update code produces a tags hstore that looks like this: `"name"=>"OpenMapTiles Secondary 2", "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"` Note the missing NULL values. This bug causes a small amount of space wastage after an update is run, because the update matching code detects the `tags` value as different, resulting in a duplicate copy of the tags value if that row is updated. This causes duplicate objects and breaks GROUP BY clauses that expect to group same-tagged features together. I've tested this by inspection of a generated mbtiles, database spot checks, and the unit test code included in this PR.
2021-11-25 09:45:11 +00:00
IF cnt < 1 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring z11 tertiary count expected >=1, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_merge_linestring_gen_z11 WHERE highway IN ('service', 'track');
IF cnt <> 0 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring z11 minor road count expected 0, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_merge_linestring_gen_z9
WHERE is_bridge = TRUE
AND toll = TRUE
AND layer = 1
AND bicycle = 'no'
AND foot = 'no'
AND horse = 'no';
IF cnt <> 1 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring z9 import tags expected 1, got ' || cnt);
Framework for SQL-based unit tests for import and updates (#1249) This PR adds the ability to create SQL tests that ensure that OSM data is properly imported and updated in the OpenMapTiles data schema. The tests work by injecting test OSM data and updates into the database and checking to ensure that the data is properly loaded into the database using standard SQL statements. With this framework in place, developers can now write small tests to inject known data into the database and ensure that imports and updates are working correctly. In addition to the framework, basic tests are provided for four layers. These initial tests are in no way comprehensive, but they provide a structure and framework for developers to add their own tests or expand the existing ones to cover more cases. Usage: `make clean && make sql-test` ## How it works The SQL tests consist of the following parts: 1. **Test import data**, located in `tests/import`. This test data is in the [OSM XML](https://wiki.openstreetmap.org/wiki/OSM_XML) format and contains the data that should be initially injected into the database. The files are numbered in order to ensure that each test data file contains OSM id numbers that are distinct from the other files. For example, the file starting with `100` will use node ids from 100000-199999, way ids from 1000-1999, and relation ids from 100-199. 1. **Test update data**, located in `tests/update`. This test data is in the [osmChange XML](https://wiki.openstreetmap.org/wiki/OsmChange) format, and contains the data that will be used to update the test import data (in order to verify that the update process is working correctly. These files are also numbered using the same scheme as the test import data. 1. **Import SQL test script**, located at `tests/test-post-import.sql`. This script is executed after the test import data has been injected, and runs SQL-based checks to ensure that the import data was properly imported. If there are failures in the tests, an entry will be added to the table `omt_test_failures`, with one record per error that occurs during the import process. A test failure will also fail the build. To inspect the test failure messages, run `make psql` and issue the comment `SELECT * FROM omt_test_failures`. 1. **Update SQL test script**, located at `tests/test-post-update.sql`. This script performs the same function as the import test script, except that it occurs after the test update data has been applied to the database. Note that script will only run if the import script passes all tests.
2021-10-28 11:38:46 +00:00
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_merge_linestring_gen_z9
WHERE highway = 'trunk'
AND expressway = TRUE;
IF cnt < 1 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring z9 import expressway expected >=1, got ' || cnt);
END IF;
BUGFIX: Fix name-based way fragmentation in transportation_name (#1295) I discovered this bug while investigating issues with the updates process related to #1190 #1292, and #814. The `transportation_name` layer produces slightly different `tags` hstore values in the `osm_transportation_name_linestring` table during the initial import versus when running an update. As currently written, the import code produces null-value keys in the `tags` column, while the update code suppresses them. This PR removes that difference and makes the import code use same method that is currently used in the update code. With a test case I've written, the import code produces a tags hstore that looks like this: `"name"=>"OpenMapTiles Secondary 2", "name:de"=>NULL, "name:en"=>NULL, "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"` ...while the update code produces a tags hstore that looks like this: `"name"=>"OpenMapTiles Secondary 2", "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"` Note the missing NULL values. This bug causes a small amount of space wastage after an update is run, because the update matching code detects the `tags` value as different, resulting in a duplicate copy of the tags value if that row is updated. This causes duplicate objects and breaks GROUP BY clauses that expect to group same-tagged features together. I've tested this by inspection of a generated mbtiles, database spot checks, and the unit test code included in this PR.
2021-11-25 09:45:11 +00:00
-- Same-named road split into 3 parts, because the middle segment is tagged toll=yes
SELECT COUNT(*) INTO cnt FROM osm_transportation_name_linestring WHERE tags->'name' = 'OpenMapTiles Secondary 3';
IF cnt <> 2 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_linestring split road count expected 2, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_name_linestring
WHERE tags->'name' = 'OpenMapTiles Path z13'
AND route_rank = 2;
IF cnt <> 1 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_name_linestring z13 route_rank expected 1, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_transportation_name_linestring
WHERE tags->'name' = 'OpenMapTiles Track z12'
AND route_rank = 1;
IF cnt <> 1 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'osm_transportation_name_linestring z12 route_rank expected 1, got ' || cnt);
END IF;
-- Duplicate route concurrencies collapsed
SELECT COUNT(*) INTO cnt FROM transportation_route_member_coalesced
WHERE network='US:I' AND ref='95';
IF cnt <> 1 THEN
INSERT INTO omt_test_failures VALUES(500, 'import', 'transportation_route_member_coalesced 1 route membership expected, got ' || cnt);
END IF;
-- Test 600
-- verify that atms are imported with correct name which can come from tags like operator or network
SELECT COUNT(*) INTO cnt FROM osm_poi_point
WHERE subclass = 'atm'
AND tags->'name' = 'OpenMapTiles ATM';
IF cnt <> 3 THEN
INSERT INTO omt_test_failures VALUES(600, 'import', 'osm_poi_point atm with name "OpenMapTiles ATM" expected 3, got ' || cnt);
END IF;
-- verify that parcel lockers are imported with correct name which can come from tags like brand or operator and can contain ref
SELECT COUNT(*) INTO cnt FROM osm_poi_point
WHERE subclass = 'parcel_locker'
AND tags->'name' like 'OpenMapTiles Parcel Locker%';
IF cnt <> 3 THEN
INSERT INTO omt_test_failures VALUES(600, 'import', 'osm_poi_point parcel_locker with name like "OpenMapTiles Parcel Locker%" expected 3, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_poi_point
WHERE subclass = 'parcel_locker'
AND tags->'name' like 'OpenMapTiles Parcel Locker PL00%';
IF cnt <> 1 THEN
INSERT INTO omt_test_failures VALUES(600, 'import', 'osm_poi_point parcel_locker with name like "OpenMapTiles Parcel Locker PL00%" expected 1, got ' || cnt);
END IF;
-- verify that charging stations are imported with correct name which can come from tags like brand or operator and can contain ref
SELECT COUNT(*) INTO cnt FROM osm_poi_point
WHERE subclass = 'charging_station'
AND tags->'name' = 'OpenMapTiles Charging Station';
IF cnt <> 2 THEN
INSERT INTO omt_test_failures VALUES(600, 'import', 'osm_poi_point charging_station with name "OpenMapTiles Charging Station" expected 2, got ' || cnt);
END IF;
SELECT COUNT(*) INTO cnt FROM osm_poi_polygon
WHERE subclass = 'charging_station'
AND tags->'name' = 'OpenMapTiles Charging Station Brand';
IF cnt <> 1 THEN
INSERT INTO omt_test_failures VALUES(600, 'import', 'osm_poi_polygon charging_station with name "OpenMapTiles Charging Station Brand" expected 1, got ' || cnt);
END IF;
Framework for SQL-based unit tests for import and updates (#1249) This PR adds the ability to create SQL tests that ensure that OSM data is properly imported and updated in the OpenMapTiles data schema. The tests work by injecting test OSM data and updates into the database and checking to ensure that the data is properly loaded into the database using standard SQL statements. With this framework in place, developers can now write small tests to inject known data into the database and ensure that imports and updates are working correctly. In addition to the framework, basic tests are provided for four layers. These initial tests are in no way comprehensive, but they provide a structure and framework for developers to add their own tests or expand the existing ones to cover more cases. Usage: `make clean && make sql-test` ## How it works The SQL tests consist of the following parts: 1. **Test import data**, located in `tests/import`. This test data is in the [OSM XML](https://wiki.openstreetmap.org/wiki/OSM_XML) format and contains the data that should be initially injected into the database. The files are numbered in order to ensure that each test data file contains OSM id numbers that are distinct from the other files. For example, the file starting with `100` will use node ids from 100000-199999, way ids from 1000-1999, and relation ids from 100-199. 1. **Test update data**, located in `tests/update`. This test data is in the [osmChange XML](https://wiki.openstreetmap.org/wiki/OsmChange) format, and contains the data that will be used to update the test import data (in order to verify that the update process is working correctly. These files are also numbered using the same scheme as the test import data. 1. **Import SQL test script**, located at `tests/test-post-import.sql`. This script is executed after the test import data has been injected, and runs SQL-based checks to ensure that the import data was properly imported. If there are failures in the tests, an entry will be added to the table `omt_test_failures`, with one record per error that occurs during the import process. A test failure will also fail the build. To inspect the test failure messages, run `make psql` and issue the comment `SELECT * FROM omt_test_failures`. 1. **Update SQL test script**, located at `tests/test-post-update.sql`. This script performs the same function as the import test script, except that it occurs after the test update data has been applied to the database. Note that script will only run if the import script passes all tests.
2021-10-28 11:38:46 +00:00
END;
$$
LANGUAGE plpgsql;
DO $$
DECLARE
cnt integer;
BEGIN
SELECT COUNT(*) INTO cnt FROM omt_test_failures;
IF cnt > 0 THEN
RAISE '% unit test(s) failed on imports. Details can be found in table omt_test_failures.', cnt USING ERRCODE = '0Z000';
END IF;
END;
$$;