From 178b4ff6dacbc829c662bf6a2a96b9e715c1147d Mon Sep 17 00:00:00 2001 From: mazano Date: Wed, 20 Feb 2019 12:28:14 +0200 Subject: [PATCH] Align Develop to PostgreSQL 11 (#121) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Fix typo in helper script * Port work in develop to 10 branch (#90) * Port 9.6 to develop (#89) * Part one of porting work from 9.6 to 10 * Backported more scripts from 9.6 branch * Added missing apt update in dockerfile * Updates to entrypoint to reference image and update docker-compose to reference 10 pg * Added sample and docs from 9.6 branch * Removed my diagram as Rizky had already added one * Fix env paths for pg 10 * Fixes for backporting work from 9.6 to 10 - dbb now spins up and accepts connections properly * Update README.md * Backport from branch: 9.6-2.4 Fix default datadir - Change into default datadir - Add small unittest * Optimise PostgreSQL performance and align with the changes done in 9.6 version * Fix version numbers * Minor change to correct env file * Merged 10 branch into develop * Remove reduntant conf file from dockerfile * Remove reduntant conf file directive from setup script * Remove reduntant conf file directive from script * Migrate settings to be compatible with PostgreSQL version 11 * fix ssl setup * Align changes to postgresql version 11 * Set destroy database to false * Commented out code for destroy database * Fix version number for build * Merge develop to master to align with PostgreSQL 11 (#117) * Added note about version number * README update * Revert to using pg 9.3 in latest build - prevent breaking downstream apps for now * Revert to pg 9.3 * Fix references to 9.4 to make them 9.3 rather * Fix incorrect version in 9.4 branch * Fix TOPOLOGY conditional typo * Update run-postgis-docker.sh * Update run-postgis-docker.sh added IPADDRESS * Updates from Marco * Allow connections from 192.168 private network too * start temporary server in local-only mode, poll instead of blind wait, preserve pid 1 * Enable PostGIS Out DB raster support http://postgis.net/2015/05/02/tip_enable_raster_drivers/ http://postgis.net/docs/postgis_installation.html#install_short_version * Remove hardcoded reference to container name "docker" * upgrade postgres to 9.5 and postgis to 2.2 * Added flexible IP range as arg on docker run * Added better description to docker env vars * Updated references to pg and postgis versions in readme * Followup 0745c488, fix references to 9.4 * problem version postgresql version is 9.4, instead of 9.5 * Added note about allowing external ports. * Update README.md * Cleanups to 9.5 branch - remove old supervisor files * Fix for README * 9.5 2.2 (#49) * Allow connections from 192.168 private network too * Updated references to pg and postgis versions in readme * Cleanups to 9.5 branch - remove old supervisor files * Fix for README * Dont add supervisor stuff which is deprecated * Fix 9.4 reference * Allow changing the default database name (#50) * Allow changing the default database name (#50) * Allow changing the default database name (#50) (#52) * Commented out legacy scripts since it seems they are removed in 2.2 * 9.5 2.2 (#58) * Allow changing the default database name (#50) * Commented out legacy scripts since it seems they are removed in 2.2 * 9.5 2.2 (#59) * Allow changing the default database name (#50) * Commented out legacy scripts since it seems they are removed in 2.2 * Improve start (#57) * Make initial dir optional * Improve termination of background initial process #56 * Merge improvements from 9.5 branch (#60) * Allow changing the default database name (#50) * Commented out legacy scripts since it seems they are removed in 2.2 * 9.5 2.2 (#59) * Allow changing the default database name (#50) * Commented out legacy scripts since it seems they are removed in 2.2 * Improve start (#57) * Make initial dir optional * Improve termination of background initial process #56 * Added more options to convenience run script * Added missing l from getopts * Added missing l from getopts * 9.5 2.2 (#61) * Allow changing the default database name (#50) * Allow changing the default database name (#50) * Allow changing the default database name (#50) (#52) * Commented out legacy scripts since it seems they are removed in 2.2 * Added more options to convenience run script * Added missing l from getopts * Added missing l from getopts * Tweak convenience scripts (#62) * Added missing l from getopts * allow connection when using docker compose (#65) see https://github.com/docker/compose/issues/4336 and https://github.com/kartoza/docker-postgis/issues/40 * Install gnupg for fetching keys first * Fix key fetching * Updated to 9.6 and postgis 2.4 * Ditch apt-cacher stuff * Upgraded to PG 10 * change version from postgresql-10.0 to 10 and fix typo in run-postgis… (#76) * change version from postgresql-10.0 to 10 and fix typo in run-postgis file * edit setup.sh to version 10 * Change from version 10.0 to 10 in Dockerfile (#75) I checked that postgresql-10.0 is an invalid name. It should be just postgresql-10 * Fixes for pg 10 to start nicely * Port 9.6 to develop (#89) * Part one of porting work from 9.6 to 10 * Backported more scripts from 9.6 branch * Added missing apt update in dockerfile * Updates to entrypoint to reference image and update docker-compose to reference 10 pg * Added sample and docs from 9.6 branch * Removed my diagram as Rizky had already added one * Fix env paths for pg 10 * Fixes for backporting work from 9.6 to 10 - dbb now spins up and accepts connections properly * Update README.md * Fix #90 and replace references to slave with replicant (Slave is a pejorative term) * Backport from branch: 9.6-2.4 Fix default datadir - Change into default datadir - Add small unittest * Merge branch 10 into develop (#113) * Fix typo in helper script * Port work in develop to 10 branch (#90) * Port 9.6 to develop (#89) * Part one of porting work from 9.6 to 10 * Backported more scripts from 9.6 branch * Added missing apt update in dockerfile * Updates to entrypoint to reference image and update docker-compose to reference 10 pg * Added sample and docs from 9.6 branch * Removed my diagram as Rizky had already added one * Fix env paths for pg 10 * Fixes for backporting work from 9.6 to 10 - dbb now spins up and accepts connections properly * Update README.md * Backport from branch: 9.6-2.4 Fix default datadir - Change into default datadir - Add small unittest * Optimise PostgreSQL performance and align with the changes done in 9.6 version * Fix version numbers * Minor change to correct env file * Merged 10 branch into develop * Remove reduntant conf file from dockerfile * Remove reduntant conf file directive from setup script * Remove reduntant conf file directive from script * Port changes for Postgres version 11 (#114) * Fix typo in helper script * Port work in develop to 10 branch (#90) * Port 9.6 to develop (#89) * Part one of porting work from 9.6 to 10 * Backported more scripts from 9.6 branch * Added missing apt update in dockerfile * Updates to entrypoint to reference image and update docker-compose to reference 10 pg * Added sample and docs from 9.6 branch * Removed my diagram as Rizky had already added one * Fix env paths for pg 10 * Fixes for backporting work from 9.6 to 10 - dbb now spins up and accepts connections properly * Update README.md * Backport from branch: 9.6-2.4 Fix default datadir - Change into default datadir - Add small unittest * Optimise PostgreSQL performance and align with the changes done in 9.6 version * Fix version numbers * Minor change to correct env file * Merged 10 branch into develop * Remove reduntant conf file from dockerfile * Remove reduntant conf file directive from setup script * Remove reduntant conf file directive from script * Migrate settings to be compatible with PostgreSQL version 11 * fix ssl setup * Port changes to 11 branch (#115) * Fix typo in helper script * Port work in develop to 10 branch (#90) * Port 9.6 to develop (#89) * Part one of porting work from 9.6 to 10 * Backported more scripts from 9.6 branch * Added missing apt update in dockerfile * Updates to entrypoint to reference image and update docker-compose to reference 10 pg * Added sample and docs from 9.6 branch * Removed my diagram as Rizky had already added one * Fix env paths for pg 10 * Fixes for backporting work from 9.6 to 10 - dbb now spins up and accepts connections properly * Update README.md * Backport from branch: 9.6-2.4 Fix default datadir - Change into default datadir - Add small unittest * Optimise PostgreSQL performance and align with the changes done in 9.6 version * Fix version numbers * Minor change to correct env file * Merged 10 branch into develop * Remove reduntant conf file from dockerfile * Remove reduntant conf file directive from setup script * Remove reduntant conf file directive from script * Migrate settings to be compatible with PostgreSQL version 11 * fix ssl setup * Align changes to postgresql version 11 * Set destroy database to false * Commented out code for destroy database * Fix version number for build * Implement conf lock file check (#116) It will make sure that the conf file will only be generated once for a given container. * Added new configuration in recovery.conf and postgres optimisations for master-slave replication (#118) * Remove template logic and use plain create extension (#119) * Remove template logic and added option to create multiple databases and extensions * Fix travis error * More optimisations for replication * Fix logic for checking if database exists since we now can create multiple databases. --- README.md | 6 +- docker-compose.yml | 24 +++++--- env-data.sh | 7 ++- sample/replication/docker-compose.yml | 3 + setup-conf.sh | 10 ++-- setup-database.sh | 85 ++++++++------------------- setup-replication.sh | 2 + 7 files changed, 61 insertions(+), 76 deletions(-) diff --git a/README.md b/README.md index 9b2f1a6..7d7a182 100644 --- a/README.md +++ b/README.md @@ -11,10 +11,11 @@ differentiates itself by: * provides ssl support out of the box * connections are restricted to the docker subnet -* template_postgis database template is created for you * a default database 'gis' is created for you so you can use this container 'out of the box' when it runs with e.g. QGIS * replication support included +* Ability to create multiple database when you spin the database. +* Enable multiple extensions in the database when setting it up We will work to add more security features to this container in the future with the aim of making a PostGIS image that is ready to be used in a production @@ -87,11 +88,12 @@ sudo docker run --name "postgis" -p 25432:5432 -d -t kartoza/postgis ## Environment variables You can also use the following environment variables to pass a -user name, password and/or default database name. +user name, password and/or default database name(or multiple databases coma separated). * -e POSTGRES_USER= * -e POSTGRES_PASS= * -e POSTGRES_DBNAME= +* -e POSTGRES_MULTIPLE_EXTENSIONS=postgis,hstore,postgis_topology # You can pass as many extensions as you need. These will be used to create a new superuser with your preferred credentials. If these are not specified then the postgresql diff --git a/docker-compose.yml b/docker-compose.yml index 09a3ab3..40f60e9 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,6 +1,6 @@ # docker-compose build # docker-compose up -d web -version: '2' +version: '2.1' volumes: dbbackups: postgis-data: @@ -8,21 +8,26 @@ volumes: services: db: - image: kartoza/postgis:10.0-2.4 + image: kartoza/postgis:11.0-2.5 volumes: - - 'postgis-data:/var/lib/postgresql' - - 'dbbackups:/backups' + - postgis-data:/var/lib/postgresql + - dbbackups:/backups environment: + # If you need to create multiple database you can add coma separated databases eg gis,data - POSTGRES_DB=gis - POSTGRES_USER=docker - POSTGRES_PASS=docker - ALLOW_IP_RANGE=0.0.0.0/0 + # Add extensions you need to be enabled by default in the DB. Default are the three specified below + #- POSTGRES_MULTIPLE_EXTENSIONS=postgis,hstore,postgis_topology ports: - - 25432:5432 - restart: unless-stopped + - 5434:5432 + restart: on-failure + healthcheck: + test: "exit 0" dbbackups: - image: kartoza/pg-backup:10.0 + image: kartoza/pg-backup:11.0 hostname: pg-backups volumes: - dbbackups:/backups @@ -35,4 +40,7 @@ services: - PGDATABASE=gis - PGPORT=5432 - PGHOST=db - restart: unless-stopped + restart: on-failure + depends_on: + db: + condition: service_healthy diff --git a/env-data.sh b/env-data.sh index be1731b..f2e8bc3 100644 --- a/env-data.sh +++ b/env-data.sh @@ -44,16 +44,19 @@ if [ -z "${DESTROY_DATABASE_ON_RESTART}" ]; then DESTROY_DATABASE_ON_RESTART=true fi if [ -z "${PG_MAX_WAL_SENDERS}" ]; then - PG_MAX_WAL_SENDERS=8 + PG_MAX_WAL_SENDERS=10 fi if [ -z "${PG_WAL_KEEP_SEGMENTS}" ]; then - PG_WAL_KEEP_SEGMENTS=100 + PG_WAL_KEEP_SEGMENTS=250 fi if [ -z "${IP_LIST}" ]; then IP_LIST='*' fi +if [ -z "${POSTGRES_MULTIPLE_EXTENSIONS}" ]; then + POSTGRES_MULTIPLE_EXTENSIONS='postgis,hstore,postgis_topology' +fi # Compatibility with official postgres variable # Official postgres variable gets priority if [ ! -z "${POSTGRES_PASSWORD}" ]; then diff --git a/sample/replication/docker-compose.yml b/sample/replication/docker-compose.yml index b424b7e..08c94f9 100644 --- a/sample/replication/docker-compose.yml +++ b/sample/replication/docker-compose.yml @@ -24,6 +24,9 @@ services: # We can specify optional credentials POSTGRES_USER: 'docker' POSTGRES_PASS: 'docker' + # Setup master replication variables + #PG_MAX_WAL_SENDERS: 8 + #PG_WAL_KEEP_SEGMENTS: 100 # You can expose the port to observe it in your local machine ports: - "7777:5432" diff --git a/setup-conf.sh b/setup-conf.sh index 6de4912..73034f3 100644 --- a/setup-conf.sh +++ b/setup-conf.sh @@ -15,21 +15,21 @@ cat $CONF.template > $CONF # This script will setup necessary configuration to optimise for PostGIS and to enable replications cat >> $CONF < ${DATADIR}/recovery.conf <