Align Develop to PostgreSQL 11 (#121)

* Fix typo in helper script

* Port work in develop to 10 branch (#90)

* Port 9.6 to develop (#89)

* Part one of porting work from 9.6 to 10

* Backported more scripts from 9.6 branch

* Added missing apt update in dockerfile

* Updates to entrypoint to reference image and update docker-compose to reference 10 pg

* Added sample and docs from 9.6 branch

* Removed my diagram as Rizky had already added one

* Fix env paths for pg 10

* Fixes for backporting work from 9.6 to 10 - dbb now spins up and accepts connections properly

* Update README.md

* Backport from branch: 9.6-2.4

Fix default datadir

- Change into default datadir
- Add small unittest

* Optimise PostgreSQL performance and align with the changes done in 9.6
version

* Fix version numbers

* Minor change to correct env file

* Merged 10 branch into develop

* Remove reduntant conf file from dockerfile

* Remove reduntant conf file directive from setup script

* Remove reduntant conf file directive from  script

* Migrate settings to be compatible with PostgreSQL version 11

* fix ssl setup

* Align changes to postgresql version 11

* Set destroy database to false

* Commented out code for destroy database

* Fix version number for build

* Merge develop to master to align with PostgreSQL 11 (#117)

* Added note about version number

* README update

* Revert to using pg 9.3 in latest build - prevent breaking downstream apps for now

* Revert to pg 9.3

* Fix references to 9.4 to make them 9.3 rather

* Fix incorrect version in 9.4 branch

* Fix TOPOLOGY conditional typo

* Update run-postgis-docker.sh

* Update run-postgis-docker.sh

added IPADDRESS

* Updates from Marco

* Allow connections from 192.168 private network too

* start temporary server in local-only mode, poll instead of blind wait, preserve pid 1

* Enable PostGIS Out DB raster support

http://postgis.net/2015/05/02/tip_enable_raster_drivers/
http://postgis.net/docs/postgis_installation.html#install_short_version

* Remove hardcoded reference to container name "docker"

* upgrade postgres to 9.5 and postgis to 2.2

* Added flexible IP range as arg on docker run

* Added better description to docker env vars

* Updated references to pg and postgis versions in readme

* Followup 0745c488, fix references to 9.4

* problem version

postgresql version is 9.4, instead of 9.5

* Added note about allowing external ports.

* Update README.md

* Cleanups to 9.5 branch - remove old supervisor files

* Fix for README

* 9.5 2.2 (#49)

* Allow connections from 192.168 private network too

* Updated references to pg and postgis versions in readme

* Cleanups to 9.5 branch - remove old supervisor files

* Fix for README

* Dont add supervisor stuff which is deprecated

* Fix 9.4 reference

* Allow changing the default database name (#50)

* Allow changing the default database name (#50)

* Allow changing the default database name (#50) (#52)

* Commented out legacy scripts since it seems they are removed in 2.2

* 9.5 2.2 (#58)

* Allow changing the default database name (#50)

* Commented out legacy scripts since it seems they are removed in 2.2

* 9.5 2.2 (#59)

* Allow changing the default database name (#50)

* Commented out legacy scripts since it seems they are removed in 2.2

* Improve start (#57)

* Make initial dir optional

* Improve termination of background initial process #56

* Merge improvements from 9.5 branch (#60)

* Allow changing the default database name (#50)

* Commented out legacy scripts since it seems they are removed in 2.2

* 9.5 2.2 (#59)

* Allow changing the default database name (#50)

* Commented out legacy scripts since it seems they are removed in 2.2

* Improve start (#57)

* Make initial dir optional

* Improve termination of background initial process #56

* Added more options to convenience run script

* Added missing l from getopts

* Added missing l from getopts

* 9.5 2.2 (#61)

* Allow changing the default database name (#50)

* Allow changing the default database name (#50)

* Allow changing the default database name (#50) (#52)

* Commented out legacy scripts since it seems they are removed in 2.2

* Added more options to convenience run script

* Added missing l from getopts

* Added missing l from getopts

* Tweak convenience scripts (#62)


* Added missing l from getopts

* allow connection when using docker compose (#65)

see https://github.com/docker/compose/issues/4336
and https://github.com/kartoza/docker-postgis/issues/40

* Install gnupg for fetching keys first

* Fix key fetching

* Updated to 9.6 and postgis 2.4

* Ditch apt-cacher stuff

* Upgraded to PG 10

* change version from postgresql-10.0 to 10 and fix typo in run-postgis… (#76)

* change version from postgresql-10.0 to 10 and fix typo in run-postgis file

* edit setup.sh to version 10

* Change from version 10.0 to 10 in Dockerfile (#75)

I checked that postgresql-10.0 is an invalid name. It should be just postgresql-10

* Fixes for pg 10 to start nicely

* Port 9.6 to develop (#89)

* Part one of porting work from 9.6 to 10

* Backported more scripts from 9.6 branch

* Added missing apt update in dockerfile

* Updates to entrypoint to reference image and update docker-compose to reference 10 pg

* Added sample and docs from 9.6 branch

* Removed my diagram as Rizky had already added one

* Fix env paths for pg 10

* Fixes for backporting work from 9.6 to 10 - dbb now spins up and accepts connections properly

* Update README.md

* Fix #90 and replace references to slave with replicant

(Slave is a pejorative term)

* Backport from branch: 9.6-2.4

Fix default datadir

- Change into default datadir
- Add small unittest

* Merge branch 10 into develop (#113)

* Fix typo in helper script

* Port work in develop to 10 branch (#90)

* Port 9.6 to develop (#89)

* Part one of porting work from 9.6 to 10

* Backported more scripts from 9.6 branch

* Added missing apt update in dockerfile

* Updates to entrypoint to reference image and update docker-compose to reference 10 pg

* Added sample and docs from 9.6 branch

* Removed my diagram as Rizky had already added one

* Fix env paths for pg 10

* Fixes for backporting work from 9.6 to 10 - dbb now spins up and accepts connections properly

* Update README.md

* Backport from branch: 9.6-2.4

Fix default datadir

- Change into default datadir
- Add small unittest

* Optimise PostgreSQL performance and align with the changes done in 9.6
version

* Fix version numbers

* Minor change to correct env file

* Merged 10 branch into develop

* Remove reduntant conf file from dockerfile

* Remove reduntant conf file directive from setup script

* Remove reduntant conf file directive from  script

* Port changes for Postgres version 11 (#114)

* Fix typo in helper script

* Port work in develop to 10 branch (#90)

* Port 9.6 to develop (#89)

* Part one of porting work from 9.6 to 10

* Backported more scripts from 9.6 branch

* Added missing apt update in dockerfile

* Updates to entrypoint to reference image and update docker-compose to reference 10 pg

* Added sample and docs from 9.6 branch

* Removed my diagram as Rizky had already added one

* Fix env paths for pg 10

* Fixes for backporting work from 9.6 to 10 - dbb now spins up and accepts connections properly

* Update README.md

* Backport from branch: 9.6-2.4

Fix default datadir

- Change into default datadir
- Add small unittest

* Optimise PostgreSQL performance and align with the changes done in 9.6
version

* Fix version numbers

* Minor change to correct env file

* Merged 10 branch into develop

* Remove reduntant conf file from dockerfile

* Remove reduntant conf file directive from setup script

* Remove reduntant conf file directive from  script

* Migrate settings to be compatible with PostgreSQL version 11

* fix ssl setup

* Port changes to 11 branch (#115)

* Fix typo in helper script

* Port work in develop to 10 branch (#90)

* Port 9.6 to develop (#89)

* Part one of porting work from 9.6 to 10

* Backported more scripts from 9.6 branch

* Added missing apt update in dockerfile

* Updates to entrypoint to reference image and update docker-compose to reference 10 pg

* Added sample and docs from 9.6 branch

* Removed my diagram as Rizky had already added one

* Fix env paths for pg 10

* Fixes for backporting work from 9.6 to 10 - dbb now spins up and accepts connections properly

* Update README.md

* Backport from branch: 9.6-2.4

Fix default datadir

- Change into default datadir
- Add small unittest

* Optimise PostgreSQL performance and align with the changes done in 9.6
version

* Fix version numbers

* Minor change to correct env file

* Merged 10 branch into develop

* Remove reduntant conf file from dockerfile

* Remove reduntant conf file directive from setup script

* Remove reduntant conf file directive from  script

* Migrate settings to be compatible with PostgreSQL version 11

* fix ssl setup

* Align changes to postgresql version 11

* Set destroy database to false

* Commented out code for destroy database

* Fix version number for build

* Implement conf lock file check (#116)

It will make sure that the conf file will only be generated once
for a given container.

* Added new configuration in recovery.conf and postgres optimisations for master-slave replication (#118)

* Remove template logic and use plain create extension (#119)

* Remove template logic and added option to create multiple databases and extensions

* Fix travis error

* More optimisations for replication

* Fix logic for checking if database exists since we now can create
multiple databases.
pull/123/head
mazano 2019-02-20 12:28:14 +02:00 zatwierdzone przez GitHub
rodzic 5a845e2998
commit 178b4ff6da
Nie znaleziono w bazie danych klucza dla tego podpisu
ID klucza GPG: 4AEE18F83AFDEB23
7 zmienionych plików z 61 dodań i 76 usunięć

Wyświetl plik

@ -11,10 +11,11 @@ differentiates itself by:
* provides ssl support out of the box * provides ssl support out of the box
* connections are restricted to the docker subnet * connections are restricted to the docker subnet
* template_postgis database template is created for you
* a default database 'gis' is created for you so you can use this container 'out of the * a default database 'gis' is created for you so you can use this container 'out of the
box' when it runs with e.g. QGIS box' when it runs with e.g. QGIS
* replication support included * replication support included
* Ability to create multiple database when you spin the database.
* Enable multiple extensions in the database when setting it up
We will work to add more security features to this container in the future with We will work to add more security features to this container in the future with
the aim of making a PostGIS image that is ready to be used in a production the aim of making a PostGIS image that is ready to be used in a production
@ -87,11 +88,12 @@ sudo docker run --name "postgis" -p 25432:5432 -d -t kartoza/postgis
## Environment variables ## Environment variables
You can also use the following environment variables to pass a You can also use the following environment variables to pass a
user name, password and/or default database name. user name, password and/or default database name(or multiple databases coma separated).
* -e POSTGRES_USER=<PGUSER> * -e POSTGRES_USER=<PGUSER>
* -e POSTGRES_PASS=<PGPASSWORD> * -e POSTGRES_PASS=<PGPASSWORD>
* -e POSTGRES_DBNAME=<PGDBNAME> * -e POSTGRES_DBNAME=<PGDBNAME>
* -e POSTGRES_MULTIPLE_EXTENSIONS=postgis,hstore,postgis_topology # You can pass as many extensions as you need.
These will be used to create a new superuser with These will be used to create a new superuser with
your preferred credentials. If these are not specified then the postgresql your preferred credentials. If these are not specified then the postgresql

Wyświetl plik

@ -1,6 +1,6 @@
# docker-compose build # docker-compose build
# docker-compose up -d web # docker-compose up -d web
version: '2' version: '2.1'
volumes: volumes:
dbbackups: dbbackups:
postgis-data: postgis-data:
@ -8,21 +8,26 @@ volumes:
services: services:
db: db:
image: kartoza/postgis:10.0-2.4 image: kartoza/postgis:11.0-2.5
volumes: volumes:
- 'postgis-data:/var/lib/postgresql' - postgis-data:/var/lib/postgresql
- 'dbbackups:/backups' - dbbackups:/backups
environment: environment:
# If you need to create multiple database you can add coma separated databases eg gis,data
- POSTGRES_DB=gis - POSTGRES_DB=gis
- POSTGRES_USER=docker - POSTGRES_USER=docker
- POSTGRES_PASS=docker - POSTGRES_PASS=docker
- ALLOW_IP_RANGE=0.0.0.0/0 - ALLOW_IP_RANGE=0.0.0.0/0
# Add extensions you need to be enabled by default in the DB. Default are the three specified below
#- POSTGRES_MULTIPLE_EXTENSIONS=postgis,hstore,postgis_topology
ports: ports:
- 25432:5432 - 5434:5432
restart: unless-stopped restart: on-failure
healthcheck:
test: "exit 0"
dbbackups: dbbackups:
image: kartoza/pg-backup:10.0 image: kartoza/pg-backup:11.0
hostname: pg-backups hostname: pg-backups
volumes: volumes:
- dbbackups:/backups - dbbackups:/backups
@ -35,4 +40,7 @@ services:
- PGDATABASE=gis - PGDATABASE=gis
- PGPORT=5432 - PGPORT=5432
- PGHOST=db - PGHOST=db
restart: unless-stopped restart: on-failure
depends_on:
db:
condition: service_healthy

Wyświetl plik

@ -44,16 +44,19 @@ if [ -z "${DESTROY_DATABASE_ON_RESTART}" ]; then
DESTROY_DATABASE_ON_RESTART=true DESTROY_DATABASE_ON_RESTART=true
fi fi
if [ -z "${PG_MAX_WAL_SENDERS}" ]; then if [ -z "${PG_MAX_WAL_SENDERS}" ]; then
PG_MAX_WAL_SENDERS=8 PG_MAX_WAL_SENDERS=10
fi fi
if [ -z "${PG_WAL_KEEP_SEGMENTS}" ]; then if [ -z "${PG_WAL_KEEP_SEGMENTS}" ]; then
PG_WAL_KEEP_SEGMENTS=100 PG_WAL_KEEP_SEGMENTS=250
fi fi
if [ -z "${IP_LIST}" ]; then if [ -z "${IP_LIST}" ]; then
IP_LIST='*' IP_LIST='*'
fi fi
if [ -z "${POSTGRES_MULTIPLE_EXTENSIONS}" ]; then
POSTGRES_MULTIPLE_EXTENSIONS='postgis,hstore,postgis_topology'
fi
# Compatibility with official postgres variable # Compatibility with official postgres variable
# Official postgres variable gets priority # Official postgres variable gets priority
if [ ! -z "${POSTGRES_PASSWORD}" ]; then if [ ! -z "${POSTGRES_PASSWORD}" ]; then

Wyświetl plik

@ -24,6 +24,9 @@ services:
# We can specify optional credentials # We can specify optional credentials
POSTGRES_USER: 'docker' POSTGRES_USER: 'docker'
POSTGRES_PASS: 'docker' POSTGRES_PASS: 'docker'
# Setup master replication variables
#PG_MAX_WAL_SENDERS: 8
#PG_WAL_KEEP_SEGMENTS: 100
# You can expose the port to observe it in your local machine # You can expose the port to observe it in your local machine
ports: ports:
- "7777:5432" - "7777:5432"

Wyświetl plik

@ -15,21 +15,21 @@ cat $CONF.template > $CONF
# This script will setup necessary configuration to optimise for PostGIS and to enable replications # This script will setup necessary configuration to optimise for PostGIS and to enable replications
cat >> $CONF <<EOF cat >> $CONF <<EOF
wal_level = hot_standby wal_level = hot_standby
max_wal_senders = $PG_MAX_WAL_SENDERS max_wal_senders = ${PG_MAX_WAL_SENDERS}
wal_keep_segments = $PG_WAL_KEEP_SEGMENTS wal_keep_segments = ${PG_WAL_KEEP_SEGMENTS}
superuser_reserved_connections= 10
max_wal_size= 2GB
wal_keep_segments= 64
hot_standby = on hot_standby = on
listen_addresses = '${IP_LIST}' listen_addresses = '${IP_LIST}'
shared_buffers = 500MB shared_buffers = 500MB
work_mem = 16MB work_mem = 16MB
maintenance_work_mem = 128MB maintenance_work_mem = 128MB
wal_buffers = 1MB wal_buffers = 1MB
# uncomment checkpoint_segments below if postgresql <= 9.4
# checkpoint_segments = 6
random_page_cost = 2.0 random_page_cost = 2.0
xmloption = 'document' xmloption = 'document'
#archive_mode=on #archive_mode=on
#archive_command = 'test ! -f ${WAL_ARCHIVE}/%f && cp -r %p ${WAL_ARCHIVE}/%f' #archive_command = 'test ! -f ${WAL_ARCHIVE}/%f && cp -r %p ${WAL_ARCHIVE}/%f'
#user-settings
EOF EOF
# Optimise PostgreSQL shared memory for PostGIS # Optimise PostgreSQL shared memory for PostGIS

Wyświetl plik

@ -6,8 +6,8 @@ source /env-data.sh
# test if DATADIR is existent # test if DATADIR is existent
if [[ ! -d ${DATADIR} ]]; then if [[ ! -d ${DATADIR} ]]; then
echo "Creating Postgres data at ${DATADIR}" echo "Creating Postgres data at ${DATADIR}"
mkdir -p ${DATADIR} mkdir -p ${DATADIR}
fi fi
@ -18,12 +18,12 @@ chown -R postgres:postgres ${DATADIR}
# test if DATADIR has content # test if DATADIR has content
if [[ ! "$(ls -A ${DATADIR})" ]]; then if [[ ! "$(ls -A ${DATADIR})" ]]; then
# No content yet - first time pg is being run! # No content yet - first time pg is being run!
# No Replicate From settings. Assume that this is a master database. # No Replicate From settings. Assume that this is a master database.
# Initialise db # Initialise db
echo "Initializing Postgres Database at ${DATADIR}" echo "Initializing Postgres Database at ${DATADIR}"
#chown -R postgres $DATADIR #chown -R postgres $DATADIR
su - postgres -c "$INITDB ${DATADIR}" su - postgres -c "$INITDB ${DATADIR}"
fi fi
# test database existing # test database existing
@ -35,67 +35,34 @@ su - postgres -c "${POSTGRES} -D ${DATADIR} -c config_file=${CONF} ${LOCALONLY}
# wait for postgres to come up # wait for postgres to come up
until su - postgres -c "psql -l"; do until su - postgres -c "psql -l"; do
sleep 1 sleep 1
done done
echo "postgres ready" echo "postgres ready"
RESULT=`su - postgres -c "psql -l | grep -w template_postgis | wc -l"`
if [[ ${RESULT} == '1' ]]
then
echo 'Postgis Already There'
if [[ ${HSTORE} == "true" ]]; then
echo 'HSTORE is only useful when you create the postgis database.'
fi
if [[ ${TOPOLOGY} == "true" ]]; then
echo 'TOPOLOGY is only useful when you create the postgis database.'
fi
else
echo "Postgis is missing, installing now"
# Note the dockerfile must have put the postgis.sql and spatialrefsys.sql scripts into /root/
# We use template0 since we want different encoding to template1
echo "Creating template postgis"
su - postgres -c "createdb template_postgis -E UTF8 -T template0"
echo "Enabling template_postgis as a template"
CMD="UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template_postgis';"
su - postgres -c "psql -c \"$CMD\""
echo "Loading postgis extension"
su - postgres -c "psql template_postgis -c 'CREATE EXTENSION postgis;'"
if [[ ${HSTORE} == "true" ]]
then
echo "Enabling hstore in the template"
su - postgres -c "psql template_postgis -c 'CREATE EXTENSION hstore;'"
fi
if [[ ${TOPOLOGY} == "true" ]]
then
echo "Enabling topology in the template"
su - postgres -c "psql template_postgis -c 'CREATE EXTENSION postgis_topology;'"
fi
# Needed when importing old dumps using e.g ndims for constraints
# Ignore error if it doesn't exists
echo "Loading legacy sql"
su - postgres -c "psql template_postgis -f ${SQLDIR}/legacy_minimal.sql" || true
su - postgres -c "psql template_postgis -f ${SQLDIR}/legacy_gist.sql" || true
fi
# Setup user # Setup user
source /setup-user.sh source /setup-user.sh
# Create a default db called 'gis' or $POSTGRES_DBNAME that you can use to get up and running quickly # Create a default db called 'gis' or $POSTGRES_DBNAME that you can use to get up and running quickly
# It will be owned by the docker db user # It will be owned by the docker db user
# Since we now pass a comma separated list in database creation we need to search for all databases as a test
RESULT=`su - postgres -c "psql -l | grep -w ${POSTGRES_DBNAME} | wc -l"` for db in $(echo ${POSTGRES_DBNAME} | tr ',' ' '); do
echo "Check default db exists" RESULT=`su - postgres -c "psql -l | grep -w ${db} | wc -l"`
if [[ ! ${RESULT} == '1' ]]; then if [[ ! ${RESULT} == '1' ]]; then
echo "Create default db ${POSTGRES_DBNAME}" echo "Create db ${db}"
su - postgres -c "createdb -O ${POSTGRES_USER} -T template_postgis ${POSTGRES_DBNAME}" su - postgres -c "createdb -O ${POSTGRES_USER} ${db}"
else for ext in $(echo ${POSTGRES_MULTIPLE_EXTENSIONS} | tr ',' ' '); do
echo "${POSTGRES_DBNAME} db already exists" echo "Enabling ${ext} in the database ${db}"
fi su - postgres -c "psql -c 'CREATE EXTENSION IF NOT EXISTS ${ext} cascade;' $db"
done
echo "Loading legacy sql"
su - postgres -c "psql ${db} -f ${SQLDIR}/legacy_minimal.sql" || true
su - postgres -c "psql ${db} -f ${SQLDIR}/legacy_gist.sql" || true
else
echo "${db} db already exists"
fi
done
# This should show up in docker logs afterwards # This should show up in docker logs afterwards
su - postgres -c "psql -l" su - postgres -c "psql -l"

Wyświetl plik

@ -74,6 +74,8 @@ cat > ${DATADIR}/recovery.conf <<EOF
standby_mode = on standby_mode = on
primary_conninfo = 'host=${REPLICATE_FROM} port=${REPLICATE_PORT} user=${POSTGRES_USER} password=${POSTGRES_PASS} sslmode=${PGSSLMODE}' primary_conninfo = 'host=${REPLICATE_FROM} port=${REPLICATE_PORT} user=${POSTGRES_USER} password=${POSTGRES_PASS} sslmode=${PGSSLMODE}'
trigger_file = '${PROMOTE_FILE}' trigger_file = '${PROMOTE_FILE}'
recovery_target_timeline='latest'
recovery_target_action='promote'
#restore_command = 'cp /opt/archive/%f "%p"' Use if you are syncing the wal segments from master #restore_command = 'cp /opt/archive/%f "%p"' Use if you are syncing the wal segments from master
EOF EOF
# Setup permissions. Postgres won't start without this. # Setup permissions. Postgres won't start without this.