kopia lustrzana https://github.com/kartoza/docker-postgis
fix promote to master in streaming replication (#370)
* fix promote to master in streaming replication * remove tests in example compose * Fix grammar errorpull/371/head
rodzic
55137f4418
commit
2be8456411
38
README.md
38
README.md
|
@ -597,7 +597,7 @@ The two main replication methods allowed are
|
|||
|
||||
|
||||
### Database permissions and password authentication
|
||||
Replication uses a dedicated user `REPLICATION_USER`. The role ${REPLICATION_USER}
|
||||
Replication uses a dedicated user `REPLICATION_USER`. The role `${REPLICATION_USER}`
|
||||
uses the default group role `pg_read_all_data`. You can read more about this
|
||||
from the [PostgreSQL documentation](https://www.postgresql.org/docs/14/predefined-roles.html)
|
||||
|
||||
|
@ -616,7 +616,7 @@ layer are saved, they are automatically propagated to the replicant. Note also t
|
|||
replicant is read-only.
|
||||
|
||||
```shell
|
||||
docker run --name "streaming-replication" -e REPLICATION=true -e WAL_LEVEL='replica' -d -p 25432:5432 kartoza/postgis:13.0
|
||||
docker run --name "streaming-replication" -e REPLICATION=true -e WAL_LEVEL='replica' -d -p 25432:5432 kartoza/postgis:14.3.2
|
||||
```
|
||||
|
||||
**Note** If you do not pass the env variable `REPLICATION_PASS` a random password
|
||||
|
@ -635,20 +635,20 @@ we can't write new data to it. The whole database cluster will be replicated.
|
|||
|
||||
#### Database permissions
|
||||
|
||||
Since we are using a role ${REPLICATION_USER}, we need to ensure that it has access to all
|
||||
Since we are using a role `${REPLICATION_USER}`, we need to ensure that it has access to all
|
||||
the tables in a particular schema. So if a user adds another schema called `data`
|
||||
to the database `gis` he also has to update the permission for the user
|
||||
with the following SQL assuming the ${REPLICATION_USER} is called replicator
|
||||
with the following SQL assuming the `${REPLICATION_USER}` is called replicator
|
||||
|
||||
```sql
|
||||
ALTER DEFAULT PRIVILEGES IN SCHEMA data GRANT SELECT ON TABLES TO replicator;
|
||||
```
|
||||
|
||||
**NB** You need to set up a strong password for replication otherwise the
|
||||
default password for ${REPLICATION_USER} will default to `replicator`
|
||||
**Note** You need to set up a strong password for replication otherwise the
|
||||
default password for `${REPLICATION_USER}` will default to random generated string
|
||||
|
||||
To experiment with the replication abilities, you can see a [docker-compose.yml](sample/replication/docker-compose.yml)
|
||||
sample. There are several environment variables that you can set, such as:
|
||||
To experiment with the streaming replication abilities, you can see a [docker-compose.yml](replication_examples/replication/docker-compose.yml).
|
||||
There are several environment variables that you can set, such as:
|
||||
|
||||
Master settings:
|
||||
- **ALLOW_IP_RANGE**: A `pg_hba.conf` domain format which will allow specified host(s)
|
||||
|
@ -676,7 +676,7 @@ Slave settings:
|
|||
- **REPLICATION_USER** User to initiate streaming replication
|
||||
- **REPLICATION_PASS** Password for a user with streaming replication role
|
||||
|
||||
To run the sample replication, follow these instructions:
|
||||
To run the example streaming_replication, follow these instructions:
|
||||
|
||||
Do a manual image build by executing the `build.sh` script
|
||||
|
||||
|
@ -684,7 +684,7 @@ Do a manual image build by executing the `build.sh` script
|
|||
./build.sh
|
||||
```
|
||||
|
||||
Go into the `sample/replication` directory and experiment with the following Make
|
||||
Go into the `replication_examples/streaming_replication` directory and experiment with the following Make
|
||||
command to run both master and slave services.
|
||||
|
||||
```shell
|
||||
|
@ -701,7 +701,7 @@ To view logs for master and slave respectively, use the following command:
|
|||
|
||||
```shell
|
||||
make master-log
|
||||
make slave-log
|
||||
make node-log
|
||||
```
|
||||
|
||||
You can try experiment with several scenarios to see how replication works
|
||||
|
@ -709,15 +709,15 @@ You can try experiment with several scenarios to see how replication works
|
|||
#### Sync changes from master to replicant
|
||||
|
||||
You can use any postgres database tools to create new tables in master, by
|
||||
connecting using POSTGRES_USER and POSTGRES_PASS credentials using exposed port.
|
||||
In the sample, the master database was exposed on port 7777.
|
||||
connecting using `POSTGRES_USER` and `POSTGRES_PASS` credentials using exposed port.
|
||||
In the streaming_replication example, the master database was exposed on port 7777.
|
||||
Or you can do it via command line, by entering the shell:
|
||||
|
||||
```shell
|
||||
make master-shell
|
||||
```
|
||||
|
||||
Then made any database changes using psql.
|
||||
Then make any database changes using psql.
|
||||
|
||||
After that, you can see that the replicant follows the changes by inspecting the
|
||||
slave database. You can, again, use database management tools using connection
|
||||
|
@ -725,7 +725,7 @@ credentials, hostname, and ports for replicant. Or you can do it via command lin
|
|||
by entering the shell:
|
||||
|
||||
```shell
|
||||
make slave-shell
|
||||
make node-shell
|
||||
```
|
||||
|
||||
Then view your changes using psql.
|
||||
|
@ -738,14 +738,14 @@ into slave environment and set `DESTROY_DATABASE_ON_RESTART: 'False'`.
|
|||
|
||||
After this, you can make changes to your replicant, but master and replicant will not
|
||||
be in sync anymore. This is useful if the replicant needs to take over a failover master.
|
||||
However it is recommended to take additional action, such as creating a backup from the
|
||||
However, it is recommended to take additional action, such as creating a backup from the
|
||||
slave so a dedicated master can be created again.
|
||||
|
||||
#### Preventing replicant database destroy on restart
|
||||
|
||||
You can optionally set `DESTROY_DATABASE_ON_RESTART: 'False'` after successful sync
|
||||
to prevent the database from being destroyed on restart. With this setting you can
|
||||
shut down your replicant and restart it later and it will continue to sync using the existing
|
||||
shut down your replicant and restart it later, and it will continue to sync using the existing
|
||||
database (as long as there are no consistencies conflicts).
|
||||
|
||||
However, you should note that this option doesn't mean anything if you didn't
|
||||
|
@ -762,7 +762,7 @@ To activate the following you need to use the environment variable
|
|||
docker run --name "logical-replication" -e WAL_LEVEL=logical -d kartoza/postgis:13.0
|
||||
```
|
||||
|
||||
For a detailed example see the docker-compose in the folder `sample/logical_replication`.
|
||||
For a detailed example see the docker-compose in the folder `replication_examples/logical_replication`.
|
||||
|
||||
### Docker image versions
|
||||
|
||||
|
@ -786,4 +786,4 @@ please consider taking out a [Support Level Agreeement](https://kartoza.com/en/s
|
|||
- Rizky Maulana (rizky@kartoza.com)
|
||||
- Admire Nyakudya (admire@kartoza.com)
|
||||
|
||||
March 2021
|
||||
April 2022
|
||||
|
|
|
@ -1,5 +1,5 @@
|
|||
# docker-compose build
|
||||
version: '2.1'
|
||||
version: '3.9'
|
||||
volumes:
|
||||
dbbackups:
|
||||
postgis-data:
|
||||
|
@ -7,7 +7,7 @@ volumes:
|
|||
services:
|
||||
|
||||
db:
|
||||
image: kartoza/postgis:14-3.1
|
||||
image: kartoza/postgis:14-3.2
|
||||
volumes:
|
||||
- postgis-data:/var/lib/postgresql
|
||||
- dbbackups:/backups
|
||||
|
@ -20,13 +20,13 @@ services:
|
|||
# Add extensions you need to be enabled by default in the DB. Default are the five specified below
|
||||
- POSTGRES_MULTIPLE_EXTENSIONS=postgis,hstore,postgis_topology,postgis_raster,pgrouting
|
||||
ports:
|
||||
- 25432:5432
|
||||
- "25432:5432"
|
||||
restart: on-failure
|
||||
healthcheck:
|
||||
test: "exit 0"
|
||||
|
||||
dbbackups:
|
||||
image: kartoza/pg-backup:14-3.1
|
||||
image: kartoza/pg-backup:14-3.2
|
||||
hostname: pg-backups
|
||||
volumes:
|
||||
- dbbackups:/backups
|
||||
|
@ -36,7 +36,6 @@ services:
|
|||
- POSTGRES_PASS=docker
|
||||
- POSTGRES_PORT=5432
|
||||
- POSTGRES_HOST=db
|
||||
- POSTGRES_DBNAME=gis
|
||||
restart: on-failure
|
||||
depends_on:
|
||||
db:
|
||||
|
|
|
@ -0,0 +1,27 @@
|
|||
up:
|
||||
docker-compose up -d
|
||||
|
||||
down:
|
||||
docker-compose down
|
||||
|
||||
scale:
|
||||
docker-compose up -d --scale pg-node=3
|
||||
|
||||
unscale:
|
||||
docker-compose up -d --scale pg-node=1
|
||||
|
||||
status:
|
||||
docker-compose ps
|
||||
|
||||
master-shell:
|
||||
docker-compose exec pg-master /bin/bash
|
||||
|
||||
node-shell:
|
||||
docker-compose exec pg-node /bin/bash
|
||||
|
||||
master-log:
|
||||
docker-compose logs -f --tail=30 pg-master
|
||||
|
||||
node-log:
|
||||
docker-compose logs -f --tail=30 pg-node
|
||||
|
|
@ -1,20 +1,20 @@
|
|||
|
||||
version: '2.1'
|
||||
version: '3.9'
|
||||
|
||||
volumes:
|
||||
pg-master-data-dir:
|
||||
pg-slave-data-dir:
|
||||
pg-node-data-dir:
|
||||
|
||||
|
||||
services:
|
||||
pg-master:
|
||||
image: kartoza/postgis:14-3.1
|
||||
image: kartoza/postgis:14-3.2
|
||||
restart: 'always'
|
||||
# You can optionally mount to volume, to play with the persistence and
|
||||
# observe how the slave will behave after restarts.
|
||||
volumes:
|
||||
- pg-master-data-dir:/var/lib/postgresql
|
||||
- ./tests:/tests
|
||||
- ./scripts/setup-master.sql:/docker-entrypoint-initdb.d/setup-master.sql
|
||||
environment:
|
||||
# ALLOW_IP_RANGE option is used to specify additionals allowed domains
|
||||
# in pg_hba.
|
||||
|
@ -34,16 +34,15 @@ services:
|
|||
healthcheck:
|
||||
test: "exit 0"
|
||||
|
||||
pg-slave:
|
||||
image: kartoza/postgis:14-3.1
|
||||
pg-node:
|
||||
image: kartoza/postgis:14-3.2
|
||||
restart: 'always'
|
||||
# You can optionally mount to volume, but we're not able to scale it
|
||||
# in that case.
|
||||
# The slave will always destroy its database and copy from master at
|
||||
# runtime
|
||||
volumes:
|
||||
- pg-slave-data-dir:/var/lib/postgresql
|
||||
- ./tests:/tests
|
||||
- pg-node-data-dir:/var/lib/postgresql
|
||||
|
||||
environment:
|
||||
# ALLOW_IP_RANGE option is used to specify additionals allowed domains
|
||||
|
@ -77,7 +76,7 @@ services:
|
|||
# If specified with any value, then it will convert current slave into
|
||||
# a writable state. Useful if master is down and the current slave needs
|
||||
# to be promoted until manual recovery.
|
||||
# PROMOTE_MASTER: 'True'
|
||||
#PROMOTE_MASTER: 'True'
|
||||
|
||||
# For now we don't support different credentials for replication
|
||||
# so we use the same credentials as master's superuser, or anything that
|
|
@ -0,0 +1,11 @@
|
|||
-- Create a table
|
||||
CREATE TABLE IF NOT EXISTS sweets
|
||||
(
|
||||
id SERIAL,
|
||||
name TEXT,
|
||||
price DECIMAL,
|
||||
CONSTRAINT sweets_pkey PRIMARY KEY (id)
|
||||
);
|
||||
|
||||
-- Inserts records into the table
|
||||
INSERT INTO sweets (name, price) VALUES ('strawberry', 4.50), ('Coffee', 6.20), ('lollipop', 3.80);
|
|
@ -1,50 +0,0 @@
|
|||
up:
|
||||
docker-compose up -d
|
||||
|
||||
down:
|
||||
docker-compose down
|
||||
|
||||
scale:
|
||||
docker-compose up -d --scale pg-slave=3
|
||||
|
||||
unscale:
|
||||
docker-compose up -d --scale pg-slave=1
|
||||
|
||||
status:
|
||||
docker-compose ps
|
||||
|
||||
master-shell:
|
||||
docker-compose exec pg-master /bin/bash
|
||||
|
||||
slave-shell:
|
||||
docker-compose exec pg-slave /bin/bash
|
||||
|
||||
master-log:
|
||||
docker-compose logs -f --tail=30 pg-master
|
||||
|
||||
slave-log:
|
||||
docker-compose logs -f --tail=30 pg-slave
|
||||
|
||||
|
||||
# These commands were used for travis
|
||||
|
||||
master-log-tail :
|
||||
docker-compose logs --tail=30 pg-master
|
||||
|
||||
slave-log-tail:
|
||||
docker-compose logs --tail=30 pg-slave
|
||||
|
||||
check-master-running:
|
||||
@echo "Check master is running"
|
||||
@docker-compose logs --tail=1 pg-master | grep 'database system is ready to accept connections' &> /dev/null
|
||||
|
||||
check-master-replication:
|
||||
@docker-compose exec pg-master /bin/sh -c "su - postgres -c \"/tests/replication_test_master.sh\""
|
||||
|
||||
check-slave-running:
|
||||
@echo "Check slave is running"
|
||||
@docker-compose logs pg-slave | grep 'database system is ready to accept read only connections' &> /dev/null
|
||||
@docker-compose logs pg-slave | grep 'started streaming WAL from primary' &> /dev/null
|
||||
|
||||
check-slave-replication:
|
||||
@docker-compose exec pg-slave /bin/sh -c "su - postgres -c \"/tests/replication_test_slave.sh\""
|
|
@ -1,19 +0,0 @@
|
|||
#!/usr/bin/env bash
|
||||
|
||||
. /scripts/env-data.sh
|
||||
|
||||
set -e
|
||||
|
||||
echo "Check master replication"
|
||||
|
||||
# Create a new table
|
||||
|
||||
echo "Create new table"
|
||||
psql -d ${POSTGRES_DBNAME} -c "CREATE TABLE test_replication_table ();"
|
||||
|
||||
# Check table exists in master
|
||||
|
||||
echo "Check table exists"
|
||||
psql -d ${POSTGRES_DBNAME} -c "\dt" | grep test_replication_table
|
||||
|
||||
exit $?
|
|
@ -1,14 +0,0 @@
|
|||
#!/usr/bin/env bash
|
||||
|
||||
. /scripts/env-data.sh
|
||||
|
||||
set -e
|
||||
|
||||
echo "Check slave replication"
|
||||
|
||||
# Check table exists in slave
|
||||
|
||||
echo "Check table exists"
|
||||
psql -d ${POSTGRES_DBNAME} -c "\dt" | grep test_replication_table
|
||||
|
||||
exit $?
|
|
@ -17,7 +17,7 @@ SQLDIR="/usr/share/postgresql/${POSTGRES_MAJOR_VERSION}/contrib/postgis-${POSTGI
|
|||
SETVARS="POSTGIS_ENABLE_OUTDB_RASTERS=1 POSTGIS_GDAL_ENABLED_DRIVERS=ENABLE_ALL"
|
||||
LOCALONLY="-c listen_addresses='127.0.0.1'"
|
||||
PG_BASEBACKUP="/usr/bin/pg_basebackup"
|
||||
PROMOTE_FILE="/tmp/pg_promote_master"
|
||||
NODE_PROMOTION="/usr/lib/postgresql/${POSTGRES_MAJOR_VERSION}/bin/pg_ctl"
|
||||
PGSTAT_TMP="/var/run/postgresql/"
|
||||
PG_PID="/var/run/postgresql/${POSTGRES_MAJOR_VERSION}-main.pid"
|
||||
|
||||
|
|
|
@ -35,7 +35,7 @@ if [[ "$WAL_LEVEL" == 'replica' && "${REPLICATION}" =~ [Tt][Rr][Uu][Ee] ]]; then
|
|||
fi
|
||||
# Promote to master if desired
|
||||
if [[ ! -z "${PROMOTE_MASTER}" ]]; then
|
||||
touch ${PROMOTE_FILE}
|
||||
su - postgres -c "${NODE_PROMOTION} promote -D ${DATADIR}"
|
||||
fi
|
||||
|
||||
fi
|
||||
|
|
Ładowanie…
Reference in New Issue