diff --git a/README.md b/README.md index 2b1a9cc..0ad0f00 100644 --- a/README.md +++ b/README.md @@ -64,7 +64,7 @@ volumes: grafana-data: ``` -This configuration will back up to AWS S3 instead. +This configuration will back up to AWS S3 instead. See below for additional tips about [S3 Bucket setup](#s3-bucket-setup). ### Stopping containers while backing up @@ -177,6 +177,36 @@ If so configured, they can also be shipped to an InfluxDB instance. This allows ![Backup dashboard sample](doc/backup-dashboard-sample.png) +## S3 Bucket setup + +Amazon S3 has [Versioning](https://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html) and [Object Lifecycle Management](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html) features that can be useful for backups. + +First, you can enable versioning for your backup bucket: + +![S3 versioning](doc/s3-versioning.png) + +Then, you can change your backup filename to a static one, for example: + +```yml +environment: + BACKUP_FILENAME: latest.tar.gz +``` + +This allows you to retain previous versions of the backup file, but the _most recent_ version is always available with the same filename: + + $ aws s3 cp s3://my-backup-bucket/latest.tar.gz . + download: s3://my-backup-bucket/latest.tar.gz to ./latest.tar.gz + +To make sure your bucket doesn't continue to grow indefinitely, you can enable some lifecycle rules: + +![S3 lifecycle](doc/s3-lifecycle.png) + +These rules will: + +- Move non-latest backups to a cheaper, long-term storage class ([Glacier](https://aws.amazon.com/glacier/)) +- Permanently remove backups after a year +- Still always keep the latest backup available (even after a year has passed) + ## Testing A bunch of test cases exist under [`test`](test/). To run them: diff --git a/doc/s3-lifecycle.png b/doc/s3-lifecycle.png new file mode 100644 index 0000000..7bef3ef Binary files /dev/null and b/doc/s3-lifecycle.png differ diff --git a/doc/s3-versioning.png b/doc/s3-versioning.png new file mode 100644 index 0000000..7864197 Binary files /dev/null and b/doc/s3-versioning.png differ diff --git a/test/backing-up-to-s3/docker-compose.yml b/test/backing-up-to-s3/docker-compose.yml index 2c75cdb..fe06d31 100644 --- a/test/backing-up-to-s3/docker-compose.yml +++ b/test/backing-up-to-s3/docker-compose.yml @@ -13,6 +13,7 @@ services: build: ../.. environment: BACKUP_CRON_EXPRESSION: "* * * * *" + BACKUP_FILENAME: latest.tar.gz AWS_S3_BUCKET_NAME: docker-volume-backup-test-bucket AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID} AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}