From f2a4a4954c0681ce5f284fbb78e8874dd97fa478 Mon Sep 17 00:00:00 2001 From: Andras Schmelczer Date: Sun, 2 Jun 2024 13:11:35 +0100 Subject: [PATCH] Improve README --- README.md | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index 8f56751..ed37a99 100644 --- a/README.md +++ b/README.md @@ -1,31 +1,31 @@ # My backup container -Snapshot a [BTRFS](https://docs.kernel.org/filesystems/btrfs.html) volume from a [Docker container](https://www.docker.com/) and robustly back it up to multiple [BorgBackup](https://borgbackup.readthedocs.io/en/stable/index.html) repositories on a schedule. +Create a snapshot of a [BTRFS](https://docs.kernel.org/filesystems/btrfs.html) volume from a [Docker container](https://www.docker.com/) and robustly back it up to multiple [BorgBackup](https://borgbackup.readthedocs.io/en/stable/index.html) repositories on a schedule. ## Quick start -- Check out [docker-compose.yml](docker-compose.yml) and set the environment variables according to your needs -- Update the password in the [.env](config/.env) file -- Adjust the [exclude.conf](conifg/exclude.conf) file to match your needs -- Run `docker compose --env-file ./config/default.env up` +1. Review and modify the [docker-compose.yml](docker-compose.yml) file to set your environment variables as needed +2. Update the password in the [.env](config/.env) file +3. Customise the [exclude.conf](conifg/exclude.conf) according to your requirements +4. Execute the command `docker compose --env-file ./config/default.env up` to spin up the container ## Background -I've been using this setup for the past year and have managed to avoid disaster by restoring all my data. This is all thanks to the excellent backup tool called BorgBackup. The scripts in the this repository serve as an opinionated thin wrapper around it. The Docker image I provide here is generic enough to be used verbatim in many self-hosting setups. However, it's just a thin wrapper and a set of configs that work together well. It's been a long journey for me to find a backup setup that works for me, so I make this available to save others from pitfalls and serve as a potenrial starting point for your own self-hosted backup container. +Over the past year, this backup setup has enabled me to successfully restore all my data after incidents. This is all thanks to the excellent backup tool called BorgBackup. The scripts in the this repository serve as an opinionated thin wrapper around it. The Docker image provided can be used directly in various self-hosting setups. It's designed to be a simple and effective tool for those looking to establish a reliable backup system, saving time and avoiding common pitfalls. ## Features -- Back up the snapshot of a BTRFS volume to avoid files changing during backup - > I self-host multiple databases and this is the most feasible way of avoiding data corruption -- Schedule the backup so that it keeps running automatically -- Keep a weekly rotated log of all previous runs -- Back up the same source to multiple BorgBackup repositories +- **Snapshotting**: Takes a snapshot of a BTRFS volume to ensure file consistency during backups. + > I self-host multiple databases and this is the most feasible way of avoiding data corruption. +- **Scheduled Backups**: Automates backups according to a defined schedule. +- **Log Rotation**: Maintains weekly logs of all backup activities. +- **Multi-Repository Backups**: Allows backups to multiple BorgBackup repositories simultaneously. ### Multi-target backups -In case we don't have disk-level redundancy and would still wish to follow the [3-2-1 rule](https://en.wikipedia.org/wiki/Backup), we have to back up the same source folder to multiple targets. For instance, I don't have a RAID setup but instead back up an SSD to [rsync.net](rsync.net) and a local HDD at the same time. +To adhere to the [3-2-1 backup rule](https://en.wikipedia.org/wiki/Backup) without disk-level redundancy, you can configure backups to multiple destinations. For example, backups can be sent simultaneously to [rsync.net](rsync.net) and a local HDD. -You can see in the [docker-compose.yml](docker-compose.yml) file that we can configure multi-target backups by setting the environment variables: `BORG_REPO_0`, `BORG_REPO_1`, `BORG_PASSPHRASE_0`, `BORG_PASSPHRASE_1`, etc. +The [`docker-compose.yml`](docker-compose.yml) file demonstrates how to set up multiple backup targets using environment variables such as `BORG_REPO_0`, `BORG_REPO_1`, `BORG_PASSPHRASE_0`, `BORG_PASSPHRASE_1`, and so forth. The backup script sequentially handles each repository defined by the environment variables, ensuring your source volume is backed up across all specified targets. The backup script first takes `BORG_REPO_0` and the corresponding env vars and sets up the [`BORG_REPO`](https://borgbackup.readthedocs.io/en/stable/usage/general.html#repository-urls), `BORG_REMOTE_PATH`, and `BORG_PASSPHRASE` environment variables for `borg`. Once the backup finished (successfully or otherwise), the script checks whether `BORG_REPO_1` exists, if so, it sets `BORG_REPO` and the other env vars to their expected values and backs up again. The script keeps going to `BORG_REPO_2`, `BORG_REPO_3` and so on as long as these are set. Otherwise, it unsets the previous `BORG_REPO` and corresponding env vars and goes to sleep. @@ -69,12 +69,12 @@ Thus, the following sets of environment variables are valid for multi-target bac ## Repository layout - src - - backup.sh: Creates a BorgBackup repository if it doesn't exist, snapshots the BTRFS volume, backs up your data, and removes old backups. - - backup-wrapper.sh: Runs backup.sh for every BorgBackup repository provided through environment variables. - - schedule.sh: Persists the logs of backup-wrapper.sh and runs it in a while True. + - [backup.sh](src/backup.sh): Creates a new BorgBackup repository if none exists, takes a snapshot of the BTRFS volume, performs the backup, and prunes old backups. + - [backup-wrapper.sh](src/backup-wrapper.sh): Executes backup.sh for each repository configured via environment variables. + - [schedule.sh](src/schedule.sh): Manages and logs the operation of backup-wrapper.sh and runs it in a continuous loop. ## Related resources -- I like this video on how to install Debian with BTRFS https://www.youtube.com/watch?v=MoWApyUb5w8. Of course, a BTRFS disk can be created after installation as well. -- rsync.net has a special discount when using BorgBackup: https://www.rsync.net/products/borg.html -- Checking out the documentation of BorgBackup is worth it: https://www.borgbackup.org/demo.html, especially the description of [borg create](https://borgbackup.readthedocs.io/en/stable/usage/create.html#description) +- Learn how to install Debian with BTRFS in this helpful [video tutorial](https://www.youtube.com/watch?v=MoWApyUb5w8). Note that a BTRFS disk can also be created post-installation. +- rsync.net offers a special discount for BorgBackup users: [BorgBackup at rsync.net](https://www.rsync.net/products/borg.html). +- Explore detailed BorgBackup documentation and demos: [BorgBackup Documentation](https://www.borgbackup.org/demo.html), including a comprehensive guide on [`borg create`](https://borgbackup.readthedocs.io/en/stable/usage/create.html#description).