backup

Remote Backup Manager

This script uses rsync to backup files from any source (e.g. Remote) to any target (usually the host, the script runs on). Once the

Dependencies

sendmail needs to be installed and configured for sending backup notifications.

Installation

Pull the repository to a location of your choice on the host. Configure your ssh client, such that rsync neither requires passwords, ports, etc. Use the config file under your .ssh folder.

Host sebastianvendt.de
        HostName sebastianvendt.de
        Port 1234
        IdentityFile /home/sebastian/sshKeys/rsyncOnSebastianvendt.de.pem

How to run the script

Create a configuration file, which holds the source and destination paths. An example configuration file is given in the following. Please note, this is also a shell script and it also must be executable!

#!/bin/bash
declare -A backup_folders #Do not change this name
backup_folders["rsync@sebastianvendt.de:/var/lib/psa/dumps/domains/sebastianvendt.de/"]="/home/sebastian/ARCHIVE/sebastian/3000_BACKUP/sebastianvendt.de/configuration"
backup_folders["backup@serles.local:/var/lib/openhab/backup"]="/home/sebastian/ARCHIVE/sebastian/3000_BACKUP/serles.local/backup"

Add the script with the path to your configuration file to your crontab-file. I suggest to run it in a screen. You can then always reattach to the screen and check the current status for long backup jobs.

10 2 * * * sebastian screen -d -m /home/sebastian/scripts/backup/getRemoteBackup.sh /home/sebastian/scripts/backup/backupFolders.conf

The core element

The heart of the script is an rsync command:

rsync --delete --out-format="%t %f %''b" -avz $sourceFolder ${backup_folders[$sourceFolder]} | tee -a $tmpLogFilePath 2>&1

--delete: This tells rsync to delete extraneous files from the receiving side (ones that aren’t on the sending side), but only for the directories that are being synchronized. You must have asked rsync to send the whole directory (e.g. "dir" or "dir/") without using a wildcard for the directory’s contents (e.g. "dir/*") since the wildcard is expanded by the shell and rsync thus gets a request to transfer individual files, not the files’ parent directory. Files that are excluded from the transfer are also excluded from being deleted unless you use the --delete-excluded option or mark the rules as only matching on the sending side (see the include/exclude modifiers in the FILTER RULES section).

-a, --archive: This is equivalent to -rlptgoD. It is a quick way of saying you want recursion and want to preserve almost everything (with -H being a notable omission). The only exception to the above equivalence is when --files-from is specified, in which case -r is not implied. Note that -a does not preserve hardlinks, because finding multiply-linked files is expensive. You must separately specify -H.

-z, --compress: With this option, rsync compresses the file data as it is sent to the destination machine, which reduces the amount of data being transmitted -- something that is useful over a slow connection. Note that this option typically achieves better compression ratios than can be achieved by using a compressing remote shell or a compressing transport because it takes advantage of the implicit information in the matching data blocks that are not explicitly sent over the connection. See the --skip-compress option for the default list of file suffixes that will not be compressed.

-v, --verbose: Be verbose :)

--out-format="%t %f %''b": Defines the output format as follows 2024/03/10 02:10:14 appdata_ocn6dip2flyl/appstore/apps.json 16.80K