|
|
@@ -1,78 +1,8 @@
|
|
|
-# Remote Backup Manager
|
|
|
-This script uses rsync to backup files from any source (e.g. Remote) to any target (usually the host, the script runs on). On completion (either successful or on error), the scripts sends a mail with the output of rsync to a specified mail address.
|
|
|
+# Overview
|
|
|
+This repository contains several scripts to manage backups.
|
|
|
|
|
|
-## Dependencies
|
|
|
-sendmail needs to be installed and configured for sending backup notifications.
|
|
|
+# backupMgr
|
|
|
+This is the core component, syncing several (remote) source folder with local backup folders
|
|
|
|
|
|
-## Installation
|
|
|
-Pull the repository to a location of your choice on the host.
|
|
|
+# joplinBackup
|
|
|
|
|
|
-## Configuration
|
|
|
-Configure your ssh client for all remote source or target directories specified in your configuration,
|
|
|
-such that rsync neither requires passwords, ports, etc. Use the ```config``` file under your ```.ssh``` folder.
|
|
|
-An example configuration for a host is given as follows.
|
|
|
-```
|
|
|
-Host sebastianvendt.de
|
|
|
- HostName sebastianvendt.de
|
|
|
- Port 1234
|
|
|
- IdentityFile /home/sebastian/sshKeys/rsyncOnSebastianvendt.de.pem
|
|
|
-```
|
|
|
-
|
|
|
-Create a configuration file, which holds the source and destination paths for the backup jobs as well as several configuration variables.
|
|
|
-An example configuration file with all *required* variables is given in the following.
|
|
|
-```
|
|
|
-#Backup directories
|
|
|
-declare -A backup_folders
|
|
|
-backup_folders["backupuser@hostA.com:/opt/SourceFolderA"]="opt/backup/folderA"
|
|
|
-backup_folders["/opt/SourceFolderB"]="opt/backup/folderB"
|
|
|
-
|
|
|
-#Configuration
|
|
|
-logPath=/home/user/logs/backupjob/
|
|
|
-logFileName="$(date +'%Y-%m-%d_%H-%M')_backupjobs.log"
|
|
|
-tmpLogFilePath=/tmp/tmp_backup.log
|
|
|
-retryAttempts=2
|
|
|
-mailFrom="backup@backupserver.com"
|
|
|
-mailTo="user@domain.com"
|
|
|
-
|
|
|
-#Delete logs older than rotationDays
|
|
|
-rotationDays=14
|
|
|
-```
|
|
|
-
|
|
|
-## How to run the script
|
|
|
-The script requires as argument the path to the configuration file
|
|
|
-```
|
|
|
-/path/of/script/getRemoteBackup.sh /path/to/configuration
|
|
|
-```
|
|
|
-
|
|
|
-## Automate the backup job
|
|
|
-To automate your backup, add the script with the path to your configuration file to your crontab-file. I suggest to run it in a detached screen session.
|
|
|
-You can then always reattach to the screen and check the current status for long backup jobs.
|
|
|
-```
|
|
|
-10 2 * * * sebastian screen -d -m /home/sebastian/scripts/backup/getRemoteBackup.sh /home/sebastian/scripts/backup/backupFolders.conf
|
|
|
-```
|
|
|
-
|
|
|
-## The core element
|
|
|
-The heart of the script is an rsync command:
|
|
|
-```
|
|
|
-rsync --delete --out-format="%t %f %''b" -avz $sourceFolder ${backup_folders[$sourceFolder]} | tee -a $tmpLogFilePath 2>&1
|
|
|
-```
|
|
|
-```--delete```: This tells rsync to delete extraneous files from the receiving side (ones that aren’t on the
|
|
|
-sending side), but only for the directories that are being synchronized. You must have asked
|
|
|
-rsync to send the whole directory (e.g. "dir" or "dir/") without using a wildcard for the
|
|
|
-directory’s contents (e.g. "dir/*") since the wildcard is expanded by the shell and rsync thus
|
|
|
-gets a request to transfer individual files, not the files’ parent directory. Files that are
|
|
|
-excluded from the transfer are also excluded from being deleted unless you use the
|
|
|
-**--delete-excluded** option or mark the rules as only matching on the sending side (see the
|
|
|
-include/exclude modifiers in the FILTER RULES section).
|
|
|
-
|
|
|
- ```-a, --archive```: This is equivalent to **-rlptgoD**. It is a quick way of saying you want recursion and want to preserve almost everything (with -H being a notable omission). The only exception to the above equivalence is when **--files-from** is specified, in which case **-r** is not implied.
|
|
|
-Note that **-a** **does** **not** **preserve** **hardlinks**, because finding multiply-linked files is expensive. You must separately specify **-H**.
|
|
|
-
|
|
|
-```-z, --compress```: With this option, rsync compresses the file data as it is sent to the destination machine, which reduces the amount of data being transmitted -- something that is useful over a slow connection. Note that this option typically achieves better compression ratios than can be achieved by using a
|
|
|
-compressing remote shell or a compressing transport because it takes advantage of the implicit
|
|
|
-information in the matching data blocks that are not explicitly sent over the connection.
|
|
|
-See the **--skip-compress** option for the default list of file suffixes that will not be compressed.
|
|
|
-
|
|
|
-```-v, --verbose```: Be verbose :)
|
|
|
-
|
|
|
-```--out-format="%t %f %''b"```: Defines the output format as follows ```2024/03/10 02:10:14 appdata_ocn6dip2flyl/appstore/apps.json 16.80K```
|
Sebastian Vendt (Manaslu)