|
|
@@ -1,80 +1,8 @@
|
|
|
-# Remote Backup Manager
|
|
|
-This script uses rsync to backup files from any source (e.g. Remote) to any target (usually the host, the script runs on).
|
|
|
-On completion (either successful or on error), the scripts sends a mail with the output of rsync to a specified mail address.
|
|
|
+# Overview
|
|
|
+This repository contains several scripts to manage backups.
|
|
|
|
|
|
-## Dependencies
|
|
|
-sendmail needs to be installed and configured for sending backup notifications.
|
|
|
-rsync needs to be installed on all remote sources/targets and on the host which runs the script.
|
|
|
+# backupMgr
|
|
|
+This is the core component, syncing several (remote) source folder with local backup folders
|
|
|
|
|
|
-## Installation
|
|
|
-Pull the repository to a location of your choice on the host.
|
|
|
+# joplinBackup
|
|
|
|
|
|
-## Configuration
|
|
|
-Configure your ssh client for all remote source or target directories specified in your configuration,
|
|
|
-such that rsync neither requires passwords, ports, etc. Use the ```config``` file under your ```.ssh``` folder.
|
|
|
-An example configuration for a host is given as follows.
|
|
|
-```
|
|
|
-Host sebastianvendt.de
|
|
|
- HostName sebastianvendt.de
|
|
|
- Port 1234
|
|
|
- IdentityFile /home/sebastian/sshKeys/rsyncOnSebastianvendt.de.pem
|
|
|
-```
|
|
|
-
|
|
|
-Create a configuration file, which holds the source and destination paths for the backup jobs as well as several configuration variables.
|
|
|
-An example configuration file with all **required** variables is given in the following.
|
|
|
-```
|
|
|
-#Backup directories
|
|
|
-declare -A backup_folders
|
|
|
-backup_folders["backupuser@hostA.com:/opt/SourceFolderA"]="opt/backup/folderA"
|
|
|
-backup_folders["/opt/SourceFolderB"]="opt/backup/folderB"
|
|
|
-
|
|
|
-#Configuration
|
|
|
-logPath=/home/user/logs/backupjob/
|
|
|
-logFileName="$(date +'%Y-%m-%d_%H-%M')_backupjobs.log"
|
|
|
-tmpLogFilePath=/tmp/tmp_backup.log
|
|
|
-retryAttempts=2
|
|
|
-mailFrom="backup@backupserver.com"
|
|
|
-mailTo="user@domain.com"
|
|
|
-
|
|
|
-#Delete logs older than rotationDays
|
|
|
-rotationDays=14
|
|
|
-```
|
|
|
-
|
|
|
-## How to run the script
|
|
|
-The script requires as argument the path to the configuration file
|
|
|
-```
|
|
|
-/path/of/script/getRemoteBackup.sh /path/to/configuration
|
|
|
-```
|
|
|
-
|
|
|
-## Automate the backup job
|
|
|
-To automate your backup, add the script with the path to your configuration to your crontab-file. I suggest to run it in a detached screen session.
|
|
|
-You can then always reattach to the screen and check the current status for long backup jobs.
|
|
|
-```
|
|
|
-10 2 * * * sebastian screen -d -m /home/sebastian/scripts/backup/getRemoteBackup.sh /home/sebastian/scripts/backup/backupFolders.conf
|
|
|
-```
|
|
|
-
|
|
|
-## The core element
|
|
|
-The heart of the script is an rsync command:
|
|
|
-```
|
|
|
-rsync --delete --out-format="%t %f %''b" -avz $sourceFolder ${backup_folders[$sourceFolder]} | tee -a $tmpLogFilePath 2>&1
|
|
|
-```
|
|
|
-```--delete```: This tells rsync to delete extraneous files from the receiving side (ones that aren’t on the
|
|
|
-sending side), but only for the directories that are being synchronized. You must have asked
|
|
|
-rsync to send the whole directory (e.g. "dir" or "dir/") without using a wildcard for the
|
|
|
-directory’s contents (e.g. "dir/*") since the wildcard is expanded by the shell and rsync thus
|
|
|
-gets a request to transfer individual files, not the files’ parent directory. Files that are
|
|
|
-excluded from the transfer are also excluded from being deleted unless you use the
|
|
|
-**--delete-excluded** option or mark the rules as only matching on the sending side (see the
|
|
|
-include/exclude modifiers in the FILTER RULES section).
|
|
|
-
|
|
|
- ```-a, --archive```: This is equivalent to **-rlptgoD**. It is a quick way of saying you want recursion and want to preserve almost everything (with -H being a notable omission). The only exception to the above equivalence is when **--files-from** is specified, in which case **-r** is not implied.
|
|
|
-Note that **-a** **does** **not** **preserve** **hardlinks**, because finding multiply-linked files is expensive. You must separately specify **-H**.
|
|
|
-
|
|
|
-```-z, --compress```: With this option, rsync compresses the file data as it is sent to the destination machine, which reduces the amount of data being transmitted -- something that is useful over a slow connection. Note that this option typically achieves better compression ratios than can be achieved by using a
|
|
|
-compressing remote shell or a compressing transport because it takes advantage of the implicit
|
|
|
-information in the matching data blocks that are not explicitly sent over the connection.
|
|
|
-See the **--skip-compress** option for the default list of file suffixes that will not be compressed.
|
|
|
-
|
|
|
-```-v, --verbose```: Be verbose :)
|
|
|
-
|
|
|
-```--out-format="%t %f %''b"```: Defines the output format as follows ```2024/03/10 02:10:14 appdata_ocn6dip2flyl/appstore/apps.json 16.80K```
|
