The files below are what I use to back up my machines, it’s quick, simple, and effective – I’ve used this for years to recover a lot of data, including bare-metal on more than one occasion. The files listed toward the bottom (Summary, Backups, and Filelist) are generated and don’t need to exist beforehand.
/Backup/SiteWideBackup, executable (chmod a+x), this script is called by a cron job, it calls the other scripts that do the actual backups.
#!/bin/bash # make sure we're running as root if (( `id -u` != 0 )); then { echo "Sorry, must be root. Exiting..."; exit; } fi; # First, make a backup of all of the MySQL Databases /Backup/MySQL/MySQLBackup # Finally, backup everything else; # Run a backup for every .conf file in the /Backup directory cd /Backup for Script in *.conf do ./Snapshot $Script done # And send an e-mail with the list of Summaries; cat /Backup/*/Summary |
/Backup/MySQL/MySQLBackup, this backs up all of MySQL.
#!/bin/bash # This script is an attempt to make good backups of all of the databases within MySQL # SRJ 2017-05-06 # # # First step, create a user solely for executing backups with; # MariaDB [(none)]> GRANT LOCK TABLES, SELECT ON *.* TO 'MySQLBackup'@'localhost' IDENTIFIED BY 'eRCG57*2F&m(dP'; # Query OK, 0 rows affected (0.00 sec) # # MariaDB [(none)]> flush privileges; # Query OK, 0 rows affected (0.00 sec) # # MariaDB [(none)]> quit; # Bye cd /Backup/MySQL Date=$(date +"%Y-%m-%d-%H-%M") User='MySQLBackup' Pass='eRCG57*2F&m(dP' MySQL=/usr/bin/mysql MySQLDump=/usr/bin/mysqldump Databases=$(${MySQL} --user=${User} -p${Pass} -e "SHOW DATABASES;" | grep -Ev "(Database|information_schema|performance_schema)") # This creates a dated file for each db, these are text files that can be restored with mysql. for db in ${Databases} ; do ${MySQLDump} --force --opt --user=${User} -p${Pass} --databases "${db}" > ${Date}-"${db}".sql done # This last line deletes files older than two weeks - touching a file refreshes that clock. find . -type f -name '*.sql' -atime +14 -exec rm -f {} \; |
/Backup/01-Root.conf, this configuration file defines the backups for a partition. The rsync application can’t cross filesystem boundaries because that would break the incremental nature of this backup.
# =-=-=-=-=-=- File locations and variables. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= # The physical device where backups will exist. MOUNT_DEVICE="LABEL=Backup"; # Where to mount the physical device. MOUNT_DIR=/Backup; # Where to write the backups SNAPSHOT_RW=/Backup/01-Root; # What directory to back up, begins and ends in / SNAP_DIR="/"; # Where to store the list of files and directories that we backed up. FILELIST=$SNAPSHOT_RW/Filelist # Summary file, a short list of actions taken by rsync to complete the backup. SUMMARY=$SNAPSHOT_RW/Summary # Stuff not to back up, there is always stuff to not back up! EXCLUDES=$SNAPSHOT_RW/Excludes; # How many snapshot backups do we want to keep? Must be at least 2! NUM_SNAP=10; |
Snapshot, this is the script that backs up regular files and folders.
#!/bin/bash # =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= # 2017-01-06-SRJ # For information on how this script works, and what it does, read the # file README in this folder # =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= # # suggestion from H. Milz: avoid accidental use of $PATH unset PATH # # =-=-=-=- System commands used by this script. -=-=-=-=-=-=-=-= ID=/usr/bin/id; ECHO=/bin/echo; MOUNT=/bin/mount; RM=/bin/rm; MV=/bin/mv; CP=/bin/cp; TOUCH=/bin/touch; RSYNC=/usr/bin/rsync; DATE=/bin/date; WC=/usr/bin/wc; CAT=/bin/cat; TAIL=/usr/bin/tail; HEAD=/usr/bin/head; GREP=/bin/grep; CHMOD=/bin/chmod # =-=-=-=-=-=-=- Make sure we're running as root -=-=-=-=-=-=-=-=-= if (( `$ID -u` != 0 )); then { $ECHO "Sorry, must be root. Exiting..."; exit; } fi; # =-=-=-=-=-=-=-=-=-=- Read in the Variables -=-=-=-=-=-=-=-=-=-=-= if [ ! -e $1 ] ; then { $ECHO "No viable config file given."; exit 0 } fi; # If we're here, there must be a config file to read, the line below # reads the variables from that file into this bash environment. . $1 # # # Update 2017-01-01 SRJ The line above sources the variables from # # # the config file into this script - see /Backup/00-System.conf # # # for more information. # Make sure that the folder list exists, create it if needed; if [ ! -f $SNAPSHOT_RW/Backups ] ; then { $ECHO Nothing >> $SNAPSHOT_RW/Backups } fi; # The full name of the oldest snapshot, first line in Backups [ $($CAT $SNAPSHOT_RW/Backups | $WC --lines) -ge $NUM_SNAP ] && \ OLDEST=$($HEAD -1 $SNAPSHOT_RW/Backups) || OLDEST="Nothing" # The full name of the last snapshot, last line in Backups LAST=$($TAIL -1 $SNAPSHOT_RW/Backups) # The full name of the snapshot for this itteration. (2006-06-22-09-47) CURRENT=$SNAPSHOT_RW/$($DATE +%F-%H-%M) # What time is it, in seconds past the epoch? StartTime=$($DATE +%s) # Hour, minute, second function; _hms() { local S=${1} # this S is used only in this sub-routine ((h=S/3600)) # fills h with the integer of seconds / 3600 ((m=S%3600/60)) # fills m with the integer of the modula from above / 60 ((s=S%60)) # fills s with the integer of seconds / 60 printf "%02d:%02d:%02d\n" $h $m $s # print 2 digit zero padded numbers } # =-=-=-=-=-=-=-=-=- The script itself. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= # I stopped mounting and unmounting the filesystem, I now do this by # hand on an as-needed basis - these backup scripts are now on the # backup drives themselves. # Attempt to remount the RO mount point as RW; else abort # $MOUNT -o remount,rw $MOUNT_DEVICE $MOUNT_DIR ; # if (( $? )); then { # $ECHO "Could not remount $MOUNT_DIR read-write"; exit; # } fi; # Creating the snapshot. # Delete the oldest snapshot, if it exists: if [ -e ${OLDEST} ] ; then { $RM -rf ${OLDEST} ;} fi ; # Delete the last filelist and create a new one with proper permissions. if [ -f $FILELIST ] ; then { $RM -f $FILELIST ;} fi ; $TOUCH $FILELIST ; $CHMOD 600 $FILELIST # Rsync from the system into the latest snapshot (notice that rsync # behaves like cp --remove-destination by default, so the destination is # unlinked first. If it were not so, this would copy over the other # snapshot(s) too! RSARGS=" --archive --verbose --delete --delete-excluded --numeric-ids \ --modify-window=1 --hard-links --one-file-system --acls --xattrs \ --exclude-from="$EXCLUDES" --link-dest=$LAST/ " # This is the line that does everything we're doing here... $RSYNC $RSARGS $SNAP_DIR $CURRENT >> $FILELIST; # Add today's snapshot to the list, and remove the oldest; $ECHO $CURRENT >> $SNAPSHOT_RW/Backups $GREP -v $OLDEST $SNAPSHOT_RW/Backups >> $SNAPSHOT_RW/$$.tmp $MV -f $SNAPSHOT_RW/$$.tmp $SNAPSHOT_RW/Backups EndTime=$($DATE +%s) ; ElapsedTime=$(( $EndTime - $StartTime )) TimeWellSpent=$( _hms ${ElapsedTime}) # Write out the summary file. $HEAD -2 $FILELIST > $SUMMARY; $ECHO $(( $($GREP -ve /$ $FILELIST | $WC --lines )-5)) \ New files backed up. >> $SUMMARY $ECHO $OLDEST was removed and $CURRENT was created >> $SUMMARY $ECHO Elapsed time: $TimeWellSpent >> $SUMMARY $TAIL -3 $FILELIST >> $SUMMARY $MV $FILELIST $CURRENT # And thats it for SNAP_DIR. Now remount the RW snapshot mountpoint as RO # $MOUNT -o remount,ro $MOUNT_DEVICE $MOUNT_DIR ; # if (( $? )); then { # $ECHO "Could not remount $MOUNT_DIR readonly"; exit; # } fi; |
/Backup/01-Root/Excludes, this file lists things we don’t want backing up.
/Backup/* /home/Shared/Backup */Cache/* /dev/* /flows/* /initrd* /home/*/.gvfs */lost+found* /media/* /mnt/* /run/* /tmp/* /proc/* /var/virus* /var/lib/vmware* /var/run/cups/certs* |
/Backup/01-Root/Summary, is just the best bits of the Filelist, below.
sending incremental file list created directory /Backup/01-Root/2019-08-14-02-02 718 New files backed up. /Backup/01-Root/2019-08-04-02-02 was removed and /Backup/01-Root/2019-08-14-02-02 was created Elapsed time: 00:14:56 sent 1,158,536,441 bytes received 127,738 bytes 2,045,303.05 bytes/sec total size is 773,557,672,398 speedup is 667.63 |
/Backups/01-Root/Backups, This is just a list of the backups that exist on the external drive.
/Backup/01-Root/2019-08-05-02-02 /Backup/01-Root/2019-08-06-02-02 /Backup/01-Root/2019-08-07-02-02 /Backup/01-Root/2019-08-08-02-02 /Backup/01-Root/2019-08-09-02-02 /Backup/01-Root/2019-08-10-02-02 /Backup/01-Root/2019-08-11-02-02 /Backup/01-Root/2019-08-12-02-02 /Backup/01-Root/2019-08-13-02-02 /Backup/01-Root/2019-08-14-02-02 |
/Backup/01-Root/Filelist, this file is moved to /Backup/01-Root/2019-08-14-02-02/Filelist after the rsync has finished.
sending incremental file list created directory /Backup/01-Root/2019-08-14-02-02 ./ etc/.git/ etc/letsencrypt/ opt/mattermost/logs/mattermost.log root/ root/.PlotLog root/.SpeedTest.log ... ... var/spool/postfix/public/pickup var/spool/postfix/public/qmgr var/tmp/ var/tmp/sclhoIPgx var/tmp/systemd-private-23f097ce9d024a4cbcc72c823a582844-mysqld.service-p5VWfi/tmp/ sent 1,158,536,441 bytes received 127,738 bytes 2,045,303.05 bytes/sec total size is 773,557,672,398 speedup is 667.63 |