Consider my use case. I have all my important digital stuff encrypted in a TrueCrypt volume. I carry that volume around with me on a usb drive. All my projects, programming and others, are in that volume. It allows me to pick up my work and continue from any computer. For practical purposes I have a working copy of folders I currently work on, copied to a hard drive on my home computer as well. This is where it might get messy. Some changes for project could be on usb drive, some on home machine hard drive. For this I have written a simple bash script, that synchronizes all desired folders on both/all drives, while allowing each copy to be modified.
tl;dr:
#! /bin/sh
# Description:
# Synchronizes important stuff from local machine to (encrypted) usb drive.
# Before synchronizing, script creates backup of local version.
#
# Notes:
# No file gets deleted by this script. Deleting files require manual delete from all locations.
# With delete option, there would be no easy way to sync new files from usb to local copy.
# Instead, they would get deleted.
# Whatever file is modified on one location will get modified on other.
# Conflict resolution is simple: local modification will override usb drive modification.
# Warning: usb drive version is NOT backed up!
#
# Warning:
# rsync is NOT version control! Local modification will override those on usb drive.
# usb drive copy of data is NOT backed up prior to sync!
# Locations settings
TRUECRYPT_VOLUME_MOUNTPOINT="/media/truecrypt1/docs";
LOCAL_MOUNTPOINT="/home/matej"
BACKUP_MOUNTPOINT="/home/matej/backup/volume_backup"
LOG_FILE="/home/matej/logs/sync_volume.log"
# all folders that should be synced are enumerated here
FOLDERS=( \
'programming' \
'skeptics' \
'accounting' \
);
# If not sudo, exit. to prevent accidental runs
if [ $UID != 0 ]; then
echo "Please run this script with sudo:"
echo "sudo $0 $*"
exit 1
fi
# if truecrypt volume mounted
if [ -d $TRUECRYPT_VOLUME_MOUNTPOINT ]; then
for folder in ${FOLDERS[@]}
do
# first backup to local place, if anything unwanted happens during sync
# what can go wrong: unwanted file gets deleted/modified, because it was (accidentaly) deleted/modified on other location
# create backup destination string
BACKUP_DEST="$BACKUP_MOUNTPOINT/$(date +"%Y-%m-%d-%H-%M")/$folder"
echo $(date +"%Y/%m/%d %T")" [$$] Creating backup folder $BACKUP_DEST" >> $LOG_FILE
# actually create backup folder
# -p option means no error if existing, make parent directories as needed
mkdir -p $BACKUP_DEST
echo $(date +"%Y/%m/%d %T")" [$$] <a class="zem_slink" title="Rsync" href="http://rsync.samba.org" target="_blank" rel="homepage">Rsync</a> to backup" >> $LOG_FILE
# copy all files and folders to backup destination
rsync -avzh "$LOCAL_MOUNTPOINT/$folder/" $BACKUP_DEST
# then sync local copy with one on usb drive
# changes on local copy have priority, because they are done first
echo $(date +"%Y/%m/%d %T")" [$$] Rsync to truecrypt volume" >> $LOG_FILE
# first, sync from local copy to usb drive.
rsync -avzh --update --progress --log-file=$LOG_FILE "$LOCAL_MOUNTPOINT/$folder/" "$TRUECRYPT_VOLUME_MOUNTPOINT/$folder"
# then, sync from usb copy to local one.
rsync -avzh --update --progress --log-file=$LOG_FILE "$TRUECRYPT_VOLUME_MOUNTPOINT/$folder/" "$LOCAL_MOUNTPOINT/$folder"
done
else
echo $(date +"%Y/%m/%d %T")" [$$] Truecrypt volume not mounted" >> $LOG_FILE
echo "Truecrypt volume not mounted."
fi
exit
A workhorse for this script is rsync. First rsync synchronizes from local copy to usb drive, immediately after that the folders are rsynced in reverse direction. So, firstly changes on local copy get copied to usb, then reverse. Rsync is not a version control, which means it wasn't designed to handle conflicts. If a file is modified on local copy, it will simply overwrite usb copy regardless of whether it was modified too or not. Clumsy conflict resolution is something you have to live with, unless you're prepared to write more complex procedure, but it was a show stopper for handling deletes.
My first version had a -delete option included to command call, but a specific use case was a show stopper. If inside synchronized folder on usb drive I created a new folder, it got simply deleted by this script. Whether a folder has been added to usb drive, or if it was first on both drives, but then got deleted on local hard drive, the current state is the same. It cannot be known which behavior led to such a state, so the script based solely on rsync cannot correctly decide what to do: copy the folder or delete it from target. For this, there is a need to record a history of a folder, so that the script could compare states before with states now, but that would complicate matters further. I simply removed -delete option. Now I need to delete stuff from both locations before running script, but this doesn't happen that often, so it is a price I am prepared to pay.
For this reason the script copies everything that it is about to synchronize to a temporary backup folder. In case wrong files get overwritten or deleted or whatever, there is a chance to undo that. Usb version of a folder is not backed up, so here is a slight chance of loosing work. If I do work on all drives, it is important to sync regularly to avoid this. You can run this manually from time to time, set up a cronjob, or set it up so that it starts when usb drive is mounted. You can also copy this script in a folder that syncs, so that wherever you introduce some modifications to it, it too is propagated to all sync folders. It is also very simple to modify for other tasks. For example, I use a variant of this script to synchronize photos from personal computer to media center computer.
Previous:
How to create tea timer for Linux console
Next:
Some useful conditional branches conversions