Secure remote backups using Rsync…..

While the world was busy trying to figure out how to sync their palm pilots to Windows 98 (or was it 95) Andrew Tridgell was working on his thesis……Rsync. The endall-beall in the world of syncing. You see, if you’ve ever tried any syncing program before you’ll appreciate the ease with which rsync works. You’ll also see the efficiency of it’s algorithm in dealing with changes/updates, and for that you can thank Andrew. I’m just gonna show you how to use this excellent tool along with ssh (another must have/must use free software utility) to setup automated secure backups between two different platforms.

For the purpose of this article I’m using a PowerMac G5 with a 1TB Firewire disk hanging off it as my backup server. As my client I’m using the departmental research server (yes, forget that it’s a server, in this exercise it’s a CLIENT).

So first we need to make sure we have the tools: ssh and rsync. They are already part of my distribution (SuSE Linux 10.1 on the client and OSX 10.5.1 on the server) so I didn’t have to install anything. You want to make sure that your rsync uses protocol version 2.x.x on both sides. (type: rsync –version on your machines to see the protocol version).

Note: You can also grab RsyncX for older OSX versions and/or cwRsync for windows.

Step 1. Is to remove the interactivity that happens during an ssh session (ie: password prompt). To do this we must generate passphraseless keys that we can initially manually exchange between our machines. So in our case we want to grant access to our client machine — via ssh — from the server. Now here is were things get a bit OS specific. Under OSX there is no root account (atleast not one you can log into) by default. In my setup I want to allow root@server.domain.ca (OSX machine) to have access to root@client.domain.ca (Linux machine), so here is what I had to do.

On the OSX machine (server or machine initiating the rsync session) login as the standard user/admin and issue the following commands:
$ sudo su - (you'll be asked for your password and then be in a root shell....be careful).
# cd ~ (this is just to make sure we're in root's home directory which is /var/root under OSX).
# ssh-keygen -t dsa -b 1024 (do NOT put in a passphrase when prompted....just press Enter).
Generating public/private dsa key pair.
Enter file in which to save the key (/var/root/.ssh/id_dsa):
Created directory '/var/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /var/root/.ssh/id_dsa.
Your public key has been saved in /var/root/.ssh/id_dsa.pub.
The key fingerprint is:
03:cc:52:7a:ed:ae:bf:53:48:9c:dd:45:c7:a9:bd:f1 root@server.domain.ca
Make sure you don’t enter a passphrase when prompted (remember we want a automated operation here). If everything goes right you should have two new files in your .ssh directory named id_dsa (your private key….don’t ever give this to anyone) and id_dsa.pub (your public key which we are going to use).

Now you need to transfer the id_dsa.pub key file to your client somehow. I do this just using ssh again. If you want to be totally super secure you can stick it on a usb key and walk it over to your client machine. So still on the server (OSX) machine I do the following (still in roots account, so be careful):
# scp ~/.ssh/id_dsa.pub root@client.domain.ca:/root/.ssh/authorized_keys
# ssh client.domain.ca -l root "chmod 600 ~/.ssh/authorized_keys"
So the first command transfers id_dsa.pub from the server and copies it to the root’s .ssh directory on the client (and renames it to authorized_keys). The second command issues a remote chmod command to set the right permissions on the authorized_keys file on the client (Linux box).

Now, to verify, while still logged into root on the server (OSX) try to ssh to your client machine (using root’s account on client). If everything works you should be able to log into root’s account on the client (Linux machine) without a password prompt. If you’re prompted for password, stop here, and redo/recheck your procedure.

Step 2. Pheew…..You made it, that was the hard part….the rest is pretty simple. Now we just need to test to make sure things are running smoothly and once that’s done we can automate everything using a cron job (Step 3.). So lets start our test run. For this I’ve chosen to backup the /etc directory on the server (lots of small files, nothing earth shattering) and my chosen path on the server is /Volumes/TERADISK. This is where OSX mounts my firewire drive (your milage may vary). I’ve created a folder on TERADISK called FULLBACKUP, which I use to have rsync house my backups. So to test issue the following command as root (yes still, so be carefull) on the server:
# /usr/bin/rsync -a -v -z -e ssh "root@client.domain.ca:/etc/" /Volumes/TERADISK/FULLBACKUP/etc
Notice I use /usr/bin to make sure I’m running the kosher version of rsync under OSX Leopard (I have an older rsync in /usr/local/bin that rsyncX installed). Now a word about slashes in rsync. Note that I follow the first /etc with a slash, but not the second occurrence. That’s because I’m telling rsync to grab the files INSIDE /etc on the client (hence the extra / at the end) and sync them to files in /Volumes/TERADISK/FULLBACKUP/etc existing directory on the server (hence the missing / at the end). You can think of a trailing / on a source as meaning “copy the contents of this directory” as opposed to “copy the directory by name”. So hopefully this command worked for you and after a few minutes you have a copy of the /etc directory on the server. If not please stop and review your steps. Do NOT do Step 3. unless Steps 1&2 are verified and working.

Step 3. Well we’ve reached the end. All you have to do now is to use crontab -e command while still logged in as root on the server to add rsync entries for folders you want synced between the client and the server. Note that I do NOT backup absolutely everything off the client, only specific folders (and their subfolders ofcourse). I’m pretty sure that rsync will get confused if you try to sync things like your dev directory from the linux client. In my cron I have the following:
0 4 * * 6 /usr/local/bin/rsyncmanu.sh
Which runs the rsync shell script on the server (as root) every saturday at 4:00 am.

That’s it. Hopefully this has been useful…..