Backup using rsync
Overview
Backing up your critical data is essential. This is one way to do it, describing a way to automize backups from one Linux machine to another (in my example two Bubba servers, but it could be any two Linux machines).
I use rsync, which contains a clever algorithm identifying what's changed since the last backup, and only copying the changed files. This means that a complete file copy doesn't have to be transferred on every backup occation, it only transfers the files that have changed.
The rsync command is initiated with a bash script, that in this case also does some copying and emails the backup status to the a user.
Finally, the script is added as a cron entry, making it run regularly, in my case, once every night.
I need to backup the email handled by my main Bubba, to a secondary Bubba acting (among other things) as a backup server. Files are copied from my main Bubba (used as email and web server) to my backup-Bubba. The script is run at the backup-Bubba, thus, the files are "pulled" to that machine. In the following guide, the machines are denoted as main-Bubba and backup-Bubba. I have called the users on each machine "main-user" and "backup-user".
Note: I give no warranties that this actually works, or even does what I intend. Use this guide at your own risk, for educational purposes only. Test it thoroughly before trusting it as a backup solution. Excito is not responsible for the content herein.
Now, to the guide:
Setting up "backup-Bubba"
Log on as backup-user, then do 'su' to become root. Uncomment the debian sources from /etc/apt/sources.list, then do:
- Code: Select all
apt-get install rsync
Also install cron and open ssh packages, if not already installed (as they are if you are using Bubba).
Type exit to return as your normal user, "backup-user".
Now, create a directory holding your script:
- Code: Select all
mkdir /home/backup-user/cron
Also create a backup directory, holding your backed up files:
- Code: Select all
mkdir /home/backup-user/backup
To test the rsync connection (note: rsync needs to be installed on both systems for this to work):
- Code: Select all
rsync -av -e ssh main-user@main-Bubba:/home/main-user/something_to_backup /home/backup-user/backup/
(type your remote password upon request).
Since we want this to be automated, typing your password every time isn't an option. This can be bypassed by depositing ssh keys on main-Bubba (google for 'ssh-keygen' if you wan't to dig deeper). To generate ssh keys:
- Code: Select all
ssh-keygen -t dsa -b 1024 -f /home/backup-user/cron/backup-Bubba-rsync-key
(Note that this will take a minuter or two. You may wish to use a longer key than 1024 bits, but be prepared to wait for a while.).
Copy the public key file from backup-Bubba to main-Bubba, using scp:
- Code: Select all
scp /home/backup-user/cron/backup-Bubba-rsync-key.pub main-user@main-Bubba:/home/main-user/.ssh
(If the .ssh directory isn't already created, you may have to do so manually, on main-Bubba).
Setting up "main-Bubba"
Log on as main-user, then do 'su' to become root. Uncomment the debian sources as before, then do:
- Code: Select all
apt-get install rsync
Also install cron and open ssh packages, if not already installed (as they are if you are using Bubba). Type 'exit' to return as a normal user (main-user).
Add the key to the authorized_keys file:
- Code: Select all
cat bubba-01-rsync-key.pub >> .ssh/authorized_keys
Note that this makes anyone at your LAN able to log in using the user 'main-user', using that key. If you consider this being a security risc, it is possible to prohibit ssh logons from other machines than backup-Bubba, but this isn't covered by this guide.
Now, back to backup-Bubba for further testing:
Testing the ssh and rsync connection
To test the ssh-connection, without having to use your password:
- Code: Select all
ssh -i /home/backup-user/cron/backup-Bubba-rsync-key main-user@main-Bubba
Just type exit to return to your local session.
And to test an rsync transfer in the same manner:
- Code: Select all
rsync -av -e "ssh -i /home/backup-user/cron/backup-Bubba-rsync-key" main-user@main-Bubba:/home/main-user/something_to_backup backup
Check that the files located in your /something_to_backup/ folder ended up in /backup/ on backup-Bubba.
Setting up a cron job to do this regularly
Create the script file:
- Code: Select all
nano /home/backup-user/cron/backup-script
And put something like this in there:
- Code: Select all
#!/bin/sh
rsync -av -e "ssh -i /home/backup-user/cron/backup-Bubba-rsync-key" main-user@main-Bubba:/home/main-user/something_to_backup/
/home/backup-user/backup/
Make this script executable:
- Code: Select all
chmod +x backup-script
And test it:
- Code: Select all
./backup-script
Now put this in the cron-tab:
- Code: Select all
crontab -e
And put something like:
- Code: Select all
0 23 * * * /home/backup-user/cron/backup-script
in there. This example means that your script will run every night at 11:00 PM (23:00), every day of the week all year around. Using:
- Code: Select all
42 07 19 * * /home/backup-user/cron/backup-script
means that it will run once every month, the 19:th at 07:42 AM. To read more about how to configure cron, read here.
Other functionality in your backup script
You can add just about anything there, for example emailing of the backup status, using a script looking something like this:
- Code: Select all
#!/bin/sh
MESSAGE="/tmp/emailmessage.txt" #A temp file storing the contents of your email message
echo "Your backup started at " > $MESSAGE
DATE=`date +%y%m%d-%T`
echo $DATE >> $MESSAGE
echo "..with the following log information:" >> $MESSAGE
#Do the actual file transfer
#(The --delete option means that files deleted at main-Bubba also will be deleted at backup-Bubba, something that fits me better
#since I'm backing up email.)
rsync -avz --delete -e "ssh -i /home/backup-user/cron/backup-Bubba-rsync-key" main-user@main-Bubba:/home/main-user/Mail/ current
>> $MESSAGE
#Create a new folder with todays date and time
DATE=`date +%y%m%d-%T`
mkdir $DATE >> $MESSAGE
#Copy the contents of your current backup to todays folder
cp -r current/ $DATE/ >> $MESSAGE
echo "..and ended at" >> $MESSAGE
echo $DATE >> $MESSAGE
mail -s "Backup done" you@yourdomain.com < $MESSAGE
In this case, you would probably want to delete some of the older backup folders, keeping a cyclic repository of backups, but I'll leave this for your to solve.
Also, don't forget to restore /etc/apt/sources.list when you are done!
