I hope that most of you would be aware of DropBox, just in case you don’t have an idea about what dropbox does the following post would help you understand dropbox and its functionalities.
DropBox is basically a online backup system in which the users have an account associated with them. A folder named dropbox is created in the user’s documents folder(In windows 7). The files and folders that are copied to this folder is synced to a back end cloud storage system. (We do not care on how dropbox manages its user’s and stuff :D). First of all the major advantage of dropbox is that it syncs data on to the cloud when the user’s bandwidth is idle or under minimal usage. This post will tell you on how you can exploit Amazon EC2 cloud to have your own storage system that can be used to back up data. Using this method you would be able to access your data from any place (provided you have an internet connection :D). That is your data becomes portable. Now lets get to the task.
First of all i recommend using either CentOS or Ubuntu (10.10). It’s totally up to you regarding which distribution you use for this task. First of all I use root to access my files and I don’t follow certain ethics as no one else is gonna use my laptop.
And before I forget credits to fak3r for coming up with such a great idea. I had tried this one and it was like very useful and that’s why I thought of writing it on my own blog with some extensions to it.
First install packages rsync(must be included with the linux box), openssh-server and lsync.
Lsync is expanded as live syncing daemon. I have no expertise on this particular package AFAIK its a package that monitors the XML structure of the file system and it has the ability to run a particular command or another system call. The lsyncd makes use of the inotify feature of the linux kernel that is used to watch folders and report events.
Rsync is a package that is used to transfer files between folders or systems (local or remote). The algorithm is designed to check for changes that have occurred to the file and transfers the changed parts of the files and folders. Rsync can be used to sync data both locally and in remote systems and hence widely used in backing up systems and servers on a large scale.
Assuming you use Ubuntu
apt-get install lsync openssh-server
I assume lsync is available with apt-get else you need to download the source code from code.google.com and compile it.
wget http://lsyncd.googlecode.com/files/lsyncd-1.26.tar.gz
tar -zxf lsyncd-1.26.tar.gz
cd lsyncd-1.26
./configure
make
make install
You must be having an example lsyncd.conf.xml in the folder from which you extracted. Just copy it to /etc/lsyncd.conf.xml
Next you need to make alternations to the configuration file in your system. Open the configuration file lsyncd.conf.xml. Go the portion that tells you about the source and target. Now you job is to set the source to the folder you wish to backup on the Amazon Cloud. Say /home/ananth/syncdata . So your configuration file must have this line. (I mean modify).
<source path=”/home/ananth/syncdata” />
Next is the target path. Here you need to provide the IP address of the system with which you want to sync to. Here give the IP Address of the Amazon EC2 instance. I’ll talk about Amazon EC2 instances and IP Addresses in my next post and I’ll make sure I’ll the post the link here.
So your target should essentially look like this
<target path=”10.15.16.17:/home/ec2-user/syncbackup”/ >
Now in your terminal run
lsyncd –config /etc/lsyncd.conf.xml –debug.
If you get any errors then there is some problem with the setup. Since your are syncing data to a machine on the internet it should be fine. Make sure your read the /var/log/messages for any clue if the setup goes wrong. You must be able to figure out the errors.
Everything that I have done is in your system, the remote host configuration is up next.
Now hoping that you have an Amazon account I proceed with this post. Set up an Amazon EC2 instance. Attach it to an elastic IP address. Just make sure that your security considerations are fine. Make sure that port 22 is open and am sure it must be.
I’ll make sure that I update.
Ananthakrishnan Ravi 