How To : Mount Amazon S3 Bucket using S3FS

S3FS is a FUSE file system that allows you to mount an Amazon S3 bucket as a local file system. It stores files natively and transparently in S3 (i.e., you can use other programs to access the same files). Maximum file size=64GB (limited by s3fs, not Amazon).

Why use Simple Storage Service over EBS ?

Simple Storage Service provides redundancy by replicating data between multiple Avalibility Zone.

S3fs is stable and is being used in number of production environments, e.g., rsync , backup to s3 , s3 as a file system

Important Note: Your kernel must support FUSE, kernels earlier than 2.6.18-164 may not have FUSE support Virtual Private Servers (VPS) may not have FUSE support compiled into their kernels.
To use it:
1. Get an Amazon S3 account! http://aws.amazon.com/s3/
2. Download, compile and install.
3. Specify your Security Credentials (Access Key ID & Secret Access Key) by one of the following methods:
4. using the passwd_file command line option
set the AWSACCESSKEYID and AWSSECRETACCESSKEY environment variables
5. use a .passwd-s3fs file in your home directory
6. use system-wide /etc/passwd-s3fs file

Mounting S3 through S3FS STEPS:-

1 -Install Required Dependencies:-
#apt-get build-essential libfuse-dev fuse-utils libcurl4 openssl-dev libxml2-dev mime-support
2 -Grab the source
#wget http://s3fs.googlecode.com/files/s3fs-1.72.tar.gz
3 –Extract from Source
#tar xvzf s3fs-1.72.tar.gz
4 –Configure and compile
#cd s3fs-1.72

#./configure
#make
#make install
5 –Password file
#touch /etc/passwd-s3fs ; chmod 600 ~/.passwd-s3fs; chmod 640 /etc/passwd-s3fs; echo accessKeyId:secretAccessKey >/etc/passwd-s3fs
6 –Create a directory for it and mount it
#mkdir /mnt/s3
#/usr/bin/s3fs <bucketname> /mnt/s3

7- Permanent Mount (Entry in fstab)
#vi /etc/fstab
Append the following entry in fstab:-
For empty directory
s3fs#<bucketname> /mnt/s3 fuse allow_other,use_cache=/tmp 0 0
For nonempty directory
s3fs#<bucketname> /mnt/s3 fuse _netdev,allow_other,nonempty,use_cache=/tmp 0 0