I recently replaced my home server (which was old and was running Server 2008 Operating System) with a faster, so much quieter, high-end workstation. It has taken me some time (quite a few days) to install a new (updated) Operating System, reinstall all my required software (and re-configure my licences) and then install all the necessary updates and security patches. Finally, I had to transfer all my data between computers… (Thankfully I was spared that by using a good disk imaging utility!)
The final step in the process was to setup my backup and do my first (full) backup of a now re-engineered hierarchal file structure. I backup to Amazon S3 for many reasons, one being the durability and security of my data and the other important reason is being able to access my data from anywhere in the world (great if my laptop has a problem – I can still get to my files from any internet enabled device and work can carry on…)
Copying a large amount of data to S3 over my internet connection took just over 4 full days the last time I did a new full backup and I was keen to avoid the same issue again. Using S3 Transfer Acceleration, I was able to shrink this time to a more than manageable amount (just under 2 days - over the weekend), S3 Transfer Acceleration uses Amazon CloudFront (which is more typically considered a solution for caching outbound data and content as a Content Distribution Network (CDN) solution.
AWS S3 Transfer Acceleration
Amazon S3 Transfer Acceleration enables fast, easy, and secure transfers of files over long distances, or transfer of large amounts of data, between your client and an S3 bucket. Transfer Acceleration takes advantage of Amazon CloudFront’s globally distributed edge locations. As the data arrives at an edge location, data is routed to Amazon S3 over an optimized network path.
Use cases for Amazon S3 Transfer Acceleration?
Like me, you might want to use Transfer Acceleration on a bucket for various reasons, including the following:
- You have customers that upload to a centralized bucket from all over the world.
- You transfer gigabytes to terabytes of data on a regular basis across continents.
- You are unable to utilize all of your available bandwidth over the Internet when uploading to Amazon S3.
How to use Amazon S3 Transfer Acceleration
This is so easy – all you have to do is enable Transfer Acceleration on your bucket. (Obviously, for your bucket to work with transfer acceleration, the bucket name must conform to DNS naming requirements). Then transfer your data to (and also from) the acceleration-enabled bucket using a s3-accelerate endpoint domain name: yourbucketname.s3-accelerate.amazonaws.com
Does it Cost More?
Yes – ever so slight, the Data Transfer IN to Amazon S3 from the Internet (normally free, but using Transfer Acceleration over CloudFront = additional: (USD) $0.04 per GB (United States, Europe, and Japan).
If you move a lot of data into or out of Amazon S3, you HAVE to use S3 Transfer Acceleration!
