Why I wont be using Amazon S3 … it’s not a complete bittorrent service.

After a lot of thought, quite a lot of code writing, lots of questions on the Amazon S3 forums, and more thought, I’ve decided not to use Amazon S3 to store the Ambisonic and DTS files from ambisonia.com.

Let me explain why.

The big feature that sucked me into considering Amazon S3 was the bittorrent support. Any file uploaded to Amazon S3 is available as a torrent. And on top of that, you can tweak (apparently) whether Amazon or other peers take priority in seeding the files (i.e. you can minimise your Amazon S3 bandwidth costs).

BUT! …. but but but …..

  1. You have no access to download statistics for the torrents. That means I will have no idea how many times something has been downloaded. I find this to be a huge limitation. The ‘currency’ of the internet is traffic/eye-balls/attention. We need to know how many downloads there has been.
  2. Uploading 200 MB (or more) files to Amazon often breaks (it is done by http post), and there is no resume upload feature. Everytime an upload is broken, it has to start again from scratch. That means that the people using the site will have to pay more in upload bandwidth, ambisonia.com will have to pay more in Amazon download bandwidth and everyone gets frustrated by an unreliable process.
  3. Amazon S3 can not act as a Bittorrent client. That means that it can not download files to itself (which is how ambisonia.com currently works). This means that one has to have a 100% complete upload before anyone else can download. In ambisonia’s current system, as soon as someone submits a torrent, anyone else can start downloading it instantly.

To me it is a no-brainer that Amazon should implement a full set of bittorrent features. For a start, allowing Amazon S3 to act as a bittorrent client would instantly solve the missing ‘resume upload feature’. Bittorrent is a rock-solid protocol, it can survive drop-outs, lost connections etc. etc. You can turn your computer off, turn it back on again, and bittorrent will effortlessly resume any uploads/downloads you were doing before.

Its taken a lot of foresight for Amazon to include _some_ bittorrent features into S3. Its a shame that what they’ve left out includes some of the more powerful features of bittorrent. And given that more complete bittorrent support could solve the missing ‘resume upload feature’, I reckon they’re mad not to do it.

So I will have to stay with running my own torrent tracker, and will have to find some other mechanism for scaling torrent seeding.

This entry was posted in Uncategorized. Bookmark the permalink.

2 Responses to Why I wont be using Amazon S3 … it’s not a complete bittorrent service.

  1. suntzu says:

    Maybe EC2 + SQS could be the solution you are looking for

  2. I can understand how EC2 could help … but how does the SQS fit in?

    EC2 would be somewhat expensive though …

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s