Easy user uploads with direct S3 uploading

If your site has a lot of user-generated uploaded content, such as a video or photo sharing site, Amazon's S3 cloud storage platform has a pretty phenomenal feature that lets them do all the heavy lifting involved with uploading. This means that your app can support massive uploads ranging from multi-gigabyte video files to thousands of photos in one sitting, without  touching your application servers!

The theory is simple, although it may take a bit of time to set up and get working correctly. The client side is easy: Simply point your form at the amazon servers, and include a few extra fields generated by your server. The server side is a little more involved, but in a nutshell, your server will serve up a signed policy document that lets the Amazon S3 servers know that this upload has been authorized by you. The policy document can limit size, destination filename (i.e. S3 key), filetype, and a variety of other useful parameters to make sure your users don't abuse the feature and upload something that doesn't belong.

To get your feet wet, follow Amazon S3's official documentation. This is easy to set up, but quite limited as it only supports one file at a time and does not fit easily into your application structure.

Modern web app users want multiple file uploads, drag and drop support, progress bars, thumbnails, upload resuming, and other features we've grown to expect from using media distribution sites like YouTube or Flickr. Fortunately, with the right JS libraries, your site can get these features with S3 direct uploading too.

For the ultimate uploading experience, I would recommend setting up Widen's Fine Uploader, an stupendous JS app that has direct S3 handling built-in. Although the site does not make it clear, you can download the GPL 3.0 sourcecode and compile it yourself using grunt, or you can  purchase a pre-compiled version that comes with a full-license. Also confusingly, there are several JS upload scripts all named "Fine Uploader" — Widen's  is the only one with direct S3 uploads, so it's the one you want.

Test out the Fine Uploader demo, see if you like it. If you do, and want to get this on your site with direct S3 uploading support as quickly as possible, then follow these steps:

  1. Either purchase or compile Widen's Fine Uploader
  2. Follow this excellent guide carefully to get your Amazon S3 settings configured correctly.
  3. Include Fine Uploader in your form, customizing it to your hearts content.
  4. Set up your server to sign policy documents for the browser to get proper authentication, using these server side examples (which includes Django, albeit unidomatically).

And you're good to go! Enjoy gigabytes of blazing-fast uploads, paying only how much you would normally for storing them on S3.