OKCOOL

For the last month or so we have been involved in a large scale overhaul of a public facing website. The site in question was pushing the 1000 page mark and contains plenty of large PDF documents.

Unusually for a site of this size there has been very little in the way of server side script (apart from an integration with Google search that we’ve done – love google!), it’s all been HTML.

When we put the new site live, there was a very high demand for it and some of the documents that were available. Unfortunately, the dedicated server it was on ground to a halt! It was frustrating because we weren’t hosting it ourselves and had no access to the box, we were told that the server was running fine, but clearly it wasn’t.

When we thought about the problem for a while, it was fairly obvious what was happening. The server itself was almost certainly fine because there was little or no server side processing to do as it was all static HTML. It should have been able to serve up 100’s of pages a second without an issue. Nope, the problem was purely bandwidth.

It’s difficult to tell exactly how high the demand was initially as it wasn’t our server, but we suspect that, within a few hours, there was somewhere between 1,000 and 10,000 requests for a 10mb document. Even spread over a 1 hour period, this meant anything from about 3mb to 30mb a second was trying to go through the server. Assuming the server had, at best, 100mbit throughput from end to end, this just wasn’t going to happen.

My first thought was to throttle IIS to only allow a trickle of data per connection. This might have solved it but would have resulted in painfully slow downloads for the users and without knowing the actual demand, it was difficult to gauge how heavily to throttle each connection.

Our solution was very simple, and it offered immediate resolution. We moved all of the key documents on to the Amazon S3 storage platform. With S3 you pay for what you store and transfer and there’s no limit on how much you can store or transfer. Also, being Amazon, there is no shortage of bandwidth.

After the slightly difficult process of updating the live site (it was of course still being hammered while we were trying to make the update) everything was resolved almost instantly and since the change the site has been performing well. In the last few days we have seem usage from S3 reach almost the 50gig mark… and it’s cost about £3!

Thanks Amazon!