OKCOOL

It’s interesting times for running online apps!

We’ve just launched an online viral competition for the clothing company Fat Face at http://www.fatfacechristmas.co.uk/

The idea behind this application is that the more friends you get to sign up from your own entry page, the more chance you stand of winning. Therefore, with any luck, it should see a sizeable amount of traffic.

I wanted to make sure this app would behave under sufficient load, so I needed to do a bit of load testing on it. I was lucky enough to spend a few days with a performance consultant a few years back who was trying to get one of my apps to run better under load. After we were done, with a few code tweaks and a new server, it went from dealing with about 5 simultaneous users to about 5000. This experience really opened my eyes to the importance of performance testing, but in this day of rapid app development, it’s difficult to find the time.

What do you need to load test?

• A working app you already think is working well - make your car road-worthy before taking it for a drive!
• Some software to run automated tests.
• A big hammer! You need to throw a lot of data at your app.

There’s no point load testing an application on your broadband connection, because you have a limited amount of connectivity. However, with the cheap availability of on-demand servers  it’s now very easy, quick, and cheap to run load testing.

Here’s what I did.

1. Used elastic fox to load up a Windows 2003 server on Amazon EC2.
2. Install my web testing software. I used a great product from Webload… it’s free and is very easy to use. There’s also loads of other great products such as WAPT but, for the price, you can’t beat Webload.
3. Record a macro of a user signing up for the competition.
4. Tweak the code to get around a primary key constraint.
5. Run some tests!

Most windows load testing software I’ve used has a very easy to use macro recording system (point 3). You hit record, carry out some actions on your website, and then close the browser. The actions are then saved as a script ready to be played back by your ‘virtual users’. The good thing about Webload is it saves these macros as javascript. This makes them very easy to tweak. For example, on our competition you can only sign up with a single email address once or an error will be thrown. I simply tweaked the macro I recorded to generate a random email adress.

Once I had my test script, I loaded it in to the app to run and set it up to throw 1, 10, 30, 100, 250, 500 and 1000 simultaneous virtual users at the application with a minute between each batch of users. It’s important to test a range of different user amounts and leave enough time between for the server to settle down so that you can see the point at which the application starts to slow down.

Here are my results from a single run:

You can see at 320 secondswhen it jumpy from 100 users to 250 simultaneous users we get a significant delay in response time.

It’s important to check what the server is up to whilst all of this is going on. I’m quite new to running linux boxes, but two good commands for doing this are free -m to show memory usage top (hold shift-m to order by memory use). This is great for seeing at what point the pysical memory runs out and the server starts swapping.

At the point it started to slow down considerably you can also see that the data throughput doesn’t change a lot. If you convert this to megabits it’s throwing about 15 megabits per second at the server. It’s quite realistic at this point that were’re coming up against the limit of what a single server can send or receive so if we wanted to take the test further we’d actually need multiple testing servers and we might need more than a single server on the app end to increase performance.

With Amazon EC2 you can easily save the state of a server (AMI - Amazon Machine Instance) so that you can quickly access it again. This means you can keep a ‘load testing’ server and call it up whenever you need it in the future.

I didn’t need to take my load testing much further than this as I was very happy with the performance.

Here’s some simple things to consider when ensuring your app runs quickly.

1. Offload static files JS, CSS, images to something like S3 or a CDN. If your server doesn’t have the throughput to serve data, it won’t make any difference how fast your app is.
2. Make sure your server isn’t swapping and has enough pysical memory.
3. Make sure you database server has enough query cache.
4. Use something like eAccellerator to cache your PHP compiles.
5. Cache output and hit your database a bit less.