Saturday, November 27, 2010

JMeter and AJAX - Part I

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part III-1 - A diversion on how to use session ids between threads 
Part IV - A more complex sample

Queries related to testing websites which use AJAX are often asked on the JMeter mailing list so here are my thoughts on this matter.

JMeter is not a browser, it simulates a HTTP request/response which means it will always struggle to replicate the finer parts of browser behavior. The advantage however is that it only needs very few resources to simulate a single browser. Testing tools that actually drive a browser will find it easier to replicate the browser however they pay for it when it comes to load tests - the amount of resources is needed is much more than the JMeter family of tools. But when testing out websites which use AJAX, JMeter with its current feature set may not be an accurate simulation because AJAX does depend on the finer parts of browser behavior.

An elegant solution would be to have the ability to define child HTTP requests of a parent HTTP sampler with a thread pool that defines how many of these will be made in parallel. But no such feature is currently available in JMeter.

But say you do have to test an AJAX enabled site today - what can one do?
It's useful in these times to be a lazy but honest tester - The honest part ensures that we will try to be accurate but the lazy part also ensures that we wont try to do any more than the minimum we need to do.

  • So the first question that should come up is do we actually need to change anything in the way we test? The vast majority of AJAX enabled sites take the form a page loads , the user does some action , an AJAX call is made and values are shown to the user without updating the rest of the page. But look at the sequence of HTTP requests. Request for the main page, get response, browser processes response,user action, AJAX call made , AJAX response, further browser processing. There isn't any request made in parallel (ignoring all the static file requests which is its own blog post) - nor is there any difference due to Asynchronous nature of the call. So a JMeter script to simulate the above would have no difference from a normal JMeter script. the AJAX call should just follow the call to the main page and we are done.
  • But hold on some AJAX enabled sites may make an AJAX call as soon as the page loads without any user action. Or the call might be made periodically.Even in these cases the onload/onready is only done after the response of the main request is made available so while the browser may make the request asynchronously in terms of the HTTP request/response there is no difference. The periodic call may only have a problem if the time it takes to return is greater than the periodic repetition. But the latter case would be a problem anyway , all you really want from JMeter is to detect if this happens. Once it happens your system is messed up and you need to fix it! There really isn't much value simulating the messed up state. To put it in another way , if your system breaks for 50 users, there isn't much value observing how much more is broken for 100 users.
  • But hold on what if multiple AJAX requests are made on load? Say a users home page has two AJAX widgets , one which shows say his tasks , one which shows his emails? Well yes two requests will be made in parallel. Usually these requests are independent of one another. The question we really need answered is "Is there a difference between a user making two independent requests in parallel and two users(maybe with the same login) making a single independent request each in parallel?" In most cases , the answer is no. The load may be slightly higher in the latter case because you may have two sessions instead of one, but in terms of the rate of requests the server sees you can normally match the rate of requests by simply increasing the number of threads (and a little bit of math).
  • But hold on what if the requests aren't independent? What if they utilize the same table/rows causing contention when its from the same user session what if widget2 depends on widget 1 for something in the session? Well one thing to do in this case is call the developer and ask him why the hell are there two dependent requests instead of a single one? That usually doesn't end well(even though the tester is right) - but yes at this time yes you might need to actually simulate requests in parallel. Note however a lot of AJAX enabled websites dont fall into this bracket and those websites can be simulated in JMeter with a reasonable degree of accuracy.
Onwards to Part - 2. Simulating AJAX requests without knowing how to modify JMeter core.

No comments: