JMeter and AJAX - Part III

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part III-1 - A diversion on how to use session ids between threads
Part IV - A more complex sample

In Part II we developed a sampler that would let us synchronize selected threads. I split that out because someday I might be tempted to actually write that timer. So lets now try and execute some AJAX requests in parallel. The demo site we will use is http://www.ajaxdaddy.com/demo-dhtml-autocomplete.html (Please don't run extensive load tests against this site - it isn't mine)
The site has an AJAX based autocomplete for countries. Access the site in FireFox, turn on Firebug and type au into the textbox and you can see the AJAX calls being made in firebug

We can see that two AJAX calls are made, one when we typed 'a' and one when we typed 'u' (There is an AJAX design pattern that applies here which doesn't seem to have been followed). So we will access this page and try to simulate two parallel calls.


The Init and Delay Sampler are from Part II. The Init sampler ensures only 1 thread does the initialization bits. The delay sampler has the rest of the code. After the Init sampler we just make all the threads wait so that no threads run before we are ready to begin (so don't use any ramp up time as that would be nullified here). Its essential the sync timer has a count equal to the count of threads in the threadgroup.
The IF controller is used so that only the main threads perform the request to the main page. Recall for 6 threads to run 2 at a time , we have the groups as Thread(1,2) , Threads(3,4), Threads(5,6). The even numbered threads will only be used for the AJAX requests. The Odd numbered threads will be used for the main sample and the AJAX requests. In our case the landing page needs to be accessed by the odd numbered threads. The even numbered threads will skip and land at the delay sampler waiting for their respective soulmates to join them(I'm currently waiting to be reunited with my wife hence the flowery language).
Anyway we have added a Gaussian random timer to allow the odd numbered threads to arrive at different times. When the odd numbered thread does arrive at the delay sampler, it along with its even numbered pair will be released to execute the Make AJAX request sampler (which chooses a word at random from the CSV file we have defined text,assert e.g. a,australia).

Hence two requests will be executed in parallel for the AJAX request sampler. If we had further requests after this we would have added another IF controller(only odd numbered threads) and coded the samples under this. if we wanted 3 requests in parallel we would have to adjust the Init sampler to say we want groups of 3.
Run the test. Though the samples may appear interleaved , note down the sample start times of the groups of threads that were supposed to execute in parallel, they will be more or less the same.

But I have cheated. The samples in parallel didn't really depend on each others data. The AJAX requests were exactly the same , differing in data. The AJAX requests didn't depend on the user's session thereby reducing the complexity of the script.

Lets attempt an example which needs the above. Onwards to Part IV (as soon as I find a good demo!)

JMeter and AJAX - Part II

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part III-1 - A diversion on how to use session ids between threads 
Part IV - A more complex sample

As mentioned in Part I , the most elegant way to simulate AJAX in JMeter would be to customize the HTTPSampler to allow child HTTPSampler requests to be executed in parallel.
Since customization of JMeter core isn't an option for me, we'll try and simulate this directly without customizing JMeter. Note that this is a toy, done for amusement rather than any real solution to an existing problem.

The first thing we should note that anything that we need done in parallel in JMeter needs its own thread. Suppose we have some requests that make two AJAX requests in parallel then the way we are going to work this is
Thread odd numbered(1,3,...) - Main thread that executes most of the requests
Thread even numbered(2,4,...) - Thread used only when needed for AJAX parallel requests so Thread(1,2) is a group as is Thread(3,4) and so on.
If we needed to make 3 AJAX requests in parallel then we would have grouped up Threads(1,2,3) and Threads (4,5,6) - but the main thread would have been Thread1, Thread4,...
You can immediately see resources are being wasted - Well we did say this was a toy.
Making the even numbered threads not execute some samplers is easy, we just need an IF controller to check. But how do you make Thread 1 and Thread 2 for e.g. execute a sampler in between at exactly the same time? In the Java world , this would usually mean reading up everything that Doug Lea and Brian Goetz have written - but in lazy developer mode this means downloading the JMeter source code and looking at SyncTimer (which blocks a fixed number of threads but not specific ones) and using the time honored technique of Copy Paste.
So before diving into the problem at hand lets see if we can simulate the following.
6 threads will arrive at varying times into one sampler. At which points they will wait till groups are ready i.e. Threads 1 and 2 will go together and Threads 3 and 4 will go together and so on.
Here's the Beanshell sampler which is actually a customized Sync Timer in disguise

import java.util.HashMap;
int totalThreads = 6; // MUST match total number defined in Thread Group
int groupsOf = 2; //MUST match number of requests to make in parallel

if (bsh.shared.myObj == void){
    // not yet defined, so create it:
    bsh.shared.myObj =new HashMap();
    bsh.shared.timerCounter =new HashMap();
    int numObjectsNeeded = totalThreads/groupsOf;
    for(int i = 0;i<numObjectsNeeded;i++) {
           bsh.shared.myObj.put(String.valueOf(i),new Object());
           bsh.shared.timerCounter.put(String.valueOf(i),new int[] { 0 });
    }
    print("populated");
} else {
   print("already init");
}

print(${__threadNum()});

int grpNo = (int)(${__threadNum()} -1)/groupsOf;
print(grpNo);
String groupNumber = String.valueOf(grpNo);
print(groupNumber);
Object syncObject = bsh.shared.myObj.get(groupNumber);
print(syncObject);
synchronized (syncObject) {
   int[] timerCount = (int[])bsh.shared.timerCounter.get(groupNumber);
   timerCount[0]++;
   final int count = timerCount[0];
   if (count >= groupsOf) {
                syncObject.notifyAll();
            } else {
                try {
                    syncObject.wait();
                } catch (InterruptedException e) {
                    log.warn(e.getLocalizedMessage());
                }
            }
            timerCount[0]=0; // Reset for next time   
}
SampleResult.setResponseData("sunk!");

The bsh.shared namespace is used to share objects between threads. So in our case we have 6 threads and we want 2 threads in parallel which means there are 3 groups of two threads each. To synchronize each group we need an object (hence 3 objects) and a counter each - to count how many threads are already available. Thats what the first part of the code sets up.(The first part is not thread safe but because we are going to test this by guaranteeing that threads arrive at different times , the snippet need not be thread safe).

The second part of the code then calculates which Group a thread belongs to, gets the object for the group, increases its count. If the number of threads is less than that we want in parallel it suspends the thread (by calling wait) otherwise it does a notifyAll which will wake up all suspended threads and allow these two threads to move in parallel

Heres what the test looks like


We have artificially added a timer to make sure requests to the Delay sampler arrive 1 thread at a time. We run the test and we can see that Threads(1,2) run the After Delay sampler at almost the same time , as do Threads(3,4) and as do Threads(5,6)
If we change the groupsOf in the delay sampler to be 3 then we see that three threads run in parallel.

And with that we are ready to use this delay sampler to make AJAX calls in parallel.
Onwards to Part III - An actual example of JMeter and AJAX

JMeter and AJAX - Part I

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part III-1 - A diversion on how to use session ids between threads 
Part IV - A more complex sample

Queries related to testing websites which use AJAX are often asked on the JMeter mailing list so here are my thoughts on this matter.

JMeter is not a browser, it simulates a HTTP request/response which means it will always struggle to replicate the finer parts of browser behavior. The advantage however is that it only needs very few resources to simulate a single browser. Testing tools that actually drive a browser will find it easier to replicate the browser however they pay for it when it comes to load tests - the amount of resources is needed is much more than the JMeter family of tools. But when testing out websites which use AJAX, JMeter with its current feature set may not be an accurate simulation because AJAX does depend on the finer parts of browser behavior.

An elegant solution would be to have the ability to define child HTTP requests of a parent HTTP sampler with a thread pool that defines how many of these will be made in parallel. But no such feature is currently available in JMeter.

But say you do have to test an AJAX enabled site today - what can one do?
It's useful in these times to be a lazy but honest tester - The honest part ensures that we will try to be accurate but the lazy part also ensures that we wont try to do any more than the minimum we need to do.

• So the first question that should come up is do we actually need to change anything in the way we test? The vast majority of AJAX enabled sites take the form a page loads , the user does some action , an AJAX call is made and values are shown to the user without updating the rest of the page. But look at the sequence of HTTP requests. Request for the main page, get response, browser processes response,user action, AJAX call made , AJAX response, further browser processing. There isn't any request made in parallel (ignoring all the static file requests which is its own blog post) - nor is there any difference due to Asynchronous nature of the call. So a JMeter script to simulate the above would have no difference from a normal JMeter script. the AJAX call should just follow the call to the main page and we are done.
• But hold on some AJAX enabled sites may make an AJAX call as soon as the page loads without any user action. Or the call might be made periodically.Even in these cases the onload/onready is only done after the response of the main request is made available so while the browser may make the request asynchronously in terms of the HTTP request/response there is no difference. The periodic call may only have a problem if the time it takes to return is greater than the periodic repetition. But the latter case would be a problem anyway , all you really want from JMeter is to detect if this happens. Once it happens your system is messed up and you need to fix it! There really isn't much value simulating the messed up state. To put it in another way , if your system breaks for 50 users, there isn't much value observing how much more is broken for 100 users.
• But hold on what if multiple AJAX requests are made on load? Say a users home page has two AJAX widgets , one which shows say his tasks , one which shows his emails? Well yes two requests will be made in parallel. Usually these requests are independent of one another. The question we really need answered is "Is there a difference between a user making two independent requests in parallel and two users(maybe with the same login) making a single independent request each in parallel?" In most cases , the answer is no. The load may be slightly higher in the latter case because you may have two sessions instead of one, but in terms of the rate of requests the server sees you can normally match the rate of requests by simply increasing the number of threads (and a little bit of math).
• But hold on what if the requests aren't independent? What if they utilize the same table/rows causing contention when its from the same user session what if widget2 depends on widget 1 for something in the session? Well one thing to do in this case is call the developer and ask him why the hell are there two dependent requests instead of a single one? That usually doesn't end well(even though the tester is right) - but yes at this time yes you might need to actually simulate requests in parallel. Note however a lot of AJAX enabled websites dont fall into this bracket and those websites can be simulated in JMeter with a reasonable degree of accuracy.

Onwards to Part - 2. Simulating AJAX requests without knowing how to modify JMeter core.

Google isnt as good as you think it is

Exhibit A - A snippet from webmaster tools for this blog



Though I know that writing about deepika instead of JMeter would give me more page hits!