Modifying JMeter scripts programatically

Problem : Is there any scripting or GUI way to massively change from WebService(SOAP)
Request (DEPRECATED) to SOAP/XML-RCP Request? http://jmeter.512774.n5.nabble.com/Question-about-WebService-SOAP-Request-DEPRECATED-td5718693.html

Solution : The JMeter script file is an actual XML file so any solution that manipulates XML can be used to solve the above problem.  As Michael Kay's excellent books on XSLT were staring at me , the way I implemented this was in XSLT.
The first thing we need to do was use what is called an identity transform

<xsl:template match="@*|node()">
 <xsl:copy>
  <xsl:apply-templates select="@*|node()" /> 
  </xsl:copy>
</xsl:template>

i.e. essentially copy the source XML to the destination. With that done we can now get to the problem at hand - which is intercept all the deprecated requests and transform them to the new one

In order to do so we can create a JMeter script which has both of these elements and look at what XML gets generated so that we know the source and destination XML we want
The deprecated XML

 <WebServiceSampler guiclass="WebServiceSamplerGui" testclass="WebServiceSampler" testname="WebService(SOAP) Request (DEPRECATED)" enabled="true">
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
          <stringProp name="HTTPSampler.domain">server</stringProp>
          <stringProp name="HTTPSampler.port">port</stringProp>
          <stringProp name="HTTPSampler.protocol">http</stringProp>
          <stringProp name="HTTPSampler.path">path</stringProp>
          <stringProp name="WebserviceSampler.wsdl_url"></stringProp>
          <stringProp name="HTTPSampler.method">POST</stringProp>
          <stringProp name="Soap.Action">ws-soapaction</stringProp>
          <stringProp name="HTTPSamper.xml_data">ws-SOAP-DATA</stringProp>
          <stringProp name="WebServiceSampler.xml_data_file">C:\Users\dshetty\Downloads\apache-jmeter-2.9\bin\logkit.xml</stringProp>
          <stringProp name="WebServiceSampler.xml_path_loc"></stringProp>
          <stringProp name="WebserviceSampler.timeout"></stringProp>
          <stringProp name="WebServiceSampler.memory_cache">true</stringProp>
          <stringProp name="WebServiceSampler.read_response">false</stringProp>
          <stringProp name="WebServiceSampler.use_proxy">false</stringProp>
          <stringProp name="WebServiceSampler.proxy_host"></stringProp>
          <stringProp name="WebServiceSampler.proxy_port"></stringProp>
          <stringProp name="TestPlan.comments">comment</stringProp>
        </WebServiceSampler>

The non deprecated SOAP request

        <SoapSampler guiclass="SoapSamplerGui" testclass="SoapSampler" testname="SOAP/XML-RPC Request" enabled="true">
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
          <stringProp name="SoapSampler.URL_DATA">url</stringProp>
          <stringProp name="HTTPSamper.xml_data">data</stringProp>
          <stringProp name="SoapSampler.xml_data_file">c:/test.xml</stringProp>
          <stringProp name="SoapSampler.SOAP_ACTION">soapaction</stringProp>
          <stringProp name="SoapSampler.SEND_SOAP_ACTION">true</stringProp>
          <boolProp name="HTTPSampler.use_keepalive">false</boolProp>
          <stringProp name="TestPlan.comments">comment</stringProp>
        </SoapSampler>

Knowing this , the template is easy

  <xsl:template match="WebServiceSampler ">
    <xsl:element name="SoapSampler">
        <xsl:attribute name="guiclass">SoapSamplerGui</xsl:attribute>
        <xsl:attribute name="testclass">SoapSampler</xsl:attribute>
        <xsl:choose>
            <xsl:when test="@testname ='WebService(SOAP) Request (DEPRECATED)' ">
                <xsl:attribute name="testname">SOAP/XML-RPC Request</xsl:attribute>
            </xsl:when>
            <xsl:otherwise>
                <xsl:attribute name="testname"><xsl:value-of select="@testname"/></xsl:attribute>
            </xsl:otherwise>
        </xsl:choose>
        <xsl:attribute name="enabled"><xsl:value-of select="@enabled"/></xsl:attribute>
        <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
            <collectionProp name="Arguments.arguments"/>
        </elementProp>
        <stringProp name="TestPlan.comments"><xsl:value-of select="stringProp[@name='TestPlan.comments']/text()" /></stringProp>
        <stringProp name="SoapSampler.URL_DATA"><xsl:value-of select="stringProp[@name='HTTPSampler.protocol']/text()" />://<xsl:value-of select="stringProp[@name='HTTPSampler.domain']/text()" />:<xsl:value-of select="stringProp[@name='HTTPSampler.port']/text()" /><xsl:value-of select="stringProp[@name='HTTPSampler.path']/text()" /></stringProp>
        <stringProp name="HTTPSamper.xml_data"><xsl:value-of select="stringProp[@name='HTTPSamper.xml_data']/text()" /></stringProp>
        <stringProp name="SoapSampler.xml_data_file"><xsl:value-of select="stringProp[@name='WebServiceSampler.xml_data_file']/text()" /></stringProp>
        <stringProp name="SoapSampler.SOAP_ACTION"><xsl:value-of select="stringProp[@name='Soap.Action']/text()" /></stringProp>
        <stringProp name="SoapSampler.SEND_SOAP_ACTION"><xsl:value-of select="(stringProp[@name='Soap.Action']/text() != '')" /></stringProp>            
        <boolProp name="HTTPSampler.use_keepalive">false</boolProp>        
    </xsl:element>
  </xsl:template>

Where we just map the older elements into their newer forms (wherever possible). So for example we match an element named "WebServiceSampler" and replace it with "SoapSampler" . We transform the testname to either use the default if the deprecated element used the default or we use the name the user specified. We copy attributes and nested elements.
 You can then use ANT or any other way to style your input scripts into newer ones for one or many files

<target name="transformWS">
    <xslt
        in="CompareXML-RPCRequest.xml"
        out="test.jmx"
        style="transformWS.xsl"/>

</target>

An alternate approach would have been to generate java code that could parse XML and remove/replace objects but the identity transform mechanism makes this technique hard to beat. Any Object binding mechanism would run into issues when JMeter changes it schema but the XSLT technique is limited to only having to worry about source and destination.
I've also always toyed with the idea of creating a functional suite of tests that user's would  run at will without being limited to the tests defined in the file and this is the approach I would use there as well - more to come.

Example - https://skydrive.live.com/?cid=1BD02FE33F80B8AC&id=1BD02FE33F80B8AC!892

Simulating Abanadoned flows in JMeter

A user wanted to simulate abandoned flows in JMeter - e.g.a  multi step checkout scenario where some number of users drop off at every step

This was my suggestion

+Sampler1
+ThroughPutController (90 , percent executions , uncheck per user = 10% abandon)
++Sampler2
++ThroughPutController (80 , percent executions , uncheck per user = 20% more abandon at this step)
+++Sampler3

Sharing session ids across threads in jmeter

Problem i want to use the same JSESSIONID across the test plan which includes several Thread Groups(http://mail-archives.apache.org/mod_mbox/jmeter-user/201303.mbox/%3Cloom.20130311T124606-446@post.gmane.org%3E).
Which is actually two problems
a. How do I share data between threads in JMeter?
b. How do I connect to an existing session in JMeter given the session id?
Note : A good question to ask at this stage is do I really , really want to do this? Usually a Jmeter thread maps to actions that a user can take (AJAX aside) - doing something like the above goes away from the JMeter model of things and an alternative to consider is why don't I change my test so that a user's session is only needed for a thread? (mini rant - This is a common scenario in support where there is a problem X , the user thinks that doing A will solve his problem, A needs B to be done , B needs C and C needs D and the user then asks "How do I do D?"without revealing his cards - in this case why is the test plan for a user split across multiple thread groups?) The user should really be asking how do I solve problem X because some person might say why not do Action Y which is simple) .

But because we will have a blog post to write , we will assume the answer to the above question is Yes, I really want to do this and this is the best way to solve the problem I have.

How to share data between threads in JMeter
JMeter's only out of box way to share data between threads is the properties object and the shared namespace in BeanShell[2] both of which have limitations. The properties object needs you to come up with some scheme to share data and also needs you to write your own code for e.g. if you consume data much faster than you can generate it.

One technique is that if ALL the data that is to be shared is to be obtained before the next threads need it then one option is to use the CSV Data Set config - The first set of threads write to the file and the next set of threads read from this file - but essentially any shared memory(e.g. a file, a database etc) will do. However if you need to read and write this data at the same time then another way is to use the standard Java data structures which are thread safe.

In this post we will look at two ways
a. The CSV data set config
b. Using a java object
Note that there is a plugin available which will do b. from Jmeter plugins InterThreadCommunication [1] . If you aren't a programmer then the plugin is probably best for you - though I will say that good testers these days need to have programming and scripting skills.

However we will just write our own for fun. You also might need to do this to implement a complicated scheme.

Before we begin we will need an application to allow us to test whatever we write. So I have deployed a web application which has 2 URLs
a. set.jsp which sets an attribute into the session with the name uuid and the value a random generated value and also returns it as part of its output. Since this is the first page we access it will also create a session and set a cookie.

b. get.jsp which reads the value of the attribute uuid and returns it as its output. If you have the same session and previously accessed set.jsp then you should see the same value as step a.

If you dont have a session(or didnt access set.jsp) then you will see the value as "null"

So let us first write a simple test to verify that all is well. Here is a screen of the test.

We simply access set.jsp. Extract out its response . Then we access get.jsp and assert that the value we extracted out matches whatever is being returned by get.jsp. If the same session is being used for both the requests then the assertion will pass , otherwise it will fail.

Remember that Java web applications allow sessions to be maintained in two ways - either the Session ID is passed as part of the URL or as a cookie (usually named JSESSIONID) or both.
For this example we will assume we are using cookies.

Lets cause the test to fail - A simple way of doing this is by disabling the HTTP Cookie Manager. If we disable this JMeter wont accept cookies and hence wont be able to maintain sessions and every request will be a new session.

So we disable the HTTP Cookie manager and run the test. It fails as we expected it. Lets verify that it's because of the session
a. Note the first request gets a Set-Cookie back from the server - the server is trying to get the client to store the SessionID so that the next request can send the SessionID back

b. Note that the next request does not send a cookie header from JMeter

c. Note that the server responds to this request with another set-cookie (because it will try to create a new session as it didnt get any existing session cookie

Now lets enable the HTTP Cookie Manager. It works!. Lets verify the cookie was passed

Now that we have some degree of confidence that our assertions work (i.e. failures are flagged as failures and successes as successes) lets rerun the test with multiple threads and multiple iterations. All work. We can also see that different set requests get different cookies. This is because a cookie manager is scoped to a single thread (so each thread gets its own cookie and session) and because we have checked  "Clear cookies at each iteration" on the cookie manager.

Sharing Data using Java Objects in realtime
Lets first create a class that will allow data to be shared. For this purpose we use a  LinkedBlockingQueue  [4]. The javadoc for the interface BlockingQueue.
"Note that a BlockingQueue can safely be used with multiple producers and multiple consumers."
Which is a fancy way of saying something can be accessed by multiple threads without the programmer needing a degree in Computer Science. Since we will be reading and writing from different threads in different thread groups , the fact that this data structure natively supports thread safety frees us up from having to implement it - no synchronized keywords are needed in our code
However we do need to store this queue object somewhere , so we simply create a wrapper class that holds on this Queue statically and provide getters and setters. Because we dont know the rate at which we will read/write we simply put a timeout on the get.

package org.md;

import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit;

public class Queue {
    private static LinkedBlockingQueue<Object> q = new LinkedBlockingQueue<Object>();
    public static void put(Object o) throws Exception{
        q.put(o);
    }
    public static Object get(long timeout) throws Exception{
        return q.poll(timeout, TimeUnit.MILLISECONDS);
    }
    public static void clear() throws Exception{
        q.clear();
    }
    public static void main(String [] args) throws Exception{
        
        Queue.put("one");
        Queue.put("two");
        //q.clear();
        System.out.println(Queue.get(1000));
        System.out.println(Queue.get(1000));
        System.out.println(Queue.get(1000));
    }
}

We run a simple single threaded test on this class to see that everything is fine (it is), we compile this into a jar, place the jar in JMeter's lib directory and our class is ready to be used.
 The test structure is shown in the screen below

a. The setup thread group clears any data that might already be there in the queue. Because we are holding onto the Queue statically and the JMeter GUI is a single JVM , we need to clear out any data that might be there from a previous run. This is the code in the Beanshell sampler.

org.md.Queue.clear();

b. We create a threadgroup and use a request to create the sessions , extract out the data that we need - the sessionid and the value returned by the page so that we can assert that we are connecting to the same session. We then call the class we have just created to put these values into the queue (as an array with two values) and finally we introduce a random delay values
The regular expression to extract the cookie is in the screenshot above

And here is the BeanShell code to add the cookie and the extracted value to the Queue

String[] data = new String[2];
data[0] = vars.get("jsessionid");
data[1] = vars.get("uuid");
org.md.Queue.put(data);

c. Next we create a threadgroup that is going to use the session ids set from the previous threadgroup using a BeanShell preprocessor. The preprocessor also has the code to add the session id cookie so that requests will connect to the existing session. As before we assumed that the Server manages session in the form of cookies. But we could also have it as part of the URL once we are able to successfully read the SessionID.
We also configure the thread groups to have different number of threads so that there will be some variation in the rate of requests (note that if you have more Sessions created in the "set" threadgroup than you can process in the "get" threadgroup you will eventually run out of memory unless you make the queue bounded in size. If you have more in the "get" threadgroup then they will keep waiting (hence we have a timeout when we ask for a session id)

import org.apache.jmeter.protocol.http.control .CookieManager;
import org.apache.jmeter.protocol.http.control.Cookie;
String[] data = (String[])org.md.Queue.get(60000);
if(data != null) {
    vars.put("jsessionid",data[0]);
    vars.put("uuid",data[1]);
    Cookie cookie = new Cookie("JSESSIONID",vars.get("jsessionid"),"localhost","/", false,-1);
    CookieManager manager = sampler.getCookieManager();
    manager.add(cookie);
} 

The code above simply asks the queue for a value, and will wait upto 60 seconds for it (This only applies if we are consuming the data much faster than we can produce it). Then if we did get a valid value we use the JMeter API's[3] to access the Cookie Manager and add a cookie to it. Note that we do need a Cookie Manager for this to work.  Some values have been hardcoded for simplicity (like the domain name)

We run the test and its all successful! Remember we do have assertions so a success is indeed a success. We can also verify using the view results tree listener and we can also check the cookie is being sent in the GET request.

Cons : Note that anything that needs data to be shared between threads , that has wait and synchronization( irrespective of whether you do it or whether the JVM does) adds an overhead to the test - so the amount of load you can generate from a single JMeter instance will reduce. Also if you need more sophisticated schemes to share data (we used a simple FIFO model) then your code is more complicated , more prone to errors and likely to add more overhead.

Sharing data using files
Using a file is usually not a good candidate for sessionids because sessionids are short lived and may timeout. You also need to have everything available in the file before you can start reading the file. They are more suited to usecases like testing for Remember me or keep me signed in type of cookies. However the flexibility a CSV data set config provides may make it worth your time.
Here is the structure of the test.

The setUp thread group sends a set of requests to create the session and get the values as before.  To write all these values to a file we use a simple data writer and configure JMeter (by modifying jmeter.properties) to write the two variables we are interested in only. Note that this is probably not a good way to do it - in addition JMeter appends the data to an existing file so either you need to delete the file before you run this or use a property that varies dynamically so that a new file is created.

Remember that the setUp thread group will always execute before any other threadgroup , so this file will be created and have all the data before the next thread group runs.  The Simple data writer unchecks all the checkboxes and we edit jmeter.properties to have the following line

 sample_variables=jsessionid,uuid

The next threadgroup simply reads the file and attaches the session id to the cookie manager.

We can have as many threads as we want (we ensure that clear cookies at every iteration is checked so that every time we request something we use the cookie that is set by the beanshell preprocessor and not the one the cookie manager had previously saved). The Beanshell code to add the cookie is the same as before

import org.apache.jmeter.protocol.http.control .CookieManager;
import org.apache.jmeter.protocol.http.control.Cookie;

Cookie cookie = new Cookie("JSESSIONID",vars.get("jsessionid"),"localhost","/", false,-1);
CookieManager manager = sampler.getCookieManager();
manager.add(cookie);

we run the test - all successful. The file generated has the values

3C7C0BD5CA87C16FA65EAECEEBB54CF6,2db51396-6db2-44cb-9336-c732e52f8ac1
5E83FA9D286583DE32E6CC7C4E799599,f173a3ac-9779-408e-8cdb-7d23dfd5e2df
AFB458860FDF0376B9C8B9F52117A668,9f2da7f4-e848-4267-af9d-806e229f9c9f
540A42700742141A8BB8C2941666C8C9,84355a19-d2e1-45b7-ac42-23e9ab03a195
A7791553761A3613064E6BF7564B76E9,c3b14e92-93bc-4500-93be-ecbe43e70a5a
DB79953CEBA4C6E4FE708AE127E53F26,c99a880d-297d-4c00-9ff2-964449d9d106
3A7BBBC042EDB07D03901EEED8D37DC1,b2b29cb8-5261-4345-9405-80973d9f7334
913AFEC1A2D22EF6C64D7B99675DC86B,13bb3aa1-36cc-4e60-942c-d3a29a74b99f
DD06AEC9AF9D7CFF4FFB0AA2462E63C0,51b7b486-e503-43f8-a3ca-e36eb0df88a8

The cookies get added to the request

We can change the CSV data set config to keep reading the same file instead of terminating at the end - that works as well. You can also check that if you disable the setup threadgroup and rerun the test , it still works (because the sessions are still active - assuming they havent timed out) - so if the data you are saving in this file is still valid it can be used anytime and anywhere. You can also bounce the server in which the JSP web application was deployed. This causes all the sessions to get invalidated and hence the file to have invalid data. Now if you run the test , all the samples will fail because all the sessions are invalid - so even though we added the session id , the server has no data for this session.

Cons : As mentioned above , this isn't realtime sharing , it relies on you already having the data to be used , and the data being valid.

Files used in this post available at https://skydrive.live.com/#cid=1BD02FE33F80B8AC&id=1BD02FE33F80B8AC!886

References
[1] InterThread Communication- http://code.google.com/p/jmeter-plugins/wiki/InterThreadCommunication
[2] BeanShell Shared namespace - http://jmeter.apache.org/usermanual/best-practices.html#bsh_variables
[3] Cookie Manager - http://jmeter.apache.org/api/org/apache/jmeter/protocol/http/control/CookieManager.html
[4] LinkedBlockingQueue - http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/LinkedBlockingQueue.html

JSON in JMeter

http://jmeter.512774.n5.nabble.com/Require-help-in-Regular-Expression-td5713320.html
Roughly user gets a JSON response and wants to use some of that data for the next JSON request. Using a regular expression to extract out multiple related data is sometimes painful. So the alternative I suggest is to use JMeter to parse the JSON object and then use it (The other would be to have JMeter to natively support JSON , but unless JMeter also has a mapping tool , this only saves two lines of code , so I don't think its that useful).

So first we download Douglas Crockford's reference implementation from JSON in java , compile it and generate a jar and copy it into JMeter's lib directory (please ensure you compile it with a java version that is compatible with whatever you will use to run JMeter) - and don't use this library for evil.

Next launch JMeter.

To spoof this example , we create a BeanShell sampler that will return the JSON string we are interested in. To this sampler we add a BeanShell post processor that will parse the response and form the object that we need for the next request. The code is given below

import org.json.JSONArray;
import org.json.JSONObject;

String jsonString = prev.getResponseDataAsString();
JSONArray equipmentParts = new JSONArray(jsonString);
JSONArray parts = new JSONArray();

for(int i=0;i<equipmentParts.length();i++ ){
    JSONObject equipmentPart = equipmentParts.getJSONObject(i).getJSONObject("equipmentPart");
    JSONObject allAttributes = equipmentPart.getJSONObject("allAttributes");
    JSONObject part = new JSONObject();
    part.put("partId",allAttributes.getLong("equipmentPartId"));
    part.put("partNumber",allAttributes.getString("compositePartName"));
    // add more here
    parts.put(part);
}

vars.put("jsonResponse", parts.toString());

Note we first get the response as a string. Then we just use the JSON libraries to parse this string into a JSON array (equipmentParts). After which we iterate over every object, extract out the attributes we are interested in (you could even have an array of attributes and have a simpler loop) and then we start forming the object we want to post (i.e. the parts JSON array and the part JSON object)

Next we simply convert the JSON parts array into a String to be used by the next sampler by setting it into the jmeter variable jsonResponse.

The next sampler simply uses a POST and only specifies the variable ${jsonResponse} in the value field(So that it will be used as is in the body).
We run the script. The Debug Sampler is used to verify that the value we setup is correct and that it does get posted in the next request

Sample code is available here https://skydrive.live.com/#cid=1BD02FE33F80B8AC&id=1BD02FE33F80B8AC!876

JMeter

Paraphrased Question on the JMeter mailing list
While working on JMeter I prepared some Test Plans in JMeter and executed
them with different combinations and I put forward the Results,graphs etc.
so please guide me what kind of details/screens/graphs/files should my
presentation include and what should not?

The first Shakespearean to be or not to be dilemma I have when I read the above is should I try to answer this question ?. On one hand I'd like to help the person asking the question, and this is more of a theoretical exercise where I can blab away without fear of consequence or effort needed to actually do what I'm saying. On the other hand the question is so generic that much of what I say will have to have so many caveats to be accurate, there will be a bunch of follow up questions I will have to answer, that my blabbing away might mislead said person because I don't really know what the person wants and as an engineer I hate to speculate without knowing. Perhaps a brusque "what are your requirements?" with an additional dose of the sad state of software testing where people don't know their goals or their requirements , if I'm in bad mood. with a slight guilty conscience since I've been here and done this and asked such questions. Should I feed the person or teach the person to fish? Since the tester seems to be a beginner will my long drawn caveat ridden, opinionated, biased opinion help the tester or make the matter worse?

Any Road.

The single most important thing to know is when you ran the tests what did you want to know about the system? What did you expect to prove? What would you consider a successful test? What would you consider a failure? or in other words What were your requirements?
If you have ever read a technical work on Testing systems , you should know that there are various types of tests and indeed there are even various types of Performance tests. Unless you know this you don't know what type of tests you need to run and you don't know how to interpret the results, indeed you probably don't even know which system to look at.

To illustrate with some examples
a. You want to know , for a specified load, whether all your operations succeed (an operation could be anything like placing an order or searching a value).
b. You want to know, for a specified load, whether your system performs acceptably
c. You want to know, for a specified load, which has been kept on the system for a sufficiently long time, your system shows acceptable characteristics.

You will notice there are a lot of weasel words in the examples above. What is "specified" load and how do we determine that?. What is "performs acceptably" and how do we determine that? What is "acceptable characteristics" and how do we determine that?

And the answer is - it depends , hence the weasel words.
Take specified load for example - How many users? How many of them hitting the site concurrently? What think time?
And answer goes back to "why are you running your tests?".
Suppose you have changed some functionality - say you refactored some code. then usually the specified load is the same as the one you see in production. i.e. you are trying to find out if your change has no impact. Success is determined if the system performs the same or better with your change.
Suppose you are doing capacity planning - Then the specified load is the number of users you expect to access your site (perhaps sometime in the future as your site grows). Success is determined by the response time and other system characteristics
Say you are going live with a new site. Then specified load is the maximum load you expect the site to see. Success is determined by the response time being lower than those specified and system characteristics being acceptable
Say you want to check if your site can take the added shopping load when Thanksgiving arrives. Then specified load is the peak burst you expect to see and so on.

It isn't easy. Most testers don't even think this is part of their job - I guess it isn't. But asking these questions should be.

Now lets complicate it some more. Lets say you like the large majority of testers are interested in the "response time" for some load. Lets break this up into two parts
a. The accuracy of the simulation and
b. The limitation of the tool under test.

a) When you run any load test, you are running a simulation (except in a very rare number of cases). The accuracy of the simulation depends on how good your test script is, but it also depends on your test environment. This is it's own blog post but unless you know your simulation is reasonably accurate or you have safety margins built in , your tests aren't any good. This isn't particularly earth shattering - but how many testers actually validate that their simulation is accurate? Many don't. If you run a JMeter with 500 threads on a single windows machine, with no think time - and you dont get an out of memory error , is your simulation accurate? In most cases the answer would be no. The JMeter machine will not be able to generate the same load as 500 windows machines each running a single user. Having 500 machines is not a practical scenario for most people (perhaps this will change with the cloud) - but the key here is if I can live with some margin of error , what is the number of machines I should have and how do I determine this? The answer is again - it depends. It depends on the application, It depends on your test, It depends on the machine you are running from, It depends on the network etc etc.
One of the ways to validate this is run the test , see some of the values you are getting from Jmeter. Now while the test is running (this is important!) , using some other machine , that is as independent from the Jmeter machine as possible , access the site under test with a browser (or even a single user JMeter thread) and see the value you get. if this is close enough to the average Jmeter value - then great!. If it varies wildly then you might have an issue. Another way is half the load on the Jmeter client machine and add another Jmeter client machine with half the load. Do you see substantial changes in response times? Do you see many more concurrent threads on the server? if so you have a problem. Repeat this till you see no significant  difference. Or you can monitor the jmeter client machine - is memory/cpu pegged at highest value? If so probably the test isn't reliable.

b) Jmeter is not a browser yadda yadda. Do not expect to be able to tell browser render times without some additional tests. Running tests from an intranet is probably not going to give you the same time as when a user from some far off country accesses your site and gets a page of 500KB. Remember to factor in the network (either guesstimate or have different machines with different locations if possible)

Remember however that you can never have 100% accuracy (well perhaps you could - it depends :). An important part of engineering is knowing what tradeoffs to make when you have limited time and resources.

Dynamically changing sampler name in a JMeter test

This came up when a user was reading URL's from a file and needed to extract out a value from the Response that he wanted to use as the sampler name (so that his report would show this information)

The solution is roughly
a. Extract out the comment using regex post processor or equivalent
b. Add a BeanShell Post Processor  as child of the sampler write code like
prev.setSampleLabel(vars.get("

variablenamefromstepa")); => replace the name with the value you extracted

The 17th law of defects

"If a functionality is not tested for 3 months by business users , the maximum severity that can be assigned to it is S3"

Source : Deepak's book of software testing laws

And the winner is - Stripes!

Recently I had the unpleasant experience of having to select a "framework" for a J2EE application.

In my day job , the decisions have been usually made for me (Weblogic Portal aka BEA Weblogic Portal aka Oracle Weblogic Portal). Pageflow was ok (even though the stateful behavior made many normal HTTP operations painful). But the Portal parts were decent and we did have reusable(the holy grail!) portlets.

When it was time to choose something for the web application I wanted to develop , there were quite a few options. Years ago , this would have been simple. Free , documented, widely used == Struts. Now this isn't quite so simple.

The first thing I eliminated was a portal container. The standard ones use JSR-168 or 286 and in their pursuit of WSRP, Federation and other portal goodies forget that they are actually using HTTP. A web based framework that needs you to jump hoops to get the HTTPServletRequest usually indicates that the creators of the framework weren't thinking their actions through. If there ever was a J2EE spec that was worse than EJB , it's JSR 168. In their crazy pursuit of the least common denominator , we get crippled products. No thank you.

I did however want something that would let me easily reuse menu's , footers , right hand modules etc. jsp:include just doesn't do it so well , where does the underlying code that fetches data for these common pages reside? If the JSP is to be just a dumb view, this is a problem.

Struts shows it age (and you need Tiles to support the above and tiles is painful) . Perhaps my views are clouded due to a project that I had worked in a past life. I assume the architect(s) wanted to beef up their CV so they had Struts, Tiles, Spring, Hibernate, iBatis , ofBiz, freemarker, ofbiz's persistence framework and every single open source technology that was somewhat popular.  Getting all of these to work together was an exercise in frustration (note this was years ago) - but as soon as I am told to use Tiles I remember this project and I stop.

Enter Spring. When I first used it , I was impressed with how well it did basic stuff. I loved Spring JDBC - this was better than crappy ORM frameworks(yes Hibernate , you too). It let people who knew SQL and enjoyed SQL work with SQL without the normal associated JDBC trappings (to this day , my homegrown SQL utilities closely resemble Spring JDBC API's). Spring MVC should be good , right? Perhaps it is. But the documentation sucks and I don't have that much time to invest. It also felt that Spring MVC was more concerned with Spring Integration and the MVC pattern than it was in solving the problem of a easy to use web framework (totally biased , totally unjustified view , based on a cursory reading of documentations - and apologies to the Spring team, who I salute - but really your documentation should be better if you want to draw in people like me)

Enter Wicket - But then two lines from the documentation
"JSP is by far the worst offender" "Wicket does not introduce any special syntax to HTML. Instead, it extends HTML in a standards-compliant way via a Wicket namespace that is fully compliant with the XHTML standard".
Look at the feature list - http://wicket.apache.org/meet/features.html . Does it start with simple to use , web framework , supports all web related features? No it starts with we use POJO's (i.e. you can unit test!). Wow. Because Unit testing a web application is value for money.
Bye Bye Wicket. You've been clean bowled. I never understood the dislike JSP gang. It's like looking at bad java code saying Java is by far the worst offender. I like tag libraries when they dont attempt to be
HTML encapsulators. I like being able to read Java code instead of the mental shift associated with interpreting yet another template framework syntax. I like the ability to embed Java code instead of new syntax , and I use it sparingly, and it works well thank you very much. I understand you might have a valid different philosophy , but it's not for me.

Enter the Play framework. Seemed simple (though non JSP). All in all looked to be great and simple. Seemed to get web applications. Targets restful structures. It would be nice to learn a new framework and see what it does better and what it does worse. But the tutorials seemed to be too command line heavy. There's nothing wrong with that of course. One problem I had is since I was trying to implement this project for a friend and I would have to hand it over to someone, I needed to use something that could be easily learnt or was widely used. Play could be it - but it is so different from the rest, that I hesistate.

Enter Stripes. Did I like the documentation. The HowTo seemed to cover everything I want, and the framework seemed to hit the right balance between what it did and what you had to do. Yay we have a winner. The proof of simplicity , my wife who is a PHP developer with rudimentary java knowledge picked up this framework in a couple of hours (or she is exceptionally smart and yes she reads this blog)

Java based web frameworks

A quick rant on the state of web based frameworks in the J2EE world
a. Imposing Maven on me is not nice.
b. If your "web" based framework cannot support clean URL's you are in the wrong business when you are writing frameworks
c. A J2EE standard doesn't mean its a good standard (JSF - Im looking at you)
d. When you want a view technology please please use JSP. Its mature and it works. It is as good or as bad as you want it to be. Making your own template language which is either inflexible or just as bad as JSP is just plain silly. Reinventing JSTL with a different archaic syntax is silly. Doing it in the guise of "no java in JSP" is silly - There isn't a framework that I can't write bad code in.
e. Make simple things simpler and make hard things easier if possible and it's ok if hard things remain hard. But do not make simple things simpler(oh look how quickly I can do CRUD) and hard things harder(because lets face it , every application does have corner cases).  Do not make hard thing's simpler and simple things harder (I have to set up how many files for Hello World?) - because lets face it not everything in a web application is Rocket Science.
f. Who would have thought Spring would turn out to be more heavyweight than EJB.
g. Who would have thought that Struts 1 would still be so heavily used
h. Who would have thought that I would now prefer to code in ASP.net than identify which Web Framework I should use.

Random Value Selection using JMeter XPath Post Processor

JMeter has two post processor extractors which are usually used when we need some data extracted from the response. The XPath extractor is useful when the Regex would match other elements in a page (e.g. a link inside an li element). However the XPath extractor is missing a functionality that the Regex Post Processor has. It only returns all the values (equivalent to specifying -1 in a Regex Post Processor). XPath also natively supports returning the "nth" value using the position() function. However it doesn't support returning a value at random.
However using normal JMeter functions we can easily implement this functionality.
Assume that you have an XPath post processor that is attached to a HTTP Sampler and is returning n values into a variable called testXPATH. if we now want to select a value at random what we need to use is

${__V(testXPATH_${__Random(1,${testXPATH_matchNr})})}

But thats a mouthfull and needs an explanation

${testXPATH_matchNr}

returns the count of how many elements matched. We are assuming that some values are going to be selected, otherwise you need an IF controller to check the condition.

${__Random(1,${testXPATH_matchNr})}

Will then select an integer between 1 and the count of the values (TODO check if we need to add 1)

${__V(testXPATH_${__Random(1,${testXPATH_matchNr})})}}

Finally we form a string of the form testXPATH_n e.g. testXPATH_3 which is then passed to the __V function which will get us the value for the variable
http://jakarta.apache.org/jmeter/usermanual/functions.html

Note that running TIDY on HTML , followed by parsing it to create a DOM structure and finally running an XPATH query against it is signifcantly slower than just running a Regex against a String. If you need to generate a large load and you have limited resources available to you, then the XPATH extractor is probably not the best alternative.

Looping through all output of a RegExp extractor (without the for each controller)

I forget why the user couldnt use a For Each controller , but heres how to loop through the result of a Regex Post Processor that returns multiple values

ThreadGroup
+HttpSampler
++Regex Extractor (variableName = links)
+WhileController(${__javaScript(${C} < ${links_matchNr})})
++HTTPSampler use ${__V(links_${C})} to access the current result
++Counter (start=1,increment=1,maximum=${links_matchNr},referenceName=C)

Dynamic values within files

I have a HTTP Post request sample to which we are passing an XML file through option "Send Files With Request". The XML file to be sent with request contain on Tag "Order id" which need to be modified [ dynamically during test plan executing] before sending the request. Original
Typically in JMeter we would use a ${variableName} notation. However JMeter only replaces these values when the the ${variableName} is within the script itself , not when it is referenced in some other file. As the user wants to use a separate (template) file which is added to the HTTP request, this technique wont work.
One option to consider is whether we could generate all the files we need before we run the test. this is possible for example when the dynamic data is a set of id's that is accessible say from a database. In this case the better (in the more efficient sense) is to have a initialisation step (perhaps as part of the build or as a separate step in Jmeter) to create all the files that the test would need. Once this is done it is easy to make the JMeter script pick one file (usually using the CSV DataSet Config).
However this isnt always possible. In the question above , the Order ID could be dynamically generated by the system under test in ways that might not be predictable. In this case we cannot generate the files in advance. We have to extend JMeter.
The cleaner more elegant way to do this would be to write a custom pre-processor of some sort because it sounds as if this functionality could be reused in some other context (or extend the HTTPSampler). But we will choose the quick and dirty way of writing a BeanShell preprocessor that does this for us.

Briefly the solution will
a. Have a template XML file. We will use @variableName@ within the file for what we want replaced. We will currently only support replacing a single variable (but it is easy to support multiple variables).
b. We will configure the HTTPSampler to use a generated file. At runtime a pre - processor will read the template file and generate this new file. This new file will be read and posted by JMeter

The Structure of the test is shown below
The HTTP Sampler adds a currently non existent file to be posted. This is the file the Pre Processor will create. The name of the file is varied based on the ThreadNumber (we could have added timestamps etc as well) so that each thread will get its own file name

We use user defined variables to define the location of the temp file, the regex that we want to run on the file and the replacement value.

The BeanShell pre-processor

import java.io.BufferedReader;
import org.apache.jmeter.protocol.http.util.HTTPFileArg;
import java.io.File;
import java.io.FileInputStream;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.charset.Charset;
import java.io.BufferedWriter;
import java.io.FileOutputStream;
import java.nio.channels.Channels;

//read the file as string
FileInputStream stream = new FileInputStream(new File(vars.get("templatePath")));
String fileAsString = "";
try {
      FileChannel fc = stream.getChannel();
      MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());     
      fileAsString = Charset.defaultCharset().decode(bb).toString();
}
finally {
        stream.close();
}
//perform the replacement
fileAsString = fileAsString.replaceAll(vars.get("replaceRegex"), vars.get(vars.get("variableName")));

//assumes first file
HTTPFileArg arg = sampler.getHTTPFiles()[0];
//write it out to the file we want
FileChannel fout = new FileOutputStream(arg.getPath()).getChannel();
BufferedWriter bf = new BufferedWriter(Channels.newWriter(fout, Charset.defaultCharset().name()));
bf.write(fileAsString);
bf.close();

Disclaimer : File Reading writing snippets swiped from Google, you can probably make it more efficient.

• Briefly the snippet reads the template file(location defined as a variable) as a String (using the default charset , if you deal with multiple languages you might want to force this to UTF-8).
• It then runs the replacement using a regex (also a variable) and the subsitution value (also a variable which for this example we have just spoofed)
• Finally it writes the file(assuming default charset , again you could force this to be UTF-8). To determine the location of the file , it reads the location from the HTTPSampler. This is currently set as the first file , but we could have used a name or any other technique.

And finally the sampler will read the file that we have created and post it as usual.

Sample JMX available here

java memory leaks

One of my colleagues asked how I diagnosed a memory leak that was impacting his application , so this is what i said. Maybe Ill write it tutorial form some time


The version of java we are running doesnt make this easy so this is what I did on dev (its hard to make this a coherent narrative).

Almost any memory leak debugging takes the following form(Im assuming the Out of Memory says the HEAP is out of memory)
a. Run your application steps that are causing the memory leak (detecting this is actually the biggest problem but if you dont know what step is causing the leak , you basically have to access all the functionality of your application). This ensures that all statics/caches/classes are loaded. Then force a garbage collection. Then check the memory for the Java VM and take a heap dump (this is a binary form of all the objects on the heap including the allocation graph)
b. Now repeat the step causing the problem. make sure that the step ends correctly (for e.g. if the step is a user goes and saves the quote, ensure that you logout and invalidate his session otherwise you get false readings). Now run a full GC again. Take another heap dump.
c. Compare the two heap dumps. if there is a difference that means you have a leak - ideally there shouldnt be a difference. In practice on a server like tomcat there are a lot of things happening in the background even if no user is accessing the site so even after a full gc , you might still see some tomcat classes or objects that appear to be new. The way around this is that you have to loop the step in b so that you will get to see significant differences for your classes rather than the app server classes.

For our env however taking a heap dump seems problematic because of the version of JDK seems really old(1.5 _07) and I wasnt able to load the taken heap dump in any tool.

Heap Dump
To take a heap dump you have the following options
a. Some JVM's support -XX:+HeapDumpOnCtrlBreak (kill -3 pid on linux). However our version doesnt
b. Some profilers(e.g. Jprobe) allow you to take heap dump - You have to pay for this but there are some free ones like NetBeans. however all of them need you to run some install steps so I didnt use this
c. use jmap . This is a utility that Java provides. Later versions let you say JAVA_BIN/jmap -dump processid. For our version we have to say jmap -heap:format=b processid . however this isnt working correctly since i couldnt load this dump into any tool. what I finally used was jmap -histo processid. This is a poor mans heap dump . it shows the classes and the number and the size they occupy, but it doesn't show who allocated the classes (so for e.g. I knew it was the concurrency related classes that were leaking , but I didnt know who was creating them - that was thanks to Google)

d. +XX:HeapDumpOnOutOfMemory - This will automatically take a heap dump when you run out of memory but debugging with this is slightly harder because you have live objects
e. use HPROF (this is an earlier version of the tool) when you start up java but it will slow the process.

Monitoring
To look at how much memory is being used, and to force GC you have the following options
a. Look at the probe we installed (specific to tomcat) - lambda/psi probe . Click System information . under overview you can see the memory being used and you can Force GC as well. You can also click Memory Utilization and it will show you the breakup. I had to add some JMX parameters to startup. This is tomcat specific.
It also lets you see sessions and their footprints (sometimes the leak isnt actually a leak. Code puts too much data in session , the user doesnt log out , the session remains active for say 30 minutes , then that much memory is used up for 30 minutes. technically this isnt aleak , when the session times out , the objects will get collected , but it looks like a leak because the memory keeps getting used , but doesnt get reclaimed)
b. use JConsole . This is in the bin directory of your java. When you start it on your machine you can add a remote connection
These are the settings I added on dev
export JAVA_OPTS="-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/local/apache-tomcat-5.5.30/logs -XX:PermSize=256m -XX:MaxPermSize=256m -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=12345 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dfile.encoding=UTF8"
This will also let you see some things that the probe does.
Using this I could find out
a. That it wasnt the sessions causing the problem (since the probe showed that only 1 MB was being used in sessions).
b. I also used it to check and see whether threads were causing issues , not the case.

Testing
I used JMeter to simulate a web user going and approving a quote and simply looped that 4000 times. I couldnt not use say SOAPUI because we needed to simulate a stub being created whereas SOAP acts as the client so it wouldnt have shown the problem. Ordinarily you would not know the Stub causes the problem , but the histograph had shown such a large number of the concurrent objects on the heap , that a google search showed that other people had faced this problem with axis 1.4.1. If I didnt have this I would have had to write a script that would have placed a quote, gone and approved it , saved it , modified it etc.

The histographs are in /root/histbeftest and /root/hist92per and are easily compared to verify.

Analysis
The histograph files are plain text and can be compared in a text editor. If I had a heap dump we could use
a. JHat - this comes in the bin directory of java (1.6 onwards and some update of 1.5 onwards but it can read 1.5 dumps). This will let you browse the heap and even compare them
b. Profilers like JProbe
c. Other tools like VisualVM, HPJmeter NetBeans, extensions in eclipse all let you load heap dumps.
One of the issues you might run into is that the Heap Dump file is quite large and your client tool may not have enough memory. To verify a memory leak you usually reduce the amount of memory you give to the JVM so that you get smaller heap dump files (so for e.g. the iquote app once it is running and all the classes are loaded takes about 20 to 30% of 512MB which is about 150 MB. So I could reduce the memory to 256MB instead of 512 and run the tests to create a much smaller heap dump)

a memory leak in sample code and then trying to detect it using tools is usually the best way to learn.

JMeter and AJAX - Part III

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part III-1 - A diversion on how to use session ids between threads
Part IV - A more complex sample

In Part II we developed a sampler that would let us synchronize selected threads. I split that out because someday I might be tempted to actually write that timer. So lets now try and execute some AJAX requests in parallel. The demo site we will use is http://www.ajaxdaddy.com/demo-dhtml-autocomplete.html (Please don't run extensive load tests against this site - it isn't mine)
The site has an AJAX based autocomplete for countries. Access the site in FireFox, turn on Firebug and type au into the textbox and you can see the AJAX calls being made in firebug

We can see that two AJAX calls are made, one when we typed 'a' and one when we typed 'u' (There is an AJAX design pattern that applies here which doesn't seem to have been followed). So we will access this page and try to simulate two parallel calls.


The Init and Delay Sampler are from Part II. The Init sampler ensures only 1 thread does the initialization bits. The delay sampler has the rest of the code. After the Init sampler we just make all the threads wait so that no threads run before we are ready to begin (so don't use any ramp up time as that would be nullified here). Its essential the sync timer has a count equal to the count of threads in the threadgroup.
The IF controller is used so that only the main threads perform the request to the main page. Recall for 6 threads to run 2 at a time , we have the groups as Thread(1,2) , Threads(3,4), Threads(5,6). The even numbered threads will only be used for the AJAX requests. The Odd numbered threads will be used for the main sample and the AJAX requests. In our case the landing page needs to be accessed by the odd numbered threads. The even numbered threads will skip and land at the delay sampler waiting for their respective soulmates to join them(I'm currently waiting to be reunited with my wife hence the flowery language).
Anyway we have added a Gaussian random timer to allow the odd numbered threads to arrive at different times. When the odd numbered thread does arrive at the delay sampler, it along with its even numbered pair will be released to execute the Make AJAX request sampler (which chooses a word at random from the CSV file we have defined text,assert e.g. a,australia).

Hence two requests will be executed in parallel for the AJAX request sampler. If we had further requests after this we would have added another IF controller(only odd numbered threads) and coded the samples under this. if we wanted 3 requests in parallel we would have to adjust the Init sampler to say we want groups of 3.
Run the test. Though the samples may appear interleaved , note down the sample start times of the groups of threads that were supposed to execute in parallel, they will be more or less the same.

But I have cheated. The samples in parallel didn't really depend on each others data. The AJAX requests were exactly the same , differing in data. The AJAX requests didn't depend on the user's session thereby reducing the complexity of the script.

Lets attempt an example which needs the above. Onwards to Part IV (as soon as I find a good demo!)

JMeter and AJAX - Part II

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part III-1 - A diversion on how to use session ids between threads 
Part IV - A more complex sample

As mentioned in Part I , the most elegant way to simulate AJAX in JMeter would be to customize the HTTPSampler to allow child HTTPSampler requests to be executed in parallel.
Since customization of JMeter core isn't an option for me, we'll try and simulate this directly without customizing JMeter. Note that this is a toy, done for amusement rather than any real solution to an existing problem.

The first thing we should note that anything that we need done in parallel in JMeter needs its own thread. Suppose we have some requests that make two AJAX requests in parallel then the way we are going to work this is
Thread odd numbered(1,3,...) - Main thread that executes most of the requests
Thread even numbered(2,4,...) - Thread used only when needed for AJAX parallel requests so Thread(1,2) is a group as is Thread(3,4) and so on.
If we needed to make 3 AJAX requests in parallel then we would have grouped up Threads(1,2,3) and Threads (4,5,6) - but the main thread would have been Thread1, Thread4,...
You can immediately see resources are being wasted - Well we did say this was a toy.
Making the even numbered threads not execute some samplers is easy, we just need an IF controller to check. But how do you make Thread 1 and Thread 2 for e.g. execute a sampler in between at exactly the same time? In the Java world , this would usually mean reading up everything that Doug Lea and Brian Goetz have written - but in lazy developer mode this means downloading the JMeter source code and looking at SyncTimer (which blocks a fixed number of threads but not specific ones) and using the time honored technique of Copy Paste.
So before diving into the problem at hand lets see if we can simulate the following.
6 threads will arrive at varying times into one sampler. At which points they will wait till groups are ready i.e. Threads 1 and 2 will go together and Threads 3 and 4 will go together and so on.
Here's the Beanshell sampler which is actually a customized Sync Timer in disguise

import java.util.HashMap;
int totalThreads = 6; // MUST match total number defined in Thread Group
int groupsOf = 2; //MUST match number of requests to make in parallel

if (bsh.shared.myObj == void){
    // not yet defined, so create it:
    bsh.shared.myObj =new HashMap();
    bsh.shared.timerCounter =new HashMap();
    int numObjectsNeeded = totalThreads/groupsOf;
    for(int i = 0;i<numObjectsNeeded;i++) {
           bsh.shared.myObj.put(String.valueOf(i),new Object());
           bsh.shared.timerCounter.put(String.valueOf(i),new int[] { 0 });
    }
    print("populated");
} else {
   print("already init");
}

print(${__threadNum()});

int grpNo = (int)(${__threadNum()} -1)/groupsOf;
print(grpNo);
String groupNumber = String.valueOf(grpNo);
print(groupNumber);
Object syncObject = bsh.shared.myObj.get(groupNumber);
print(syncObject);
synchronized (syncObject) {
   int[] timerCount = (int[])bsh.shared.timerCounter.get(groupNumber);
   timerCount[0]++;
   final int count = timerCount[0];
   if (count >= groupsOf) {
                syncObject.notifyAll();
            } else {
                try {
                    syncObject.wait();
                } catch (InterruptedException e) {
                    log.warn(e.getLocalizedMessage());
                }
            }
            timerCount[0]=0; // Reset for next time   
}
SampleResult.setResponseData("sunk!");

The bsh.shared namespace is used to share objects between threads. So in our case we have 6 threads and we want 2 threads in parallel which means there are 3 groups of two threads each. To synchronize each group we need an object (hence 3 objects) and a counter each - to count how many threads are already available. Thats what the first part of the code sets up.(The first part is not thread safe but because we are going to test this by guaranteeing that threads arrive at different times , the snippet need not be thread safe).

The second part of the code then calculates which Group a thread belongs to, gets the object for the group, increases its count. If the number of threads is less than that we want in parallel it suspends the thread (by calling wait) otherwise it does a notifyAll which will wake up all suspended threads and allow these two threads to move in parallel

Heres what the test looks like


We have artificially added a timer to make sure requests to the Delay sampler arrive 1 thread at a time. We run the test and we can see that Threads(1,2) run the After Delay sampler at almost the same time , as do Threads(3,4) and as do Threads(5,6)
If we change the groupsOf in the delay sampler to be 3 then we see that three threads run in parallel.

And with that we are ready to use this delay sampler to make AJAX calls in parallel.
Onwards to Part III - An actual example of JMeter and AJAX

JMeter and AJAX - Part I

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part III-1 - A diversion on how to use session ids between threads 
Part IV - A more complex sample

Queries related to testing websites which use AJAX are often asked on the JMeter mailing list so here are my thoughts on this matter.

JMeter is not a browser, it simulates a HTTP request/response which means it will always struggle to replicate the finer parts of browser behavior. The advantage however is that it only needs very few resources to simulate a single browser. Testing tools that actually drive a browser will find it easier to replicate the browser however they pay for it when it comes to load tests - the amount of resources is needed is much more than the JMeter family of tools. But when testing out websites which use AJAX, JMeter with its current feature set may not be an accurate simulation because AJAX does depend on the finer parts of browser behavior.

An elegant solution would be to have the ability to define child HTTP requests of a parent HTTP sampler with a thread pool that defines how many of these will be made in parallel. But no such feature is currently available in JMeter.

But say you do have to test an AJAX enabled site today - what can one do?
It's useful in these times to be a lazy but honest tester - The honest part ensures that we will try to be accurate but the lazy part also ensures that we wont try to do any more than the minimum we need to do.

• So the first question that should come up is do we actually need to change anything in the way we test? The vast majority of AJAX enabled sites take the form a page loads , the user does some action , an AJAX call is made and values are shown to the user without updating the rest of the page. But look at the sequence of HTTP requests. Request for the main page, get response, browser processes response,user action, AJAX call made , AJAX response, further browser processing. There isn't any request made in parallel (ignoring all the static file requests which is its own blog post) - nor is there any difference due to Asynchronous nature of the call. So a JMeter script to simulate the above would have no difference from a normal JMeter script. the AJAX call should just follow the call to the main page and we are done.
• But hold on some AJAX enabled sites may make an AJAX call as soon as the page loads without any user action. Or the call might be made periodically.Even in these cases the onload/onready is only done after the response of the main request is made available so while the browser may make the request asynchronously in terms of the HTTP request/response there is no difference. The periodic call may only have a problem if the time it takes to return is greater than the periodic repetition. But the latter case would be a problem anyway , all you really want from JMeter is to detect if this happens. Once it happens your system is messed up and you need to fix it! There really isn't much value simulating the messed up state. To put it in another way , if your system breaks for 50 users, there isn't much value observing how much more is broken for 100 users.
• But hold on what if multiple AJAX requests are made on load? Say a users home page has two AJAX widgets , one which shows say his tasks , one which shows his emails? Well yes two requests will be made in parallel. Usually these requests are independent of one another. The question we really need answered is "Is there a difference between a user making two independent requests in parallel and two users(maybe with the same login) making a single independent request each in parallel?" In most cases , the answer is no. The load may be slightly higher in the latter case because you may have two sessions instead of one, but in terms of the rate of requests the server sees you can normally match the rate of requests by simply increasing the number of threads (and a little bit of math).
• But hold on what if the requests aren't independent? What if they utilize the same table/rows causing contention when its from the same user session what if widget2 depends on widget 1 for something in the session? Well one thing to do in this case is call the developer and ask him why the hell are there two dependent requests instead of a single one? That usually doesn't end well(even though the tester is right) - but yes at this time yes you might need to actually simulate requests in parallel. Note however a lot of AJAX enabled websites dont fall into this bracket and those websites can be simulated in JMeter with a reasonable degree of accuracy.

Onwards to Part - 2. Simulating AJAX requests without knowing how to modify JMeter core.

Google isnt as good as you think it is

Exhibit A - A snippet from webmaster tools for this blog



Though I know that writing about deepika instead of JMeter would give me more page hits!

WLST to dump free heap memory

from java.io import FileWriter
from java.io import BufferedWriter
from java.util import Date
connect(eval('username'),eval('password'),eval('url'))
domainRuntime()
cd('/ServerRuntimes/' + eval('managedServerName') + '/JVMRuntime/' + eval('managedServerName'))
heapFreeCurrentPerOld = str(cmo.getHeapFreePercent())
heapFreeCurrentValOld = str(cmo.getHeapFreeCurrent())
cmo.runGC()
java.lang.Thread.sleep(int(eval('sleepTime')))

heapFreeCurrentPerNew = str(cmo.getHeapFreePercent())
heapFreeCurrentValNew = str(cmo.getHeapFreeCurrent())
newDate = Date()
fstream = FileWriter(eval('outputFile'),true)
out =  BufferedWriter(fstream)
out.write(eval('managedServerName') + "," + str(newDate.getTime()) + "," + heapFreeCurrentPerOld + "," + heapFreeCurrentValOld + "," + heapFreeCurrentPerNew + "," + heapFreeCurrentValNew + "\n");
out.close();
disconnect()
exit()

There is also a property file which contains the configuration information

managedServerName=<MANAGED>
username=<USER>
password=<PASSWORD>
url=<URL>
sleepTime=<PAUSE>
outputFile=<PathToFile.csv>

And finally to run

$WL_SERVER_HOME/common/bin>wlst -loadProperties


This was written to output the free memory before and after GC for a particular managed server (which used to run jobs) on weblogic. It was cron'ed to run at fixed intervals. The generated file is used to check if there is a memory leak. The heap free current value should remain fairly constant. if it keeps reducing , we have a leak

Getting the current thread group name

${__BeanShell(ctx.getThreadGroup().getName())}

Academic and Happy

via http://urlai.com/url/theworkaholic.blogspot.com

Text analysis
theworkaholic.blogspot.com is probably written by a male somewhere between 36-50 years old. The writing style is academic and happy most of the time.