Java based web frameworks

A quick rant on the state of web based frameworks in the J2EE world
a. Imposing Maven on me is not nice.
b. If your "web" based framework cannot support clean URL's you are in the wrong business when you are writing frameworks
c. A J2EE standard doesn't mean its a good standard (JSF - Im looking at you)
d. When you want a view technology please please use JSP. Its mature and it works. It is as good or as bad as you want it to be. Making your own template language which is either inflexible or just as bad as JSP is just plain silly. Reinventing JSTL with a different archaic syntax is silly. Doing it in the guise of "no java in JSP" is silly - There isn't a framework that I can't write bad code in.
e. Make simple things simpler and make hard things easier if possible and it's ok if hard things remain hard. But do not make simple things simpler(oh look how quickly I can do CRUD) and hard things harder(because lets face it , every application does have corner cases).  Do not make hard thing's simpler and simple things harder (I have to set up how many files for Hello World?) - because lets face it not everything in a web application is Rocket Science.
f. Who would have thought Spring would turn out to be more heavyweight than EJB.
g. Who would have thought that Struts 1 would still be so heavily used
h. Who would have thought that I would now prefer to code in ASP.net than identify which Web Framework I should use.

Random Value Selection using JMeter XPath Post Processor

JMeter has two post processor extractors which are usually used when we need some data extracted from the response. The XPath extractor is useful when the Regex would match other elements in a page (e.g. a link inside an li element). However the XPath extractor is missing a functionality that the Regex Post Processor has. It only returns all the values (equivalent to specifying -1 in a Regex Post Processor). XPath also natively supports returning the "nth" value using the position() function. However it doesn't support returning a value at random.
However using normal JMeter functions we can easily implement this functionality.
Assume that you have an XPath post processor that is attached to a HTTP Sampler and is returning n values into a variable called testXPATH. if we now want to select a value at random what we need to use is

${__V(testXPATH_${__Random(1,${testXPATH_matchNr})})}

But thats a mouthfull and needs an explanation

${testXPATH_matchNr}

returns the count of how many elements matched. We are assuming that some values are going to be selected, otherwise you need an IF controller to check the condition.

${__Random(1,${testXPATH_matchNr})}

Will then select an integer between 1 and the count of the values (TODO check if we need to add 1)

${__V(testXPATH_${__Random(1,${testXPATH_matchNr})})}}

Finally we form a string of the form testXPATH_n e.g. testXPATH_3 which is then passed to the __V function which will get us the value for the variable
http://jakarta.apache.org/jmeter/usermanual/functions.html

Note that running TIDY on HTML , followed by parsing it to create a DOM structure and finally running an XPATH query against it is signifcantly slower than just running a Regex against a String. If you need to generate a large load and you have limited resources available to you, then the XPATH extractor is probably not the best alternative.

Looping through all output of a RegExp extractor (without the for each controller)

I forget why the user couldnt use a For Each controller , but heres how to loop through the result of a Regex Post Processor that returns multiple values

ThreadGroup
+HttpSampler
++Regex Extractor (variableName = links)
+WhileController(${__javaScript(${C} < ${links_matchNr})})
++HTTPSampler use ${__V(links_${C})} to access the current result
++Counter (start=1,increment=1,maximum=${links_matchNr},referenceName=C)

Dynamic values within files

I have a HTTP Post request sample to which we are passing an XML file through option "Send Files With Request". The XML file to be sent with request contain on Tag "Order id" which need to be modified [ dynamically during test plan executing] before sending the request. Original
Typically in JMeter we would use a ${variableName} notation. However JMeter only replaces these values when the the ${variableName} is within the script itself , not when it is referenced in some other file. As the user wants to use a separate (template) file which is added to the HTTP request, this technique wont work.
One option to consider is whether we could generate all the files we need before we run the test. this is possible for example when the dynamic data is a set of id's that is accessible say from a database. In this case the better (in the more efficient sense) is to have a initialisation step (perhaps as part of the build or as a separate step in Jmeter) to create all the files that the test would need. Once this is done it is easy to make the JMeter script pick one file (usually using the CSV DataSet Config).
However this isnt always possible. In the question above , the Order ID could be dynamically generated by the system under test in ways that might not be predictable. In this case we cannot generate the files in advance. We have to extend JMeter.
The cleaner more elegant way to do this would be to write a custom pre-processor of some sort because it sounds as if this functionality could be reused in some other context (or extend the HTTPSampler). But we will choose the quick and dirty way of writing a BeanShell preprocessor that does this for us.

Briefly the solution will
a. Have a template XML file. We will use @variableName@ within the file for what we want replaced. We will currently only support replacing a single variable (but it is easy to support multiple variables).
b. We will configure the HTTPSampler to use a generated file. At runtime a pre - processor will read the template file and generate this new file. This new file will be read and posted by JMeter

The Structure of the test is shown below
The HTTP Sampler adds a currently non existent file to be posted. This is the file the Pre Processor will create. The name of the file is varied based on the ThreadNumber (we could have added timestamps etc as well) so that each thread will get its own file name

We use user defined variables to define the location of the temp file, the regex that we want to run on the file and the replacement value.

The BeanShell pre-processor

import java.io.BufferedReader;
import org.apache.jmeter.protocol.http.util.HTTPFileArg;
import java.io.File;
import java.io.FileInputStream;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.charset.Charset;
import java.io.BufferedWriter;
import java.io.FileOutputStream;
import java.nio.channels.Channels;

//read the file as string
FileInputStream stream = new FileInputStream(new File(vars.get("templatePath")));
String fileAsString = "";
try {
      FileChannel fc = stream.getChannel();
      MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());     
      fileAsString = Charset.defaultCharset().decode(bb).toString();
}
finally {
        stream.close();
}
//perform the replacement
fileAsString = fileAsString.replaceAll(vars.get("replaceRegex"), vars.get(vars.get("variableName")));

//assumes first file
HTTPFileArg arg = sampler.getHTTPFiles()[0];
//write it out to the file we want
FileChannel fout = new FileOutputStream(arg.getPath()).getChannel();
BufferedWriter bf = new BufferedWriter(Channels.newWriter(fout, Charset.defaultCharset().name()));
bf.write(fileAsString);
bf.close();

Disclaimer : File Reading writing snippets swiped from Google, you can probably make it more efficient.

• Briefly the snippet reads the template file(location defined as a variable) as a String (using the default charset , if you deal with multiple languages you might want to force this to UTF-8).
• It then runs the replacement using a regex (also a variable) and the subsitution value (also a variable which for this example we have just spoofed)
• Finally it writes the file(assuming default charset , again you could force this to be UTF-8). To determine the location of the file , it reads the location from the HTTPSampler. This is currently set as the first file , but we could have used a name or any other technique.

And finally the sampler will read the file that we have created and post it as usual.

Sample JMX available here

java memory leaks

One of my colleagues asked how I diagnosed a memory leak that was impacting his application , so this is what i said. Maybe Ill write it tutorial form some time


The version of java we are running doesnt make this easy so this is what I did on dev (its hard to make this a coherent narrative).

Almost any memory leak debugging takes the following form(Im assuming the Out of Memory says the HEAP is out of memory)
a. Run your application steps that are causing the memory leak (detecting this is actually the biggest problem but if you dont know what step is causing the leak , you basically have to access all the functionality of your application). This ensures that all statics/caches/classes are loaded. Then force a garbage collection. Then check the memory for the Java VM and take a heap dump (this is a binary form of all the objects on the heap including the allocation graph)
b. Now repeat the step causing the problem. make sure that the step ends correctly (for e.g. if the step is a user goes and saves the quote, ensure that you logout and invalidate his session otherwise you get false readings). Now run a full GC again. Take another heap dump.
c. Compare the two heap dumps. if there is a difference that means you have a leak - ideally there shouldnt be a difference. In practice on a server like tomcat there are a lot of things happening in the background even if no user is accessing the site so even after a full gc , you might still see some tomcat classes or objects that appear to be new. The way around this is that you have to loop the step in b so that you will get to see significant differences for your classes rather than the app server classes.

For our env however taking a heap dump seems problematic because of the version of JDK seems really old(1.5 _07) and I wasnt able to load the taken heap dump in any tool.

Heap Dump
To take a heap dump you have the following options
a. Some JVM's support -XX:+HeapDumpOnCtrlBreak (kill -3 pid on linux). However our version doesnt
b. Some profilers(e.g. Jprobe) allow you to take heap dump - You have to pay for this but there are some free ones like NetBeans. however all of them need you to run some install steps so I didnt use this
c. use jmap . This is a utility that Java provides. Later versions let you say JAVA_BIN/jmap -dump processid. For our version we have to say jmap -heap:format=b processid . however this isnt working correctly since i couldnt load this dump into any tool. what I finally used was jmap -histo processid. This is a poor mans heap dump . it shows the classes and the number and the size they occupy, but it doesn't show who allocated the classes (so for e.g. I knew it was the concurrency related classes that were leaking , but I didnt know who was creating them - that was thanks to Google)

d. +XX:HeapDumpOnOutOfMemory - This will automatically take a heap dump when you run out of memory but debugging with this is slightly harder because you have live objects
e. use HPROF (this is an earlier version of the tool) when you start up java but it will slow the process.

Monitoring
To look at how much memory is being used, and to force GC you have the following options
a. Look at the probe we installed (specific to tomcat) - lambda/psi probe . Click System information . under overview you can see the memory being used and you can Force GC as well. You can also click Memory Utilization and it will show you the breakup. I had to add some JMX parameters to startup. This is tomcat specific.
It also lets you see sessions and their footprints (sometimes the leak isnt actually a leak. Code puts too much data in session , the user doesnt log out , the session remains active for say 30 minutes , then that much memory is used up for 30 minutes. technically this isnt aleak , when the session times out , the objects will get collected , but it looks like a leak because the memory keeps getting used , but doesnt get reclaimed)
b. use JConsole . This is in the bin directory of your java. When you start it on your machine you can add a remote connection
These are the settings I added on dev
export JAVA_OPTS="-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/local/apache-tomcat-5.5.30/logs -XX:PermSize=256m -XX:MaxPermSize=256m -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=12345 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dfile.encoding=UTF8"
This will also let you see some things that the probe does.
Using this I could find out
a. That it wasnt the sessions causing the problem (since the probe showed that only 1 MB was being used in sessions).
b. I also used it to check and see whether threads were causing issues , not the case.

Testing
I used JMeter to simulate a web user going and approving a quote and simply looped that 4000 times. I couldnt not use say SOAPUI because we needed to simulate a stub being created whereas SOAP acts as the client so it wouldnt have shown the problem. Ordinarily you would not know the Stub causes the problem , but the histograph had shown such a large number of the concurrent objects on the heap , that a google search showed that other people had faced this problem with axis 1.4.1. If I didnt have this I would have had to write a script that would have placed a quote, gone and approved it , saved it , modified it etc.

The histographs are in /root/histbeftest and /root/hist92per and are easily compared to verify.

Analysis
The histograph files are plain text and can be compared in a text editor. If I had a heap dump we could use
a. JHat - this comes in the bin directory of java (1.6 onwards and some update of 1.5 onwards but it can read 1.5 dumps). This will let you browse the heap and even compare them
b. Profilers like JProbe
c. Other tools like VisualVM, HPJmeter NetBeans, extensions in eclipse all let you load heap dumps.
One of the issues you might run into is that the Heap Dump file is quite large and your client tool may not have enough memory. To verify a memory leak you usually reduce the amount of memory you give to the JVM so that you get smaller heap dump files (so for e.g. the iquote app once it is running and all the classes are loaded takes about 20 to 30% of 512MB which is about 150 MB. So I could reduce the memory to 256MB instead of 512 and run the tests to create a much smaller heap dump)

a memory leak in sample code and then trying to detect it using tools is usually the best way to learn.

JMeter and AJAX - Part III

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part III-1 - A diversion on how to use session ids between threads
Part IV - A more complex sample

In Part II we developed a sampler that would let us synchronize selected threads. I split that out because someday I might be tempted to actually write that timer. So lets now try and execute some AJAX requests in parallel. The demo site we will use is http://www.ajaxdaddy.com/demo-dhtml-autocomplete.html (Please don't run extensive load tests against this site - it isn't mine)
The site has an AJAX based autocomplete for countries. Access the site in FireFox, turn on Firebug and type au into the textbox and you can see the AJAX calls being made in firebug

We can see that two AJAX calls are made, one when we typed 'a' and one when we typed 'u' (There is an AJAX design pattern that applies here which doesn't seem to have been followed). So we will access this page and try to simulate two parallel calls.


The Init and Delay Sampler are from Part II. The Init sampler ensures only 1 thread does the initialization bits. The delay sampler has the rest of the code. After the Init sampler we just make all the threads wait so that no threads run before we are ready to begin (so don't use any ramp up time as that would be nullified here). Its essential the sync timer has a count equal to the count of threads in the threadgroup.
The IF controller is used so that only the main threads perform the request to the main page. Recall for 6 threads to run 2 at a time , we have the groups as Thread(1,2) , Threads(3,4), Threads(5,6). The even numbered threads will only be used for the AJAX requests. The Odd numbered threads will be used for the main sample and the AJAX requests. In our case the landing page needs to be accessed by the odd numbered threads. The even numbered threads will skip and land at the delay sampler waiting for their respective soulmates to join them(I'm currently waiting to be reunited with my wife hence the flowery language).
Anyway we have added a Gaussian random timer to allow the odd numbered threads to arrive at different times. When the odd numbered thread does arrive at the delay sampler, it along with its even numbered pair will be released to execute the Make AJAX request sampler (which chooses a word at random from the CSV file we have defined text,assert e.g. a,australia).

Hence two requests will be executed in parallel for the AJAX request sampler. If we had further requests after this we would have added another IF controller(only odd numbered threads) and coded the samples under this. if we wanted 3 requests in parallel we would have to adjust the Init sampler to say we want groups of 3.
Run the test. Though the samples may appear interleaved , note down the sample start times of the groups of threads that were supposed to execute in parallel, they will be more or less the same.

But I have cheated. The samples in parallel didn't really depend on each others data. The AJAX requests were exactly the same , differing in data. The AJAX requests didn't depend on the user's session thereby reducing the complexity of the script.

Lets attempt an example which needs the above. Onwards to Part IV (as soon as I find a good demo!)

JMeter and AJAX - Part II

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part III-1 - A diversion on how to use session ids between threads 
Part IV - A more complex sample

As mentioned in Part I , the most elegant way to simulate AJAX in JMeter would be to customize the HTTPSampler to allow child HTTPSampler requests to be executed in parallel.
Since customization of JMeter core isn't an option for me, we'll try and simulate this directly without customizing JMeter. Note that this is a toy, done for amusement rather than any real solution to an existing problem.

The first thing we should note that anything that we need done in parallel in JMeter needs its own thread. Suppose we have some requests that make two AJAX requests in parallel then the way we are going to work this is
Thread odd numbered(1,3,...) - Main thread that executes most of the requests
Thread even numbered(2,4,...) - Thread used only when needed for AJAX parallel requests so Thread(1,2) is a group as is Thread(3,4) and so on.
If we needed to make 3 AJAX requests in parallel then we would have grouped up Threads(1,2,3) and Threads (4,5,6) - but the main thread would have been Thread1, Thread4,...
You can immediately see resources are being wasted - Well we did say this was a toy.
Making the even numbered threads not execute some samplers is easy, we just need an IF controller to check. But how do you make Thread 1 and Thread 2 for e.g. execute a sampler in between at exactly the same time? In the Java world , this would usually mean reading up everything that Doug Lea and Brian Goetz have written - but in lazy developer mode this means downloading the JMeter source code and looking at SyncTimer (which blocks a fixed number of threads but not specific ones) and using the time honored technique of Copy Paste.
So before diving into the problem at hand lets see if we can simulate the following.
6 threads will arrive at varying times into one sampler. At which points they will wait till groups are ready i.e. Threads 1 and 2 will go together and Threads 3 and 4 will go together and so on.
Here's the Beanshell sampler which is actually a customized Sync Timer in disguise

import java.util.HashMap;
int totalThreads = 6; // MUST match total number defined in Thread Group
int groupsOf = 2; //MUST match number of requests to make in parallel

if (bsh.shared.myObj == void){
    // not yet defined, so create it:
    bsh.shared.myObj =new HashMap();
    bsh.shared.timerCounter =new HashMap();
    int numObjectsNeeded = totalThreads/groupsOf;
    for(int i = 0;i<numObjectsNeeded;i++) {
           bsh.shared.myObj.put(String.valueOf(i),new Object());
           bsh.shared.timerCounter.put(String.valueOf(i),new int[] { 0 });
    }
    print("populated");
} else {
   print("already init");
}

print(${__threadNum()});

int grpNo = (int)(${__threadNum()} -1)/groupsOf;
print(grpNo);
String groupNumber = String.valueOf(grpNo);
print(groupNumber);
Object syncObject = bsh.shared.myObj.get(groupNumber);
print(syncObject);
synchronized (syncObject) {
   int[] timerCount = (int[])bsh.shared.timerCounter.get(groupNumber);
   timerCount[0]++;
   final int count = timerCount[0];
   if (count >= groupsOf) {
                syncObject.notifyAll();
            } else {
                try {
                    syncObject.wait();
                } catch (InterruptedException e) {
                    log.warn(e.getLocalizedMessage());
                }
            }
            timerCount[0]=0; // Reset for next time   
}
SampleResult.setResponseData("sunk!");

The bsh.shared namespace is used to share objects between threads. So in our case we have 6 threads and we want 2 threads in parallel which means there are 3 groups of two threads each. To synchronize each group we need an object (hence 3 objects) and a counter each - to count how many threads are already available. Thats what the first part of the code sets up.(The first part is not thread safe but because we are going to test this by guaranteeing that threads arrive at different times , the snippet need not be thread safe).

The second part of the code then calculates which Group a thread belongs to, gets the object for the group, increases its count. If the number of threads is less than that we want in parallel it suspends the thread (by calling wait) otherwise it does a notifyAll which will wake up all suspended threads and allow these two threads to move in parallel

Heres what the test looks like


We have artificially added a timer to make sure requests to the Delay sampler arrive 1 thread at a time. We run the test and we can see that Threads(1,2) run the After Delay sampler at almost the same time , as do Threads(3,4) and as do Threads(5,6)
If we change the groupsOf in the delay sampler to be 3 then we see that three threads run in parallel.

And with that we are ready to use this delay sampler to make AJAX calls in parallel.
Onwards to Part III - An actual example of JMeter and AJAX

JMeter and AJAX - Part I

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part III-1 - A diversion on how to use session ids between threads 
Part IV - A more complex sample

Queries related to testing websites which use AJAX are often asked on the JMeter mailing list so here are my thoughts on this matter.

JMeter is not a browser, it simulates a HTTP request/response which means it will always struggle to replicate the finer parts of browser behavior. The advantage however is that it only needs very few resources to simulate a single browser. Testing tools that actually drive a browser will find it easier to replicate the browser however they pay for it when it comes to load tests - the amount of resources is needed is much more than the JMeter family of tools. But when testing out websites which use AJAX, JMeter with its current feature set may not be an accurate simulation because AJAX does depend on the finer parts of browser behavior.

An elegant solution would be to have the ability to define child HTTP requests of a parent HTTP sampler with a thread pool that defines how many of these will be made in parallel. But no such feature is currently available in JMeter.

But say you do have to test an AJAX enabled site today - what can one do?
It's useful in these times to be a lazy but honest tester - The honest part ensures that we will try to be accurate but the lazy part also ensures that we wont try to do any more than the minimum we need to do.

• So the first question that should come up is do we actually need to change anything in the way we test? The vast majority of AJAX enabled sites take the form a page loads , the user does some action , an AJAX call is made and values are shown to the user without updating the rest of the page. But look at the sequence of HTTP requests. Request for the main page, get response, browser processes response,user action, AJAX call made , AJAX response, further browser processing. There isn't any request made in parallel (ignoring all the static file requests which is its own blog post) - nor is there any difference due to Asynchronous nature of the call. So a JMeter script to simulate the above would have no difference from a normal JMeter script. the AJAX call should just follow the call to the main page and we are done.
• But hold on some AJAX enabled sites may make an AJAX call as soon as the page loads without any user action. Or the call might be made periodically.Even in these cases the onload/onready is only done after the response of the main request is made available so while the browser may make the request asynchronously in terms of the HTTP request/response there is no difference. The periodic call may only have a problem if the time it takes to return is greater than the periodic repetition. But the latter case would be a problem anyway , all you really want from JMeter is to detect if this happens. Once it happens your system is messed up and you need to fix it! There really isn't much value simulating the messed up state. To put it in another way , if your system breaks for 50 users, there isn't much value observing how much more is broken for 100 users.
• But hold on what if multiple AJAX requests are made on load? Say a users home page has two AJAX widgets , one which shows say his tasks , one which shows his emails? Well yes two requests will be made in parallel. Usually these requests are independent of one another. The question we really need answered is "Is there a difference between a user making two independent requests in parallel and two users(maybe with the same login) making a single independent request each in parallel?" In most cases , the answer is no. The load may be slightly higher in the latter case because you may have two sessions instead of one, but in terms of the rate of requests the server sees you can normally match the rate of requests by simply increasing the number of threads (and a little bit of math).
• But hold on what if the requests aren't independent? What if they utilize the same table/rows causing contention when its from the same user session what if widget2 depends on widget 1 for something in the session? Well one thing to do in this case is call the developer and ask him why the hell are there two dependent requests instead of a single one? That usually doesn't end well(even though the tester is right) - but yes at this time yes you might need to actually simulate requests in parallel. Note however a lot of AJAX enabled websites dont fall into this bracket and those websites can be simulated in JMeter with a reasonable degree of accuracy.

Onwards to Part - 2. Simulating AJAX requests without knowing how to modify JMeter core.

Google isnt as good as you think it is

Exhibit A - A snippet from webmaster tools for this blog



Though I know that writing about deepika instead of JMeter would give me more page hits!

WLST to dump free heap memory

from java.io import FileWriter
from java.io import BufferedWriter
from java.util import Date
connect(eval('username'),eval('password'),eval('url'))
domainRuntime()
cd('/ServerRuntimes/' + eval('managedServerName') + '/JVMRuntime/' + eval('managedServerName'))
heapFreeCurrentPerOld = str(cmo.getHeapFreePercent())
heapFreeCurrentValOld = str(cmo.getHeapFreeCurrent())
cmo.runGC()
java.lang.Thread.sleep(int(eval('sleepTime')))

heapFreeCurrentPerNew = str(cmo.getHeapFreePercent())
heapFreeCurrentValNew = str(cmo.getHeapFreeCurrent())
newDate = Date()
fstream = FileWriter(eval('outputFile'),true)
out =  BufferedWriter(fstream)
out.write(eval('managedServerName') + "," + str(newDate.getTime()) + "," + heapFreeCurrentPerOld + "," + heapFreeCurrentValOld + "," + heapFreeCurrentPerNew + "," + heapFreeCurrentValNew + "\n");
out.close();
disconnect()
exit()

There is also a property file which contains the configuration information

managedServerName=<MANAGED>
username=<USER>
password=<PASSWORD>
url=<URL>
sleepTime=<PAUSE>
outputFile=<PathToFile.csv>

And finally to run

$WL_SERVER_HOME/common/bin>wlst -loadProperties


This was written to output the free memory before and after GC for a particular managed server (which used to run jobs) on weblogic. It was cron'ed to run at fixed intervals. The generated file is used to check if there is a memory leak. The heap free current value should remain fairly constant. if it keeps reducing , we have a leak

Getting the current thread group name

${__BeanShell(ctx.getThreadGroup().getName())}

Academic and Happy

via http://urlai.com/url/theworkaholic.blogspot.com

Text analysis
theworkaholic.blogspot.com is probably written by a male somewhere between 36-50 years old. The writing style is academic and happy most of the time.

Jmeter with Spring Webflow

Motivation
The application uses URL containing dynamic parameters and so a record/replay fails
http://www.mail-archive.com/jmeter-user@jakarta.apache.org/msg30317.html

Solution
This short tutorial deals with test scripts that interact with dynamic data, and we will choose a Spring Webflow example to write a test script against.
So we look around on Google for a sample Spring Webflow application which leads us to
Spring By Example Sample
I'm not going to use the JMeter Proxy Recorder because it seems to harm beginners more than it helps. Testers seem to get the impression that all they need to do is record and replay the script and they are done. So I shall use Firefox and LiveHttpHeaders. We are going to Login to the site and create a User

Step 1: Request the Login Page

http://www.springbyexample.org/simple-webflow/login.jsp
GET /simple-webflow/login.jsp HTTP/1.1

Response 
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=[uniquevalue1]; Path=/simple-webflow

So the first request response shows that the system sets a cookie and the name should tell us that this is a session id. [TODO]. This should also indicate to us that we need to add a cookie manager. Note the status is 200 in the Response. This is the server telling us all is well.

Step 2: Submit Login

http://www.springbyexample.org/simple-webflow/loginProcess;jsessionid=[uniquevalue1]

POST /simple-webflow/loginProcess;jsessionid=[uniquevalue1]
HTTP/1.1
Cookie: JSESSIONID=[uniquevalue1]
Content-Type: application/x-www-form-urlencoded
Content-Length: 48
j_username=david&j_password=newyork&submit=Login

Response
HTTP/1.1 302 Moved Temporarily
Date: Tue, 17 Aug 2010 05:03:48 GMT
Set-Cookie: JSESSIONID=[uniquevalue2]; Path=/simple-webflow
Location: http://www.springbyexample.org/simple-webflow/index.jsp
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/index.jsp
GET /simple-webflow/index.jsp HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 302 Moved Temporarily
Location: http://www.springbyexample.org/simple-webflow/index.html
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/index.html
GET /simple-webflow/index.html HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]
HTTP/1.1 200 OK

a. The browser sends back the cookie that was previously set. However note that the URL also has a jsessionid which has the same value as the cookie that was set. This is known as URL rewriting. Our test will ignore this value because we will be using a Cookie manager
b. The server sends a 302 response to the request. This basically asks the browser to request a new URL (the one that the Location Response Header has). Note this can happen more than once as the next request too results in a 302 response, which finally results in a 200 status code.
HTTP Status Codes are discussed here
c. The server also sends a Set-Cookie for JSessionID again with a new value. This is usually done for security reasons. We can ignore this for JMeter because the Cookie manager will handle this for us. However the sampler that we use needs to have follow redirects set. If redirect automatically is set , JMeter would not get to see this Set-Cookie header and since the application has switched the session id , if you sent it the old session id (which was associated to the before login session)

Click the Create Link

http://www.springbyexample.org/simple-webflow/person.html
GET /simple-webflow/person.html HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

Response
HTTP/1.1 302 Moved Temporarily
Location: http://www.springbyexample.org/simple-webflow/person.html?execution=e1s1

----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/person.html?execution=e1s1
GET /simple-webflow/person.html?execution=e2s1 HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 200 OK
Date: Tue, 17 Aug 2010 05:06:04 GMT

Clicking the Create link also resulted in a 302 code so the browser issued a new request. But take a look at the URL. It has a parameter named execution with a value e1s1. Now click the link again.

Click Create link again

http://www.springbyexample.org/simple-webflow/person.html
GET /simple-webflow/person.html HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

Response
HTTP/1.1 302 Moved Temporarily
Location: http://www.springbyexample.org/simple-webflow/person.html?execution=e2s1
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/person.html?execution=e2s1

GET /simple-webflow/person.html?execution=e2s1 HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 200 OK

The "execution" parameter value changes! So if you run the same test multiple times , the value has to be different. This is a common cause of failure of Record / Replay tests. Some of the data is dynamic. This data can be of various forms
a. Something the application framework needs
b. Something where the data varies (E.g. the UserId, the Product ID). Typically a user friendly string is shown on the screen but when the user performs an action , an id (usually a key in the database) is what is used by the application
c. Dynamic fields (E.g. DHTML based applications) which get added on the fly.
To deal with this parameter/URL we need to be able to extract the value from previous requests in our JMeter scripts.
If you haven't developed the application it may be difficult to identify whats dynamic and whats not. The easy way is to record the script twice(With different data) and observe what changes. e.g. if you browse a product catalog , change the product.

Submit the form

http://www.springbyexample.org/simple-webflow/person.html?execution=e2s1

POST /simple-webflow/person.html?execution=e2s1 
HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]
Content-Length: 56
id=&firstName=deepak&lastName=shetty&_eventId_save=Save

Response
HTTP/1.1 302 Moved Temporarily
Location: http://www.springbyexample.org/simple-webflow/person/search.html
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/person/search.html
GET /simple-webflow/person/search.html HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 200 OK

Unfortunately the application has a visual bug, the added user does not show. However if you do a view source on the page and search for the username you created you can find it.

Click Logout

http://www.springbyexample.org/simple-webflow/logout
GET /simple-webflow/logout HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

Response
HTTP/1.1 302 Moved Temporarily
Set-Cookie: SPRING_SECURITY_REMEMBER_ME_COOKIE=""; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/simple-webflow
Location: http://www.springbyexample.org/simple-webflow/logoutSuccess.jsp
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/logoutSuccess.jsp
GET /simple-webflow/logoutSuccess.jsp HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=[uniquevalue3]; Path=/simple-webflow

The JMeter Script
Now that we know the above we can get to writing the script

Most of the script is pretty much standard(All Samplers use follow redirects and we have added a Cookie Manager), till we get to the point where we need to extract out the URL.
The Request Create Sampler extracts out the URL. Since we know that the application is going to redirect us , the extracter will extract out the last URL we will be on



The Post Create Data Sampler simply uses this extracted URL



Note that all samplers have assertions that verify the behavior, so if something goes wrong the Assertion should fail. For e.g. after creating the user we verify that the user we created exists on the page. When requesting a page the assertions verify some text on the page that is unique to the page.
Run the test, we get no errors! Modify the ThreadGroup to have two threads and two iterations. Run the test , everything should pass. Now look at the View Results Tree listener. You should see the redirects being followed (multiple children results under a parent). But do you see different values for the execution parameter? No - This is because each thread is its own session and each user will start with the same parameter value. Even having multiple iterations on the thread group doesnt give you different values because we are logging out at the end of the script so the next iteration is a new session.

Try adding a Loop controller to create 10 users. Note that the loop controller must enclose the Request Create and Post Create samplers (because we need to extract a new value for the execution parameter every time!). Using View results tree you can now observe the execution parameter changing.



Sample Script
Spring Webflow.jmx

Spring Webflow with loop.jmx

First weblogic portal expert

YAY!


Thankfully a lot of posters don't use Google, otherwise I couldn't answer the same questions again!

Updated JEE Interview Questions - Draft

JSP
What are the various way's to handle exceptions in a web application. What would you use when. Demonstrate with examples from any project you have worked on
What are the various way's you have implemented security in your web application. Also mention alternatives that can be used
Which framework have you used. Please describe the shortcomings in the framework and the ways to work around them
What is the difference between a static include, jsp:include, jsp:forward and redirect. Please give one example of each.
Describe some Tag Libraries you have written. What are advantages / disadvantages of tag library. If possible give an example of a badly written open source JSP tag library(or any of your own) and how you would improve it.
Give one example of when you would use a filter. How do you execute a filter after the request?
Give one example of when you would use a listener. Which listeners have you used in your project?
Besides defining servlets , name one feature that can be specified in the web deployment descriptor
Name JSP implicit object. Give one example of the use of each implicit object. Is exception an implicit object? If so can i write exception.printStackTrace in any JSP? what will it print?
Comment whether a JSP web application works with cookies disabled.
Which web container did you use. Please describe how to use a container specific feature.
Describe some JSTL tags you have used and what they are useful for
How do you use a datasource in a JSP?
How do you internationalize a web application. What all do you need to take care of

EJB
How do you implement security in an EJB . Why does the framework provided security not work in most real world scenarios? (or does it?)
If an MDB throws an Exception what happens to the message?
If an EJB has two methods M1 and M2 both with RequiresNew and M1 makes a call to M2 by calling it (because M2 is a method in the same class) how many transactions are there?
What limitations are

JDBC
Lets say you want to execute a query and show some data. Walk through the steps to do this in Raw JDBC.
What do you prefer using of the following and why? CMP/JDO/ORMs (like hibernate/toplink)/SQL helpers (like Spring/iBatis)/Raw SQL?
In the context of ORM what is lazy loading/eager load. What do you use in your project? Why? What factors determine what is to be used.
Your ORM based system is slow. How would you diagnose the problem?
Why is pessimistic locking not useful in a web-application (or is it?)

JMS
Say you have a queue. The Producer can produce messages much faster than the consumer can consume. What are your options?


JMX
Give any example where you have used JMX

WEB
Given a large data set , what options do you have for displaying this dataset. Which one do you prefer and why?
Suppose you are tasked to implement an AJAX based system where the user types in a word in a search box and you want to show him possible queries (e.,g. in Google if you type dee you get deepak chopra or deepika padukone)in a drop down , what considerations would you have in implementing the AJAX parts of the client (Assume that you have a server side component that can give you the results)
If you want Google to search your site easily , what do you need to take care of in your web app

Misc
You have a clustered J2EE web application. You need to batch process a file which has a billion records. Each record is independent . How do you do it?
You have a table which has UserID (primary key) and UserName. The requirement is to display this in a drop down box on a page so that the user will select the name but the value posted to the next page will be the userid. The data displayed must be sorted by username. The DB developer asks you what data structure he needs to return the results in. What is your answer? Assuming that you get this data structure , is there any comment you wish to make on your implementation?
Name a few design patterns that you see being used in the Java API with examples.

Simplified procedure for Syncing Embedded LDAP with Weblogic Portal Database

a. Shut down servers
b. On the admin server, backup the LDAP directory \servers\AdminServer\data\ to another location
c. Delete the internal LDAP directory at \servers\AdminServer\data\ldap (also from managed
servers if applicable)
d. Delete data from the p13n entitlement tables (delete from tablename)
( namely p13n_entitlement_role, p13n_entitlement_resource,p13n_entitlement_policy, p13n_entitlement_application and p13n_delegated_hierarchy )
e. Start the servers
f. Redeploy the application
g. Recreate the SAML relying party configuration
h. Recreate Entitlements

Dynamic Parameters in JMeter

Motivation

You need to send a variable number of parameters with an HTTP request i.e. the number of parameters are not known when the script is written. For example a User can register with a number of accounts , but this number varies per user. Or perhaps you want to Post process extract a number of hidden fields on the previous response and post all of them in the next Request, but you either don't know the number or there are too many fields to enter manually into the HTTPSampler.

Solution

Use a BeanShell PreProcessor to dynamically add variables to the sampler. The relevant code is sampler.addArgument(name,value)[1]

Sample

We will write a script to access a page , extract out data based on a pattern and send these as separate parameters to the next request. The script is as shown below.

We make a request to some dummy page and then we have a regex extractor to extract out multiple values. In this case the Regex extracts out all words(Match No = -1) beginning with test

Then we have a Debug Sampler to verify the regex is working, notice the results in View Results Tree Listener. The Regex has a reference variable name of inputTerms. The response of the Debug sampler is shown below

You can see from the Listeners response tab that the total number of results is 14 , available under the key inputTerms_matchNr and that each result is available as inputTerms_$resultnumber.

The matching groups which we usually do not need are available as inputTerms_$resultnumber_g$groupnumber. Armed with this information, the beanshell preprocessor script is trivial as shown below

int count = Integer.parseInt(vars.get("inputTerms_matchNr"));
for(int i=1;i<=count;i++) { //regex counts are 1 based
sampler.addArgument("hardcodedkey", vars.get("inputTerms_" + i));
}

Heres the next request to which the beanshell pre processor is attached. We have one predefined variable and the rest will be dynamically added by beanshell.

Next we check the request data we are posting using the Request tab of the View Results Tree Listener

Notice that the argument we added to the next request was sent, as were all the parameters we added dynamically. The parameter values were also encoded.
Sample Script : https://skydrive.live.com/#cid=1BD02FE33F80B8AC&id=1BD02FE33F80B8AC!268

Reference
[1] Javadocs for HTTP Sample Base

Asserting MS Office formats

As a follow up to the PDF post, you can do something similar using Apache POI

import org.apache.poi.POITextExtractor;
import org.apache.poi.extractor.ExtractorFactory;
import java.io.ByteArrayInputStream;

ByteArrayInputStream bais = new ByteArrayInputStream(in);
POITextExtractor extractor = ExtractorFactory.createExtractor(bais);
prev.setResponseData(extractor.getText());

You should be able to parse word documents and excel and ppt's using this beanshell post processor. Remember to copy the POI libraries to JMeter/lib

Future work
Look into Apache Tika for a unified interface (may need to sacrifice some functionality like the startPage endPage of PDFBox)

Asserting PDF's

A question on the JMeter mailing list regarding extracting/asserting text inside a PDF file and since I ran into this for a functional scenario , I wrote up my attempt. I use PDFBox as the library. Download the binaries and copy pdfbox-1.0.0.jar and external/fontbox-1.0.0.jar into JMeter's lib directory. We'll access the PDF at http://jakarta.apache.org/jmeter/usermanual/jmeter_distributed_testing_step_by_step.pdf and check that the PDF does contain "Distributed Testing Step-by-step" Here's the JMeter sample script So we have a transaction controller 'Check PDF' so that we only get a result item. The HTTP Request Sampler 'Request PDF' requests the PDF. The bulk of the code is in the beanshell post processor titled 'Extract Text'

import java.io.ByteArrayInputStream;
import java.io.StringWriter;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.util.PDFTextStripper;


PDDocument document = null;
StringWriter sw = new StringWriter();
try {
ByteArrayInputStream bais = new ByteArrayInputStream(data);
document = PDDocument.load(bais);
PDFTextStripper stripper = new PDFTextStripper("UTF-8");
stripper.setSortByPosition( false );
stripper.setShouldSeparateByBeads( true );
stripper.setStartPage( 1 );
stripper.setEndPage(Integer.MAX_VALUE );
stripper.writeText( document, sw );
} catch (Throwable t) {
t.printStackTrace();
sw.append("ERROR");
} finally {
sw.close();
document.close();
}
vars.put("extractedText", sw.toString());

Update - for PDFBox 2.0.26

import java.io.ByteArrayInputStream;
import java.io.StringWriter;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.text.PDFTextStripper;


PDDocument document = null;
StringWriter sw = new StringWriter();
try {
	ByteArrayInputStream bais = new ByteArrayInputStream(data);
	document = PDDocument.load(bais);
	PDFTextStripper stripper = new PDFTextStripper();		
	stripper.setSortByPosition( false );
	stripper.setShouldSeparateByBeads( true );
	stripper.setStartPage( 1 );
	stripper.setEndPage(Integer.MAX_VALUE );        
	stripper.writeText( document, sw );
} catch (Throwable t) {
                             t.printStackTrace();
	sw.append("ERROR");
} finally {
	sw.close();
	document.close();
}
vars.put("extractedText", sw.toString());

All this does is use the PDFBox API to extract text from the PDF (bytes are present in the data object) and write it to a variable in JMeter , extractedText. The next sampler is a Java Sampler which sets the ResultData to ${extracted text}. This will echo back the contents of this variable as the response of the sampler. Once this is done , you can use a normal response assertion, regex extractor or whatever else you need to process the text Update: as mentioned by Milamber in the comments , you can instead have the last line of the beanshell as

prev.setResponseData(sw.toString());

This will set the response of the HTTPSampler to be the text value and allow you to directly specify the assertion (and you can also eliminate the transaction controller). Update : if you are interested in MS office formats then follow the steps in this post, just change the BeanShell post processor to that mentioned in Asserting MS Office Formats Future work a. make a custom PDF(or any format sampler) with the options that are hardcoded (e.g. startPage or endPage) b. experiment with ways to use less memory. Typically should extract and write to file and use that.

Random stuff on jmeter testing

Update : I realized that much of what I wanted to say has already been said and in a far better manner in Perfomance testing patterns and practices.
One of the questions asked on the JMeter mailing list was

How do I analyse my application with JMeter?

There are various reasons one might run a load test broadly detection, simulation, verification and analysis. The type of test or the scripts you might write, what you would measure or track would vary a bit depending on what you want to do. JMeter is good for simulation and verification. It is an aid to detection and analysis but you usually need other tools to help you out.

Detection

• You want to find out if your application behaves correctly when accessed by multiple threads. Whether your database starts showing deadlocks or whether you might have race conditions.
• You want to find out if your application has memory leaks.

Simulation

• You want to find out how your application behaves under standard load or how it behaves under some peak load(e.g. shopping during the holidays). Response times etc.

• In the specific case of Java based web applications you want to know how often your GC cycles might run or how long.

• You application makes remote calls(e.g. webservice calls) and you want to know whether all the resources are recovered correctly. perhaps you have some throttling mechanisms in place and you want to see that its working correctly

Verification

• You want to verify the result of changing some tuning parameters.
• You want to check your SLA's are met
  

Analysis

• You might already know there is a problem and you want to simulate load while you are profiling the application.

These areas overlap and I've used the above categories quite broadly to merely illustrate that your objective and hence your test script will vary based on your objective.

For e.g.
You want to find out if your application behaves correctly when accessed by multiple threads - In this case your test script would only be concerned with running some parts of the applications at exactly the same time. You'd want to exercise multiple parts of your application. Perhaps you are aware that some part of the application internally spawns threads and you'd run a test that exercise that area for a long time or with a high load. You don't at this point really care whether these are unrealistic scenarios or non representative scenarios, nor are you really looking at what the response times are. All you care about is do you see stuck threads or deadlocks. Do you see a really long wait time for most threads though some threads finish really fast.

Or perhaps you want to tune a memory parameter and you want to verify the change
In this case Response Times / Throughput really matter (for the same test of course). You'd first take a baseline reading without the tuning, and then another with the tuning. The test scripts must be representative of actual user behavior.

Perhaps you want to check whether your site can handle holiday shopping onslaughts. In this case you would modify your tests to show bursts of activity, you'd closely monitor response times but you also want to check what happens on the server. How much memory, How much CPU. You might also want to see what load might actually bring down your servers. You might want to check if your load balancers evenly distribute the load.

Or perhaps you have certain Service Level Agreements and you need to know response times accurately for the load specified in your agreement. In this case you need a representative user journey and you also need representative background users.

All of which means there is no easy answer to ' How do I analyse my application with JMeter'. It can only be answered by What is it that you want to analyse (normally answered as well, performance).

Lets take the most common use case, what is the 'response time' for my application.
However actually getting the response time is more difficult than reading the response time calculation from the JMeter test results.
This is problematic due to
a. JMeter is not a browser and does not render the page. Different browsers take different times to render the same page. Compare older versions of Internet explorer with Chrome for e.g.
b. A returning user with some files cached will probably show lesser times than a first time user.
c. The network / connection speed from which the user is accessing the application may be significant. And your users may be spread out throughout the world.
d AJAX based applications / DHTML applications are difficult to predict because not only does it vary by browser , but the number of calls that a browser may make in parallel is also different, but some calls will be made in parallel and its difficult to know that.

So any response time would have (roughly speaking)
a. The time it takes for the application to actually respond with all the data
b. The time it takes for this data to be transferred over the network
c. The time it takes to download static files (bearing in mind that not all files may be downloaded and that browsers may request multiple static files in parallel)
d. The time it actually takes to render the page.

JMeter can help you out with a, b, and c. but what it is good at, is finding out a. for the network on which it is running on.

Typically your requirements might define an Service Level agreement for your site as Browsing operations must take < 6seconds 90% of the time and shopping operations must take <8 seconds 90% of the time. You also know how much large your pages are and you can guesstimate how much time it would take for the page to be transferred over the internet. You might take an average with some safety factor or you might take a worst case scenario. Using a browser tool like YSlow or Googles PageSpeed , you can also have some insight on how your static are downloaded , how long they take etc. And you might add some time for how long the browser takes to render. After considering all of this you might arrive at a new figure that on a high bandwidth intranet (which thereby eliminates most of the network variables) your browsing operations must take < 2 seconds just to get the data and your shopping operations must take < 4 seconds for your SLA's to hold because the rest of the time has already been used by the other factors.
After this you would have to write a script which generates representative loads (for the operations being verified and the operations that would happen in the background), run the test and verify the 90% percentile lies below the value you have calculated above. But perhaps it doesn't. Static files can be optimised by reducing their number, their size, gzipping them adding expiry headers etc, but maybe you have already done this. The Clients network and browser aren't within your control so there isn't much you can do there. The next step is figuring out where your problem lies. JMeter can't help you there, you need a different set of tools. But JMeter can help you to simulate load or parts of it so that you can monitor your application with the tools of your choice. Some of your findings may be infrastructure related, Some may be code you'd have to make changes and retest and repeat.