And the winner is - Stripes!

Recently I had the unpleasant experience of having to select a "framework" for a J2EE application.

In my day job , the decisions have been usually made for me (Weblogic Portal aka BEA Weblogic Portal aka Oracle Weblogic Portal). Pageflow was ok (even though the stateful behavior made many normal HTTP operations painful). But the Portal parts were decent and we did have reusable(the holy grail!) portlets.

When it was time to choose something for the web application I wanted to develop , there were quite a few options. Years ago , this would have been simple. Free , documented, widely used == Struts. Now this isn't quite so simple.

The first thing I eliminated was a portal container. The standard ones use JSR-168 or 286 and in their pursuit of WSRP, Federation and other portal goodies forget that they are actually using HTTP. A web based framework that needs you to jump hoops to get the HTTPServletRequest usually indicates that the creators of the framework weren't thinking their actions through. If there ever was a J2EE spec that was worse than EJB , it's JSR 168. In their crazy pursuit of the least common denominator , we get crippled products. No thank you.

I did however want something that would let me easily reuse menu's , footers , right hand modules etc. jsp:include just doesn't do it so well , where does the underlying code that fetches data for these common pages reside? If the JSP is to be just a dumb view, this is a problem.

Struts shows it age (and you need Tiles to support the above and tiles is painful) . Perhaps my views are clouded due to a project that I had worked in a past life. I assume the architect(s) wanted to beef up their CV so they had Struts, Tiles, Spring, Hibernate, iBatis , ofBiz, freemarker, ofbiz's persistence framework and every single open source technology that was somewhat popular.  Getting all of these to work together was an exercise in frustration (note this was years ago) - but as soon as I am told to use Tiles I remember this project and I stop.

Enter Spring. When I first used it , I was impressed with how well it did basic stuff. I loved Spring JDBC - this was better than crappy ORM frameworks(yes Hibernate , you too). It let people who knew SQL and enjoyed SQL work with SQL without the normal associated JDBC trappings (to this day , my homegrown SQL utilities closely resemble Spring JDBC API's). Spring MVC should be good , right? Perhaps it is. But the documentation sucks and I don't have that much time to invest. It also felt that Spring MVC was more concerned with Spring Integration and the MVC pattern than it was in solving the problem of a easy to use web framework (totally biased , totally unjustified view , based on a cursory reading of documentations - and apologies to the Spring team, who I salute - but really your documentation should be better if you want to draw in people like me)

Enter Wicket - But then two lines from the documentation
"JSP is by far the worst offender" "Wicket does not introduce any special syntax to HTML. Instead, it extends HTML in a standards-compliant way via a Wicket namespace that is fully compliant with the XHTML standard".
Look at the feature list - http://wicket.apache.org/meet/features.html . Does it start with simple to use , web framework , supports all web related features? No it starts with we use POJO's (i.e. you can unit test!). Wow. Because Unit testing a web application is value for money.
Bye Bye Wicket. You've been clean bowled. I never understood the dislike JSP gang. It's like looking at bad java code saying Java is by far the worst offender. I like tag libraries when they dont attempt to be
HTML encapsulators. I like being able to read Java code instead of the mental shift associated with interpreting yet another template framework syntax. I like the ability to embed Java code instead of new syntax , and I use it sparingly, and it works well thank you very much. I understand you might have a valid different philosophy , but it's not for me.

Enter the Play framework. Seemed simple (though non JSP). All in all looked to be great and simple. Seemed to get web applications. Targets restful structures. It would be nice to learn a new framework and see what it does better and what it does worse. But the tutorials seemed to be too command line heavy. There's nothing wrong with that of course. One problem I had is since I was trying to implement this project for a friend and I would have to hand it over to someone, I needed to use something that could be easily learnt or was widely used. Play could be it - but it is so different from the rest, that I hesistate.

Enter Stripes. Did I like the documentation. The HowTo seemed to cover everything I want, and the framework seemed to hit the right balance between what it did and what you had to do. Yay we have a winner. The proof of simplicity , my wife who is a PHP developer with rudimentary java knowledge picked up this framework in a couple of hours (or she is exceptionally smart and yes she reads this blog)

Java based web frameworks

A quick rant on the state of web based frameworks in the J2EE world
a. Imposing Maven on me is not nice.
b. If your "web" based framework cannot support clean URL's you are in the wrong business when you are writing frameworks
c. A J2EE standard doesn't mean its a good standard (JSF - Im looking at you)
d. When you want a view technology please please use JSP. Its mature and it works. It is as good or as bad as you want it to be. Making your own template language which is either inflexible or just as bad as JSP is just plain silly. Reinventing JSTL with a different archaic syntax is silly. Doing it in the guise of "no java in JSP" is silly - There isn't a framework that I can't write bad code in.
e. Make simple things simpler and make hard things easier if possible and it's ok if hard things remain hard. But do not make simple things simpler(oh look how quickly I can do CRUD) and hard things harder(because lets face it , every application does have corner cases).  Do not make hard thing's simpler and simple things harder (I have to set up how many files for Hello World?) - because lets face it not everything in a web application is Rocket Science.
f. Who would have thought Spring would turn out to be more heavyweight than EJB.
g. Who would have thought that Struts 1 would still be so heavily used
h. Who would have thought that I would now prefer to code in ASP.net than identify which Web Framework I should use.

Random Value Selection using JMeter XPath Post Processor

JMeter has two post processor extractors which are usually used when we need some data extracted from the response. The XPath extractor is useful when the Regex would match other elements in a page (e.g. a link inside an li element). However the XPath extractor is missing a functionality that the Regex Post Processor has. It only returns all the values (equivalent to specifying -1 in a Regex Post Processor). XPath also natively supports returning the "nth" value using the position() function. However it doesn't support returning a value at random.
However using normal JMeter functions we can easily implement this functionality.
Assume that you have an XPath post processor that is attached to a HTTP Sampler and is returning n values into a variable called testXPATH. if we now want to select a value at random what we need to use is

${__V(testXPATH_${__Random(1,${testXPATH_matchNr})})}

But thats a mouthfull and needs an explanation

${testXPATH_matchNr}

returns the count of how many elements matched. We are assuming that some values are going to be selected, otherwise you need an IF controller to check the condition.

${__Random(1,${testXPATH_matchNr})}

Will then select an integer between 1 and the count of the values (TODO check if we need to add 1)

${__V(testXPATH_${__Random(1,${testXPATH_matchNr})})}}

Finally we form a string of the form testXPATH_n e.g. testXPATH_3 which is then passed to the __V function which will get us the value for the variable
http://jakarta.apache.org/jmeter/usermanual/functions.html

Note that running TIDY on HTML , followed by parsing it to create a DOM structure and finally running an XPATH query against it is signifcantly slower than just running a Regex against a String. If you need to generate a large load and you have limited resources available to you, then the XPATH extractor is probably not the best alternative.

Looping through all output of a RegExp extractor (without the for each controller)

I forget why the user couldnt use a For Each controller , but heres how to loop through the result of a Regex Post Processor that returns multiple values

ThreadGroup
+HttpSampler
++Regex Extractor (variableName = links)
+WhileController(${__javaScript(${C} < ${links_matchNr})})
++HTTPSampler use ${__V(links_${C})} to access the current result
++Counter (start=1,increment=1,maximum=${links_matchNr},referenceName=C)

Dynamic values within files

I have a HTTP Post request sample to which we are passing an XML file through option "Send Files With Request". The XML file to be sent with request contain on Tag "Order id" which need to be modified [ dynamically during test plan executing] before sending the request. Original
Typically in JMeter we would use a ${variableName} notation. However JMeter only replaces these values when the the ${variableName} is within the script itself , not when it is referenced in some other file. As the user wants to use a separate (template) file which is added to the HTTP request, this technique wont work.
One option to consider is whether we could generate all the files we need before we run the test. this is possible for example when the dynamic data is a set of id's that is accessible say from a database. In this case the better (in the more efficient sense) is to have a initialisation step (perhaps as part of the build or as a separate step in Jmeter) to create all the files that the test would need. Once this is done it is easy to make the JMeter script pick one file (usually using the CSV DataSet Config).
However this isnt always possible. In the question above , the Order ID could be dynamically generated by the system under test in ways that might not be predictable. In this case we cannot generate the files in advance. We have to extend JMeter.
The cleaner more elegant way to do this would be to write a custom pre-processor of some sort because it sounds as if this functionality could be reused in some other context (or extend the HTTPSampler). But we will choose the quick and dirty way of writing a BeanShell preprocessor that does this for us.

Briefly the solution will
a. Have a template XML file. We will use @variableName@ within the file for what we want replaced. We will currently only support replacing a single variable (but it is easy to support multiple variables).
b. We will configure the HTTPSampler to use a generated file. At runtime a pre - processor will read the template file and generate this new file. This new file will be read and posted by JMeter

The Structure of the test is shown below
The HTTP Sampler adds a currently non existent file to be posted. This is the file the Pre Processor will create. The name of the file is varied based on the ThreadNumber (we could have added timestamps etc as well) so that each thread will get its own file name

We use user defined variables to define the location of the temp file, the regex that we want to run on the file and the replacement value.

The BeanShell pre-processor

import java.io.BufferedReader;
import org.apache.jmeter.protocol.http.util.HTTPFileArg;
import java.io.File;
import java.io.FileInputStream;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.charset.Charset;
import java.io.BufferedWriter;
import java.io.FileOutputStream;
import java.nio.channels.Channels;

//read the file as string
FileInputStream stream = new FileInputStream(new File(vars.get("templatePath")));
String fileAsString = "";
try {
      FileChannel fc = stream.getChannel();
      MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());     
      fileAsString = Charset.defaultCharset().decode(bb).toString();
}
finally {
        stream.close();
}
//perform the replacement
fileAsString = fileAsString.replaceAll(vars.get("replaceRegex"), vars.get(vars.get("variableName")));

//assumes first file
HTTPFileArg arg = sampler.getHTTPFiles()[0];
//write it out to the file we want
FileChannel fout = new FileOutputStream(arg.getPath()).getChannel();
BufferedWriter bf = new BufferedWriter(Channels.newWriter(fout, Charset.defaultCharset().name()));
bf.write(fileAsString);
bf.close();

Disclaimer : File Reading writing snippets swiped from Google, you can probably make it more efficient.

• Briefly the snippet reads the template file(location defined as a variable) as a String (using the default charset , if you deal with multiple languages you might want to force this to UTF-8).
• It then runs the replacement using a regex (also a variable) and the subsitution value (also a variable which for this example we have just spoofed)
• Finally it writes the file(assuming default charset , again you could force this to be UTF-8). To determine the location of the file , it reads the location from the HTTPSampler. This is currently set as the first file , but we could have used a name or any other technique.

And finally the sampler will read the file that we have created and post it as usual.

Sample JMX available here

java memory leaks

One of my colleagues asked how I diagnosed a memory leak that was impacting his application , so this is what i said. Maybe Ill write it tutorial form some time


The version of java we are running doesnt make this easy so this is what I did on dev (its hard to make this a coherent narrative).

Almost any memory leak debugging takes the following form(Im assuming the Out of Memory says the HEAP is out of memory)
a. Run your application steps that are causing the memory leak (detecting this is actually the biggest problem but if you dont know what step is causing the leak , you basically have to access all the functionality of your application). This ensures that all statics/caches/classes are loaded. Then force a garbage collection. Then check the memory for the Java VM and take a heap dump (this is a binary form of all the objects on the heap including the allocation graph)
b. Now repeat the step causing the problem. make sure that the step ends correctly (for e.g. if the step is a user goes and saves the quote, ensure that you logout and invalidate his session otherwise you get false readings). Now run a full GC again. Take another heap dump.
c. Compare the two heap dumps. if there is a difference that means you have a leak - ideally there shouldnt be a difference. In practice on a server like tomcat there are a lot of things happening in the background even if no user is accessing the site so even after a full gc , you might still see some tomcat classes or objects that appear to be new. The way around this is that you have to loop the step in b so that you will get to see significant differences for your classes rather than the app server classes.

For our env however taking a heap dump seems problematic because of the version of JDK seems really old(1.5 _07) and I wasnt able to load the taken heap dump in any tool.

Heap Dump
To take a heap dump you have the following options
a. Some JVM's support -XX:+HeapDumpOnCtrlBreak (kill -3 pid on linux). However our version doesnt
b. Some profilers(e.g. Jprobe) allow you to take heap dump - You have to pay for this but there are some free ones like NetBeans. however all of them need you to run some install steps so I didnt use this
c. use jmap . This is a utility that Java provides. Later versions let you say JAVA_BIN/jmap -dump processid. For our version we have to say jmap -heap:format=b processid . however this isnt working correctly since i couldnt load this dump into any tool. what I finally used was jmap -histo processid. This is a poor mans heap dump . it shows the classes and the number and the size they occupy, but it doesn't show who allocated the classes (so for e.g. I knew it was the concurrency related classes that were leaking , but I didnt know who was creating them - that was thanks to Google)

d. +XX:HeapDumpOnOutOfMemory - This will automatically take a heap dump when you run out of memory but debugging with this is slightly harder because you have live objects
e. use HPROF (this is an earlier version of the tool) when you start up java but it will slow the process.

Monitoring
To look at how much memory is being used, and to force GC you have the following options
a. Look at the probe we installed (specific to tomcat) - lambda/psi probe . Click System information . under overview you can see the memory being used and you can Force GC as well. You can also click Memory Utilization and it will show you the breakup. I had to add some JMX parameters to startup. This is tomcat specific.
It also lets you see sessions and their footprints (sometimes the leak isnt actually a leak. Code puts too much data in session , the user doesnt log out , the session remains active for say 30 minutes , then that much memory is used up for 30 minutes. technically this isnt aleak , when the session times out , the objects will get collected , but it looks like a leak because the memory keeps getting used , but doesnt get reclaimed)
b. use JConsole . This is in the bin directory of your java. When you start it on your machine you can add a remote connection
These are the settings I added on dev
export JAVA_OPTS="-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/local/apache-tomcat-5.5.30/logs -XX:PermSize=256m -XX:MaxPermSize=256m -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=12345 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dfile.encoding=UTF8"
This will also let you see some things that the probe does.
Using this I could find out
a. That it wasnt the sessions causing the problem (since the probe showed that only 1 MB was being used in sessions).
b. I also used it to check and see whether threads were causing issues , not the case.

Testing
I used JMeter to simulate a web user going and approving a quote and simply looped that 4000 times. I couldnt not use say SOAPUI because we needed to simulate a stub being created whereas SOAP acts as the client so it wouldnt have shown the problem. Ordinarily you would not know the Stub causes the problem , but the histograph had shown such a large number of the concurrent objects on the heap , that a google search showed that other people had faced this problem with axis 1.4.1. If I didnt have this I would have had to write a script that would have placed a quote, gone and approved it , saved it , modified it etc.

The histographs are in /root/histbeftest and /root/hist92per and are easily compared to verify.

Analysis
The histograph files are plain text and can be compared in a text editor. If I had a heap dump we could use
a. JHat - this comes in the bin directory of java (1.6 onwards and some update of 1.5 onwards but it can read 1.5 dumps). This will let you browse the heap and even compare them
b. Profilers like JProbe
c. Other tools like VisualVM, HPJmeter NetBeans, extensions in eclipse all let you load heap dumps.
One of the issues you might run into is that the Heap Dump file is quite large and your client tool may not have enough memory. To verify a memory leak you usually reduce the amount of memory you give to the JVM so that you get smaller heap dump files (so for e.g. the iquote app once it is running and all the classes are loaded takes about 20 to 30% of 512MB which is about 150 MB. So I could reduce the memory to 256MB instead of 512 and run the tests to create a much smaller heap dump)

a memory leak in sample code and then trying to detect it using tools is usually the best way to learn.