Workaholic

And the winner is - Stripes!

2011-11-26T22:10:00.001-08:00

Recently I had the unpleasant experience of having to select a "framework" for a J2EE application.

In my day job , the decisions have been usually made for me (Weblogic Portal aka BEA Weblogic Portal aka Oracle Weblogic Portal). Pageflow was ok (even though the stateful behavior made many normal HTTP operations painful). But the Portal parts were decent and we did have reusable(the holy grail!) portlets.

When it was time to choose something for the web application I wanted to develop , there were quite a few options. Years ago , this would have been simple. Free , documented, widely used == Struts. Now this isn't quite so simple.

The first thing I eliminated was a portal container. The standard ones use JSR-168 or 286 and in their pursuit of WSRP, Federation and other portal goodies forget that they are actually using HTTP. A web based framework that needs you to jump hoops to get the HTTPServletRequest usually indicates that the creators of the framework weren't thinking their actions through. If there ever was a J2EE spec that was worse than EJB , it's JSR 168. In their crazy pursuit of the least common denominator , we get crippled products. No thank you.

I did however want something that would let me easily reuse menu's , footers , right hand modules etc. jsp:include just doesn't do it so well , where does the underlying code that fetches data for these common pages reside? If the JSP is to be just a dumb view, this is a problem.

Struts shows it age (and you need Tiles to support the above and tiles is painful) . Perhaps my views are clouded due to a project that I had worked in a past life. I assume the architect(s) wanted to beef up their CV so they had Struts, Tiles, Spring, Hibernate, iBatis , ofBiz, freemarker, ofbiz's persistence framework and every single open source technology that was somewhat popular. Getting all of these to work together was an exercise in frustration (note this was years ago) - but as soon as I am told to use Tiles I remember this project and I stop.

Enter Spring. When I first used it , I was impressed with how well it did basic stuff. I loved Spring JDBC - this was better than crappy ORM frameworks(yes Hibernate , you too). It let people who knew SQL and enjoyed SQL work with SQL without the normal associated JDBC trappings (to this day , my homegrown SQL utilities closely resemble Spring JDBC API's). Spring MVC should be good , right? Perhaps it is. But the documentation sucks and I don't have that much time to invest. It also felt that Spring MVC was more concerned with Spring Integration and the MVC pattern than it was in solving the problem of a easy to use web framework (totally biased , totally unjustified view , based on a cursory reading of documentations - and apologies to the Spring team, who I salute - but really your documentation should be better if you want to draw in people like me)

Enter Wicket - But then two lines from the documentation
"JSP is by far the worst offender" "Wicket does not introduce any special syntax to HTML. Instead, it extends HTML in a standards-compliant way via a Wicket namespace that is fully compliant with the XHTML standard".
Look at the feature list - http://wicket.apache.org/meet/features.html . Does it start with simple to use , web framework , supports all web related features? No it starts with we use POJO's (i.e. you can unit test!). Wow. Because Unit testing a web application is value for money.
Bye Bye Wicket. You've been clean bowled. I never understood the dislike JSP gang. It's like looking at bad java code saying Java is by far the worst offender. I like tag libraries when they dont attempt to be
HTML encapsulators. I like being able to read Java code instead of the mental shift associated with interpreting yet another template framework syntax. I like the ability to embed Java code instead of new syntax , and I use it sparingly, and it works well thank you very much. I understand you might have a valid different philosophy , but it's not for me.

Enter the Play framework. Seemed simple (though non JSP). All in all looked to be great and simple. Seemed to get web applications. Targets restful structures. It would be nice to learn a new framework and see what it does better and what it does worse. But the tutorials seemed to be too command line heavy. There's nothing wrong with that of course. One problem I had is since I was trying to implement this project for a friend and I would have to hand it over to someone, I needed to use something that could be easily learnt or was widely used. Play could be it - but it is so different from the rest, that I hesistate.

Enter Stripes. Did I like the documentation. The HowTo seemed to cover everything I want, and the framework seemed to hit the right balance between what it did and what you had to do. Yay we have a winner. The proof of simplicity , my wife who is a PHP developer with rudimentary java knowledge picked up this framework in a couple of hours (or she is exceptionally smart and yes she reads this blog)

Java based web frameworks

2011-09-23T11:38:00.000-07:00

A quick rant on the state of web based frameworks in the J2EE world
a. Imposing Maven on me is not nice.
b. If your "web" based framework cannot support clean URL's you are in the wrong business when you are writing frameworks
c. A J2EE standard doesn't mean its a good standard (JSF - Im looking at you)
d. When you want a view technology please please use JSP. Its mature and it works. It is as good or as bad as you want it to be. Making your own template language which is either inflexible or just as bad as JSP is just plain silly. Reinventing JSTL with a different archaic syntax is silly. Doing it in the guise of "no java in JSP" is silly - There isn't a framework that I can't write bad code in.
e. Make simple things simpler and make hard things easier if possible and it's ok if hard things remain hard. But do not make simple things simpler(oh look how quickly I can do CRUD) and hard things harder(because lets face it , every application does have corner cases). Do not make hard thing's simpler and simple things harder (I have to set up how many files for Hello World?) - because lets face it not everything in a web application is Rocket Science.
f. Who would have thought Spring would turn out to be more heavyweight than EJB.
g. Who would have thought that Struts 1 would still be so heavily used
h. Who would have thought that I would now prefer to code in ASP.net than identify which Web Framework I should use.

Random Value Selection using JMeter XPath Post Processor

2011-08-11T21:33:00.000-07:00

JMeter has two post processor extractors which are usually used when we need some data extracted from the response. The XPath extractor is useful when the Regex would match other elements in a page (e.g. a link inside an li element). However the XPath extractor is missing a functionality that the Regex Post Processor has. It only returns all the values (equivalent to specifying -1 in a Regex Post Processor). XPath also natively supports returning the "nth" value using the position() function. However it doesn't support returning a value at random.
However using normal JMeter functions we can easily implement this functionality.
Assume that you have an XPath post processor that is attached to a HTTP Sampler and is returning n values into a variable called testXPATH. if we now want to select a value at random what we need to use is

${__V(testXPATH_${__Random(1,${testXPATH_matchNr})})}

But thats a mouthfull and needs an explanation

${testXPATH_matchNr}

returns the count of how many elements matched. We are assuming that some values are going to be selected, otherwise you need an IF controller to check the condition.

${__Random(1,${testXPATH_matchNr})}

Will then select an integer between 1 and the count of the values (TODO check if we need to add 1)

${__V(testXPATH_${__Random(1,${testXPATH_matchNr})})}}

Finally we form a string of the form testXPATH_n e.g. testXPATH_3 which is then passed to the __V function which will get us the value for the variable
http://jakarta.apache.org/jmeter/usermanual/functions.html

Note that running TIDY on HTML , followed by parsing it to create a DOM structure and finally running an XPATH query against it is signifcantly slower than just running a Regex against a String. If you need to generate a large load and you have limited resources available to you, then the XPATH extractor is probably not the best alternative.

Looping through all output of a RegExp extractor (without the for each controller)

2011-07-11T14:33:00.000-07:00

I forget why the user couldnt use a For Each controller , but heres how to loop through the result of a Regex Post Processor that returns multiple values

ThreadGroup
+HttpSampler
++Regex Extractor (variableName = links)
+WhileController(${__javaScript(${C} < ${links_matchNr})})
++HTTPSampler use ${__V(links_${C})} to access the current result
++Counter (start=1,increment=1,maximum=${links_matchNr},referenceName=C)

Dynamic values within files

2011-02-10T09:56:00.000-08:00

I have a HTTP Post request sample to which we are passing an XML file through option "Send Files With Request". The XML file to be sent with request contain on Tag "Order id" which need to be modified [ dynamically during test plan executing] before sending the request. Original

Typically in JMeter we would use a ${variableName} notation. However JMeter only replaces these values when the the ${variableName} is within the script itself , not when it is referenced in some other file. As the user wants to use a separate (template) file which is added to the HTTP request, this technique wont work.
One option to consider is whether we could generate all the files we need before we run the test. this is possible for example when the dynamic data is a set of id's that is accessible say from a database. In this case the better (in the more efficient sense) is to have a initialisation step (perhaps as part of the build or as a separate step in Jmeter) to create all the files that the test would need. Once this is done it is easy to make the JMeter script pick one file (usually using the CSV DataSet Config).
However this isnt always possible. In the question above , the Order ID could be dynamically generated by the system under test in ways that might not be predictable. In this case we cannot generate the files in advance. We have to extend JMeter.
The cleaner more elegant way to do this would be to write a custom pre-processor of some sort because it sounds as if this functionality could be reused in some other context (or extend the HTTPSampler). But we will choose the quick and dirty way of writing a BeanShell preprocessor that does this for us.

Briefly the solution will
a. Have a template XML file. We will use @variableName@ within the file for what we want replaced. We will currently only support replacing a single variable (but it is easy to support multiple variables).
b. We will configure the HTTPSampler to use a generated file. At runtime a pre - processor will read the template file and generate this new file. This new file will be read and posted by JMeter

The Structure of the test is shown below

The HTTP Sampler adds a currently non existent file to be posted. This is the file the Pre Processor will create. The name of the file is varied based on the ThreadNumber (we could have added timestamps etc as well) so that each thread will get its own file name

We use user defined variables to define the location of the temp file, the regex that we want to run on the file and the replacement value.

The BeanShell pre-processor

import java.io.BufferedReader;
import org.apache.jmeter.protocol.http.util.HTTPFileArg;
import java.io.File;
import java.io.FileInputStream;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.charset.Charset;
import java.io.BufferedWriter;
import java.io.FileOutputStream;
import java.nio.channels.Channels;

//read the file as string
FileInputStream stream = new FileInputStream(new File(vars.get("templatePath")));
String fileAsString = "";
try {
      FileChannel fc = stream.getChannel();
      MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());     
      fileAsString = Charset.defaultCharset().decode(bb).toString();
}
finally {
        stream.close();
}
//perform the replacement
fileAsString = fileAsString.replaceAll(vars.get("replaceRegex"), vars.get(vars.get("variableName")));

//assumes first file
HTTPFileArg arg = sampler.getHTTPFiles()[0];
//write it out to the file we want
FileChannel fout = new FileOutputStream(arg.getPath()).getChannel();
BufferedWriter bf = new BufferedWriter(Channels.newWriter(fout, Charset.defaultCharset().name()));
bf.write(fileAsString);
bf.close();

Disclaimer : File Reading writing snippets swiped from Google, you can probably make it more efficient.

Briefly the snippet reads the template file(location defined as a variable) as a String (using the default charset , if you deal with multiple languages you might want to force this to UTF-8).
It then runs the replacement using a regex (also a variable) and the subsitution value (also a variable which for this example we have just spoofed)
Finally it writes the file(assuming default charset , again you could force this to be UTF-8). To determine the location of the file , it reads the location from the HTTPSampler. This is currently set as the first file , but we could have used a name or any other technique.

And finally the sampler will read the file that we have created and post it as usual.

Sample JMX available here

java memory leaks

2011-01-26T10:38:00.001-08:00

One of my colleagues asked how I diagnosed a memory leak that was impacting his application , so this is what i said. Maybe Ill write it tutorial form some time

The version of java we are running doesnt make this easy so this is what I did on dev (its hard to make this a coherent narrative).

Almost any memory leak debugging takes the following form(Im assuming the Out of Memory says the HEAP is out of memory)
a. Run your application steps that are causing the memory leak (detecting this is actually the biggest problem but if you dont know what step is causing the leak , you basically have to access all the functionality of your application). This ensures that all statics/caches/classes are loaded. Then force a garbage collection. Then check the memory for the Java VM and take a heap dump (this is a binary form of all the objects on the heap including the allocation graph)
b. Now repeat the step causing the problem. make sure that the step ends correctly (for e.g. if the step is a user goes and saves the quote, ensure that you logout and invalidate his session otherwise you get false readings). Now run a full GC again. Take another heap dump.
c. Compare the two heap dumps. if there is a difference that means you have a leak - ideally there shouldnt be a difference. In practice on a server like tomcat there are a lot of things happening in the background even if no user is accessing the site so even after a full gc , you might still see some tomcat classes or objects that appear to be new. The way around this is that you have to loop the step in b so that you will get to see significant differences for your classes rather than the app server classes.

For our env however taking a heap dump seems problematic because of the version of JDK seems really old(1.5 _07) and I wasnt able to load the taken heap dump in any tool.

Heap Dump
To take a heap dump you have the following options
a. Some JVM's support -XX:+HeapDumpOnCtrlBreak (kill -3 pid on linux). However our version doesnt
b. Some profilers(e.g. Jprobe) allow you to take heap dump - You have to pay for this but there are some free ones like NetBeans. however all of them need you to run some install steps so I didnt use this
c. use jmap . This is a utility that Java provides. Later versions let you say JAVA_BIN/jmap -dump processid. For our version we have to say jmap -heap:format=b processid . however this isnt working correctly since i couldnt load this dump into any tool. what I finally used was jmap -histo processid. This is a poor mans heap dump . it shows the classes and the number and the size they occupy, but it doesn't show who allocated the classes (so for e.g. I knew it was the concurrency related classes that were leaking , but I didnt know who was creating them - that was thanks to Google)

d. +XX:HeapDumpOnOutOfMemory - This will automatically take a heap dump when you run out of memory but debugging with this is slightly harder because you have live objects
e. use HPROF (this is an earlier version of the tool) when you start up java but it will slow the process.

Monitoring
To look at how much memory is being used, and to force GC you have the following options
a. Look at the probe we installed (specific to tomcat) - lambda/psi probe . Click System information . under overview you can see the memory being used and you can Force GC as well. You can also click Memory Utilization and it will show you the breakup. I had to add some JMX parameters to startup. This is tomcat specific.
It also lets you see sessions and their footprints (sometimes the leak isnt actually a leak. Code puts too much data in session , the user doesnt log out , the session remains active for say 30 minutes , then that much memory is used up for 30 minutes. technically this isnt aleak , when the session times out , the objects will get collected , but it looks like a leak because the memory keeps getting used , but doesnt get reclaimed)
b. use JConsole . This is in the bin directory of your java. When you start it on your machine you can add a remote connection
These are the settings I added on dev
export JAVA_OPTS="-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/local/apache-tomcat-5.5.30/logs -XX:PermSize=256m -XX:MaxPermSize=256m -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=12345 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dfile.encoding=UTF8"
This will also let you see some things that the probe does.
Using this I could find out
a. That it wasnt the sessions causing the problem (since the probe showed that only 1 MB was being used in sessions).
b. I also used it to check and see whether threads were causing issues , not the case.

Testing
I used JMeter to simulate a web user going and approving a quote and simply looped that 4000 times. I couldnt not use say SOAPUI because we needed to simulate a stub being created whereas SOAP acts as the client so it wouldnt have shown the problem. Ordinarily you would not know the Stub causes the problem , but the histograph had shown such a large number of the concurrent objects on the heap , that a google search showed that other people had faced this problem with axis 1.4.1. If I didnt have this I would have had to write a script that would have placed a quote, gone and approved it , saved it , modified it etc.

The histographs are in /root/histbeftest and /root/hist92per and are easily compared to verify.

Analysis
The histograph files are plain text and can be compared in a text editor. If I had a heap dump we could use
a. JHat - this comes in the bin directory of java (1.6 onwards and some update of 1.5 onwards but it can read 1.5 dumps). This will let you browse the heap and even compare them
b. Profilers like JProbe
c. Other tools like VisualVM, HPJmeter NetBeans, extensions in eclipse all let you load heap dumps.
One of the issues you might run into is that the Heap Dump file is quite large and your client tool may not have enough memory. To verify a memory leak you usually reduce the amount of memory you give to the JVM so that you get smaller heap dump files (so for e.g. the iquote app once it is running and all the classes are loaded takes about 20 to 30% of 512MB which is about 150 MB. So I could reduce the memory to 256MB instead of 512 and run the tests to create a much smaller heap dump)

a memory leak in sample code and then trying to detect it using tools is usually the best way to learn.

JMeter and AJAX - Part III

2010-11-28T04:01:00.000-08:00

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part IV - A more complex sample

In Part II we developed a sampler that would let us synchronize selected threads. I split that out because someday I might be tempted to actually write that timer. So lets now try and execute some AJAX requests in parallel. The demo site we will use is http://www.ajaxdaddy.com/demo-dhtml-autocomplete.html (Please don't run extensive load tests against this site - it isn't mine)
The site has an AJAX based autocomplete for countries. Access the site in FireFox, turn on Firebug and type au into the textbox and you can see the AJAX calls being made in firebug

We can see that two AJAX calls are made, one when we typed 'a' and one when we typed 'u' (There is an AJAX design pattern that applies here which doesn't seem to have been followed). So we will access this page and try to simulate two parallel calls.

The Init and Delay Sampler are from Part II. The Init sampler ensures only 1 thread does the initialization bits. The delay sampler has the rest of the code. After the Init sampler we just make all the threads wait so that no threads run before we are ready to begin (so don't use any ramp up time as that would be nullified here). Its essential the sync timer has a count equal to the count of threads in the threadgroup.
The IF controller is used so that only the main threads perform the request to the main page. Recall for 6 threads to run 2 at a time , we have the groups as Thread(1,2) , Threads(3,4), Threads(5,6). The even numbered threads will only be used for the AJAX requests. The Odd numbered threads will be used for the main sample and the AJAX requests. In our case the landing page needs to be accessed by the odd numbered threads. The even numbered threads will skip and land at the delay sampler waiting for their respective soulmates to join them(I'm currently waiting to be reunited with my wife hence the flowery language).
Anyway we have added a Gaussian random timer to allow the odd numbered threads to arrive at different times. When the odd numbered thread does arrive at the delay sampler, it along with its even numbered pair will be released to execute the Make AJAX request sampler (which chooses a word at random from the CSV file we have defined text,assert e.g. a,australia).

Hence two requests will be executed in parallel for the AJAX request sampler. If we had further requests after this we would have added another IF controller(only odd numbered threads) and coded the samples under this. if we wanted 3 requests in parallel we would have to adjust the Init sampler to say we want groups of 3.
Run the test. Though the samples may appear interleaved , note down the sample start times of the groups of threads that were supposed to execute in parallel, they will be more or less the same.

But I have cheated. The samples in parallel didn't really depend on each others data. The AJAX requests were exactly the same , differing in data. The AJAX requests didn't depend on the user's session thereby reducing the complexity of the script.

Lets attempt an example which needs the above. Onwards to Part IV (as soon as I find a good demo!)

JMeter and AJAX - Part II

2010-11-27T05:08:00.000-08:00

import java.util.HashMap;
int totalThreads = 6; // MUST match total number defined in Thread Group
int groupsOf = 2; //MUST match number of requests to make in parallel

if (bsh.shared.myObj == void){
    // not yet defined, so create it:
    bsh.shared.myObj =new HashMap();
    bsh.shared.timerCounter =new HashMap();
    int numObjectsNeeded = totalThreads/groupsOf;
    for(int i = 0;i<numObjectsNeeded;i++) {
           bsh.shared.myObj.put(String.valueOf(i),new Object());
           bsh.shared.timerCounter.put(String.valueOf(i),new int[] { 0 });
    }
    print("populated");
} else {
   print("already init");
}

print(${__threadNum()});

int grpNo = (int)(${__threadNum()} -1)/groupsOf;
print(grpNo);
String groupNumber = String.valueOf(grpNo);
print(groupNumber);
Object syncObject = bsh.shared.myObj.get(groupNumber);
print(syncObject);
synchronized (syncObject) {
   int[] timerCount = (int[])bsh.shared.timerCounter.get(groupNumber);
   timerCount[0]++;
   final int count = timerCount[0];
   if (count >= groupsOf) {
                syncObject.notifyAll();
            } else {
                try {
                    syncObject.wait();
                } catch (InterruptedException e) {
                    log.warn(e.getLocalizedMessage());
                }
            }
            timerCount[0]=0; // Reset for next time   
}
SampleResult.setResponseData("sunk!");

The bsh.shared namespace is used to share objects between threads. So in our case we have 6 threads and we want 2 threads in parallel which means there are 3 groups of two threads each. To synchronize each group we need an object (hence 3 objects) and a counter each - to count how many threads are already available. Thats what the first part of the code sets up.(The first part is not thread safe but because we are going to test this by guaranteeing that threads arrive at different times , the snippet need not be thread safe).

The second part of the code then calculates which Group a thread belongs to, gets the object for the group, increases its count. If the number of threads is less than that we want in parallel it suspends the thread (by calling wait) otherwise it does a notifyAll which will wake up all suspended threads and allow these two threads to move in parallel

Heres what the test looks like

We have artificially added a timer to make sure requests to the Delay sampler arrive 1 thread at a time. We run the test and we can see that Threads(1,2) run the After Delay sampler at almost the same time , as do Threads(3,4) and as do Threads(5,6)
If we change the groupsOf in the delay sampler to be 3 then we see that three threads run in parallel.

And with that we are ready to use this delay sampler to make AJAX calls in parallel.
Onwards to Part III - An actual example of JMeter and AJAX

JMeter and AJAX - Part I

2010-11-27T04:59:00.000-08:00

So the first question that should come up is do we actually need to change anything in the way we test? The vast majority of AJAX enabled sites take the form a page loads , the user does some action , an AJAX call is made and values are shown to the user without updating the rest of the page. But look at the sequence of HTTP requests. Request for the main page, get response, browser processes response,user action, AJAX call made , AJAX response, further browser processing. There isn't any request made in parallel (ignoring all the static file requests which is its own blog post) - nor is there any difference due to Asynchronous nature of the call. So a JMeter script to simulate the above would have no difference from a normal JMeter script. the AJAX call should just follow the call to the main page and we are done.
But hold on some AJAX enabled sites may make an AJAX call as soon as the page loads without any user action. Or the call might be made periodically.Even in these cases the onload/onready is only done after the response of the main request is made available so while the browser may make the request asynchronously in terms of the HTTP request/response there is no difference. The periodic call may only have a problem if the time it takes to return is greater than the periodic repetition. But the latter case would be a problem anyway , all you really want from JMeter is to detect if this happens. Once it happens your system is messed up and you need to fix it! There really isn't much value simulating the messed up state. To put it in another way , if your system breaks for 50 users, there isn't much value observing how much more is broken for 100 users.
But hold on what if multiple AJAX requests are made on load? Say a users home page has two AJAX widgets , one which shows say his tasks , one which shows his emails? Well yes two requests will be made in parallel. Usually these requests are independent of one another. The question we really need answered is "Is there a difference between a user making two independent requests in parallel and two users(maybe with the same login) making a single independent request each in parallel?" In most cases , the answer is no. The load may be slightly higher in the latter case because you may have two sessions instead of one, but in terms of the rate of requests the server sees you can normally match the rate of requests by simply increasing the number of threads (and a little bit of math).
But hold on what if the requests aren't independent? What if they utilize the same table/rows causing contention when its from the same user session what if widget2 depends on widget 1 for something in the session? Well one thing to do in this case is call the developer and ask him why the hell are there two dependent requests instead of a single one? That usually doesn't end well(even though the tester is right) - but yes at this time yes you might need to actually simulate requests in parallel. Note however a lot of AJAX enabled websites dont fall into this bracket and those websites can be simulated in JMeter with a reasonable degree of accuracy.

Onwards to Part - 2. Simulating AJAX requests without knowing how to modify JMeter core.

Google isnt as good as you think it is

2010-11-16T16:18:00.003-08:00

Exhibit A - A snippet from webmaster tools for this blog

Though I know that writing about deepika instead of JMeter would give me more page hits!

WLST to dump free heap memory

2010-10-21T17:27:00.000-07:00

from java.io import FileWriter
from java.io import BufferedWriter
from java.util import Date
connect(eval('username'),eval('password'),eval('url'))
domainRuntime()
cd('/ServerRuntimes/' + eval('managedServerName') + '/JVMRuntime/' + eval('managedServerName'))
heapFreeCurrentPerOld = str(cmo.getHeapFreePercent())
heapFreeCurrentValOld = str(cmo.getHeapFreeCurrent())
cmo.runGC()
java.lang.Thread.sleep(int(eval('sleepTime')))

heapFreeCurrentPerNew = str(cmo.getHeapFreePercent())
heapFreeCurrentValNew = str(cmo.getHeapFreeCurrent())
newDate = Date()
fstream = FileWriter(eval('outputFile'),true)
out =  BufferedWriter(fstream)
out.write(eval('managedServerName') + "," + str(newDate.getTime()) + "," + heapFreeCurrentPerOld + "," + heapFreeCurrentValOld + "," + heapFreeCurrentPerNew + "," + heapFreeCurrentValNew + "\n");
out.close();
disconnect()
exit()

There is also a property file which contains the configuration information

managedServerName=<MANAGED>
username=<USER>
password=<PASSWORD>
url=<URL>
sleepTime=<PAUSE>
outputFile=<PathToFile.csv>

And finally to run

$WL_SERVER_HOME/common/bin>wlst -loadProperties

This was written to output the free memory before and after GC for a particular managed server (which used to run jobs) on weblogic. It was cron'ed to run at fixed intervals. The generated file is used to check if there is a memory leak. The heap free current value should remain fairly constant. if it keeps reducing , we have a leak

Getting the current thread group name

2010-10-06T20:56:00.001-07:00

${__BeanShell(ctx.getThreadGroup().getName())}

Academic and Happy

2010-09-11T20:22:00.000-07:00

via http://urlai.com/url/theworkaholic.blogspot.com

Text analysis
theworkaholic.blogspot.com is probably written by a male somewhere between 36-50 years old. The writing style is academic and happy most of the time.

Jmeter with Spring Webflow

2010-08-03T23:44:00.000-07:00

Motivation
The application uses URL containing dynamic parameters and so a record/replay fails
http://www.mail-archive.com/jmeter-user@jakarta.apache.org/msg30317.html

Solution
This short tutorial deals with test scripts that interact with dynamic data, and we will choose a Spring Webflow example to write a test script against.
So we look around on Google for a sample Spring Webflow application which leads us to
Spring By Example Sample
I'm not going to use the JMeter Proxy Recorder because it seems to harm beginners more than it helps. Testers seem to get the impression that all they need to do is record and replay the script and they are done. So I shall use Firefox and LiveHttpHeaders. We are going to Login to the site and create a User

Step 1: Request the Login Page


http://www.springbyexample.org/simple-webflow/login.jsp
GET /simple-webflow/login.jsp HTTP/1.1

Response 
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=[uniquevalue1]; Path=/simple-webflow

So the first request response shows that the system sets a cookie and the name should tell us that this is a session id. [TODO]. This should also indicate to us that we need to add a cookie manager. Note the status is 200 in the Response. This is the server telling us all is well.

Step 2: Submit Login


http://www.springbyexample.org/simple-webflow/loginProcess;jsessionid=[uniquevalue1]

POST /simple-webflow/loginProcess;jsessionid=[uniquevalue1]
HTTP/1.1
Cookie: JSESSIONID=[uniquevalue1]
Content-Type: application/x-www-form-urlencoded
Content-Length: 48
j_username=david&j_password=newyork&submit=Login

Response
HTTP/1.1 302 Moved Temporarily
Date: Tue, 17 Aug 2010 05:03:48 GMT
Set-Cookie: JSESSIONID=[uniquevalue2]; Path=/simple-webflow
Location: http://www.springbyexample.org/simple-webflow/index.jsp
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/index.jsp
GET /simple-webflow/index.jsp HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 302 Moved Temporarily
Location: http://www.springbyexample.org/simple-webflow/index.html
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/index.html
GET /simple-webflow/index.html HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]
HTTP/1.1 200 OK

a. The browser sends back the cookie that was previously set. However note that the URL also has a jsessionid which has the same value as the cookie that was set. This is known as URL rewriting. Our test will ignore this value because we will be using a Cookie manager
b. The server sends a 302 response to the request. This basically asks the browser to request a new URL (the one that the Location Response Header has). Note this can happen more than once as the next request too results in a 302 response, which finally results in a 200 status code.
HTTP Status Codes are discussed here
c. The server also sends a Set-Cookie for JSessionID again with a new value. This is usually done for security reasons. We can ignore this for JMeter because the Cookie manager will handle this for us. However the sampler that we use needs to have follow redirects set. If redirect automatically is set , JMeter would not get to see this Set-Cookie header and since the application has switched the session id , if you sent it the old session id (which was associated to the before login session)

Click the Create Link


http://www.springbyexample.org/simple-webflow/person.html
GET /simple-webflow/person.html HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

Response
HTTP/1.1 302 Moved Temporarily
Location: http://www.springbyexample.org/simple-webflow/person.html?execution=e1s1

----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/person.html?execution=e1s1
GET /simple-webflow/person.html?execution=e2s1 HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 200 OK
Date: Tue, 17 Aug 2010 05:06:04 GMT

Clicking the Create link also resulted in a 302 code so the browser issued a new request. But take a look at the URL. It has a parameter named execution with a value e1s1. Now click the link again.

Click Create link again


http://www.springbyexample.org/simple-webflow/person.html
GET /simple-webflow/person.html HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

Response
HTTP/1.1 302 Moved Temporarily
Location: http://www.springbyexample.org/simple-webflow/person.html?execution=e2s1
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/person.html?execution=e2s1

GET /simple-webflow/person.html?execution=e2s1 HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 200 OK

The "execution" parameter value changes! So if you run the same test multiple times , the value has to be different. This is a common cause of failure of Record / Replay tests. Some of the data is dynamic. This data can be of various forms
a. Something the application framework needs
b. Something where the data varies (E.g. the UserId, the Product ID). Typically a user friendly string is shown on the screen but when the user performs an action , an id (usually a key in the database) is what is used by the application
c. Dynamic fields (E.g. DHTML based applications) which get added on the fly.
To deal with this parameter/URL we need to be able to extract the value from previous requests in our JMeter scripts.
If you haven't developed the application it may be difficult to identify whats dynamic and whats not. The easy way is to record the script twice(With different data) and observe what changes. e.g. if you browse a product catalog , change the product.

Submit the form


http://www.springbyexample.org/simple-webflow/person.html?execution=e2s1

POST /simple-webflow/person.html?execution=e2s1 
HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]
Content-Length: 56
id=&firstName=deepak&lastName=shetty&_eventId_save=Save

Response
HTTP/1.1 302 Moved Temporarily
Location: http://www.springbyexample.org/simple-webflow/person/search.html
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/person/search.html
GET /simple-webflow/person/search.html HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 200 OK

Unfortunately the application has a visual bug, the added user does not show. However if you do a view source on the page and search for the username you created you can find it.

Click Logout


http://www.springbyexample.org/simple-webflow/logout
GET /simple-webflow/logout HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

Response
HTTP/1.1 302 Moved Temporarily
Set-Cookie: SPRING_SECURITY_REMEMBER_ME_COOKIE=""; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/simple-webflow
Location: http://www.springbyexample.org/simple-webflow/logoutSuccess.jsp
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/logoutSuccess.jsp
GET /simple-webflow/logoutSuccess.jsp HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=[uniquevalue3]; Path=/simple-webflow

The JMeter Script
Now that we know the above we can get to writing the script

Most of the script is pretty much standard(All Samplers use follow redirects and we have added a Cookie Manager), till we get to the point where we need to extract out the URL.
The Request Create Sampler extracts out the URL. Since we know that the application is going to redirect us , the extracter will extract out the last URL we will be on

The Post Create Data Sampler simply uses this extracted URL

Note that all samplers have assertions that verify the behavior, so if something goes wrong the Assertion should fail. For e.g. after creating the user we verify that the user we created exists on the page. When requesting a page the assertions verify some text on the page that is unique to the page.
Run the test, we get no errors! Modify the ThreadGroup to have two threads and two iterations. Run the test , everything should pass. Now look at the View Results Tree listener. You should see the redirects being followed (multiple children results under a parent). But do you see different values for the execution parameter? No - This is because each thread is its own session and each user will start with the same parameter value. Even having multiple iterations on the thread group doesnt give you different values because we are logging out at the end of the script so the next iteration is a new session.

Try adding a Loop controller to create 10 users. Note that the loop controller must enclose the Request Create and Post Create samplers (because we need to extract a new value for the execution parameter every time!). Using View results tree you can now observe the execution parameter changing.

Sample Script
Spring Webflow.jmx

Spring Webflow with loop.jmx

First weblogic portal expert

2010-06-17T21:52:00.000-07:00

YAY!

Thankfully a lot of posters don't use Google, otherwise I couldn't answer the same questions again!

Updated JEE Interview Questions - Draft

2010-05-27T14:07:00.000-07:00

JSP
What are the various way's to handle exceptions in a web application. What would you use when. Demonstrate with examples from any project you have worked on
What are the various way's you have implemented security in your web application. Also mention alternatives that can be used
Which framework have you used. Please describe the shortcomings in the framework and the ways to work around them
What is the difference between a static include, jsp:include, jsp:forward and redirect. Please give one example of each.
Describe some Tag Libraries you have written. What are advantages / disadvantages of tag library. If possible give an example of a badly written open source JSP tag library(or any of your own) and how you would improve it.
Give one example of when you would use a filter. How do you execute a filter after the request?
Give one example of when you would use a listener. Which listeners have you used in your project?
Besides defining servlets , name one feature that can be specified in the web deployment descriptor
Name JSP implicit object. Give one example of the use of each implicit object. Is exception an implicit object? If so can i write exception.printStackTrace in any JSP? what will it print?
Comment whether a JSP web application works with cookies disabled.
Which web container did you use. Please describe how to use a container specific feature.
Describe some JSTL tags you have used and what they are useful for
How do you use a datasource in a JSP?
How do you internationalize a web application. What all do you need to take care of

EJB
How do you implement security in an EJB . Why does the framework provided security not work in most real world scenarios? (or does it?)
If an MDB throws an Exception what happens to the message?
If an EJB has two methods M1 and M2 both with RequiresNew and M1 makes a call to M2 by calling it (because M2 is a method in the same class) how many transactions are there?
What limitations are

JDBC
Lets say you want to execute a query and show some data. Walk through the steps to do this in Raw JDBC.
What do you prefer using of the following and why? CMP/JDO/ORMs (like hibernate/toplink)/SQL helpers (like Spring/iBatis)/Raw SQL?
In the context of ORM what is lazy loading/eager load. What do you use in your project? Why? What factors determine what is to be used.
Your ORM based system is slow. How would you diagnose the problem?
Why is pessimistic locking not useful in a web-application (or is it?)

JMS
Say you have a queue. The Producer can produce messages much faster than the consumer can consume. What are your options?

JMX
Give any example where you have used JMX

WEB
Given a large data set , what options do you have for displaying this dataset. Which one do you prefer and why?
Suppose you are tasked to implement an AJAX based system where the user types in a word in a search box and you want to show him possible queries (e.,g. in Google if you type dee you get deepak chopra or deepika padukone)in a drop down , what considerations would you have in implementing the AJAX parts of the client (Assume that you have a server side component that can give you the results)
If you want Google to search your site easily , what do you need to take care of in your web app

Misc
You have a clustered J2EE web application. You need to batch process a file which has a billion records. Each record is independent . How do you do it?
You have a table which has UserID (primary key) and UserName. The requirement is to display this in a drop down box on a page so that the user will select the name but the value posted to the next page will be the userid. The data displayed must be sorted by username. The DB developer asks you what data structure he needs to return the results in. What is your answer? Assuming that you get this data structure , is there any comment you wish to make on your implementation?
Name a few design patterns that you see being used in the Java API with examples.

Simplified procedure for Syncing Embedded LDAP with Weblogic Portal Database

2010-04-07T11:15:00.001-07:00

a. Shut down servers
b. On the admin server, backup the LDAP directory \servers\AdminServer\data\ to another location
c. Delete the internal LDAP directory at \servers\AdminServer\data\ldap (also from managed
servers if applicable)
d. Delete data from the p13n entitlement tables (delete from tablename)
( namely p13n_entitlement_role, p13n_entitlement_resource,p13n_entitlement_policy, p13n_entitlement_application and p13n_delegated_hierarchy )
e. Start the servers
f. Redeploy the application
g. Recreate the SAML relying party configuration
h. Recreate Entitlements

Dynamic Parameters in JMeter

2010-03-19T18:32:00.000-07:00

Motivation

You need to send a variable number of parameters with an HTTP request i.e. the number of parameters are not known when the script is written. For example a User can register with a number of accounts , but this number varies per user. Or perhaps you want to Post process extract a number of hidden fields on the previous response and post all of them in the next Request, but you either don't know the number or there are too many fields to enter manually into the HTTPSampler.

Solution

Use a BeanShell PreProcessor to dynamically add variables to the sampler. The relevant code is sampler.addArgument(name,value)[1]

Sample

We will write a script to access a page , extract out data based on a pattern and send these as separate parameters to the next request. The script is as shown below.

We make a request to some dummy page and then we have a regex extractor to extract out multiple values. In this case the Regex extracts out all words(Match No = -1) beginning with test

Then we have a Debug Sampler to verify the regex is working, notice the results in View Results Tree Listener. The Regex has a reference variable name of inputTerms. The response of the Debug sampler is shown below

You can see from the Listeners response tab that the total number of results is 14 , available under the key inputTerms_matchNr and that each result is available as inputTerms_$resultnumber.

The matching groups which we usually do not need are available as inputTerms_$resultnumber_g$groupnumber. Armed with this information, the beanshell preprocessor script is trivial as shown below

int count = Integer.parseInt(vars.get("inputTerms_matchNr"));
for(int i=1;i<=count;i++) { //regex counts are 1 based
sampler.addArgument("hardcodedkey", vars.get("inputTerms_" + i));
}

Heres the next request to which the beanshell pre processor is attached. We have one predefined variable and the rest will be dynamically added by beanshell.

Next we check the request data we are posting using the Request tab of the View Results Tree Listener

Notice that the argument we added to the next request was sent, as were all the parameters we added dynamically. The parameter values were also encoded.

Reference
[1] Javadocs for HTTP Sample Base

Asserting MS Office formats

2010-03-15T16:14:00.000-07:00

As a follow up to the PDF post, you can do something similar using Apache POI

import org.apache.poi.POITextExtractor;
import org.apache.poi.extractor.ExtractorFactory;
import java.io.ByteArrayInputStream;

ByteArrayInputStream bais = new ByteArrayInputStream(in);
POITextExtractor extractor = ExtractorFactory.createExtractor(bais);
prev.setResponseData(extractor.getText());

You should be able to parse word documents and excel and ppt's using this beanshell post processor. Remember to copy the POI libraries to JMeter/lib

Future work
Look into Apache Tika for a unified interface (may need to sacrifice some functionality like the startPage endPage of PDFBox)

Asserting PDF's

2010-03-15T13:15:00.000-07:00

A question on the JMeter mailing list regarding extracting/asserting text inside a PDF file and since I ran into this for a functional scenario , I wrote up my attempt.
I use PDFBox as the library. Download the binaries and copy pdfbox-1.0.0.jar and external/fontbox-1.0.0.jar into JMeter's lib directory.
We'll access the PDF at http://jakarta.apache.org/jmeter/usermanual/jmeter_distributed_testing_step_by_step.pdf and check that the PDF does contain "Distributed Testing Step-by-step"

Here's the JMeter sample script

So we have a transaction controller 'Check PDF' so that we only get a result item. The HTTP Request Sampler 'Request PDF' requests the PDF. The bulk of the code is in the beanshell post processor titled 'Extract Text'

import java.io.ByteArrayInputStream;
import java.io.StringWriter;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.util.PDFTextStripper;


PDDocument document = null;
StringWriter sw = new StringWriter();
try {
ByteArrayInputStream bais = new ByteArrayInputStream(data);
document = PDDocument.load(bais);
PDFTextStripper stripper = new PDFTextStripper("UTF-8");
stripper.setSortByPosition( false );
stripper.setShouldSeparateByBeads( true );
stripper.setStartPage( 1 );
stripper.setEndPage(Integer.MAX_VALUE );
stripper.writeText( document, sw );
} catch (Throwable t) {
t.printStackTrace();
sw.append("ERROR");
} finally {
sw.close();
document.close();
}
vars.put("extractedText", sw.toString());

All this does is use the PDFBox API to extract text from the PDF (bytes are present in the data object) and write it to a variable in JMeter , extractedText.

The next sampler is a Java Sampler which sets the ResultData to ${extracted text}. This will echo back the contents of this variable as the response of the sampler.

Once this is done , you can use a normal response assertion, regex extractor or whatever else you need to process the text

Update: as mentioned by Milamber in the comments , you can instead have the last line of the beanshell as

prev.setResponseData(sw.toString());

This will set the response of the HTTPSampler to be the text value and allow you to directly specify the assertion (and you can also eliminate the transaction controller).

Update : if you are interested in MS office formats then follow the steps in this post, just change the BeanShell post processor to that mentioned in Asserting MS Office Formats
Future work
a. make a custom PDF(or any format sampler) with the options that are hardcoded (e.g. startPage or endPage)
b. experiment with ways to use less memory. Typically should extract and write to file and use that.

Random stuff on jmeter testing

2010-02-07T19:43:00.000-08:00

Update : I realized that much of what I wanted to say has already been said and in a far better manner in Perfomance testing patterns and practices.
One of the questions asked on the JMeter mailing list was

How do I analyse my application with JMeter?

There are various reasons one might run a load test broadly detection, simulation, verification and analysis. The type of test or the scripts you might write, what you would measure or track would vary a bit depending on what you want to do. JMeter is good for simulation and verification. It is an aid to detection and analysis but you usually need other tools to help you out.

Detection

You want to find out if your application behaves correctly when accessed by multiple threads. Whether your database starts showing deadlocks or whether you might have race conditions.
You want to find out if your application has memory leaks.

Simulation

You want to find out how your application behaves under standard load or how it behaves under some peak load(e.g. shopping during the holidays). Response times etc.

In the specific case of Java based web applications you want to know how often your GC cycles might run or how long.

You application makes remote calls(e.g. webservice calls) and you want to know whether all the resources are recovered correctly. perhaps you have some throttling mechanisms in place and you want to see that its working correctly

Verification

You want to verify the result of changing some tuning parameters.
You want to check your SLA's are met

Analysis

You might already know there is a problem and you want to simulate load while you are profiling the application.

These areas overlap and I've used the above categories quite broadly to merely illustrate that your objective and hence your test script will vary based on your objective.

For e.g.
You want to find out if your application behaves correctly when accessed by multiple threads - In this case your test script would only be concerned with running some parts of the applications at exactly the same time. You'd want to exercise multiple parts of your application. Perhaps you are aware that some part of the application internally spawns threads and you'd run a test that exercise that area for a long time or with a high load. You don't at this point really care whether these are unrealistic scenarios or non representative scenarios, nor are you really looking at what the response times are. All you care about is do you see stuck threads or deadlocks. Do you see a really long wait time for most threads though some threads finish really fast.

Or perhaps you want to tune a memory parameter and you want to verify the change
In this case Response Times / Throughput really matter (for the same test of course). You'd first take a baseline reading without the tuning, and then another with the tuning. The test scripts must be representative of actual user behavior.

Perhaps you want to check whether your site can handle holiday shopping onslaughts. In this case you would modify your tests to show bursts of activity, you'd closely monitor response times but you also want to check what happens on the server. How much memory, How much CPU. You might also want to see what load might actually bring down your servers. You might want to check if your load balancers evenly distribute the load.

Or perhaps you have certain Service Level Agreements and you need to know response times accurately for the load specified in your agreement. In this case you need a representative user journey and you also need representative background users.

All of which means there is no easy answer to ' How do I analyse my application with JMeter'. It can only be answered by What is it that you want to analyse (normally answered as well, performance).

Lets take the most common use case, what is the 'response time' for my application.
However actually getting the response time is more difficult than reading the response time calculation from the JMeter test results.
This is problematic due to
a. JMeter is not a browser and does not render the page. Different browsers take different times to render the same page. Compare older versions of Internet explorer with Chrome for e.g.
b. A returning user with some files cached will probably show lesser times than a first time user.
c. The network / connection speed from which the user is accessing the application may be significant. And your users may be spread out throughout the world.
d AJAX based applications / DHTML applications are difficult to predict because not only does it vary by browser , but the number of calls that a browser may make in parallel is also different, but some calls will be made in parallel and its difficult to know that.

So any response time would have (roughly speaking)
a. The time it takes for the application to actually respond with all the data
b. The time it takes for this data to be transferred over the network
c. The time it takes to download static files (bearing in mind that not all files may be downloaded and that browsers may request multiple static files in parallel)
d. The time it actually takes to render the page.

JMeter can help you out with a, b, and c. but what it is good at, is finding out a. for the network on which it is running on.

Typically your requirements might define an Service Level agreement for your site as Browsing operations must take < 6seconds 90% of the time and shopping operations must take <8 seconds 90% of the time. You also know how much large your pages are and you can guesstimate how much time it would take for the page to be transferred over the internet. You might take an average with some safety factor or you might take a worst case scenario. Using a browser tool like YSlow or Googles PageSpeed , you can also have some insight on how your static are downloaded , how long they take etc. And you might add some time for how long the browser takes to render. After considering all of this you might arrive at a new figure that on a high bandwidth intranet (which thereby eliminates most of the network variables) your browsing operations must take < 2 seconds just to get the data and your shopping operations must take < 4 seconds for your SLA's to hold because the rest of the time has already been used by the other factors.
After this you would have to write a script which generates representative loads (for the operations being verified and the operations that would happen in the background), run the test and verify the 90% percentile lies below the value you have calculated above. But perhaps it doesn't. Static files can be optimised by reducing their number, their size, gzipping them adding expiry headers etc, but maybe you have already done this. The Clients network and browser aren't within your control so there isn't much you can do there. The next step is figuring out where your problem lies. JMeter can't help you there, you need a different set of tools. But JMeter can help you to simulate load or parts of it so that you can monitor your application with the tools of your choice. Some of your findings may be infrastructure related, Some may be code you'd have to make changes and retest and repeat.

JSP Interview questions

2010-01-23T13:23:00.000-08:00

1. What are the various way's to handle exceptions in a web application. What would you use when. Demonstrate with examples from any project you have worked on
2. What are the various way's you have implemented security in your web application. Also mention alternatives that can be used
3. When would you use servlets. name one thing you can do with a servlet that you cannot do with a JSP.
4. Which framework have you used. Please describe the shortcomings in the framework and the ways to work around them
5. What is the difference between a static include, jsp:include, jsp:forward and redirect. Please give one example of each.
6 Describe some Tag Libraries you have written. What are advantages / disadvantages of tag library. If possible give an example of a badly written open source JSP tag library(or any of your own) and how you would improve it.
7. Give one example of when you would use a filter. How do you execute a filter after the request?
8. Give one example of when you would use a listener. Which listeners have you used in your project?
9. Besides defining servlets , name one feature that can be specified in the web deployment descriptor
10. Name JSP implicit object. Give one example of the use of each implicit object. Is exception an implicit object? If so can i write exception.printStackTrace in any JSP? what will it print?
11. Comment whether a JSP web application works with cookies disabled.
12. Which web container did you use. Please describe how to use a container specific feature.
13. Describe some JSTL tags you have used and what they are useful for
14. How do you use a datasource in a JSP?
15. How do you internationalize a web application. What all do you need to take care of

JMeter Graphs and ANT

2010-01-11T12:17:00.000-08:00

A follow up to the previous posts, I've integrated the graph code with ANT so that the HTML reports provided by running JMeter from the command line and styling them can be extended to include the graphs.

The sample report that I generated is shown below
The custom report hyperlinks the titles of the normal summary tables to lead to graphs for the same.
To implement this we need to
a. Allow the graph code to be invoked from ANT. This can be done by writing a simple java class with a main method that passes parameters on the command line, or we could write a custom ant task. I wrote a custom ant task as a proof of concept. We need to customise the build script as well
b.Modify the XSLT to write out image anchors.

These steps are described below.
The custom ant Task

public class AggregateGraphTask extends Task {
private String outputDir;
private String outputFilePrefix;
private Boolean showThreshold = Boolean.TRUE;
private Double threshold = 500D;
private String jmeterResultFile;
private String jmeterHome;

public String getJmeterHome() {
    return jmeterHome;
}

public void setJmeterHome(String jmeterHome) {
    this.jmeterHome = jmeterHome;
}

public String getJmeterResultFile() {
    return jmeterResultFile;
}

public void setJmeterResultFile(String jmeterResultFile) {
    this.jmeterResultFile = jmeterResultFile;
}

public String getOutputDir() {
    return outputDir;
}

public void setOutputDir(String outputDir) {
    this.outputDir = outputDir;
}

public String getOutputFilePrefix() {
    return outputFilePrefix;
}

public void setOutputFilePrefix(String outputFilePrefix) {
    this.outputFilePrefix = outputFilePrefix;
}

public Boolean getShowThreshold() {
    return showThreshold;
}

public void setShowThreshold(Boolean showThreshold) {
    this.showThreshold = showThreshold;
}

public Double getThreshold() {
    return threshold;
}

public void setThreshold(Double threshold) {
    this.threshold = threshold;
}

@Override
public void execute() throws BuildException {
    try {
        GraphClient.init(jmeterHome);
        String outputPrefix = outputDir +  File.separator + outputFilePrefix;
        if(Boolean.TRUE.equals(showThreshold))  {
            GraphClient.writeAggregateChartWithThreshold(jmeterResultFile, outputPrefix, showThreshold, threshold);
        } else {
            GraphClient.writeAggregateChart(jmeterResultFile, outputPrefix)    ;
        }
    } catch(Exception e) {
        throw new BuildException(e);
    }
}

}

This is a pretty straightforward class which calls our API's based on parameters passed to it. It declares fields for all the attributes it expects and then calls the Graph API's.

The ANT build.
I assume you have JMeter working from ANT, this assumes all the libraries needed for ANT and JMeter are in place

<project name="Jmeter" basedir="." default="runOfflineGraph">
<property name="lib.dir" value="${basedir}/lib"/>
<property name="report.dir" value="${basedir}/report"/>
<property name="styles.dir" value="${basedir}/styles"/>
<property name="export.dir" value="${basedir}/export"/>
<property environment="env"/>
<property name="jmeter.home.dir" value="${env.JMETER_HOME}"/>
<property name="jfreechart.home.dir" value="${env.JFREECHART_HOME}"/>

<path id="run.classpath">
<fileset dir="${jmeter.home.dir}" includes="**/*.jar"/>
<fileset dir="${jfreechart.home.dir}" includes="**/*.jar"/>
<fileset dir="${lib.dir}" includes="*.jar"/>
</path>

<target name="clean">
<delete dir="${report.dir}" />
<delete dir="${export.dir}" />
</target>

<target name="init">
<mkdir dir="${report.dir}" />
<mkdir dir="${export.dir}" />
</target>


<target name="runJMeter" depends="init">
<taskdef
    name="jmeter"
classname="org.programmerplanet.ant.taskdefs.jmeter.JMeterTask"/>
<taskdef name="aggregatechart" classname="org.md.jmeter.ant.AggregateGraphTask" classpathref="run.classpath"/>

<tstamp />
<property name="uniqueTStamp" value="${DSTAMP}${TSTAMP}" />
<property name="imageNamePrefix" value="AggregateChartThreshold-${uniqueTStamp}" />
<property name="jmeter.result.fileName" value="${run.test.report}-${uniqueTStamp}" />
<property name="jmeter.result.file" value="${report.dir}/${jmeter.result.fileName}.jtl" />
<jmeter
    jmeterhome="${jmeter.home.dir}"
    testplan="${run.test.plan}"
        resultlog="${jmeter.result.file}">
    <property name="jmeter.save.saveservice.output_format" value="xml"/>
    <property name="run.threadcount" value="${run.threadcount}" />
    <property name="run.loopcount" value="${run.loopcount}" />
    <property name="sample_variables" value="${sample_variables}" />
</jmeter>
<xslt
    in="${jmeter.result.file}"
    out="${report.dir}/${jmeter.result.fileName}.html"
    style="${styles.dir}/${xsl.file}">
    <param name="imageNamePrefix" expression="${imageNamePrefix}"/>


 <aggregatechart jmeterHome="${jmeter.home.dir}" jmeterResultFile="${jmeter.result.file}" outputDir="${report.dir}"
outputFilePrefix="${imageNamePrefix}" showThreshold="true" threshold="200"/>
</target>


<target name="runOfflineGraph" depends="init">
<antcall target="runJMeter">
<param name="run.test.plan" value="OfflineGraphs.jmx"/>
<param name="run.test.report" value="OfflineGraph"/>
<param name="sample_variables" value=""/>
<param name="run.threadcount" value="1"/>
<param name="run.loopcount" value="5"/>
<param name="xsl.file" value="OfflineGraph.xsl" />
</antcall>
</target>
</project>

Important points are
a) we define a run.classpath which has everything we need at runtime to generate the graphs.
b) we have a taskdef aggregatechart for our custom task
c) we invoke the custom chart by passing it the parameters we need. These are closely linked with the previous steps in the build. The result jog from jmeter (${jmeter.result.file}) is passed as an input to the task. The image file names to be generated are important ${imageNamePrefix} as we need to reference this in the stylesheet. The directory to which the Graph code writes must be the same as the XSLT output (or atleast the XSLT and Graph code must be consistent in where the images are referenced from in the HTML)

The XSLT stylesheet
The changes here are pretty straightforward. I've copied extras/jmeter-results-report_21.xsl and renamed it to OfflineGraph.xsl.
The important changes are

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:param name="imageNamePrefix">AggregateChartThreshold</xsl:param>

We pass in a parameter that is to be used while generating the img tag in the HTML

            <xsl:call-template name="summary" />
      <hr size="1" width="95%" align="left" />

      <xsl:call-template name="pagelist" />
      <hr size="1" width="95%" align="left" />

      <xsl:call-template name="detail" />
      <xsl:call-template name="graph-images" />

We call our custom template

            <th><a href="#AverageGraph">Average Time</a></th>
      <th><a href="#MinimumGraph">Min Time</a></th>
      <th><a href="#MaximumGraph">Max Time</a></th>

We change the titles to be anchors (named anchors that are next to the images)

<xsl:template name="output-image">
<xsl:param name="suffix" />
<xsl:element name="img">
  <xsl:attribute name="src"><xsl:value-of select="$imageNamePrefix" />-<xsl:value-of select="$suffix" />.png</xsl:attribute>
</xsl:element>
</xsl:template>

<xsl:template name="graph-images">
Graphs
<br />
<b>Minimum</b>
<br />
<a name="MinimumGraph"></a>
<xsl:call-template name="output-image">
  <xsl:with-param name="suffix">Min</xsl:with-param>
</xsl:call-template>
<br />
<b>Maximum</b>
<br />
<a name="MaximumGraph"></a>
<xsl:call-template name="output-image">
  <xsl:with-param name="suffix">Max</xsl:with-param>
</xsl:call-template>
<br />
<b>Average</b>
<br />
<a name="AverageGraph"></a>
<xsl:call-template name="output-image">
  <xsl:with-param name="suffix">Avg</xsl:with-param>
</xsl:call-template>
<br />
<b>Median</b>
<br />
<a name="MedianGraph"></a>
<xsl:call-template name="output-image">
  <xsl:with-param name="suffix">Median</xsl:with-param>
</xsl:call-template>
<br />
<b>90 percentile</b>
<br />
<a name="NinetyPerGraph"></a>
<xsl:call-template name="output-image">
  <xsl:with-param name="suffix">90</xsl:with-param>
</xsl:call-template>
</xsl:template>

Finally we output img tags. The knowledge of how the Aggregate Graph generates the suffix for the images is hardcoded into the stylesheet. The passed parameter is used to form the filename as well (this is used to allow multiple runs , so that the images don't overwrite previous ones)

Running the ANT build
This is the run.cmd I use (windows only)


set JAVA_HOME=C:\bea102\jdk150_11
set JMETER_HOME=C:\projects\R1-Portal-CMS\test\jakarta-jmeter-2.3.4
set JFREECHART_HOME=C:\work\java\jfreechart-1.0.13
set ANT_HOME=C:\work\java\apache-ant-1.7.1
set PATH=%JAVA_HOME%\bin;%ANT_HOME%\bin;%PATH%
set CLASSPATH=%JMETER_HOME%\extras\ant-jmeter-1.0.9.jar;%CLASSPATH%
ant %*

We set some environment properties that the build needs and run it.

Source Code is available here

Aggregate Graphs in JMeter using JFreeChart (3D Bar Charts) with thresholds

2010-01-10T17:19:00.000-08:00

Borrowed heavily from the demos in JFreeChart, I've modified the previous samples to also support taking in a threshold value for response. Values less than the threshold are shown in Green Bars, values greater are shown using Red Bars. The threshold is also drawn.

Sample Images

Sample Code

    public static void writeAggregateChartWithThreshold() throws Exception {
  File f = new File(JMETER_RESULT_FILE);
  ResultCollector rc = new ResultCollector();
  AggregateChartVisualizer v = new AggregateChartVisualizer(ConfigUtil
          .getOutputGraphDir()
          + "/AggregateChartThreshold",true,500);
  ResultCollectorHelper rch = new ResultCollectorHelper(rc, v);
  XStreamJTLParser p = new XStreamJTLParser(f, rch);
  p.parse();
  v.writeOutput();
}

Source code available here

Aggregate Graphs in JMeter using JFreeChart (3D Bar Charts)

2010-01-10T16:44:00.000-08:00

Now written AggregateChartVisualizer which duplicates AggregateGraph functionality in JMeter.
Sample Charts

Sample Code to be used by clients

    public static void writeAggregateChart() throws Exception {
       File f = new File(JMETER_RESULT_FILE);
       ResultCollector rc = new ResultCollector();
       AggregateChartVisualizer v = new AggregateChartVisualizer(ConfigUtil
               .getOutputGraphDir()
               + "/AggregateChart");
       ResultCollectorHelper rch = new ResultCollectorHelper(rc, v);
       XStreamJTLParser p = new XStreamJTLParser(f, rch);
       p.parse();
       v.writeOutput();
   }

Source Code for the entire workspace available here

Graphs for JMeter (parsing JMeter result logs)

2009-12-31T14:14:00.000-08:00

One of the few features lacking in JMeter is when you run the tests from the command line, the out of box reports are restricted to a stylesheet that generates a summary report.
There are workarounds, you could load the result into Excel (small files) , or you could parse the log file and use JFreeChart to generate the graphs which is what I did. See examples
The following is an explanation of the mechanisms I used. These are probably not going to be out of the box , but hopefully they will be useful to someone who can customise it. The samples are also meant to be used by developers, so if you are a Tester with little or no coding experience, get a developer from your team to help.

I haven't looked closely at the JMeter parsing details, but you don't need to the details to use the JMeter classes (which in my opinion is a hallmark of a well designed system). There are two important files , the saveservice.properties and jmeter.properties which I have copied to a different location from the JMeter home so that I could modify them if I needed to.

The basic code for parsing using JMeter classes(using the JMeter API) is

SaveService.loadTestResults(FileInputStream, ResultCollectorHelper);

where ResultCollectorHelper is passed a Visualizer. The Visualizer has one method that is important to us

add (SampleResult sampleResult)

The Visualizer interface is a simple strategy that can be implemented as we want. Since we also want to write some graphs, I created a new interface called OfflineVisualizer which adds a single method

public Object writeOutput() throws IOException

Here is the class diagram (generated using FUJABA)

Visualizer is a simple strategy pattern. I've some sample implementations for LineChartVisualizer, StackedBarChartVisualizer, MinMaxAvgGraphVisualizer respectively to draw a line chart for each response, a Stacked chart (latency plus response) or a line chart showing Min,Max, Avg along with the response time.

If we take a quick look at the LineChartVisualizer code , its pretty straightforward, it simply uses the JFreeChart API and populates the data from the SampleResult. Note that the line chart objects would use memory proportional to the number of samples

//adds a sample. JFreechart uses a TimeSeries object into which we set each data item
public void add(SampleResult sampleResult) {
String label = sampleResult.getSampleLabel();
TimeSeries s1 = map.get(label);
if (s1 == null) {
   s1 = new TimeSeries(label);
   map.put(label, s1);
}
long responseTime = sampleResult.getTime();
Date d = new Date(sampleResult.getStartTime());
s1.addOrUpdate(new Millisecond(d), responseTime);
}
//uses JFreeChartAPI to write the data into an image file
public Object writeOutput() throws IOException {
TimeSeriesCollection dataset = new TimeSeriesCollection();
for (Map.Entry<String, TimeSeries> entry : map.entrySet()) {
   dataset.addSeries(entry.getValue());
}
JFreeChart chart = createChart(dataset);
FileOutputStream fos = null;
try {
   fos = new FileOutputStream(fileName);
   ChartUtilities.writeChartAsPNG(fos, chart, WIDTH, HEIGHT);
} finally {
   if (fos != null) {
       fos.close();
   }
}
return null;
}
//use the JFreeChart API to generate a Line Chart
private static JFreeChart createChart(XYDataset dataset) {
JFreeChart chart = ChartFactory.createTimeSeriesChart("Response Chart", // title
       "Date", // x-axis label
       "Time(ms)", // y-axis label
       dataset, // data
       true, // create legend?
       true, // generate tooltips?
       false // generate URLs?
       );

chart.setBackgroundPaint(Color.white);
XYPlot plot = (XYPlot) chart.getPlot();
plot.setBackgroundPaint(Color.lightGray);
plot.setDomainGridlinePaint(Color.white);
plot.setRangeGridlinePaint(Color.white);
plot.setAxisOffset(new RectangleInsets(5.0, 5.0, 5.0, 5.0));
plot.setDomainCrosshairVisible(true);
plot.setRangeCrosshairVisible(true);
XYItemRenderer r = plot.getRenderer();
if (r instanceof XYLineAndShapeRenderer) {
   XYLineAndShapeRenderer renderer = (XYLineAndShapeRenderer) r;
   renderer.setBaseShapesVisible(true);
   renderer.setBaseShapesFilled(true);
   renderer.setDrawSeriesLineAsPath(true);
}
DateAxis axis = (DateAxis) plot.getDomainAxis();
axis.setDateFormatOverride(new SimpleDateFormat("dd-MMM-yyyy HH:mm"));
return chart;
}

We can change the data some graphs show by using the Decorator pattern. One decorator LabelFilterVisualizer is shown.

/**
* decorates the visualizer by filtering out labels
*/
public void add(SampleResult sampleResult) {
 boolean allow = labels.contains(sampleResult.getSampleLabel());
 if (!pass) {
     allow = !allow;
 }
 if (allow) {
     visualizer.add(sampleResult);
 }
}

/**
* delegates to the decorated visualizer
*
* @return whatever the decorated visualizer returns
*/
public Object writeOutput() throws IOException {
 return visualizer.writeOutput();
}

This class filters out labels and only delegates those that satisfy the criteria. The writing of the image is delegated to the decorated OfflineVisualizer
We can also use the composite pattern(CompositeVisualizer) to have multiple graphs generated with a single pass through the result log file.

/**
* adds the sample to each of the composed visualizers
*/
public void add(SampleResult sampleResult) {
  for (OfflineVisualizer visualizer : visualizers) {
      visualizer.add(sampleResult);
  }

}

/**
* @return a List of each result from the composed visualizer
*/
public Object writeOutput() throws IOException {
  List<Object> result = new ArrayList<Object>();
  for (OfflineVisualizer visualizer : visualizers) {
      result.add(visualizer.writeOutput());
  }
  return result;
}

Finally we can use all the above to process multiple files , for e.g. when we want to show trends across multiple runs with varying thread counts.

/**
* parses each file
*
* @throws Exception
*/
public void parse() throws Exception {
  // One day we might multithread this
  for (String file : files) {
      ResultCollector rc = new ResultCollector();
      TotalThroughputVisualizer ttv = new TotalThroughputVisualizer();
      visualizers.add(ttv);
      ResultCollectorHelper rch = new ResultCollectorHelper(rc, ttv);
      XStreamJTLParser p = new XStreamJTLParser(new File(file), rch);
      p.parse();
  }
}

/**
* Gets the resulting throughput from each file and combines them
*
* @return always returns null
* @throws IOException
*/
public Object writeOutput() throws IOException {
  XYSeries xyseries = new XYSeries("throughput");
  for (AbstractOfflineVisualizer visualizer : visualizers) {
      Throughput throughput = (Throughput) visualizer.writeOutput();
      xyseries.add(throughput.getThreadCount(), throughput
              .getThroughput());
  }
  XYSeriesCollection dataset = new XYSeriesCollection();
  dataset.addSeries(xyseries);
  JFreeChart chart = createChart(dataset);
  FileOutputStream fos = null;
  try {
      fos = new FileOutputStream(fileName);
      ChartUtilities.writeChartAsPNG(fos, chart, WIDTH, HEIGHT);
  } finally {
      if (fos != null) {
          fos.close();
      }
  }
  return null;
}

Here's a sample that I ran. A single thread hits 3 pages on the apache website in a loop.

Response times are plotted against each label (without considering the thread).

File f = new File(JMETER_RESULT_FILE);
ResultCollector rc = new ResultCollector();
LineChartVisualizer v = new LineChartVisualizer(OUTPUT_GRAPH_DIR + "/LineChart.png");
ResultCollectorHelper rch = new ResultCollectorHelper(rc, v);//this is the visualizer we want
XStreamJTLParser p = new XStreamJTLParser(f, rch);
p.parse();
v.writeOutput(); //write the output

The next example filters out only the Component reference request and plots the response time, the minimum time, the maximum time and the average time for this request. You could extend this to indicate the median or the 90th percentile.

The code for this graph is

File f = new File(JMETER_RESULT_FILE);
ResultCollector rc = new ResultCollector();
MinMaxAvgGraphVisualizer v = new MinMaxAvgGraphVisualizer(OUTPUT_GRAPH_DIR + "/MinMaxAvg.png");
String[] labels = {"Component reference"}; //we only want this label
LabelFilterVisualizer lv= new  LabelFilterVisualizer(Arrays.asList(labels), v);//decorate the MinMaxAvgGraphVisualizer
ResultCollectorHelper rch = new ResultCollectorHelper(rc, lv);//use the decorated visualizer
XStreamJTLParser p = new XStreamJTLParser(f, rch);
p.parse();
lv.writeOutput();//write it out

The next chart shows a stacked chart which splits the response time for the Component reference into latency and the rest of the time.

File f = new File(JMETER_RESULT_FILE);
ResultCollector rc = new ResultCollector();
StackedBarChartVisualizer v = new StackedBarChartVisualizer(OUTPUT_GRAPH_DIR + "/StackedBarChart.png");
String[] labels = {"Component reference"};//we only want this label
LabelFilterVisualizer lv= new  LabelFilterVisualizer(Arrays.asList(labels), v);//Decorate the StackedBarChartVisualizer
ResultCollectorHelper rch = new ResultCollectorHelper(rc, lv);
XStreamJTLParser p = new XStreamJTLParser(f, rch);
p.parse();
lv.writeOutput(); //write the output

We could also run all these graphs at the same time using the Composite

File f = new File(JMETER_RESULT_FILE);
ResultCollector rc = new ResultCollector();
LineChartVisualizer lcv = new LineChartVisualizer(OUTPUT_GRAPH_DIR + "/AllLineChart.png");
StackedBarChartVisualizer sbv = new StackedBarChartVisualizer(OUTPUT_GRAPH_DIR + "/AllStackedBarChart.png");
MinMaxAvgGraphVisualizer mmav = new MinMaxAvgGraphVisualizer(OUTPUT_GRAPH_DIR + "/AllMinMaxAvg.png");
String[] labels = {"Component reference"};
LabelFilterVisualizer lv= new  LabelFilterVisualizer(Arrays.asList(labels), sbv);//decorate
LabelFilterVisualizer lv2= new  LabelFilterVisualizer(Arrays.asList(labels), mmav);//decorate
OfflineVisualizer[] vs = {lcv, lv,lv2};//use these 3 visualizers
CompositeVisualizer cv = new CompositeVisualizer(Arrays.asList(vs));//create a composite
ResultCollectorHelper rch = new ResultCollectorHelper(rc, cv);
XStreamJTLParser p = new XStreamJTLParser(f, rch);
p.parse();
cv.writeOutput();//the composite will delegate to each visualizer

I also reran the same test for 1, 3,5,7 and 10 threads. Using the classes above and a new throughput visualizer (where I calculate throughput as total number of requests / total time the test ran) and plotted the throughput v/s the number of threads

String [] files = {JMETER_RESULT_DIR + "/OfflineGraphs-dev-200912311310.jtl", JMETER_RESULT_DIR + "/OfflineGraphs-dev-200912311312.jtl",JMETER_RESULT_DIR + "/OfflineGraphs-dev-200912311315.jtl",JMETER_RESULT_DIR + "/OfflineGraphs-dev-200912311316.jtl",JMETER_RESULT_DIR + "/OfflineGraphs-dev-200912311318.jtl"};
MultiFileThroughput mft = new MultiFileThroughput(Arrays.asList(files),OUTPUT_GRAPH_DIR + "/Throughput.png");
mft.parse();
mft.writeOutput();

The above examples are not exhaustive and probably wont work for you (for e.g. threads are ignored, thread groups are ignored, and these might have meaning for your test). However you should be able to use this to write your own implementation.

Running the code.
a. Download the code. This is an eclipse workspace. To get this to compile, you need to define two variables in eclipse (JMETER_HOME and JFREECHART_HOME) for the classpath. Modify config.properties to whatever is applicable for your system. Use GraphClient to see the samples
I created an additional dummy directory for Jmeter home and created a bin directory under it and copied jmeter.properties and saveservice.properties.
b. Change the client to use the visualizers you want. The sample client should give you some idea. Or create a new visualizer
c. Compile and run! If you use a different IDE or want to use ANT it should be pretty straight forward. The source code has been written and tested on Java 1.5 . There isn't any 1.5 feature I use except generics and the new for loop syntax. You could change this to be 1.4 compatible.

Further work
a. Combining results from multiple files into a single run.
b. Making the visualizers configurable
c. Canned HTML reports
d. Threads/ThreadGroups
e. Determine limits for the graphs.
f. Support custom attributes

If there are specific graph requests , I might take a look into it, day job and wife willing.

JMeter and SLA's

2009-12-16T12:43:00.001-08:00

One of the current issues on our site is that while we profiled and performance and load tested the important pages before we went live, we haven't done it for subsequent builds and releases. There are various excuses for this (lack of time, lack of representative environments, restrictions on the actions that may be performed because the site is live), none of them really justified. However no matter how much you test the site before hand, the site may still malfunction, perhaps transiently on production. For e.g. we sometimes got timeout errors between 6:00 to 10:00 a.m. (it was eventually determined to be a Database index compacting job that was creating trouble). The problem is that we had to be reactive, look at the logs, there was a timeout, run around like headless chickens because the site was working fine now, no access to the environment to see whats happening etc. Now we could have configured logs to automatically notify us when there are errors but this would only work if there was a timeout (in our 60 seconds for any remote operation). If a page that normally takes 2 seconds to load took anything under 60 seconds we would not see errors.
In previous projects , we had OVIS, which I believe is expensive, but my current project has no such commerical tool. Open source tools all seem to solve parts of the problem , but there didn't seem to be any tool that did everything I wanted.
Briefly
a. Flexible schemes to measure response times. We needed to be able to simulate accessing stand alone urls, login flows, checkout flows, search flows.
b. Ability to store the data and view trend graphs
c. Ability to run the tests on a schedule
d. Ability to specify thresholds for each page (again with a fair degree of flexibility) and mark responses as failed
e. Flexible notification schemes

The choice of technologies I used were based more on things I wanted to learn or refresh rather than the best there is, so keep in mind this is more of a toy than I would have liked
a. JMeter for response times. I'm not really interested in loading the site, nor do I want exact browser render times, I'm just looking for ballpark numbers and deviations, especially after builds or at odd hours.
b. Hudson for scheduling Jmeter builds. I chose hudson because I haven't used it before.
c. STAX for parsing JTL. I chose stax because I wanted to be able to parse large files, and i already know SAX , but I've never used STAX.
d. Tomcat with JSP + Spring. I've loved Spring JDBC ever since I've used it (take that Hibernate, JPA, JDO, EJB). It removes all the redundant code while not sacrificing the power of SQL , and there is no learning curve beyond Spring. I chose JSP over any of the MVC framework because of shortage of time. While people may insist how their preferred framework saves them tons of time , this only applies in the long run
e. JQuery for all the javascript stuff
f. JFreeChart for the chart related functionality. I've used this before and found it to be a solid library.
g. Derby for the database, this is something I have not used before, I wanted a reasonably stable database , non embedded.

Most of the things Ive written aren't really reusable, in addition this was quick and dirty, so don't expect this to work for you
a. The JMeter Script
b. Parsing the JMeter Script and loading it into the database
c. Scheduling JMeter to run in Hudson
d. Writing a UI around this
e. Allowing administrators to configure thresholds and notifications
f. Notifications

Randomly Clicking links in JMeter (sometimes known as spidering)

2009-11-20T19:59:00.001-08:00

A follow up to Spidering a site with JMeter
A user on the JMeter mailing list posted his solution using the HTML link parser[1] to spider a site. The spidering consists of clicking a link at random from the links parsed from the last accessed page.
The test looks like
Script available at Spider.jmx
The Initial Request is used by the HTML Link Parser to get the initial set of urls from which one will be chosen.
The While controllers condition is simply true , since we want it to loop forever.
The Spider HTTP Sampler has a path of .*.
The If Controller has a condition ${__javaScript(!${JMeterThread.last_sample_ok})}
This simply checks if the last sample fetched failed (by .* or because it fetched a CGI/PDF which cant be parsed for links and if so reexecutes the Initial Request.)
There are numerous tweaks you can implement , you might not execute the initial request, it might be one at random that you pick , or the last successful request. You might choose to check the request being made to restrict the paths.

Note that this clicks a link at random from a set of links acquired from the previously clicked page. This cannot ensure that a link is not repeated and cannot ensure that all links are fetched.

[1] http://jakarta.apache.org/jmeter/usermanual/component_reference.html#HTML_Link_Parser

Dependent tests in JMeter (kind of)

2009-11-11T13:58:00.000-08:00

A common use case in testing is the concept of dependent tests(except for the unit test fanatics who love JUnit and didn't realise they needed this functionality till TestNG came along). One of the requirements then becomes that these dependent test should not execute if the test that it depends on fails. To implement this in JMeter we need to use the following two pieces of information
The variable JMeterThread.last_sample_ok is set to "true" or "false" after all assertions for a sampler have been run. [1]
If Controller -Evaluate for all children - Should condition be evaluated for all children? If not checked, then the condition is only evaluated on entry.[2]

Combining the two bits of information we have
Thread Group
If Controller((${JMeterThread.last_sample_

ok}) with Evaluate for all children = checked
Req 1 --> if this errors req 1 and req 2 wont be executed
Req 2 --> if this errors , req 3 wont be executed
Req 3
Note that any assertion failing would also mark the request as failed.
Note also that you cannot have nested dependent sets but you could flatten them out as separate IF controllers.

[1] http://jakarta.apache.org/jmeter/usermanual/component_reference.html#assertions
[2] http://jakarta.apache.org/jmeter/usermanual/component_reference.html#If_Controller

Commenting Code

2009-10-30T09:11:00.000-07:00

Have you ever written code that reads from a BufferedReader? Suppose you read someone else's code that said

//There is a BufferedReader r

StringBuffer sb = new StringBuffer();
int i;
while ((i = r.read()) != -1)
sb.append((char)i);

What do you think ? will you change it to the more normal

int i;
char[] data = new char[1024];
while ((i = r.read(data,0,data.length)) != -1)
sb.append(data);
Edit: Embarassingly the above code is wrong, but the code is only illustrative , reading a character at a time v/s reading chunks of it

Which is more efficient? Which is more 'performant'?

And finally if you read the original code with an additional comment

StringBuffer sb = new StringBuffer();
int i;
// under JIT, testing seems to show this simple loop is as fast
// as any of the alternatives
while ((i = r.read()) != -1)
sb.append((char)i);

would you even bother?

Code snippet taken from ImportSupport.java - jakarta-taglibs-standard-1.1.2-src.

Spidering a site with JMeter

2009-10-29T22:54:00.000-07:00

Sometimes we need to check every link on the site and see that they all work, and this question came up a couple of times on the JMeter forum 'How do I use JMeter to spider my site?'
But before we go into the solutions, lets take a step back and see the reasons behind wanting to spider the site or skip to solution

a. You want to find out whether any urls respond with a 404. This isn't really a task for JMeter and there are various open source/free link checkers that one might use so there really isn't a need to run JMeter to solve this class of problems (http://java-source.net/open-source/crawlers for just spiders in Java. There are others too like Xenu or LinkChecker)

b. You want to generate some sort of background load and you hit upon this technique. A spider run with a specific number of threads will provide the load. While a valid scenario, this doesn't really simulate what the users are doing on the site. So it goes back to what are you trying to simulate?. It's much better to simulate actual journeys with representative loads. You might need to study your logs and your webserver monitoring tools to figure this out. It's tougher to do this, but it's more useful.

c. You want to simulate the behavior of an actual spider (like Google) and see how your site responds, whether all the pages are reachable. See a.

Other problems
A test without assertions is pretty much useless. A spidering test by its nature is difficult to assert (other than response code = 200! and perhaps the page does not contain the standard error message shown).

JMeter does not really provide good out of the box support for spidering. The documents refer to an HTML Link Parser which can be used for spiders which leads some users to try it out and complain that it doesn't work. It does(see this post) but not how you expect, and not as a spider (The reference manual needs to change).

Before we go on to trying to implement an actual Spider in JMeter, lets see some alternatives that we have (using JMeter and not a third party tool).
a. Most sites have a fixed set of URL's and a possible dynamic set e.g. a Product Catalog where each product maps to a row in the database. It is easy enough to write a query that fetches these (using a JDBC Sampler) and generating a CSV file that contains the URL's you want. The JDBC sampler is followed by a Thread Group (number of threads the spider will run) which reads each URL from the CSV. This is especially useful when you consider that it is quite possible that some links are not accessible from any other link in the site (This is bad site design, but exists , for e.g. FAQ are not browsable on my current site, they must be searched for which means that there is no URL from which the FAQ is linked to and a spider would never find them directly)

b. Some sites generate a sitemap (it may even be a sitemap that is used for Google) for the reasons mentioned above. It is trivial to parse this to obtain all the urls. A stylesheet can convert this into a CSV and the rest is the same as point a.

One last thing before we start discussing JMeter solutions. The first time I came to know anything about how spiders work is when I ran Nutch locally(and later refined with the knowledge of MapReduce).
In a simplified form
a. A first stage reads URLs that are pending from the buffer/queue and downloads the content. This is multithreaded but not too much so as to not bring down the site.
b. The second stage parses the contents for links and feeds it into the same buffer and queue.
c. A third stage indexes the content for search. This is irrelevant for our tests.
A related concept is that of depth. i.e. In how many clicks(minimum) does it take to reach the link from the root/home/starting point of the website.

Attempt 1.
Using the previous depth definition, most sites(because of menu's and sitemaps) need at the most 5-7 clicks to reach any page from the root page (kind of like Kevin Bacon's six degrees of separation). This implies that instead of a generic solution we could have a hardcoded solution which fixes the depth that we look at and use the time tested method of Copy - Paste.
Here's what this solution would like

The test plan is configured to run the Thread Groups serially
1. Thread Group L0 fetches all the urls listed in a file named L_0.csv. Each request is attached to beanshell listener which parses the response to extract all anchors and writes these anchors to a separate temp file. The code which does this is lifted from AnchorModifier and is accessed via a Beanshell script calling a Java class(JMeterSpiderUtil).
2. Thread Group L0 Consolidate (single thread) creates a unique set of all the urls from the temporary files created in step 1 and subtracts the urls already fetched from L_0.csv and writes these urls to a file named L_1.csv. This code is also in the Java class and is described below.
3. Thread Group L1 (multi thread) fetches all the urls listed in the file L_1.csv which was created in step. Each request is attached to beanshell listener which parses the response to extract all anchors and writes these anchors to a separate temp file.
4. Thread Group L1 Consolidate (single thread) creates a unique set of all the urls from the temporary files created in step 3 and subtracts the urls already fetched from L_0.csv,L_1.csv and writes these urls to a file named L_2.csv
... and so on for any number of levels/depths that you want.
If you are any sort of developer , you are probably groaning at the above. "Hasn't this guy heard about loops? What about maintaining these tests? Are we going to make any changes in 5 places ?".
We could use Module Controllers to reuse most of the test structure but it's still inelegant.
One of the reasons I've described the above is that even if the solution looks inelegant it is easy to understand and doesn't take time to implement, which means you can start testing your site pretty quickly. Note that your priority is the testing of the site , not the elegance of the testing script.

Attempt 2
Lets now see if we can increase the elegance of the script. One of the problems we run into is that the CSV data set config can't use variable names for the filename. Another problem is that in the solution above we run the Thread Groups serially and we use a single thread in a thread group to combine the results. If we want to use a single looped thread group we have to ensure only 1 thread does the combining which needs to wait for all the other threads to complete. You can probably simplify this solution by extending the CSV data set config or the looping controllers, I don't consider these approaches because I have no Swing experience at all , so the only ways I extend JMeter are via BeanShell or Java.
After some experimentation this is the solution that I've come up with

1. The Loop Controller controls the depth/level
2. The simple controller has an If controller that is only true for the thread with threadnumber 1. It defines the current level and copies the file L_${currentlevel}.csv to urls.csv
3. The wait for everyone is configured with a synchronizing timer (same as total number of threads in a threadgroup) so that all the threads are waiting till the first thread has finished in step 2
4. The while controller iterates over all the urls in the csv. The CSV Data Set is configured to read the copied urls.csv file (since we cannot make the name variable). What we will do in the subsequent steps is recreate this same file with new data. Each request is attached to beanshell listener which parses the response to extract all anchors and writes these anchors to a separate temp file. The code which does this is lifted from AnchorModifier and is accessed via a Beanshell script calling a Java class(JMeterSpiderUtil).
5. We have a copy of step 2 here, all the threads wait till everyone else is done (for that level only)
6. The If controller ensures that the consolidation is done only for the first thread, all the files written in step 4 are combined into a unique set, all the urls already processed are subtracted and a new file L_${nextlevel}.csv is written. Properties are set so that the ${currentlevel} now is the ${nextlevel} so that step 1 will now pick up this new file and copy it as urls.csv for the CSVDataSetConfig to pick up.
7. The Reset Property Bean Shell sampler is used to reset the CSV Data Set Config
FileServer server = FileServer.getFileServer(); // get the File Server
server.closeFiles(); // close everything
server.reserveFile("../spider/urls/urls.csv", null, "../spider/urls/urls.csv"); //reregister the CSV, we have chosen sharing mode as All Threads to avoid copying the alias name generation in CSVDataSet.java

This was run with a root of http://jakarta.apache.org/jmeter/index.html.
Only urls with jmeter in them were spidered and with the jakarta.apache.org host.
Level 1 - 17 urls
Level 2 - 29 urls
Level 3 - 125 urls
Level 4 - 833 urls
Level 5 - 2 urls
Level 6 - 0 urls
And I did get some failures too e.g.
http://jakarta.apache.org/jmeter/$next
http://jakarta.apache.org/jmeter/$prev
So I guess the test is successful because it found some issues!.
Which means there are no more urls that satisfy our criteria. You could change the loop to a while controller and use this condition to check whether or not the test should exit. However some sites generate unique urls (e.g. by appending a timestamp) which makes it possible that your test might not exit , so you should normally have a safety for maximum depth.

Is attempt 2 more elegant? Probably , but also less configurable and took about 2-3 days to get it working and needed some study of JMeter source code. Note that the previous solution could vary the number of threads available to each Thread Group but this can't. However by using the constant throughput timer , you can achieve variable throughput for different levels.

JMeterSpiderUtil.java
The major part of this code is from AnchorModifier
Important snippets are shown
if(isExcluded(fetchedUrl) ) //excludes stuff like PDF/.jmx files which cant be parsed
...
(Document) HtmlParsingUtils
.getDOM(responseText.substring(index)); // gets a DOM from the request
...
NodeList nodeList = html.getElementsByTagName("a"); //gets the links
...
HTTPSamplerBase newUrl = HtmlParsingUtils.createUrlFromAnchor( hrefStr, ConversionUtils.makeRelativeURL(result .getURL(), base)); //get the url
...
if(allowedHost.equalsIgnoreCase(newUrl.getDomain())) {
String currUrl = newUrl.getUrl().toString();
if(matchesPath(currUrl)) {
//currUrl = stripSessionId(currUrl);
//currUrl = stripTStamp(currUrl);
fw.write(currUrl + "\n");
}
}
//checks whether the host is the one we are interested in, whether the path is one that we want to spider, could strip out session ids or timestamp parameters in the url
...
Download Code
SpiderTest - Attempt 1.
SpiderTest2 - Attempt 2.
JMeterSpiderUtil - Java utility.

If you want to use the code
a. Ensure that the total number of Threads is specified correctly in both synchronizing timers (use a property)
b. Some directories are hardcoded. I used a directory named scripts under jmeter home, another directory called spider at the same level as scripts. Scripts has two sub directories temp and urls. L_0.csv the starting point is copied into urls.
c. If you want to rerun the test ensure you delete all directories under temp and all previously generated csv files in urls (except for L_0.csv.
d. You might have to change the java code to further filter urls /improve the code. The Jmeter path regular expression is hardcoded
e. You have to change the allowedHost , probably to an allowable list rather than a single value.
f. You probably have to honor robots.txt
g. You might want to check the fetch embedded resources or change what urls are considered to be fetched (currently only anchors no forms or ajax urls based on a pattern)

Note that the code is extremely inefficient and was only written to check if what I theorized in http://www.mail-archive.com/jmeter-user@jakarta.apache.org/msg27108.html was possible
There is a lot of work to properly parameterise this test , but hopefully this can get you started.

Code available here

Interview questions revisited

2009-10-23T16:52:00.000-07:00

Ive been experimenting with webmaster tools and analytics for this blog, and while running a Google search , I came across
http://www.experts-exchange.com/Software/Server_Software/Application_Servers/Java/BEA_WebLogic/Q_24000475.html+weblogic+portal+interview+questions (Hint use Google Cache to see the answers )
And on the BEA forums I see
http://forums.oracle.com/forums/thread.jspa?threadID=919149&tstart=15
http://ananthkannan.blogspot.com/2009/08/weblogic-portal-interview-questions_29.html
http://venkataportal.blogspot.com/2009/09/comming-soon.html
Compared with my own
http://theworkaholic.blogspot.com/2007/02/weblogic-portal-interview-questions.html
http://theworkaholic.blogspot.com/2009/10/weblogic-portal-interview-questions-ii.html

There's a pretty big difference between the kind of questions I ask and the kind of questions people seem to think will be asked or indeed do ask. A multiple choice question? really? I guess that was picked up from the BEA certification exam. (The less said about certification the better). Is there a point asking people something that's right there in the documentation or something that any respectable search engine could?
Lets get some assumptions out of the way
a. A bad resource is extremely detrimental to any software project. The contribution is negative and a big negative at that. It is better to not have the resource than have a bad resource.
b. There isn't an easy way to eliminate a bad resource at a short listing phase.
In most cases there are more people applying to the job than there are jobs. The resume is too abused to be an effective eliminator. If you look at a typical Java/EE resume , every specification in the EE umbrella is covered. Everyone has solid knowledge and expertise in all the specifications. On project experience is sometimes faked.
Would a quick multiple choice easily corrected paper help? I believe that this is actually bad. The people who aren't that knowledgeable know it, and spend their time memorizing documents/api's etc before an interview and can probably game this test. The people who I know are good in their fields usually don't have much time or patience for the minutiae, but are quite capable of doing this on demand. Project Experience would be a good indicator, but it is costly to verify this before hand. References are usually given by friends and aren't reliable. Typically a interviewee isn't going to provide a reference to someone who will give him a negative review.

So we can't rely on the short listing process to eliminate the bad apples. You must as an interviewer go to an interview thinking that you might be gamed. This means that straightforward questions might be answered by a bad candidate. This doesn't mean that you should ask the brain teaser sort of questions which only indicate that the interviewee is good at solving brain teasers (or has Googled the answers).

What then constitutes a good interview question?
Here are my criteria
a. The interviewee must be able to describe what he has worked on /is working on effectively. he must be confident in the modules he has worked. He must be able to answer questions related to his module when you vary some of the parameters. This is a deal breaker. A person who doesn't know what his project probably wont be able to handle yours either.
b. Most of the technical questions I ask are conversational and to which there probably isn't a right answer. The question is just the opening gambit, E4 for chess players. If I feel I am getting a recitation from documents , I introduce a twist or change a parameter of the problem (e.g. if the answer is something like I would design this with Spring by utilizing Dependency Injection IOC pattern and use the Hibernate ... - would be met with sorry the spring/hibernate license doesn't meet the project requirements, you can't use it).
c. Hands on experience on the technologies Im looking for is always a great plus, but it isn't a dealbreaker for me. If you can handle JSP, you can handle JSF. If you can handle Struts, you can handle other controller frameworks. What I can't stand is when someone states about all the stuff he has worked on at the start, how he was the heart and soul of the entire project, the life of the party, later changes his tune to say well I didn't really work much on that particular part. Thats a deal breaker. Dishonesty means I can't trust any of the other wonderful things you said, bye bye.
d. Never ask code questions without also providing the books, the documents, the search engine and a compiler. Writing code snippets on a whiteboard is stupid. Pseudo code questions are perfectly acceptable. Don't ask people to reinvent sorting algorithms when there are so many books (When will I ever buy that Donald Knuth book) that they could use. If you want to check analytical skills then use real life examples. There must have been numerous problems with your project, describe the circumstances and ask the resource to make suggestions.

In some ways I'm glad that I don't have to conduct interviews anymore. The last time I was proudly telling my mother of how many people I have rejected, she said why am I depriving people from working , and that you don't know how much they might need the job. While I still stand by my assumption that no resource is better than a bad one, it's still disturbing to think that I might(probably) have made errors in judgement and maybe just maybe I rejected a deserving candidate and maybe just maybe he really needed it. Like I said I'm glad I don't make hire decisions anymore

Throughout this post, I have referred to the interviewee as 'him'. Thats probably due to that fact that more than 90% of the candidates I've interviewed are male. Which is a sad state of affairs for software.

First Weblogic Portal Pro

2009-10-21T08:52:00.000-07:00

I'd like to thank.....This shouldnt give me that much happiness, but it does.

Weblogic Portal interview questions - II

2009-10-20T21:52:00.000-07:00

The following are the Portal interview questions that ive used or kept or have been asked(in no particular order , and no answers either :) )
I do not include questions (e.g. what is a nested pageflow) that can be answered with Google.
Also see Weblogic Portal interview questions - I

What options do you have for Single Sign On for a Weblogic Portal application (and in general). Give the advantages and disadvantages of each approach
If you are using WSRP, and the user is logged in to the consumer , is he also logged into the producer? If so how? If not how do you do this?
If you have standard static HTML application, how would you optimise this for performance? For each of the technique's you mention , how would this be implemented in Weblogic Portal
How do you ensure that a Weblogic Portal application is easily Searchable by external search engines like Google
What are serious problems/ drawbacks of JSR 168/ JSR 286. Under what circumstances would you not use these for your portlet implementation? Under what circumstances would you use these for your portlet implementation?
Why is asynchronous desktop a bad idea? In what situations does it become a good idea?
What circumstances can cause issues with Portal Propagation. Would you use propagation in your actual production? If not , why not?
How would you integrate Flex / Any flash based widget into your portal application?

Detecting missing files with JMeter

2009-10-12T18:34:00.000-07:00

I have lately found that JMeter is becoming my tool of choice for almost all the normal mundane programming tasks. Case in point.
On my current website's we have a bunch of PDF's (150K) which are accessible only via search and have an entry in some table for that purpose. Each PDF is linked to a language and multiple countries so the total number of rows in the database are much more than the number of files. Now years later, due to human error and other causes some of these records exist in the database but there aren't corresponding PDF files on the webserver , which allows the user to see a link when he searches for the data but a 404 error when the user actually clicks it. I had to generate a report listing all these files.

Constraints
a. Administrators wont let you run a program on the web server.
b. You could ask them to copy files to a separate directory but it takes about a week to get approval for anything related to production except for a Database copy (which is available immediately)

I initially thought of asking for a recursive file name print of all the files from the webserver to compare against the database (but writing this Java program would have taken half a day to iron out the bugs) . So I settled on JMeter

Run query to get a list of files and save it to a CSV (Squirrel SQL client)
Thread Group (10 in parallel)
CSV Data Set
Http Request (HEAD) , the web link to the PDF is read from the CSV

Run from command line with a sample_variables property set to fields from the CSV.
Time to run test = 1.5 hrs. JMeter's sample HTML was good enough to be shown to the users to fix the issues.
Whats great is that when the missing files are uploaded I can verify the data easily again.

Now I could have used JMeter's JDBC sampler to eliminate the Squirrel client.

Profiling BEA Weblogic Portal Apps

2009-10-06T15:08:00.000-07:00

Profiling a portal application running on earlier versions of BEA Weblogic has always been somewhat painful(still is) if you aren't willing to pay for a commercial profiler(It still might be painful). With Weblogic 8.1 I had used Eclipse Colorer but that doesn't seem to work with the later versions of Eclipse and hasn't been developed in a while, it crashed on Weblogic 10 (JDK 1.5). I tried out a few from the Open Source Java Profilers page but some crashed the JVM and some didn't do what I want.
The basic things were
a. I needed to check execution times.
b. I didn't want to recompile my application or make changes to code.

I'd played around a bit with TPTP so I gave it a try and used it, it worked reasonably well, I eliminated some code that didn't cache data correctly , so all in all it was a success. I haven't had time to look through all the settings in detail , and I'm sure some of the settings are redundant , but they worked for me. I've created these steps using the latest available versions of TPTP/Eclipse.
I ran the test in Windows Vista. Folks using a different O.S. are probably smart enough to not need these steps.

Steps
a. Install Eclipse IDE for Java EE Developers

b. Install the TPTP(4.6.1) plugin. There are a set of screens on how to do this - http://wiki.eclipse.org/Install_TPTP_with_Update_Manager. You could also download the All in one which has Eclipse + TPTP. Also referred to a couple of links on TPTP. Profiling J2SE 5.0 based applications and TPTP installation guide

c. Download the agent controller for TPTP. Unzip it to a folder. Call this folder $AGENT_CONTROLLER_HOME

d. Set a new environment variable
JAVA_PROFILER_HOME=$AGENT_CONTROLLER_HOME\plugins\org.eclipse.tptp.javaprofiler

e. Set up the PATH ( I did this in Control Panel --> System --> Advanced --> Environment variables)
$AGENT_CONTROLLER_HOME\plugins\org.eclipse.tptp.javaprofiler;$AGENT_CONTROLLER_HOME\bin;
You should have Java in your path somewhere. I use the same JDK as that with BEA. (i.e. Java 1.5 . I did try Java 1.6, but it didn't work for me)

I run on Windows Vista so all command prompts are launched with Run As Administrator including the BEA server.

f. In a command window cd to $AGENT_CONTROLLER_HOME\bin and run setConfig. Specify the path to Java (1.5) and the other options, I chose the options in the screen below.

g. Start the agent controller (ensure no firewall blocking or unblock) by running acserver.exe.

h. In a new command line window run SampleClient. If all is well, you should see the response. Close SampleClient command window but keep acserver running

Setting up BEA
i. Goto the BEA portal domain and change the following settings in setDomainEnv.cmd (these already exist , just change the values)
set debugFlag=false
set testConsoleFlag=false
set iterativeDevFlag=false
...
set PRODUCTION_MODE=true

Towards the bottom of the file (4-5 lines from the bottom), add the command to enable the profiler

set JAVA_OPTIONS=%JAVA_OPTIONS%
set JAVA_OPTIONS=-agentlib:JPIBootLoader=JPIAgent:server=controlled,filters=$DOMAIN_HOME\filters.txt;CGProf:execdetails=true %JAVA_OPTIONS%

Here we specify that the process should wait (server=controlled) till we connect to it, specify some filters for packages that we have no interest in (and which would cause the system to be slower), specify that we want to capture executing details.

Create a file named filters.txt in the path you have specified
org.apache.* * EXCLUDE
com.bea.* * EXCLUDE
weblogic.* * EXCLUDE
netscape.* * EXCLUDE
antlr.* * EXCLUDE
com.octetstring.* * EXCLUDE
com.rsa.* * EXCLUDE
org.omg.* * EXCLUDE
javelin.* * EXCLUDE
kodo.* * EXCLUDE
org.opensaml.* * EXCLUDE
com.pointbase.* * EXCLUDE
serp.* * EXCLUDE
com.solarmetric.* * EXCLUDE
schemacom_bea_xml.* * EXCLUDE
com.asn1c.* * EXCLUDE
com.certicom.* * EXCLUDE

When I hadn't filtered out kodo packages , I did get a ClassFormatError so at a minimum these packages must be filtered

j. Now run startWeblogic. The process should wait(we specified server=controlled remember)

k. Now start eclipse. Click Run --> Profile Configurations. Click Attach to Agent and hit new icon. A new Configuration is created

l. Now click the agents tab, if all is well you should be able to see an entry

m. Double Click it and specify the filters (same as the ones specified in filters.txt)

n. Click Next, Uncheck the run automatically, Click finish.

o. Click Apply and Profile. Switch to the profile perspective.

We haven't started profiling yet, but the Weblogic server will now continue start up. You probably have to wait about 10 minutes.

p. Once Weblogic is in running mode , you can start the profiling by clicking the run icon in the left pane. You can also click the execution statistics (though this might be empty since we have filtered most of the default BEA code that runs.

q. Now exercise your application by accessing it in the browser or by running a test e.g. a JMeter test.
You should now be able to see execution details in Eclipse. For e.g.

which shows 100 calls being made to DBService. Double click it.

which shows the method calling it (TestService.getList() 1 call here makes a 100 calls to the DB , plus some BEA security checks). The TestService is called by the Portlet Controller as shown

And you can easily conclude that there is some sort of N+1 problem here. A single request leads to 100 db calls, after which inspect the code fix the problem, rerun the profile and verify that you only invoke the DB once.

However there is a caveat here, it is far far easier to profile your code out of container. If you can separate out your code so that most of it runs outside Weblogic , then it's easier to profile it. And as we all know, this isn't always possible.

Running JMeter for a large number of concurrent threads

2009-09-10T15:42:00.000-07:00

A common repeated question (Observation JMeter users probably don't use Google) on the JMeter forums goes something like
I need to run N000 users concurrently and I have a Windows/Unix/Mac with N GB RAM , will that work? or how do I run N000 users from JMeter or Im running N000 users concurrently against my server, am I performance testing this correctly?

And the correct answer is (like all things related to performance testing) , test it out and see for yourself. This doesn't mean that there aren't rule of thumb's that you can follow but that no one can give you a definitive answer.

My rules of thumb

Know the load your client machines can generate approximately(for your test) - On my Windows PC (with Vista and 3GB RAM, running JMeter with a heap of 1 GB, dual core 2.4GHZ) if I run more than 100 threads, my machine starts to hang. I can't use any other applications so I normally wouldn't run a single JMeter instance with > 100 threads for a machine configured like mine (But my tests usually don't have think times and delays). But this is a rule of thumb not a commandment. if I run Reliability and Performance monitor and check out the health of my PC , and I find it reasonably healthy then I can increase the threads, if not I'll reduce them. I also verify visually that the response time that JMeter seems to be giving me (by directly checking the JTL or CSV file being generated) is approximately the same as if I accessed the page(just the html using a proxy or browser sniffer) from a browser running on a different machine. We did have a case where a tester used LoadRunner to generate loads for 2000 users from a single machine which started giving him average response times of > 2 minutes but people who accessed the site saw response time of 15-20 seconds. The machine which was generating the load couldn't handle these many threads/sockets and was the bottleneck rather than the application being tested. This is also dependent on your test scripts and your application under test. Test scripts that have delays between requests will obviously be able to handle larger number of threads (because there wont be as many concurrent threads).
Run separate instances of JMeter (instead of master-slave) - Jmeter allows you to run a test in master slave mode (With 1..n slaves). The advantage of this that you get all the results at the master as well as the start times etc can be more or less synchronized. However the overhead has now increased , as well as the problem that supposedly this bit of JMeter is not designed too well. Its better to run separate instances of JMeter and combine the results (if CSV is chosen as the output file format, then this is as simple as file concatenation, if XML you need to do a little more work, but its easy as well). You could split up the test itself (e.g. 100 users browsing 100 users logging in could be split up into two jmeter tests) or run the same test with some threads per machine.
Run in command line mode, Disable all Listeners and preferably use CSV as your output format. Understand JVM options. JMeter is a java application , if you want the most out of your JMeter client you should run it light and you should know how to tune a java application (the Java Heap especially).
Understand your actual requirements. Far too often testers don't know the difference between logged in users and concurrent users. Or think times. Or how http works. Or how their app works. Or why input data to the test should be varied (our App cached data and try as we might we couldn't get the tester to understand that if all the concurrent users browsed the exact same page, everyone except the first user's will get really fast times). Second normal users take a lot of time to read a page, fill in form fields, pause between requests, drink a cup of coffee whatever, The number of user's with an active session is always greater than the number of user's actually doing something at the same time (and in a lot of cases far greater). Check your site with realistic (plus safety margin )numbers. If there are 100 concurrent users at peak times in your system then check with numbers close to it first before you want to test out 10,000 concurrent users
Performance testing goes hand in hand with tuning. Sometimes questions are asked of the sort 'I increased the load , I get SocketExceptions or the Applications responds very slowly , help!'. Well this is what you expect your test to tell you right? You now need to check why your application is responding slowly , using logs, profilers whatever. Rule of thumb tune , test, tune, test, tune, test, tune as often as you can.

Performance Tuning tales

2009-08-23T22:07:00.000-07:00

When you ask a J2EE guy something about performance tuning you'd probably get something that includes JVM tuning heap space, survivor ratio's , types of GC or you might get the don't use EJB , minimise the number of remote calls , or the use Hibernate , use Prepared Statements, use Cache etc etc. Interestingly enough for my current project we had implemented most of the above, tested it locally and seemed to get <4 second times for most pages. And we missed a big problem (hint it's a web based system) , can you guess?
The system responds in under 4 seconds when accessed locally. When accessed through a browser based in New York some (Can you guess?) requests take 20+ seconds. Luckily we did run external tests as soon as the production systems were available so we found this out a couple of months before the system was actually released.
In any case , in hindsight as soon as we knew the problem existed it was relatively easy to guess and verify(YSlow) the problems. We only had few small images so we knew the problem wasn't there.
a. Whats good for development isn't necessarily good for deployment. e.g. We do normally split CSS files , Javascript files and include them separately. Yes a browser will cache these files ( you do add Etags don't you?), but the first request and HTTPS will be slow if the browser must make these connections (normally at the most two at a time). - We used Yahoo and ANT to combine the CSS into one file and the Javascript into another (at build time) drastically reducing the response times
b. GZIP. Creating Gzipped versions of the files and dynamically gzipping all the content (a flag on the webserver) also brought down the times.
Just doing the above brought down the times to under 4 seconds even when the site was accessed remotely on limited bandwidth clients.
Moral of the story : Always test , never guess when it comes to performance tuning.

Testing

2009-08-19T22:08:00.000-07:00

There are various kinds of test's that can be performed on a system , but it looks like most engineers don't agree on the definition of the test. For e.g. in my current project a 'unit test' tests out the end to end functionality but it must be performed by a developer i.e. any test that a developer performs is a unit test!
In any case this is my usage

Unit Test - Generally meant to represent that only a small subset of the code is under test. You might find a few fanatics who argue to the effect that anything that goes to the database isn't a unit test, anything that needs an external interface isn't a unit test etc etc. Ignore them. The key is that the test might not exercise the flow as it finally would. There might be mocked interfaces, hardcoded data. Generally written by developers and generally easier to automate. Unit tests are also low hanging fruit, and contrary to what most agile engineers will tell you, are pretty much useless expect to impress some manager with 'We have automated unit tests!' , 'We have continuous builds with JUnit reports!','We have 99% code coverage!'. The quality of these tests written is mostly poor (they are written by developers after all), the test data provided rarely covers boundary conditions, invalid data, exceptional conditions. Yes Yes I know you should have high code coverage. It's quite easy to game the high code coverage (which is what happens when metrics are imposed by management) but even if it wasn't, the code coverage is only as good as the code and the test.
e.g. function int add(int one, int two) {
return 4;
}
Test function with add(2,2), add(-1,5) , 100% code coverage , automated unit tests all green , mission accomplished!?
i.e. If the developer doesn't account for the boundary conditions in code, he isn't likely to test out those scenarios either.

Integration Test - Generally tests out the interface between multiple systems, though the data might be faked. If there are only two types of test that you can carry out on your system then this is one of them.
These tests are extremely hard to write (or atleast make it repeatable), and are very very useful. The earlier you can have these tests up and running on your system, the better the quality of your system. These tests are hard to write not because of technical problems(which exist) but because of people issues. Different systems are normally run by different teams (sometimes even external to your organisation) have their own schedules , develop at their own pace and make their own assumptions and are notoriously non co-operative. Technical limitations are normally due to non repeatable time sensitive data which either needs the data reset to a known state or data created from scratch.

Functional Test (End to End test). - Tests out functionality from a user's perspective. This is the second type of test that must be run and the earlier you can run these tests in your system the better the quality of the system is. Normally nothing is faked , actual data is used for the tests. When these tests are performed by a business user / stake holder these become Acceptance tests. These tests are extremely important and difficult to fully automate. e.g. These would also include UI tests (this page doesnt work on my browser!).

Performance/Load - Functional tests run in parallel (normally a good mixture) with varying concurrent users. Easy to do if the functional tests can be automated. In most cases difficult to simulate (especially if the system is already live, easier to do the first time). Depending on the duration you run this is also by different names. Some organizations refer to long running tests as smoke tests. (used to smoke out memory leaks) whereas some organisations use smoke tests to refer to some important functional tests that are run to check that no major errors exist.

System Tests - Used to check that all systems are up and running. Not really a test category by itself , but useful to abort test runs.

Google this!

2009-06-25T15:10:00.000-07:00

Life Science Research
Clinical Diagnostics
Spectroscopy
Process Separations
Food/Animal/Environment Testing
Life Science Education

Fixing URLs

2009-06-10T20:45:00.000-07:00

Motivation
For some tests, the URL to visit is extracted from the previous request using one of the PostProcessors (usually the Regex PostProcessor). However a URL containing parameters should normally have ampersands(&) escaped . This causes problems because JMeter will not automatically unescape these URL's

Solution
Use a javascript function to unescape the urls.

Sample
Assume that the Regex Post processor has extracted the url into a variable named returnUrl, then in the next HTTPSampler (Path field), instead of using ${returnUrl} use the function below.
${__javaScript('${returnUrl}'.replace(/amp;/gi\,''))}

This will simply replace amp; with a blank string so that a URL of the form http://www.yoursite.com/page?param1=value1&param2=value2 becomes
http://www.yoursite.com/page?param1=value1&param2=value2

Note that we don't need to do this for encoded values like %2F or whatever because that is taken care by the webserver

First Weblogic Portal Journeyman - WooHoo

2009-06-10T10:22:00.000-07:00

Data DrivenTesting(from a database)

2009-06-09T20:26:00.000-07:00

Motivation
Data to a test needs to be varied , however the input needs to be constrained to a set of values(normally in some database table). E.g. check the prices of the top 10 items. Here we need to provide 10 item ids , however if these values are kept in a CSV file , then the file needs to be repeatedly updated. There needs to be a way to get these values to the test at runtime

Solution
Use the JDBC Request Sampler to fetch the data at runtime. Use either the Save Responses to a file or a BeanShell post processor to write the data to a file. Finally use the CSV Data Set Config to read this data.

Sample

Varying the data to the test

2009-06-07T14:54:00.000-07:00

Motivation
The same test's need to be run, but the data we pass to it must be varied.
In addition further tests may need to change their behavior depending on the data. E.g. A user registering may provide different data , and subsequent screens may change depending on what the user has entered or may be skipped altogether. A common scenario is when a user registers to a site has various options he can choose from , and we need to test the behavior of the site for different combinations

Solution
JMeter provides multiple ways to vary the data
a. Use of User Parameters [1]
b. Use of Variables [2]
c. CSV Data Set Config [3]
The solution we will use is Option c.

The advantage of using CSV Data Set Config is that the data is externalised from the test , and can be updated by any user including a non technical business person. By making the assertion a part of the data, users can add more tests without needing the test itself to be modified. The other advantage of a CSV Data Set Config over a User Parameters pre processor is that the number of items that will be tested is fixed independent of the number of threads you will run (assuming you write the test in that fashion) OR can be made dependent on the number of threads. User Parameters is more closely tied to the number of threads.
e.g. If you wanted to create 10 distinct users , you'd only have 10 rows in your CSV data set config and you could use 1 to 10 threads. But if you needed to do this with User Parameters you'd probably need to specify exactly 10 threads.

So the solution takes the form of
a. Create a CSV Data Set Config element and point it to the CSV files.
b. Create your tests to use this data
If you want as many tests as you have rows in your CSV file, then you can either end the thread or use a Loop Controller an check for the special value ''<EOF>"

Sample

References
[1] User Parameters
[2] Variables
[3] CSV Data Set Config

Testing multiple environments with JMeter

2009-06-05T19:21:00.000-07:00

Motivation
Most projects have a number of environments through which the code moves. e.g. A Development environment, A Test environment, A User Acceptance test environment, A reference environment and finally the Production environment. Hence the same test script is to be targeted to multiple environments. In theory all of this is automated , and anything that succeeds/fails in one environment would succeed/fail consistently in all other environments. However environmental differences and human errors almost always cause one to hear "But it works on my machine"
Whats needed is a way to run the same test against different environments easily
Solution
An assumption we are going to make here is that these tests are automated and would be run from the command line, in our case using ANT.
The solution has to address two basic requirements
a. Parameterization of the various environments
b. You should still be able to run the test in GUI mode (when you are modifying/extending the test)
To implement this we will use JMeter properties[1] and use normal ANT [2],[3] features.

While writing the HTTP tests, add an HTTP Request Default element

If the Server name or IP is to be varied then enter
${__property(run.server,,yourdevserver.com)}
This will look for a property named run.server , but will use yourdevserver.com if no property is specified. For the third parameter, use the server against which you want to run the test while running in GUI mode.

Finally in your Build Script that you use to run Jmeter

<jmeter jmeterhome=".." testplan="${run.test.plan}"

resultlog="${report.dir}/${run.test.report}-${run.env}-${DSTAMP}${TSTAMP}.jtl">

<property name="run.server" value="${run.server}" />

</jmeter>

where the ANT property run.server can be varied to run this against different environment.

Sample

TODO attach ant and jmx file.

References
[1] JMeter Properties
[2] JMeter Ant task
[3] Ant Manual

JMeter Prologue

2009-06-05T18:57:00.000-07:00

I've been testing my existing work application using JMeter for the last 6 months , so what follows are a series of posts that I think isn't covered in most documentation or available online easily or in the same place (I had to figure out these solutions myself). However there is a caveat to that, the solution may not be the best possible one , may not be efficient and there may be more elegant ways to do what the solution does. In which case please inform me. However the solutions do have one thing in common, they worked for me. Your mileage may vary.

The rest of the post is a rant , feel free to move on
A few observations on a Friday(with a tip of the hat to BusyBee)

That people who keep asking for Unit Tests don't have a clue about testing web based applications.

That data driven integration testing between multiple systems is damn hard to do , but boy is it satisfying when you actually find a defect due to it, and boy is it worth your time to develop these tests.

That open source tools are so much better than some of the paid commercial tools. Except when it comes to presentation of the reports. Which surprisingly is also the difference between a Techie and a Manager.

That there still is no tool to truly perform visual tests on a website.
And That someone who could develop such a tool could make a lot of money.
And that someone wont be me.

That there is no time to test all the normal cases, much less the weird corner ones. and that there would always have been plenty of time in hindsight, if we had just gotten our act together earlier

That a successful test is the one that fails in local,development, test, QA , UAT environments. That a test thats succeeds in the above environments but fails on the production environment is a pain.

That a web based test should never underestimate the ignorance of a user.

That developers do not make good testers. And that most developers are better testers than the official testing team, especially when we test other developers code. And that scares me.

Almost there

2009-04-30T13:35:00.000-07:00

Updated

Prototype

2008-02-02T18:25:00.000-08:00

Name : Prototype

Context : Obtain a copy

Consequences :

1)Add remove products at runtime
2)Specify new objects by varying value or structure
3)Reduced Subclassing

4)Configure an application with class dynamically

Examples

Anything implementing clone(),

Class

Factory method

2008-02-02T18:21:00.000-08:00

Name : Factory Method
Context : Hide the implementation class but defer the instantiation to subclass

Consequence
a) Provides hooks for subclass

Examples

Calendar.getInstance()

Connection conn = java.sql.DriverManager.getConnection(uri, name, password);

Abstract Factory

2008-02-01T18:02:00.000-08:00

Name : Abstract Factory
Context : Creating Related family of objects
Example: Toolkit(awt)

java.sql.Connection (sort of creates Statements and PreparedStatements and CallableStatements where each is specific to the database)
Consequence
1) Exchange Product Families
2) Supporting new kinds of products
3) Factories are normally singletons! But they need not be.

4)Note that every factory must support every type of object to be created. This is sometimes problematic.

Singleton (Creational)

2008-02-01T17:03:00.000-08:00

Name : Singleton
Comment : Probably the best known (atleast if you go by answers you get in interviews to the question 'Patterns you have used') of the design patterns and probably one of the text book cases of people ignoring understand the context and understand the consequences when it comes to this pattern
Context : Need only 1(or a fixed or controllable number of instances) (e.g. memory consumption , or heavy initialization like reading a file,a requirement - there is only one Neo, a pre java 1.5 enum such that == can be used )
Examples : Single Instance (java.awt.Toolkit, Inspector)
Consequences :
a) Controlled access to state
b) Permits a variable number of instances (e.g. access to any object pool)!
c) Watch out for real world multiple managed servers, clusters etc(older versions of Expresso framework used an in memory primary key generator as an incremented singleton - no use in a cluster. Real world naive code that uses a timestamp plus ip address as unique key through a singleton)
d) Watch out for synchronized access for loading the singleton, thread safety, shared state.
e) Watch out for serializable (might break singletoness if you dont follow effective java)
f) Watch out for static block errors during initialization(if initialising eagerly). Will cause NoClassDefFoundErrors.
g) Watch out for being unable to reset a singleton (e.g. a singleton that reads a file and stores it.If your not careful , changing the file needs a bounce of the server)
h) Watch out for singletonitis. i.e. Making classes singleton even when there is no shared state, no performance or initialisation penalty.
i) Watch out for problems testing the singleton

With all the Watch out's is this pattern worth the trouble? read the context!

Design Patterns Course

2008-02-01T16:22:00.000-08:00

The following series of posts deal with a design pattern course I conducted for a few colleagues of my team who insisted that I knew enough to conduct such a course(I don't). One of the problems I had faced while attending the design patterns course conducted by probably the best Technical Manager I've worked under (Homi Bharda) was that practical examples were hard to come by, where you could see the usefullness and though the Design patterns book by Erich Gamma did point to some examples they almost always dealt with GUI heavy , stateful , highly object oriented programs while at that stage I dealt with mostly stateless CRUD web based applications.
I resolved that any course I took would show practical examples and what better example than the Java API which was quite familiar to my colleagues. I also had a lot of problem differentiating between patterns. Some of the Factory patterns looked alike. Some of the patterns like Command and Strategy looked alike. I resolved to have some comparisons at least.

What these notes dont have is my bitterly cynical comments e.g. of people who code such that all their classes have a suffix which is pattern e.g. ObjectCommand uses an ObjectFacade uses an ObjectFactory creates an ObjectService which uses an ObjectDAO and returns an ObjectAdapter which wraps an ObjectDecorator which wraps a ObjectDTO (Disclaimer I have done this too). This literally is the developer's cry look Im a designer , I use patterns , I even know their names! But ask them Give me some examples of design patterns in the Java SDK and we have a) Blank Stares b) Singleton c) A very very few Decorator.
So anyway other than the above rant , very few cynical comments

General Notes
What is a pattern ?
A Name, A Recurring problem with context, A recommended Solution(or way ahead), Consequences and Alternatives
The bit a lot of people miss out is the Context and the Consequence. A pattern cannot be used irrespective of the context . This causes problems like 'Singletonitis'. A pattern always has some consequence. A pattern probably does have alternatives which are to be evaluated.
A pattern need not always be up front design, you can arrive at many patterns with a judicious use of refactoring and some simple principles notably DRY.
Patterns are also sometimes subsumed by the language or the platform (e.g. Factory in Visual Basic COM and Iterator in java) and some people refer to patterns as deficiency in the language / platform ( I dont agree!).
Anyone who does anything with Design Patterns has to refer to Erich Gamma's et al book , and thats what I have based this on including the organisation of the patterns as Creational, Behavioral and Structural. Its quite difficult for novices (like me) to differentiate what between what constitutes a behavioral or structural pattern so dont bother right now.
Also note that any Object oriented system has available to it the mechanisms of Polymorphism and Inheritance (Implementation of Interface). Therefore all diagrams of Design patterns will look similar in a lot of case. it is necessary to understand the nuance and differences and context.
And lastly in a web project , differentiate between building a framework and solving a problem at hand. Its very easy to get carried away and over engineer and its as easy to under engineer, and it's also simple to shout TDD, refactoring , agile. Ultimately experience is what you use to decide whether you need a pattern upfront or later, Refactor or not. And anyone who tells you otherwise is either a manager or a consultant.

Testing

2008-01-25T16:18:00.000-08:00

You really appreciate the value of tests (especially regression) when you have to fix some code which is complex and is used in multiple places and you aren't sure of the various conditions under which it gets invoked (or even what the result should be). Unfortunately it is sometimes difficult to convince management. We asked and were denied initially anyway , for a separate environment to create and run tests. The separate environments were needed because of the extreme complexity of the business needed, in my opinion, an environment where the initial state is exactly known , otherwise there is no way to accurately predict the outcome of running time sensitive, old data sensitive scenarios. However the cost of maintaing said environment was deemed to be too high. Im not so sure. anyway we are getting an environment so yay, let the games begin.
It's also at time like these that you realise , how out of touch with reality , the TDD/Agile/ JUnit testing people that you see are , because they never seem to discuss anything but unit testing.
I cant see unit test's adding much value to the system I work in , but oh what wouldn't I give for a set of repeatable integration and functional tests.
Other than the really good Java Next Generation Testing book I havent seen the Testing books deal with real world stuff. (Did anyone else think that the Junit cookbook example of Money was representative of what people test in an automated way in real life)

So here are the things that make the testing complicated
a. The system run's on a cluster. Problems have been reported on the cluster , some of which have found to be environment related and some are code related. Would anything other than in container testing on representative environments find this out?. This deals directly with the people advocating 'mocking' interfaces, not running in container, having fast running tests etc.
b. One of the defects was caused when two unrelated processes ran at exactly the same time and on the same server of the cluster. The processes shared infrastructure code but were business wise and code wise unrelated. How would a test have found this out , other than sheer luck? Note that junit as a framework is really bad. Again this is not a complaint about JUnit, just that you dont see people discussing concurrent testing of unrelated data.
c. A complex Architecture. A portal sends data asynchronously (JMS) to a WLI system which has multiple processes , which in turn needs a lot of system data (histories) to perform calculations, makes its business decisions using a set of business rules which are User Configurable(using a totally different system) and then persists this data to a database.
Assuming you want to test that everything works end to end, is there any other way other than restoring the system to a known point in time(so you know what the out come should be?). The catch again is that running the same test on the exact same data might not still return the same results because the results are time sensitive (e.g. there is a different contract for 2007 as compared to 2008 so running the test in 2008 will give you different results).
d. The results depend on the sequence of tests. This isn't a code problem , this is exactly how the business works. Other than creating different data for different cases , this is difficult to solve. In addition you do want to also test what happens when a particular sequence of events occur ( a claim is in , it is paid, the cheque is sent, the claim is adjusted, a new check is sent , the first one is returned, now assert). How is it that the unit testing folks shout themselves hoarse that tests should be independent of each other. I know I know , what I have described isn't a unit test. But you see , the individual bits work. The sequence fails.
Oh I can go on. But there is an India v/s Australia cricket match. Go India!

Debugging

2008-01-25T16:06:00.000-08:00

Due to circumstances beyond our control , we've had to take up fixing someone else's code. The business rules are complex (which means the code must be so). There are no existing tests (this is a consequence of both the complex rules and a complex architecture , the latter could be simplified if it were not for the fact that the project is years late). Now the TDD guys are probably jumping up and down at this stage , pointing out that the lack of tests is the reason why everything is so complex , just ignore them , we'll get to these folks later. The bottom line is there are no tests, we are trying to create some regression tests, that will take time , meanwhile there are still bugs to be resolved. I was assigned one of them.
It took me 2 hours to setup the environment . Ones a weblogic portal domain and Ones a weblogic integration domain. The build for the portal takes about 30 minutes after which weblogic workshop duly crashes. I have to point my servers to a configuration other than their default which needs me to mess around manually. It takes another hour to understand what the defect is. it takes some more time to verify that the tester is indeed right, the system is doing something wrong. It takes some time to identify the areas of code that are possibly the cause. it takes a couple of hours to understand the nuances in the mostly undocumented code.
I then have to go and ask another dev, whether the testers expected results are correct, and it turns out they aren't. There is a defect and what the system currently does is wrong , but what should the system do is undocumented and unknown. We could make a guess but this impacts other systems so we need them to confirm. A meeting is scheduled.
The meeting lasts 2 hours. Some issues are resolved, Some new ones are raised. Some have to be followed up with other people. Armed with some more knowledge I look through the code. I get a sudden inspiration, I test out two scenarios , it looks like I'm right.

The defect is fixed by modifying all of two lines of code. Total time to fix defect ? 1.5 days.

Useful links

2007-02-14T08:56:00.001-08:00

Advanced Pageflow
CM API usage example
Faults and Contingencies
WLST offline

Weblogic Portal interview questions

2007-02-14T08:46:00.000-08:00

The following are the Portal interview questions that ive used or kept or have been asked(in no particular order , and no answers either :) )
I do not include questions (e.g. what is a nested pageflow) that can be answered with google.
Also see Weblogic Portal interview questions - II

In what situations can you not use a UUP ?. What factors determine choosing a UUP?
What are the different techniques you can use for Inter Portlet Communications?. What are the pros and cons of each technique?.
Would you choose 1 desktop with entitlements or multiple desktops? Why?
What are the options for security in Weblogic Portal?
Can you change a LookAndFeel at runtime ?, if so what are the alternatives?
What is the impact of using AJAX in a portlet (out of the box or otherwise)?
Why should pageflow variables be serializable?
If your using a file repository for content and you have a webserver in front of the application servers who serves the static files?
When do you use the refresh action?
What happens if the validate method of your form bean throws an exception
How do you use the struts validation framework with BEA pageflows?
The BEA pageflow documentation mentions that there is only 1 current pageflow. And that the current pageflow is destroyed when a new one is accessed. is that accurate?
What all features should a screen scraping portlet possess?

To be continued...

Weblogic debug flags

2007-02-13T06:48:00.000-08:00

Reproduced from Hussein Badakhchani's Blog and comments
weblogic debug flags(8.1)
IIOP
weblogic.iiop.ots
weblogic.iiop.transport
weblogic.iiop.marshal
weblogic.iiop.startup

JDBC/Data Sources
weblogic.JDBCConn
weblogic.JDBCSQL
weblogic.JDBCConnStackTrace

JMX/Deployment
weblogic.commoAdmin
weblogic.commoProxy
weblogic.deployerRuntime
weblogic.MasterDeployer
weblogic.deployTask
weblogic.deployHelper
weblogic.MasterDeployer
weblogic.OamDelta
weblogic.OamVersion
weblogic.slaveDeployer.semaphore
weblogic.slaveDeployer
weblogic.ConfigMBean
weblogic.ConfigMBeanEncrypt
weblogic.ConfigMBeanSetAttribute
weblogic.management.DynamicMBeanImpl
weblogic.management.DynamicMBeanImpl.setget
weblogic.mbeanProxyCache
weblogic.mbeanDelete
weblogic.mbeanQuery
weblogic.MBeanInteropList
weblogic.mbeanProxy
weblogic.registerMBean
weblogic.getMBeanInfo
weblogic.getMBeanAttributes
weblogic.addDependenciesRecursively
weblogic.MBeanListener
weblogic.application
weblogic.deployer
weblogic.appPoller
weblogic.appManager
weblogic.BootstrapServlet
weblogic.fileDistributionServlet

Application Deployment
weblogic.J2EEApplication
weblogic.application
weblogic.appPoller
weblogic.appManager

JTA
weblogic.JTAGateway
weblogic.JTAGatewayStackTrace
weblogic.JTA2PC
weblogic.JTA2PCStackTrace
weblogic.JTAHealth
weblogic.JTAPropagate
weblogic.JTARecovery
weblogic.JTAXA
weblogic.JTAXAStackTrace
weblogic.JTAResourceHealth
weblogic.JTAMigration
weblogic.JTARecoveryStackTrace
weblogic.JTANaming
weblogic.JTATLOG
weblogic.JTALifecycle

EJB
weblogic.ejb.cache.debug
weblogic.ejb.cache.verbose
ejb.enableCacheDump
weblogic.ejb20.cmp.rdbms.debug
weblogic.ejb20.cmp.rdbms.verbose
weblogic.ejb20.persistence.debug
weblogic.ejb20.persistence.verbose
weblogic.ejb20.compliance.debug
weblogic.ejb20.compliance.verbose
weblogic.ejb.deployment.debug
weblogic.ejb.deployment.verbose
weblogic.ejb20.dd.xml
weblogic.ejb.deployer.debug
weblogic.ejb.deployer.verbose
weblogic.ejb.verbose.deployment
weblogic.ejb20.ejbc.debug
weblogic.ejb20.ejbc.verbose
weblogic.ejb.runtime.debug
weblogic.ejb.runtime.verbose
weblogic.ejb20.jms.poll.debug
weblogic.ejb20.jms.poll.verbose
weblogic.ejb20.security.debug
weblogic.ejb20.security.verbose
weblogic.ejb.locks.debug
weblogic.ejb.locks.verbose
weblogic.ejb.bean.manager.debug
weblogic.ejb.bean.manager.verbose
weblogic.ejb.pool.InstancePool.debug
weblogic.ejb.pool.InstancePool.verbose
weblogic.ejb.swap.debug
weblogic.ejb.swap.verbose
weblogic.j2ee.dd.xml

General
weblogic.debug
weblogic.kernel.debug
weblogic.debug.DebugConnection
weblogic.debug.DebugRouting
weblogic.debug.DebugMessaging
weblogic.debug.isLogRemoteExceptionsEnabled
weblogic.StdoutDebugEnabled

WLI
wlc.debug.signature
wli.bpm.client.security.debug
wli.bpm.studio.timeprocessor.debug
wli.bpm.studio.debug
wli.bpm.server.common.timedevent.debug
wli.bpm.server.common.xmltemplate.debug
wli.bpm.server.eventprocessor.addrmsgdebug
wli.bpm.server.eventprocessor.debug
wli.bpm.server.jms.debug
wli.bpm.server.plugin.debug
wli.bpm.server.workflow.debug
wli.bpm.server.businesscalendar.debug
wli.bpm.server.busop.debug
wli.bpm.server.workflow.action.taskduedate.debug
wli.bpm.server.workflow.timedevent.debug
wli.bpm.server.xml.debug
wli.bpm.server.xslt.debug
wli.bpm.server.workflow.start.debug
wli.bpm.server.workflowprocessor.debug
wli.common.server.errorlistener.debug

Messaging Bridge
-Dweblogic.debug.DebugMessagingBridgeStartup=true -Dweblogic.debug.DebugMessagingBridgeRuntime=true And two others for stdout and stderr : -Dweblogic.Stdout= -Dweblogic.Stderr=

SSL
-Dweblogic.security.SSL.verbose=true -Dssl.debug=true

-9.x available via console plus
webservices: -Dweblogic.wsee.verbose=*

6.x,7.x
http://wldj.sys-con.com/read/42733.htm

RMI debug flags
java.rmi.server.logCalls=true
sun.rmi.loader.logLevel=[BRIEF|VERBOSE]
sun.rmi.server.exceptionTrace
sun.rmi.server.logLevel=[BRIEF|VERBOSE]

More straws

2006-11-19T08:21:00.000-08:00

http://jeffsutherland.com/scrum/2006/11/scrum-supports-cmmi-level-5.html
http://jeffsutherland.com/scrum/2006/11/is-cmmi-worth-doing.html
From the inventor of scrum
(Disclaimer i am working in a CMM Level 5 company and its possible indeed probable i am biased)
So the agile management folks see value in CMM level 5 principles. After all what could be incorrect about
Level 1 - Uncertainty. Success depends on individual effort.
Level 2 - Awakening. Basic project management practices are established.
Level 3 - Enlightenment. Standard process throughout organization.
Level 4 - Wisdom. Detailed metrics are collected and evaluated.
Level 5 - Certainty. Continuous process improvement via metrics feedback.
Wouldnt everyone want to be level 5 , certain their project succeeds?
But isnt agile about people above process?
Doesnt agile tacitly imply that if the people arent good no process is going to save anything?
Certainly there isnt anything in agile that forbids collection of metrics and forbids improvement. Indeed doesnt the automation of everything in sight actually allow you to do this.
But but but and herein lies the problem, the practice of collecting metrics is so perverted (see the timesheets which have a task called filling in timesheet) that almost always means that people will fudge it.
It also looks like agile management consultants have discovered there is money to be made by coaching people to be agile scrum cmmi pcmm bs577 compliant. Or whatever.
So will the agilisits be enlightened? Does a standard process have more value than a tweaked one? Whatever happened to the these are guidelines and must be changed for your project, use whatever works for you?

Agile no more

2006-11-12T04:05:00.000-08:00

Its amazing to see how developer friendly practices are perverted into management friendly practices as soon as a pack of managers are let loose on them.Right so lets begin at the beginning Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan (http://www.agilemanifesto.org)While we could quible over whether we collaborate with customers or negotiate with them , its fairly reasonable to assume that no software engineer is going to disagree with the spirit of what was said here.All of the above principles are developer friendly. No more follow that CMM level 5 process, No more according to my project plan we should have finished 53.23% percent of project. are we there yet?But look agile has crossed the chasm and what do we haveAgile consultants who have the "you must tailor agile to meet your needs" that means you need an agile coach or a scrum master whatnot because evidently if you tried to do it yourself your project would fail because you would do agile wrong.Oh look agile projects do need management. And it must be something new otherwise how would we consultants ever earn money?Heres a new management technique. Lets call it Scrum. We cant obviously have a position called Project Manager , lets call it scrum master. Lets not call it a requirements document, lets call it a product backlog. Lets not call it an iteration , lets call it a sprint. Oh yes how about burndown charts, yep sounds like something people would pay money to be taught what it is. Oh and wait we need to know what our teammates are upto every minute. Lets ask for status updates every day. Oh and we need to have data on the minutest of things. Hmm we cant call it a timesheet but we'll make them fill up the hours for every minute task and we'll make them constantly reestimate each task.Ok so thats an exaggeration.But think about it. Say the functionality on a website was show orders for a user. What would a developer say (lets assume the overall architecture and software and platforms have been decided). Mostly something like ok it would take about a week for me to do it. If you fell behind you 'd sit up for 2 hours more if you were ahead you'd quit and go home early. In about a week you'd be done if you were reasonably competent. Developer friendly? Id think so. if there were problem you'd escalate as soon as you knew of it. If the problem was technical you'd ask your peers at lunch or tea.But thats not what's being practised.1 week as a coarse estimate wont do any more. You have to provide finer tasks so that you know if you are in red. You have to track at an hour basis for the tasks. You have to give an update every day. You have to work uniformly for 8 hours every day because supposedly that increases productivity. Bah. Anyone who's coded knows that if the problem is interesting and you've started on it , you wont stop unless you've solved the problem Or atleast its that way for some people.Not all people are alike.And so we come to the original manifesto. For the agile projectsIs it individual and interactions over processes and tools ? Probably not. look around the debates in agile . To pair program or not to (process) , to TDD or not to (process), unit test all code (process), daily standup calls(process), use wiki for collaboration (tool), use junit(tool) etc etc
Working software over comprehensive documentation ? Product/Sprint Backlogs, Burn down charts, User stories (all documentation)
Customer collaboration over contract negotiation ? Ambiguous at best. And certainly i have yet to see any agile consultants who havent negotiated contracts.
Responding to change over following a plan? No more and no less than before, perhaps evidenced by the fact that the manifesto is signed in 2001 and 5 years hence there has been no change to it. Lessons learnt in the 5 years have definitely not caused the manifesto to change. Rather any failures are met with the standard "you arent doing it right"
Perhaps the problem is in the first item in the process. Individuals. And individuals are flawed and biased.

More Agile

2006-08-27T05:38:00.000-07:00

Burst out laughing when I read
“Soon all of our singletons were composite, we could use the factory method on our abstract factory to get the prototype of the builder which would make a proxy for the decorated adapter that acted as a facade for the visitor which took us to see all of our singletons”- http://jroller.com/page/rolsen?entry=turn_it_up_to_11
Since it reminds me of a certain project I was in where the chief designers had obviously come out of a design patterns course and so there were *Factory aplenty which then lead to the creation of a class named FactoryFactory and the clients of this API who ofcourse had their own factory came up with FactoryFactoryFactory. Ok the last class is fictional , but the one before that is an actual class.
And so yes the latest rage is Agile and Scrums and what nots. I remember reading Kent Beck’s book on extreme programming about 5 years ago when agile had not yet caught up here in India, agreeing vehemently on the accept change and disagreeing just as vehemently on pair programming (some people interpret this as side by side programming, a practice I wholeheartedly endorse , and not just for technical reasons, its how I met my wife to be :-) )
As the years went by and I saw more and more of the strict waterfall model , the im not going to start design till requirements are signed of, im not going to start development till design is signed off, I realized the agilest were right , waterfall like practices are no way to run a project. Agile was the way to go. And though we didn’t practice all the various XP practices or Scrum or whatever, most projects I worked on were pretty much flexible, with changes being accepted after the date for code freeze (which magically thawed)
Oh just to clarify , we are a service/project oriented organization , providing services to various clients. The client I work with now has decided to go Agile in a major way and to impress the client agile training is now mandatory for everyone in our group in our organization. And ive kept up with some of the articles that the agilists keep publishing. And so the inevitable conclusion. There is no practice in the world that people cant screw up.
So lets see where do we start.
First the proponents of agile itself. While they make the right noises (Each process must be tailored to what works for you), the answers given most of the time is ,(to quote cedric beust) “if you aren’t getting benefits from agile , you aren’t doing it right.”
Second the insistence by some on absolutely silly practices. Like Code the test first before you write the code. I cant and don’t want to code tests that don’t bloody well compile , I have to define the interfaces and classes implementing those first atleast! Though of course testing is good , automated tests are better, unit tests are ok, mock tests are ok, Integration tests are worth their value in gold. But try getting some agilist to talk about tests other than unit tests.
Then we have the silly rules that follow from the above. You must have 90% coverage in your tests cases, otherwise code is rejected. Heres a thought experiment. Write a function doubleIt , return the value 4 always and write a test case which passes in the value 2. You have voila 100% coverage. Beat that. No need to write more tests. If you impose silly rules , developers will work around them.
And then we have the other problems. Daily status calls, not exceeding 15 mins. Must be at the start of the day. Ok cool. Umm the delivery head is US based, the customer in UK and the dev team in India. Whats the start of the day? Oh agile doesn’t work so well for distributed teams! Well what do I do then go back to waterfall? Stupid.
And the ofcourse the famous if you aren’t doing practice x, you aren’t agile.
I think I prefer the time when in a project of duration of 1 month you could relax for 15 days and then work double for the next 15 instead of work uniformly for the 30 days.
Though people may say that quality in the latter case is better, I don’t think that’s necessary.
I don’t think agile works in the real world. Distributed teams are a fact of life. Multi vendor projects with conflicting interests are a fact of life too. I suppose agile would work with a closely knit team. But that’s fast becoming an exception in the areas we work.
And with new processes and positions (like agile coach) the Agile methodology is becoming as rigid as the other processes.
And a final thought whats remained the rule in software , good people can make any process work, incompetents can make any process fail.

Agile

2006-06-04T06:03:00.000-07:00

The other day I got a ppt from a friend who got it from someone else about a project in my company which is a “success” because how they implemented “agile methodologies” and how it helped and the practices they followed with general patting of backs and lessons learnt and improvements still to be made.
It’s a 75 slide power point.
Get the picture?
Or lets see the Extreme programming principles. On one hand embrace change and on the other you must follow x,y,z to have a successful project(with no guarantees of success). What happens if you don’t pair program or do test driven development or whatever else is the latest breakthough in software engg? Is it that you don’t embrace change?
I think agile is best described by uncle bob , “never be blocked”. And once you accept that, the agile manifestos, the programming principles the slide guides aren’t all that necessary. True they might help, but the problem is any rigid process (agile extreme or otherwise) causes blocks.
Write junits for everything. Unit test are really low value for money. Integration tests are very hard( especially if it’s a distributed system, developed by multiple disparate teams )over space and time. All the wonderful stubs and mocks so loved by the TDD guys fail when the stub and mocks are replaced by the actual system. Which isn’t to say stubs aren’t good, but they give you a false sense of security. And can hide flaws. For e.g. There was a defined web service interface. It wasn’t ready we mocked it. All was well. Till we ran it with real data. Turns out we get stuff like ##YOUSHOULDNTSEETHIS## the xml structure is the same , the web service has the same interface but surprisingly none of us thought to write stubs and mocks that returned elements with junk values.Or what to do in that case or even how to identify them.
You can and should violate principles if the need arises. That’s agile. Embrace change. a

The Fallacies of Design

2005-12-11T07:51:00.000-08:00

The Fallacies of Design
While reading the various UML/Pattern/Architecture/OO/XP/TDD books and literature, I think most authors forget some of the basic maladies that plague software projects. Or perhaps because there are no solutions for the problems, it is best to ignore them. So far there have been very few insurmountable problems as regards to the technologies involved in projects that I have worked on. They have always been workarounds , tactical pathworks and other hacks. But there are the other issues which can derail a project.
So here are the various fallacies I have seen

There is a best solution that solves your problems.

Systems that you need to integrate/interact with can be changed to implement the best possible solution

People within the project will cooperate with you. People outside the project will cooperate with you

The People defining the requirements know what they want. Once a requirement is signed of by said rule, it wont change

Budget is infinite, Time is infinite

Managers can manage, Designers can design, Analysts can analyze and coders can code.

The solution to any problem is a pattern/platform/approach/methodology

If the test cases pass and code coverage is > x% then there wont be problems in the wild

People will always tell the truth

Murphys law does not apply to software.

None of these fallacies are technical. They aren’t meant to be. Its rarely the technology that derails a project.

Design Patterns v/s Experience

2005-11-08T07:52:00.000-08:00

While reading through TestNG and comparing it with JUnit, it is obvious that TestNG is far simpler to use and at first glance looks to be designed better (or has more features). Note I haven’t written a single TestNG test yet.
And I remembered reading , when people complained about the limitations of JUnit that JUnit was designed as a framework that could be extended. And how JUnit had used all sorts of design patterns.
Which leads us to Design Patterns v/s Experience question. I do not suggest these two are mutually exclusive. The comparison of JUnit and TestNG is not strictly fair, and neither is this a feature by feature comparison, nor an indictment of design pattern approaches. Its simply that having JUnit (and its varied extensions around) allowed Cedric to create a better framework. Like not having to implement a class or interface for your tests. The different ways of running setup (as opposed to the design pattern used by Gamma and co.) . And while a catalog of Design Patterns exist, how do you quantify or catalog the Experience that allows you to select the correct Design Pattern?
Experience also teaches you when design patterns need not be used. (oh yes there are cases, but I leave that for a separate entry).
And if you had to choose a candidate with Design patterns on his CV(All candidates know factory,singleton,value object, dao and session façade) or one who seemed to have the experience, choose the guy with experience (This has nothing to do with 6 yrs experience and no DP in CV!)

Youth And Responsibility

2005-05-21T21:43:00.000-07:00

Its obvious to anyone who thinks about such things that the only hope for the future are the youth. The big picture aside, lets look at youth only in the context of the workplace. Its also painfully evident that no one trusts the young trainees just out of the trainee program.
We are of course polite. They dont have enough experience we say, we cant risk it - what if they screw up on production?. We(the experienced people, the guys who have seen it all) can't spend all our time reviewing what the trainees are delivering. In other words it is easy to find rational arguments which prevent us from delegating responsibility/tasks to the young 'uns.
The plain truth is that we don't trust them. Or we trust that they will screw up.
Which is a sorry state of affairs.
I need to break out of this mental block . And I think I'm still someone who trusts the younger people around him a lot more than others.
But it has to improve

The Anarchist Management Model

2005-05-01T02:18:00.000-07:00

One of the things that keeps occupying my thoughts these days is how much that we learn from life can be applied to our work.
One of the major problems in work can be summarised as Management. Most non managers would not hesitate in calling Management evil , and only perhaps were we really feeling charitable would we call Management a necessary evil. But Managment persists as a practice , even worse you have younger generations(am i really this old?) growing up wanting to be managers. You have people joining the project stating that they want a management role.
As such I'm against all sorts of management in principle. I think this has to do with the fact that as a form of governance I prefer Anarchy. And currently Management is a pseudo democratic dictatorship. You dont really have a say on who is your manager (hence a sort of dictatorship) but perhaps results /feedback can get the manager removed.
Id like to see a project where people take responsibility for their actions. Divide the work amongst themselves based on their competencies. Set targets for themselves and meet them. Resolve their problems without needing outside intervention.
Could it Happen ?

Culture and Design

2005-04-10T01:34:00.000-07:00

Does the culture we have been brought up with affect the design work we do?
Case In point.
One of the clients of my company is spending large amounts of money to attain homogenity in the technology/platforms/systems it uses to avoid the integration problems that arise with heterogenous systems.
My personal preference has always been to develop systems independently using whatever is best for that system. And then figure out ways to integrate them.
Both approaches have their shortcomings and advantages of course.

Im wondering if the fact that diversity that is so much a part of Indian life has anything to do with my preference. If so i'm grateful for it.

Interviews

2005-04-10T01:26:00.000-07:00

One of the things that is part of my my workstack is obliging Recruitment whenever they want some J2EE interviews conducted.
It is a highly painful experience.
On one side we have the questions that we are given to ask by Theoreticians who seem to be detached from real world programming (what is the output of i++ + ++i or how many overloaded constructors does the File object have - note these arent really questions we are supposed to ask , but you get the idea) .
On the other side we have the candidates. It looks like they come prepared to answer questions like the above. And because I always begin with what are the technical details of your project and what did you work on, 9 times out of 10 the candidate looks back with a blank stare. Or provides some inane details like There is a login screen , the user enters his password and is taken to his homepage. The End. I am not alone in this regard. Other people from my company have expressed similar sentiments. Perhaps the reason is no good candidates are applying to my company :-) .
This actually leaves only one way out if we want competent resources. Hire people out of college. Train them and then mentor them.
It remains to be seen how successful this will be

Loyalty

2005-03-13T05:17:00.000-08:00

Do you remember the time you were in school? The time when your school was the bestest there was , when all the people in your school were smarter than any other school , where the grounds were better to play in than in anyone else's school, where the teachers were the best and kindest, where the principals were stern but what the heck you liked them anyway.
I do.
One of the things you learnt in school was loyalty and that loyalty begets loyalty. And there was a thing called Honour.Do you remember the time the teacher asked who did whatever and everyone knew but no one said a word.? Atleast my school(and with that i include the campus , the teachers and my fellow students) did try to teach me that. But then i did tell you it was the best school.
One of the first things that struck me in the induction speech when i joined by company (5.5 years back) was the thing someone from senior management said. He said You should not be loyal to the company. He said the company does not even want your loyalty. It will do what is best for itself and you should do what is best for yourself. He went on to refer to the practice in my company of giving long service awards at the end of 3 years and joked about it.
The Achieving excellence session went on to teach you new keywords WIIFM WIIFU (whats in it for me , whats in it for us)
At that time i would rather listen to If(Rudyard Kipling)
If you can talk with crowds and keep your virtue,
Or walk with kings - nor lose the common touch;
I resolved that this was a case of keeping my virtue while walking with the crowd and thefact that the company doesnt care for my loyalty does not in any way prevent me from being loyal.
I suspect that i should have bowed to experience instead of indulging in youthful idealism.
So much so that the fact that i have stuck with my company for 5 and a half years is now a liability. Supposedly IT HR people feel that if a software engineer sticks in the same company for more than two years there is something wrong with him - lack of ambition, initiative whatever.
So why am i still in my company