JMeter and AJAX - Part III

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part III-1 - A diversion on how to use session ids between threads
Part IV - A more complex sample

In Part II we developed a sampler that would let us synchronize selected threads. I split that out because someday I might be tempted to actually write that timer. So lets now try and execute some AJAX requests in parallel. The demo site we will use is http://www.ajaxdaddy.com/demo-dhtml-autocomplete.html (Please don't run extensive load tests against this site - it isn't mine)
The site has an AJAX based autocomplete for countries. Access the site in FireFox, turn on Firebug and type au into the textbox and you can see the AJAX calls being made in firebug

We can see that two AJAX calls are made, one when we typed 'a' and one when we typed 'u' (There is an AJAX design pattern that applies here which doesn't seem to have been followed). So we will access this page and try to simulate two parallel calls.


The Init and Delay Sampler are from Part II. The Init sampler ensures only 1 thread does the initialization bits. The delay sampler has the rest of the code. After the Init sampler we just make all the threads wait so that no threads run before we are ready to begin (so don't use any ramp up time as that would be nullified here). Its essential the sync timer has a count equal to the count of threads in the threadgroup.
The IF controller is used so that only the main threads perform the request to the main page. Recall for 6 threads to run 2 at a time , we have the groups as Thread(1,2) , Threads(3,4), Threads(5,6). The even numbered threads will only be used for the AJAX requests. The Odd numbered threads will be used for the main sample and the AJAX requests. In our case the landing page needs to be accessed by the odd numbered threads. The even numbered threads will skip and land at the delay sampler waiting for their respective soulmates to join them(I'm currently waiting to be reunited with my wife hence the flowery language).
Anyway we have added a Gaussian random timer to allow the odd numbered threads to arrive at different times. When the odd numbered thread does arrive at the delay sampler, it along with its even numbered pair will be released to execute the Make AJAX request sampler (which chooses a word at random from the CSV file we have defined text,assert e.g. a,australia).

Hence two requests will be executed in parallel for the AJAX request sampler. If we had further requests after this we would have added another IF controller(only odd numbered threads) and coded the samples under this. if we wanted 3 requests in parallel we would have to adjust the Init sampler to say we want groups of 3.
Run the test. Though the samples may appear interleaved , note down the sample start times of the groups of threads that were supposed to execute in parallel, they will be more or less the same.

But I have cheated. The samples in parallel didn't really depend on each others data. The AJAX requests were exactly the same , differing in data. The AJAX requests didn't depend on the user's session thereby reducing the complexity of the script.

Lets attempt an example which needs the above. Onwards to Part IV (as soon as I find a good demo!)

JMeter and AJAX - Part II

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part III-1 - A diversion on how to use session ids between threads 
Part IV - A more complex sample

As mentioned in Part I , the most elegant way to simulate AJAX in JMeter would be to customize the HTTPSampler to allow child HTTPSampler requests to be executed in parallel.
Since customization of JMeter core isn't an option for me, we'll try and simulate this directly without customizing JMeter. Note that this is a toy, done for amusement rather than any real solution to an existing problem.

The first thing we should note that anything that we need done in parallel in JMeter needs its own thread. Suppose we have some requests that make two AJAX requests in parallel then the way we are going to work this is
Thread odd numbered(1,3,...) - Main thread that executes most of the requests
Thread even numbered(2,4,...) - Thread used only when needed for AJAX parallel requests so Thread(1,2) is a group as is Thread(3,4) and so on.
If we needed to make 3 AJAX requests in parallel then we would have grouped up Threads(1,2,3) and Threads (4,5,6) - but the main thread would have been Thread1, Thread4,...
You can immediately see resources are being wasted - Well we did say this was a toy.
Making the even numbered threads not execute some samplers is easy, we just need an IF controller to check. But how do you make Thread 1 and Thread 2 for e.g. execute a sampler in between at exactly the same time? In the Java world , this would usually mean reading up everything that Doug Lea and Brian Goetz have written - but in lazy developer mode this means downloading the JMeter source code and looking at SyncTimer (which blocks a fixed number of threads but not specific ones) and using the time honored technique of Copy Paste.
So before diving into the problem at hand lets see if we can simulate the following.
6 threads will arrive at varying times into one sampler. At which points they will wait till groups are ready i.e. Threads 1 and 2 will go together and Threads 3 and 4 will go together and so on.
Here's the Beanshell sampler which is actually a customized Sync Timer in disguise

import java.util.HashMap;
int totalThreads = 6; // MUST match total number defined in Thread Group
int groupsOf = 2; //MUST match number of requests to make in parallel

if (bsh.shared.myObj == void){
    // not yet defined, so create it:
    bsh.shared.myObj =new HashMap();
    bsh.shared.timerCounter =new HashMap();
    int numObjectsNeeded = totalThreads/groupsOf;
    for(int i = 0;i<numObjectsNeeded;i++) {
           bsh.shared.myObj.put(String.valueOf(i),new Object());
           bsh.shared.timerCounter.put(String.valueOf(i),new int[] { 0 });
    }
    print("populated");
} else {
   print("already init");
}

print(${__threadNum()});

int grpNo = (int)(${__threadNum()} -1)/groupsOf;
print(grpNo);
String groupNumber = String.valueOf(grpNo);
print(groupNumber);
Object syncObject = bsh.shared.myObj.get(groupNumber);
print(syncObject);
synchronized (syncObject) {
   int[] timerCount = (int[])bsh.shared.timerCounter.get(groupNumber);
   timerCount[0]++;
   final int count = timerCount[0];
   if (count >= groupsOf) {
                syncObject.notifyAll();
            } else {
                try {
                    syncObject.wait();
                } catch (InterruptedException e) {
                    log.warn(e.getLocalizedMessage());
                }
            }
            timerCount[0]=0; // Reset for next time   
}
SampleResult.setResponseData("sunk!");

The bsh.shared namespace is used to share objects between threads. So in our case we have 6 threads and we want 2 threads in parallel which means there are 3 groups of two threads each. To synchronize each group we need an object (hence 3 objects) and a counter each - to count how many threads are already available. Thats what the first part of the code sets up.(The first part is not thread safe but because we are going to test this by guaranteeing that threads arrive at different times , the snippet need not be thread safe).

The second part of the code then calculates which Group a thread belongs to, gets the object for the group, increases its count. If the number of threads is less than that we want in parallel it suspends the thread (by calling wait) otherwise it does a notifyAll which will wake up all suspended threads and allow these two threads to move in parallel

Heres what the test looks like


We have artificially added a timer to make sure requests to the Delay sampler arrive 1 thread at a time. We run the test and we can see that Threads(1,2) run the After Delay sampler at almost the same time , as do Threads(3,4) and as do Threads(5,6)
If we change the groupsOf in the delay sampler to be 3 then we see that three threads run in parallel.

And with that we are ready to use this delay sampler to make AJAX calls in parallel.
Onwards to Part III - An actual example of JMeter and AJAX

JMeter and AJAX - Part I

A four part post on JMeter and AJAX
Part I - A general discussion on why we might not need to worry about testing AJAX on JMETER
Part II - A diversion on creating a sampler that will allow us to synchronize between specific threads
Part III - A simple sample on executing some AJAX requests in parallel
Part III-1 - A diversion on how to use session ids between threads 
Part IV - A more complex sample

Queries related to testing websites which use AJAX are often asked on the JMeter mailing list so here are my thoughts on this matter.

JMeter is not a browser, it simulates a HTTP request/response which means it will always struggle to replicate the finer parts of browser behavior. The advantage however is that it only needs very few resources to simulate a single browser. Testing tools that actually drive a browser will find it easier to replicate the browser however they pay for it when it comes to load tests - the amount of resources is needed is much more than the JMeter family of tools. But when testing out websites which use AJAX, JMeter with its current feature set may not be an accurate simulation because AJAX does depend on the finer parts of browser behavior.

An elegant solution would be to have the ability to define child HTTP requests of a parent HTTP sampler with a thread pool that defines how many of these will be made in parallel. But no such feature is currently available in JMeter.

But say you do have to test an AJAX enabled site today - what can one do?
It's useful in these times to be a lazy but honest tester - The honest part ensures that we will try to be accurate but the lazy part also ensures that we wont try to do any more than the minimum we need to do.

• So the first question that should come up is do we actually need to change anything in the way we test? The vast majority of AJAX enabled sites take the form a page loads , the user does some action , an AJAX call is made and values are shown to the user without updating the rest of the page. But look at the sequence of HTTP requests. Request for the main page, get response, browser processes response,user action, AJAX call made , AJAX response, further browser processing. There isn't any request made in parallel (ignoring all the static file requests which is its own blog post) - nor is there any difference due to Asynchronous nature of the call. So a JMeter script to simulate the above would have no difference from a normal JMeter script. the AJAX call should just follow the call to the main page and we are done.
• But hold on some AJAX enabled sites may make an AJAX call as soon as the page loads without any user action. Or the call might be made periodically.Even in these cases the onload/onready is only done after the response of the main request is made available so while the browser may make the request asynchronously in terms of the HTTP request/response there is no difference. The periodic call may only have a problem if the time it takes to return is greater than the periodic repetition. But the latter case would be a problem anyway , all you really want from JMeter is to detect if this happens. Once it happens your system is messed up and you need to fix it! There really isn't much value simulating the messed up state. To put it in another way , if your system breaks for 50 users, there isn't much value observing how much more is broken for 100 users.
• But hold on what if multiple AJAX requests are made on load? Say a users home page has two AJAX widgets , one which shows say his tasks , one which shows his emails? Well yes two requests will be made in parallel. Usually these requests are independent of one another. The question we really need answered is "Is there a difference between a user making two independent requests in parallel and two users(maybe with the same login) making a single independent request each in parallel?" In most cases , the answer is no. The load may be slightly higher in the latter case because you may have two sessions instead of one, but in terms of the rate of requests the server sees you can normally match the rate of requests by simply increasing the number of threads (and a little bit of math).
• But hold on what if the requests aren't independent? What if they utilize the same table/rows causing contention when its from the same user session what if widget2 depends on widget 1 for something in the session? Well one thing to do in this case is call the developer and ask him why the hell are there two dependent requests instead of a single one? That usually doesn't end well(even though the tester is right) - but yes at this time yes you might need to actually simulate requests in parallel. Note however a lot of AJAX enabled websites dont fall into this bracket and those websites can be simulated in JMeter with a reasonable degree of accuracy.

Onwards to Part - 2. Simulating AJAX requests without knowing how to modify JMeter core.

Google isnt as good as you think it is

Exhibit A - A snippet from webmaster tools for this blog



Though I know that writing about deepika instead of JMeter would give me more page hits!

WLST to dump free heap memory

from java.io import FileWriter
from java.io import BufferedWriter
from java.util import Date
connect(eval('username'),eval('password'),eval('url'))
domainRuntime()
cd('/ServerRuntimes/' + eval('managedServerName') + '/JVMRuntime/' + eval('managedServerName'))
heapFreeCurrentPerOld = str(cmo.getHeapFreePercent())
heapFreeCurrentValOld = str(cmo.getHeapFreeCurrent())
cmo.runGC()
java.lang.Thread.sleep(int(eval('sleepTime')))

heapFreeCurrentPerNew = str(cmo.getHeapFreePercent())
heapFreeCurrentValNew = str(cmo.getHeapFreeCurrent())
newDate = Date()
fstream = FileWriter(eval('outputFile'),true)
out =  BufferedWriter(fstream)
out.write(eval('managedServerName') + "," + str(newDate.getTime()) + "," + heapFreeCurrentPerOld + "," + heapFreeCurrentValOld + "," + heapFreeCurrentPerNew + "," + heapFreeCurrentValNew + "\n");
out.close();
disconnect()
exit()

There is also a property file which contains the configuration information

managedServerName=<MANAGED>
username=<USER>
password=<PASSWORD>
url=<URL>
sleepTime=<PAUSE>
outputFile=<PathToFile.csv>

And finally to run

$WL_SERVER_HOME/common/bin>wlst -loadProperties


This was written to output the free memory before and after GC for a particular managed server (which used to run jobs) on weblogic. It was cron'ed to run at fixed intervals. The generated file is used to check if there is a memory leak. The heap free current value should remain fairly constant. if it keeps reducing , we have a leak

Getting the current thread group name

${__BeanShell(ctx.getThreadGroup().getName())}

Academic and Happy

via http://urlai.com/url/theworkaholic.blogspot.com

Text analysis
theworkaholic.blogspot.com is probably written by a male somewhere between 36-50 years old. The writing style is academic and happy most of the time.

Jmeter with Spring Webflow

Motivation
The application uses URL containing dynamic parameters and so a record/replay fails
http://www.mail-archive.com/jmeter-user@jakarta.apache.org/msg30317.html

Solution
This short tutorial deals with test scripts that interact with dynamic data, and we will choose a Spring Webflow example to write a test script against.
So we look around on Google for a sample Spring Webflow application which leads us to
Spring By Example Sample
I'm not going to use the JMeter Proxy Recorder because it seems to harm beginners more than it helps. Testers seem to get the impression that all they need to do is record and replay the script and they are done. So I shall use Firefox and LiveHttpHeaders. We are going to Login to the site and create a User

Step 1: Request the Login Page

http://www.springbyexample.org/simple-webflow/login.jsp
GET /simple-webflow/login.jsp HTTP/1.1

Response 
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=[uniquevalue1]; Path=/simple-webflow

So the first request response shows that the system sets a cookie and the name should tell us that this is a session id. [TODO]. This should also indicate to us that we need to add a cookie manager. Note the status is 200 in the Response. This is the server telling us all is well.

Step 2: Submit Login

http://www.springbyexample.org/simple-webflow/loginProcess;jsessionid=[uniquevalue1]

POST /simple-webflow/loginProcess;jsessionid=[uniquevalue1]
HTTP/1.1
Cookie: JSESSIONID=[uniquevalue1]
Content-Type: application/x-www-form-urlencoded
Content-Length: 48
j_username=david&j_password=newyork&submit=Login

Response
HTTP/1.1 302 Moved Temporarily
Date: Tue, 17 Aug 2010 05:03:48 GMT
Set-Cookie: JSESSIONID=[uniquevalue2]; Path=/simple-webflow
Location: http://www.springbyexample.org/simple-webflow/index.jsp
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/index.jsp
GET /simple-webflow/index.jsp HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 302 Moved Temporarily
Location: http://www.springbyexample.org/simple-webflow/index.html
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/index.html
GET /simple-webflow/index.html HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]
HTTP/1.1 200 OK

a. The browser sends back the cookie that was previously set. However note that the URL also has a jsessionid which has the same value as the cookie that was set. This is known as URL rewriting. Our test will ignore this value because we will be using a Cookie manager
b. The server sends a 302 response to the request. This basically asks the browser to request a new URL (the one that the Location Response Header has). Note this can happen more than once as the next request too results in a 302 response, which finally results in a 200 status code.
HTTP Status Codes are discussed here
c. The server also sends a Set-Cookie for JSessionID again with a new value. This is usually done for security reasons. We can ignore this for JMeter because the Cookie manager will handle this for us. However the sampler that we use needs to have follow redirects set. If redirect automatically is set , JMeter would not get to see this Set-Cookie header and since the application has switched the session id , if you sent it the old session id (which was associated to the before login session)

Click the Create Link

http://www.springbyexample.org/simple-webflow/person.html
GET /simple-webflow/person.html HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

Response
HTTP/1.1 302 Moved Temporarily
Location: http://www.springbyexample.org/simple-webflow/person.html?execution=e1s1

----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/person.html?execution=e1s1
GET /simple-webflow/person.html?execution=e2s1 HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 200 OK
Date: Tue, 17 Aug 2010 05:06:04 GMT

Clicking the Create link also resulted in a 302 code so the browser issued a new request. But take a look at the URL. It has a parameter named execution with a value e1s1. Now click the link again.

Click Create link again

http://www.springbyexample.org/simple-webflow/person.html
GET /simple-webflow/person.html HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

Response
HTTP/1.1 302 Moved Temporarily
Location: http://www.springbyexample.org/simple-webflow/person.html?execution=e2s1
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/person.html?execution=e2s1

GET /simple-webflow/person.html?execution=e2s1 HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 200 OK

The "execution" parameter value changes! So if you run the same test multiple times , the value has to be different. This is a common cause of failure of Record / Replay tests. Some of the data is dynamic. This data can be of various forms
a. Something the application framework needs
b. Something where the data varies (E.g. the UserId, the Product ID). Typically a user friendly string is shown on the screen but when the user performs an action , an id (usually a key in the database) is what is used by the application
c. Dynamic fields (E.g. DHTML based applications) which get added on the fly.
To deal with this parameter/URL we need to be able to extract the value from previous requests in our JMeter scripts.
If you haven't developed the application it may be difficult to identify whats dynamic and whats not. The easy way is to record the script twice(With different data) and observe what changes. e.g. if you browse a product catalog , change the product.

Submit the form

http://www.springbyexample.org/simple-webflow/person.html?execution=e2s1

POST /simple-webflow/person.html?execution=e2s1 
HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]
Content-Length: 56
id=&firstName=deepak&lastName=shetty&_eventId_save=Save

Response
HTTP/1.1 302 Moved Temporarily
Location: http://www.springbyexample.org/simple-webflow/person/search.html
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/person/search.html
GET /simple-webflow/person/search.html HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 200 OK

Unfortunately the application has a visual bug, the added user does not show. However if you do a view source on the page and search for the username you created you can find it.

Click Logout

http://www.springbyexample.org/simple-webflow/logout
GET /simple-webflow/logout HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

Response
HTTP/1.1 302 Moved Temporarily
Set-Cookie: SPRING_SECURITY_REMEMBER_ME_COOKIE=""; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/simple-webflow
Location: http://www.springbyexample.org/simple-webflow/logoutSuccess.jsp
----------------------------------------------------------
http://www.springbyexample.org/simple-webflow/logoutSuccess.jsp
GET /simple-webflow/logoutSuccess.jsp HTTP/1.1
Cookie: JSESSIONID=[uniquevalue2]

HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=[uniquevalue3]; Path=/simple-webflow

The JMeter Script
Now that we know the above we can get to writing the script

Most of the script is pretty much standard(All Samplers use follow redirects and we have added a Cookie Manager), till we get to the point where we need to extract out the URL.
The Request Create Sampler extracts out the URL. Since we know that the application is going to redirect us , the extracter will extract out the last URL we will be on



The Post Create Data Sampler simply uses this extracted URL



Note that all samplers have assertions that verify the behavior, so if something goes wrong the Assertion should fail. For e.g. after creating the user we verify that the user we created exists on the page. When requesting a page the assertions verify some text on the page that is unique to the page.
Run the test, we get no errors! Modify the ThreadGroup to have two threads and two iterations. Run the test , everything should pass. Now look at the View Results Tree listener. You should see the redirects being followed (multiple children results under a parent). But do you see different values for the execution parameter? No - This is because each thread is its own session and each user will start with the same parameter value. Even having multiple iterations on the thread group doesnt give you different values because we are logging out at the end of the script so the next iteration is a new session.

Try adding a Loop controller to create 10 users. Note that the loop controller must enclose the Request Create and Post Create samplers (because we need to extract a new value for the execution parameter every time!). Using View results tree you can now observe the execution parameter changing.



Sample Script
Spring Webflow.jmx

Spring Webflow with loop.jmx

First weblogic portal expert

YAY!


Thankfully a lot of posters don't use Google, otherwise I couldn't answer the same questions again!

Updated JEE Interview Questions - Draft

JSP
What are the various way's to handle exceptions in a web application. What would you use when. Demonstrate with examples from any project you have worked on
What are the various way's you have implemented security in your web application. Also mention alternatives that can be used
Which framework have you used. Please describe the shortcomings in the framework and the ways to work around them
What is the difference between a static include, jsp:include, jsp:forward and redirect. Please give one example of each.
Describe some Tag Libraries you have written. What are advantages / disadvantages of tag library. If possible give an example of a badly written open source JSP tag library(or any of your own) and how you would improve it.
Give one example of when you would use a filter. How do you execute a filter after the request?
Give one example of when you would use a listener. Which listeners have you used in your project?
Besides defining servlets , name one feature that can be specified in the web deployment descriptor
Name JSP implicit object. Give one example of the use of each implicit object. Is exception an implicit object? If so can i write exception.printStackTrace in any JSP? what will it print?
Comment whether a JSP web application works with cookies disabled.
Which web container did you use. Please describe how to use a container specific feature.
Describe some JSTL tags you have used and what they are useful for
How do you use a datasource in a JSP?
How do you internationalize a web application. What all do you need to take care of

EJB
How do you implement security in an EJB . Why does the framework provided security not work in most real world scenarios? (or does it?)
If an MDB throws an Exception what happens to the message?
If an EJB has two methods M1 and M2 both with RequiresNew and M1 makes a call to M2 by calling it (because M2 is a method in the same class) how many transactions are there?
What limitations are

JDBC
Lets say you want to execute a query and show some data. Walk through the steps to do this in Raw JDBC.
What do you prefer using of the following and why? CMP/JDO/ORMs (like hibernate/toplink)/SQL helpers (like Spring/iBatis)/Raw SQL?
In the context of ORM what is lazy loading/eager load. What do you use in your project? Why? What factors determine what is to be used.
Your ORM based system is slow. How would you diagnose the problem?
Why is pessimistic locking not useful in a web-application (or is it?)

JMS
Say you have a queue. The Producer can produce messages much faster than the consumer can consume. What are your options?


JMX
Give any example where you have used JMX

WEB
Given a large data set , what options do you have for displaying this dataset. Which one do you prefer and why?
Suppose you are tasked to implement an AJAX based system where the user types in a word in a search box and you want to show him possible queries (e.,g. in Google if you type dee you get deepak chopra or deepika padukone)in a drop down , what considerations would you have in implementing the AJAX parts of the client (Assume that you have a server side component that can give you the results)
If you want Google to search your site easily , what do you need to take care of in your web app

Misc
You have a clustered J2EE web application. You need to batch process a file which has a billion records. Each record is independent . How do you do it?
You have a table which has UserID (primary key) and UserName. The requirement is to display this in a drop down box on a page so that the user will select the name but the value posted to the next page will be the userid. The data displayed must be sorted by username. The DB developer asks you what data structure he needs to return the results in. What is your answer? Assuming that you get this data structure , is there any comment you wish to make on your implementation?
Name a few design patterns that you see being used in the Java API with examples.

Simplified procedure for Syncing Embedded LDAP with Weblogic Portal Database

a. Shut down servers
b. On the admin server, backup the LDAP directory \servers\AdminServer\data\ to another location
c. Delete the internal LDAP directory at \servers\AdminServer\data\ldap (also from managed
servers if applicable)
d. Delete data from the p13n entitlement tables (delete from tablename)
( namely p13n_entitlement_role, p13n_entitlement_resource,p13n_entitlement_policy, p13n_entitlement_application and p13n_delegated_hierarchy )
e. Start the servers
f. Redeploy the application
g. Recreate the SAML relying party configuration
h. Recreate Entitlements

Dynamic Parameters in JMeter

Motivation

You need to send a variable number of parameters with an HTTP request i.e. the number of parameters are not known when the script is written. For example a User can register with a number of accounts , but this number varies per user. Or perhaps you want to Post process extract a number of hidden fields on the previous response and post all of them in the next Request, but you either don't know the number or there are too many fields to enter manually into the HTTPSampler.

Solution

Use a BeanShell PreProcessor to dynamically add variables to the sampler. The relevant code is sampler.addArgument(name,value)[1]

Sample

We will write a script to access a page , extract out data based on a pattern and send these as separate parameters to the next request. The script is as shown below.

We make a request to some dummy page and then we have a regex extractor to extract out multiple values. In this case the Regex extracts out all words(Match No = -1) beginning with test

Then we have a Debug Sampler to verify the regex is working, notice the results in View Results Tree Listener. The Regex has a reference variable name of inputTerms. The response of the Debug sampler is shown below

You can see from the Listeners response tab that the total number of results is 14 , available under the key inputTerms_matchNr and that each result is available as inputTerms_$resultnumber.

The matching groups which we usually do not need are available as inputTerms_$resultnumber_g$groupnumber. Armed with this information, the beanshell preprocessor script is trivial as shown below

int count = Integer.parseInt(vars.get("inputTerms_matchNr"));
for(int i=1;i<=count;i++) { //regex counts are 1 based
sampler.addArgument("hardcodedkey", vars.get("inputTerms_" + i));
}

Heres the next request to which the beanshell pre processor is attached. We have one predefined variable and the rest will be dynamically added by beanshell.

Next we check the request data we are posting using the Request tab of the View Results Tree Listener

Notice that the argument we added to the next request was sent, as were all the parameters we added dynamically. The parameter values were also encoded.
Sample Script : https://skydrive.live.com/#cid=1BD02FE33F80B8AC&id=1BD02FE33F80B8AC!268

Reference
[1] Javadocs for HTTP Sample Base

Asserting MS Office formats

As a follow up to the PDF post, you can do something similar using Apache POI

import org.apache.poi.POITextExtractor;
import org.apache.poi.extractor.ExtractorFactory;
import java.io.ByteArrayInputStream;

ByteArrayInputStream bais = new ByteArrayInputStream(in);
POITextExtractor extractor = ExtractorFactory.createExtractor(bais);
prev.setResponseData(extractor.getText());

You should be able to parse word documents and excel and ppt's using this beanshell post processor. Remember to copy the POI libraries to JMeter/lib

Future work
Look into Apache Tika for a unified interface (may need to sacrifice some functionality like the startPage endPage of PDFBox)

Asserting PDF's

A question on the JMeter mailing list regarding extracting/asserting text inside a PDF file and since I ran into this for a functional scenario , I wrote up my attempt. I use PDFBox as the library. Download the binaries and copy pdfbox-1.0.0.jar and external/fontbox-1.0.0.jar into JMeter's lib directory. We'll access the PDF at http://jakarta.apache.org/jmeter/usermanual/jmeter_distributed_testing_step_by_step.pdf and check that the PDF does contain "Distributed Testing Step-by-step" Here's the JMeter sample script So we have a transaction controller 'Check PDF' so that we only get a result item. The HTTP Request Sampler 'Request PDF' requests the PDF. The bulk of the code is in the beanshell post processor titled 'Extract Text'

import java.io.ByteArrayInputStream;
import java.io.StringWriter;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.util.PDFTextStripper;


PDDocument document = null;
StringWriter sw = new StringWriter();
try {
ByteArrayInputStream bais = new ByteArrayInputStream(data);
document = PDDocument.load(bais);
PDFTextStripper stripper = new PDFTextStripper("UTF-8");
stripper.setSortByPosition( false );
stripper.setShouldSeparateByBeads( true );
stripper.setStartPage( 1 );
stripper.setEndPage(Integer.MAX_VALUE );
stripper.writeText( document, sw );
} catch (Throwable t) {
t.printStackTrace();
sw.append("ERROR");
} finally {
sw.close();
document.close();
}
vars.put("extractedText", sw.toString());

Update - for PDFBox 2.0.26

import java.io.ByteArrayInputStream;
import java.io.StringWriter;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.text.PDFTextStripper;


PDDocument document = null;
StringWriter sw = new StringWriter();
try {
	ByteArrayInputStream bais = new ByteArrayInputStream(data);
	document = PDDocument.load(bais);
	PDFTextStripper stripper = new PDFTextStripper();		
	stripper.setSortByPosition( false );
	stripper.setShouldSeparateByBeads( true );
	stripper.setStartPage( 1 );
	stripper.setEndPage(Integer.MAX_VALUE );        
	stripper.writeText( document, sw );
} catch (Throwable t) {
                             t.printStackTrace();
	sw.append("ERROR");
} finally {
	sw.close();
	document.close();
}
vars.put("extractedText", sw.toString());

All this does is use the PDFBox API to extract text from the PDF (bytes are present in the data object) and write it to a variable in JMeter , extractedText. The next sampler is a Java Sampler which sets the ResultData to ${extracted text}. This will echo back the contents of this variable as the response of the sampler. Once this is done , you can use a normal response assertion, regex extractor or whatever else you need to process the text Update: as mentioned by Milamber in the comments , you can instead have the last line of the beanshell as

prev.setResponseData(sw.toString());

This will set the response of the HTTPSampler to be the text value and allow you to directly specify the assertion (and you can also eliminate the transaction controller). Update : if you are interested in MS office formats then follow the steps in this post, just change the BeanShell post processor to that mentioned in Asserting MS Office Formats Future work a. make a custom PDF(or any format sampler) with the options that are hardcoded (e.g. startPage or endPage) b. experiment with ways to use less memory. Typically should extract and write to file and use that.

Random stuff on jmeter testing

Update : I realized that much of what I wanted to say has already been said and in a far better manner in Perfomance testing patterns and practices.
One of the questions asked on the JMeter mailing list was

How do I analyse my application with JMeter?

There are various reasons one might run a load test broadly detection, simulation, verification and analysis. The type of test or the scripts you might write, what you would measure or track would vary a bit depending on what you want to do. JMeter is good for simulation and verification. It is an aid to detection and analysis but you usually need other tools to help you out.

Detection

• You want to find out if your application behaves correctly when accessed by multiple threads. Whether your database starts showing deadlocks or whether you might have race conditions.
• You want to find out if your application has memory leaks.

Simulation

• You want to find out how your application behaves under standard load or how it behaves under some peak load(e.g. shopping during the holidays). Response times etc.

• In the specific case of Java based web applications you want to know how often your GC cycles might run or how long.

• You application makes remote calls(e.g. webservice calls) and you want to know whether all the resources are recovered correctly. perhaps you have some throttling mechanisms in place and you want to see that its working correctly

Verification

• You want to verify the result of changing some tuning parameters.
• You want to check your SLA's are met
  

Analysis

• You might already know there is a problem and you want to simulate load while you are profiling the application.

These areas overlap and I've used the above categories quite broadly to merely illustrate that your objective and hence your test script will vary based on your objective.

For e.g.
You want to find out if your application behaves correctly when accessed by multiple threads - In this case your test script would only be concerned with running some parts of the applications at exactly the same time. You'd want to exercise multiple parts of your application. Perhaps you are aware that some part of the application internally spawns threads and you'd run a test that exercise that area for a long time or with a high load. You don't at this point really care whether these are unrealistic scenarios or non representative scenarios, nor are you really looking at what the response times are. All you care about is do you see stuck threads or deadlocks. Do you see a really long wait time for most threads though some threads finish really fast.

Or perhaps you want to tune a memory parameter and you want to verify the change
In this case Response Times / Throughput really matter (for the same test of course). You'd first take a baseline reading without the tuning, and then another with the tuning. The test scripts must be representative of actual user behavior.

Perhaps you want to check whether your site can handle holiday shopping onslaughts. In this case you would modify your tests to show bursts of activity, you'd closely monitor response times but you also want to check what happens on the server. How much memory, How much CPU. You might also want to see what load might actually bring down your servers. You might want to check if your load balancers evenly distribute the load.

Or perhaps you have certain Service Level Agreements and you need to know response times accurately for the load specified in your agreement. In this case you need a representative user journey and you also need representative background users.

All of which means there is no easy answer to ' How do I analyse my application with JMeter'. It can only be answered by What is it that you want to analyse (normally answered as well, performance).

Lets take the most common use case, what is the 'response time' for my application.
However actually getting the response time is more difficult than reading the response time calculation from the JMeter test results.
This is problematic due to
a. JMeter is not a browser and does not render the page. Different browsers take different times to render the same page. Compare older versions of Internet explorer with Chrome for e.g.
b. A returning user with some files cached will probably show lesser times than a first time user.
c. The network / connection speed from which the user is accessing the application may be significant. And your users may be spread out throughout the world.
d AJAX based applications / DHTML applications are difficult to predict because not only does it vary by browser , but the number of calls that a browser may make in parallel is also different, but some calls will be made in parallel and its difficult to know that.

So any response time would have (roughly speaking)
a. The time it takes for the application to actually respond with all the data
b. The time it takes for this data to be transferred over the network
c. The time it takes to download static files (bearing in mind that not all files may be downloaded and that browsers may request multiple static files in parallel)
d. The time it actually takes to render the page.

JMeter can help you out with a, b, and c. but what it is good at, is finding out a. for the network on which it is running on.

Typically your requirements might define an Service Level agreement for your site as Browsing operations must take < 6seconds 90% of the time and shopping operations must take <8 seconds 90% of the time. You also know how much large your pages are and you can guesstimate how much time it would take for the page to be transferred over the internet. You might take an average with some safety factor or you might take a worst case scenario. Using a browser tool like YSlow or Googles PageSpeed , you can also have some insight on how your static are downloaded , how long they take etc. And you might add some time for how long the browser takes to render. After considering all of this you might arrive at a new figure that on a high bandwidth intranet (which thereby eliminates most of the network variables) your browsing operations must take < 2 seconds just to get the data and your shopping operations must take < 4 seconds for your SLA's to hold because the rest of the time has already been used by the other factors.
After this you would have to write a script which generates representative loads (for the operations being verified and the operations that would happen in the background), run the test and verify the 90% percentile lies below the value you have calculated above. But perhaps it doesn't. Static files can be optimised by reducing their number, their size, gzipping them adding expiry headers etc, but maybe you have already done this. The Clients network and browser aren't within your control so there isn't much you can do there. The next step is figuring out where your problem lies. JMeter can't help you there, you need a different set of tools. But JMeter can help you to simulate load or parts of it so that you can monitor your application with the tools of your choice. Some of your findings may be infrastructure related, Some may be code you'd have to make changes and retest and repeat.

JSP Interview questions

1. What are the various way's to handle exceptions in a web application. What would you use when. Demonstrate with examples from any project you have worked on
2. What are the various way's you have implemented security in your web application. Also mention alternatives that can be used
3. When would you use servlets. name one thing you can do with a servlet that you cannot do with a JSP.
4. Which framework have you used. Please describe the shortcomings in the framework and the ways to work around them
5. What is the difference between a static include, jsp:include, jsp:forward and redirect. Please give one example of each.
6 Describe some Tag Libraries you have written. What are advantages / disadvantages of tag library. If possible give an example of a badly written open source JSP tag library(or any of your own) and how you would improve it.
7. Give one example of when you would use a filter. How do you execute a filter after the request?
8. Give one example of when you would use a listener. Which listeners have you used in your project?
9. Besides defining servlets , name one feature that can be specified in the web deployment descriptor
10. Name JSP implicit object. Give one example of the use of each implicit object. Is exception an implicit object? If so can i write exception.printStackTrace in any JSP? what will it print?
11. Comment whether a JSP web application works with cookies disabled.
12. Which web container did you use. Please describe how to use a container specific feature.
13. Describe some JSTL tags you have used and what they are useful for
14. How do you use a datasource in a JSP?
15. How do you internationalize a web application. What all do you need to take care of

JMeter Graphs and ANT

A follow up to the previous posts, I've integrated the graph code with ANT so that the HTML reports provided by running JMeter from the command line and styling them can be extended to include the graphs.

The sample report that I generated is shown below
The custom report hyperlinks the titles of the normal summary tables to lead to graphs for the same.
To implement this we need to
a. Allow the graph code to be invoked from ANT. This can be done by writing a simple java class with a main method that passes parameters on the command line, or we could write a custom ant task. I wrote a custom ant task as a proof of concept. We need to customise the build script as well
b.Modify the XSLT to write out image anchors.

These steps are described below.
The custom ant Task

public class AggregateGraphTask extends Task {
private String outputDir;
private String outputFilePrefix;
private Boolean showThreshold = Boolean.TRUE;
private Double threshold = 500D;
private String jmeterResultFile;
private String jmeterHome;

public String getJmeterHome() {
    return jmeterHome;
}

public void setJmeterHome(String jmeterHome) {
    this.jmeterHome = jmeterHome;
}

public String getJmeterResultFile() {
    return jmeterResultFile;
}

public void setJmeterResultFile(String jmeterResultFile) {
    this.jmeterResultFile = jmeterResultFile;
}

public String getOutputDir() {
    return outputDir;
}

public void setOutputDir(String outputDir) {
    this.outputDir = outputDir;
}

public String getOutputFilePrefix() {
    return outputFilePrefix;
}

public void setOutputFilePrefix(String outputFilePrefix) {
    this.outputFilePrefix = outputFilePrefix;
}

public Boolean getShowThreshold() {
    return showThreshold;
}

public void setShowThreshold(Boolean showThreshold) {
    this.showThreshold = showThreshold;
}

public Double getThreshold() {
    return threshold;
}

public void setThreshold(Double threshold) {
    this.threshold = threshold;
}

@Override
public void execute() throws BuildException {
    try {
        GraphClient.init(jmeterHome);
        String outputPrefix = outputDir +  File.separator + outputFilePrefix;
        if(Boolean.TRUE.equals(showThreshold))  {
            GraphClient.writeAggregateChartWithThreshold(jmeterResultFile, outputPrefix, showThreshold, threshold);
        } else {
            GraphClient.writeAggregateChart(jmeterResultFile, outputPrefix)    ;
        }
    } catch(Exception e) {
        throw new BuildException(e);
    }
}

}

This is a pretty straightforward class which calls our API's based on parameters passed to it. It declares fields for all the attributes it expects and then calls the Graph API's.

The ANT build.
I assume you have JMeter working from ANT, this assumes all the libraries needed for ANT and JMeter are in place

<project name="Jmeter" basedir="." default="runOfflineGraph">
<property name="lib.dir" value="${basedir}/lib"/>
<property name="report.dir" value="${basedir}/report"/>
<property name="styles.dir" value="${basedir}/styles"/>
<property name="export.dir" value="${basedir}/export"/>
<property environment="env"/>
<property name="jmeter.home.dir" value="${env.JMETER_HOME}"/>
<property name="jfreechart.home.dir" value="${env.JFREECHART_HOME}"/>

<path id="run.classpath">
<fileset dir="${jmeter.home.dir}" includes="**/*.jar"/>
<fileset dir="${jfreechart.home.dir}" includes="**/*.jar"/>
<fileset dir="${lib.dir}" includes="*.jar"/>
</path>

<target name="clean">
<delete dir="${report.dir}" />
<delete dir="${export.dir}" />
</target>

<target name="init">
<mkdir dir="${report.dir}" />
<mkdir dir="${export.dir}" />
</target>


<target name="runJMeter" depends="init">
<taskdef
    name="jmeter"
classname="org.programmerplanet.ant.taskdefs.jmeter.JMeterTask"/>
<taskdef name="aggregatechart" classname="org.md.jmeter.ant.AggregateGraphTask" classpathref="run.classpath"/>

<tstamp />
<property name="uniqueTStamp" value="${DSTAMP}${TSTAMP}" />
<property name="imageNamePrefix" value="AggregateChartThreshold-${uniqueTStamp}" />
<property name="jmeter.result.fileName" value="${run.test.report}-${uniqueTStamp}" />
<property name="jmeter.result.file" value="${report.dir}/${jmeter.result.fileName}.jtl" />
<jmeter
    jmeterhome="${jmeter.home.dir}"
    testplan="${run.test.plan}"
        resultlog="${jmeter.result.file}">
    <property name="jmeter.save.saveservice.output_format" value="xml"/>
    <property name="run.threadcount" value="${run.threadcount}" />
    <property name="run.loopcount" value="${run.loopcount}" />
    <property name="sample_variables" value="${sample_variables}" />
</jmeter>
<xslt
    in="${jmeter.result.file}"
    out="${report.dir}/${jmeter.result.fileName}.html"
    style="${styles.dir}/${xsl.file}">
    <param name="imageNamePrefix" expression="${imageNamePrefix}"/>


 <aggregatechart jmeterHome="${jmeter.home.dir}" jmeterResultFile="${jmeter.result.file}" outputDir="${report.dir}"
outputFilePrefix="${imageNamePrefix}" showThreshold="true" threshold="200"/>
</target>


<target name="runOfflineGraph" depends="init">
<antcall target="runJMeter">
<param name="run.test.plan" value="OfflineGraphs.jmx"/>
<param name="run.test.report" value="OfflineGraph"/>
<param name="sample_variables" value=""/>
<param name="run.threadcount" value="1"/>
<param name="run.loopcount" value="5"/>
<param name="xsl.file" value="OfflineGraph.xsl" />
</antcall>
</target>
</project>

Important points are
a) we define a run.classpath which has everything we need at runtime to generate the graphs.
b) we have a taskdef aggregatechart for our custom task
c) we invoke the custom chart by passing it the parameters we need. These are closely linked with the previous steps in the build. The result jog from jmeter (${jmeter.result.file}) is passed as an input to the task. The image file names to be generated are important ${imageNamePrefix} as we need to reference this in the stylesheet. The directory to which the Graph code writes must be the same as the XSLT output (or atleast the XSLT and Graph code must be consistent in where the images are referenced from in the HTML)

The XSLT stylesheet
The changes here are pretty straightforward. I've copied extras/jmeter-results-report_21.xsl and renamed it to OfflineGraph.xsl.
The important changes are

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:param name="imageNamePrefix">AggregateChartThreshold</xsl:param>

We pass in a parameter that is to be used while generating the img tag in the HTML

            <xsl:call-template name="summary" />
      <hr size="1" width="95%" align="left" />

      <xsl:call-template name="pagelist" />
      <hr size="1" width="95%" align="left" />

      <xsl:call-template name="detail" />
      <xsl:call-template name="graph-images" />

We call our custom template

            <th><a href="#AverageGraph">Average Time</a></th>
      <th><a href="#MinimumGraph">Min Time</a></th>
      <th><a href="#MaximumGraph">Max Time</a></th>

We change the titles to be anchors (named anchors that are next to the images)

<xsl:template name="output-image">
<xsl:param name="suffix" />
<xsl:element name="img">
  <xsl:attribute name="src"><xsl:value-of select="$imageNamePrefix" />-<xsl:value-of select="$suffix" />.png</xsl:attribute>
</xsl:element>
</xsl:template>

<xsl:template name="graph-images">
Graphs
<br />
<b>Minimum</b>
<br />
<a name="MinimumGraph"></a>
<xsl:call-template name="output-image">
  <xsl:with-param name="suffix">Min</xsl:with-param>
</xsl:call-template>
<br />
<b>Maximum</b>
<br />
<a name="MaximumGraph"></a>
<xsl:call-template name="output-image">
  <xsl:with-param name="suffix">Max</xsl:with-param>
</xsl:call-template>
<br />
<b>Average</b>
<br />
<a name="AverageGraph"></a>
<xsl:call-template name="output-image">
  <xsl:with-param name="suffix">Avg</xsl:with-param>
</xsl:call-template>
<br />
<b>Median</b>
<br />
<a name="MedianGraph"></a>
<xsl:call-template name="output-image">
  <xsl:with-param name="suffix">Median</xsl:with-param>
</xsl:call-template>
<br />
<b>90 percentile</b>
<br />
<a name="NinetyPerGraph"></a>
<xsl:call-template name="output-image">
  <xsl:with-param name="suffix">90</xsl:with-param>
</xsl:call-template>
</xsl:template>

Finally we output img tags. The knowledge of how the Aggregate Graph generates the suffix for the images is hardcoded into the stylesheet. The passed parameter is used to form the filename as well (this is used to allow multiple runs , so that the images don't overwrite previous ones)

Running the ANT build
This is the run.cmd I use (windows only)

set JAVA_HOME=C:\bea102\jdk150_11
set JMETER_HOME=C:\projects\R1-Portal-CMS\test\jakarta-jmeter-2.3.4
set JFREECHART_HOME=C:\work\java\jfreechart-1.0.13
set ANT_HOME=C:\work\java\apache-ant-1.7.1
set PATH=%JAVA_HOME%\bin;%ANT_HOME%\bin;%PATH%
set CLASSPATH=%JMETER_HOME%\extras\ant-jmeter-1.0.9.jar;%CLASSPATH%
ant %*

We set some environment properties that the build needs and run it.


Source Code is available here

Aggregate Graphs in JMeter using JFreeChart (3D Bar Charts) with thresholds

Borrowed heavily from the demos in JFreeChart, I've modified the previous samples to also support taking in a threshold value for response. Values less than the threshold are shown in Green Bars, values greater are shown using Red Bars. The threshold is also drawn.

Sample Images








Sample Code

    public static void writeAggregateChartWithThreshold() throws Exception {
  File f = new File(JMETER_RESULT_FILE);
  ResultCollector rc = new ResultCollector();
  AggregateChartVisualizer v = new AggregateChartVisualizer(ConfigUtil
          .getOutputGraphDir()
          + "/AggregateChartThreshold",true,500);
  ResultCollectorHelper rch = new ResultCollectorHelper(rc, v);
  XStreamJTLParser p = new XStreamJTLParser(f, rch);
  p.parse();
  v.writeOutput();
}

Source code available here

Aggregate Graphs in JMeter using JFreeChart (3D Bar Charts)

Now written AggregateChartVisualizer which duplicates AggregateGraph functionality in JMeter.
Sample Charts







Sample Code to be used by clients

    public static void writeAggregateChart() throws Exception {
       File f = new File(JMETER_RESULT_FILE);
       ResultCollector rc = new ResultCollector();
       AggregateChartVisualizer v = new AggregateChartVisualizer(ConfigUtil
               .getOutputGraphDir()
               + "/AggregateChart");
       ResultCollectorHelper rch = new ResultCollectorHelper(rc, v);
       XStreamJTLParser p = new XStreamJTLParser(f, rch);
       p.parse();
       v.writeOutput();
   }

Source Code for the entire workspace available here