Problem i want to use the same JSESSIONID across the test plan which includes several Thread Groups(http://mail-archives.apache.org/mod_mbox/jmeter-user/201303.mbox/%3Cloom.20130311T124606-446@post.gmane.org%3E).
Which is actually two problems
a. How do I share data between threads in JMeter?
b. How do I connect to an existing session in JMeter given the session id?
Note : A good question to ask at this stage is do I really , really want to do this? Usually a Jmeter thread maps to actions that a user can take (AJAX aside) - doing something like the above goes away from the JMeter model of things and an alternative to consider is why don't I change my test so that a user's session is only needed for a thread? (mini rant - This is a common scenario in support where there is a problem X , the user thinks that doing A will solve his problem, A needs B to be done , B needs C and C needs D and the user then asks "How do I do D?"without revealing his cards - in this case why is the test plan for a user split across multiple thread groups?) The user should really be asking how do I solve problem X because some person might say why not do Action Y which is simple) .
But because we will have a blog post to write , we will assume the answer to the above question is Yes, I really want to do this and this is the best way to solve the problem I have.
How to share data between threads in JMeter
JMeter's only out of box way to share data between threads is the properties object and the shared namespace in
BeanShell[2] both of which have limitations. The properties object needs you to come up with some scheme to share data and also needs you to write your own code for e.g. if you consume data much faster than you can generate it.
One technique is that if ALL the data that is to be shared is to be obtained before the next threads need it then one option is to use the CSV Data Set config - The first set of threads write to the file and the next set of threads read from this file - but essentially any shared memory(e.g. a file, a database etc) will do. However if you need to read and write this data at the same time then another way is to use the standard Java data structures which are thread safe.
In this post we will look at two ways
a. The CSV data set config
b. Using a java object
Note that there is a plugin available which will do b. from Jmeter plugins
InterThreadCommunication [1] . If you aren't a programmer then the plugin is probably best for you - though I will say that good testers these days need to have programming and scripting skills.
However we will just write our own for fun. You also might need to do this to implement a complicated scheme.
Before we begin we will need an application to allow us to test whatever we write. So I have deployed a web application which has 2 URLs
a. set.jsp which sets an attribute into the session with the name uuid and the value a random generated value and also returns it as part of its output. Since this is the first page we access it will also create a session and set a cookie.
b. get.jsp which reads the value of the attribute uuid and returns it as its output. If you have the same session and previously accessed set.jsp then you should see the same value as step a.
If you dont have a session(or didnt access set.jsp) then you will see the value as "null"
So let us first write a simple test to verify that all is well. Here is a screen of the test.
We simply access set.jsp. Extract out its response . Then we access get.jsp and assert that the value we extracted out matches whatever is being returned by get.jsp. If the same session is being used for both the requests then the assertion will pass , otherwise it will fail.
Remember that Java web applications allow sessions to be maintained in two ways - either the Session ID is passed as part of the URL or as a cookie (usually named JSESSIONID) or both.
For this example we will assume we are using cookies.
Lets cause the test to fail - A simple way of doing this is by disabling the HTTP Cookie Manager. If we disable this JMeter wont accept cookies and hence wont be able to maintain sessions and every request will be a new session.
So we disable the HTTP Cookie manager and run the test. It fails as we expected it. Lets verify that it's because of the session
a. Note the first request gets a Set-Cookie back from the server - the server is trying to get the client to store the SessionID so that the next request can send the SessionID back
b. Note that the next request does not send a cookie header from JMeter
c. Note that the server responds to this request with another set-cookie (because it will try to create a new session as it didnt get any existing session cookie
Now lets enable the HTTP Cookie Manager. It works!. Lets verify the cookie was passed
Now that we have some degree of confidence that our assertions work (i.e. failures are flagged as failures and successes as successes) lets rerun the test with multiple threads and multiple iterations. All work. We can also see that different set requests get different cookies. This is because a cookie manager is scoped to a single thread (so each thread gets its own cookie and session) and because we have checked "Clear cookies at each iteration" on the cookie manager.
Sharing Data using Java Objects in realtime
Lets first create a class that will allow data to be shared. For this purpose we use a
LinkedBlockingQueue [4]. The javadoc for the interface BlockingQueue.
"Note that a BlockingQueue can safely be used with multiple
producers and multiple consumers."
Which is a fancy way of saying something can be accessed by multiple threads without the programmer needing a degree in Computer Science. Since we will be reading and writing from different threads in different thread groups , the fact that this data structure natively supports thread safety frees us up from having to implement it - no synchronized keywords are needed in our code
However we do need to store this queue object somewhere , so we simply create a wrapper class that holds on this Queue statically and provide getters and setters. Because we dont know the rate at which we will read/write we simply put a timeout on the get.
package org.md;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit;
public class Queue {
private static LinkedBlockingQueue<Object> q = new LinkedBlockingQueue<Object>();
public static void put(Object o) throws Exception{
q.put(o);
}
public static Object get(long timeout) throws Exception{
return q.poll(timeout, TimeUnit.MILLISECONDS);
}
public static void clear() throws Exception{
q.clear();
}
public static void main(String [] args) throws Exception{
Queue.put("one");
Queue.put("two");
//q.clear();
System.out.println(Queue.get(1000));
System.out.println(Queue.get(1000));
System.out.println(Queue.get(1000));
}
}
We run a simple single threaded test on this class to see that everything is fine (it is), we compile this into a jar, place the jar in JMeter's lib directory and our class is ready to be used.
The test structure is shown in the screen below
a. The setup thread group clears any data that might already be there in the queue. Because we are holding onto the Queue statically and the JMeter GUI is a single JVM , we need to clear out any data that might be there from a previous run. This is the code in the Beanshell sampler.
org.md.Queue.clear();
b. We create a threadgroup and use a request to create the sessions , extract out the data that we need - the sessionid and the value returned by the page so that we can assert that we are connecting to the same session. We then call the class we have just created to put these values into the queue (as an array with two values) and finally we introduce a random delay values
The regular expression to extract the cookie is in the screenshot above
And here is the BeanShell code to add the cookie and the extracted value to the Queue
String[] data = new String[2];
data[0] = vars.get("jsessionid");
data[1] = vars.get("uuid");
org.md.Queue.put(data);
c. Next we create a threadgroup that is going to use the session ids set from the previous threadgroup using a BeanShell preprocessor. The preprocessor also has the code to add the session id cookie so that requests will connect to the existing session. As before we assumed that the Server manages session in the form of cookies. But we could also have it as part of the URL once we are able to successfully read the SessionID.
We also configure the thread groups to have different number of threads so that there will be some variation in the rate of requests (note that if you have more Sessions created in the "set" threadgroup than you can process in the "get" threadgroup you will eventually run out of memory unless you make the queue bounded in size. If you have more in the "get" threadgroup then they will keep waiting (hence we have a timeout when we ask for a session id)
import org.apache.jmeter.protocol.http.control .CookieManager;
import org.apache.jmeter.protocol.http.control.Cookie;
String[] data = (String[])org.md.Queue.get(60000);
if(data != null) {
vars.put("jsessionid",data[0]);
vars.put("uuid",data[1]);
Cookie cookie = new Cookie("JSESSIONID",vars.get("jsessionid"),"localhost","/", false,-1);
CookieManager manager = sampler.getCookieManager();
manager.add(cookie);
}
The code above simply asks the queue for a value, and will wait upto 60 seconds for it (This only applies if we are consuming the data much faster than we can produce it). Then if we did get a valid value we use the JMeter API's[3] to access the Cookie Manager and add a cookie to it. Note that we do need a Cookie Manager for this to work. Some values have been hardcoded for simplicity (like the domain name)
We run the test and its all successful! Remember we do have assertions so a success is indeed a success. We can also verify using the view results tree listener and we can also check the cookie is being sent in the GET request.
Cons : Note that anything that needs data to be shared between threads , that has wait and synchronization( irrespective of whether you do it or whether the JVM does) adds an overhead to the test - so the amount of load you can generate from a single JMeter instance will reduce. Also if you need more sophisticated schemes to share data (we used a simple FIFO model) then your code is more complicated , more prone to errors and likely to add more overhead.
Sharing data using files
Using a file is usually not a good candidate for sessionids because sessionids are short lived and may timeout. You also need to have everything available in the file before you can start reading the file. They are more suited to usecases like testing for Remember me or keep me signed in type of cookies. However the flexibility a CSV data set config provides may make it worth your time.
Here is the structure of the test.
The setUp thread group sends a set of requests to create the session and get the values as before. To write all these values to a file we use a simple data writer and configure JMeter (by modifying jmeter.properties) to write the two variables we are interested in only. Note that this is probably not a good way to do it - in addition JMeter appends the data to an existing file so either you need to delete the file before you run this or use a property that varies dynamically so that a new file is created.
Remember that the setUp thread group will always execute before any other threadgroup , so this file will be created and have all the data before the next thread group runs. The Simple data writer unchecks all the checkboxes and we edit jmeter.properties to have the following line
sample_variables=jsessionid,uuid
The next threadgroup simply reads the file and attaches the session id to the cookie manager.
We can have as many threads as we want (we ensure that clear cookies at
every iteration is checked so that every time we request something we
use the cookie that is set by the beanshell preprocessor and not the one
the cookie manager had previously saved). The Beanshell code to add the cookie is the same as before
import org.apache.jmeter.protocol.http.control .CookieManager;
import org.apache.jmeter.protocol.http.control.Cookie;
Cookie cookie = new Cookie("JSESSIONID",vars.get("jsessionid"),"localhost","/", false,-1);
CookieManager manager = sampler.getCookieManager();
manager.add(cookie);
we run the test - all successful. The file generated has the values
3C7C0BD5CA87C16FA65EAECEEBB54CF6,2db51396-6db2-44cb-9336-c732e52f8ac1
5E83FA9D286583DE32E6CC7C4E799599,f173a3ac-9779-408e-8cdb-7d23dfd5e2df
AFB458860FDF0376B9C8B9F52117A668,9f2da7f4-e848-4267-af9d-806e229f9c9f
540A42700742141A8BB8C2941666C8C9,84355a19-d2e1-45b7-ac42-23e9ab03a195
A7791553761A3613064E6BF7564B76E9,c3b14e92-93bc-4500-93be-ecbe43e70a5a
DB79953CEBA4C6E4FE708AE127E53F26,c99a880d-297d-4c00-9ff2-964449d9d106
3A7BBBC042EDB07D03901EEED8D37DC1,b2b29cb8-5261-4345-9405-80973d9f7334
913AFEC1A2D22EF6C64D7B99675DC86B,13bb3aa1-36cc-4e60-942c-d3a29a74b99f
DD06AEC9AF9D7CFF4FFB0AA2462E63C0,51b7b486-e503-43f8-a3ca-e36eb0df88a8
The cookies get added to the request
We can change the CSV data set config to keep reading the same file instead of terminating at the end - that works as well. You can also check that if you disable the setup threadgroup and rerun the test , it still works (because the sessions are still active - assuming they havent timed out) - so if the data you are saving in this file is still valid it can be used anytime and anywhere. You can also bounce the server in which the JSP web application was deployed. This causes all the sessions to get invalidated and hence the file to have invalid data. Now if you run the test , all the samples will fail because all the sessions are invalid - so even though we added the session id , the server has no data for this session.
Cons : As mentioned above , this isn't realtime sharing , it relies on you already having the data to be used , and the data being valid.
Files used in this post available at
https://skydrive.live.com/#cid=1BD02FE33F80B8AC&id=1BD02FE33F80B8AC!886
References
[1] InterThread Communication-
http://code.google.com/p/jmeter-plugins/wiki/InterThreadCommunication
[2] BeanShell Shared namespace -
http://jmeter.apache.org/usermanual/best-practices.html#bsh_variables
[3] Cookie Manager -
http://jmeter.apache.org/api/org/apache/jmeter/protocol/http/control/CookieManager.html
[4] LinkedBlockingQueue -
http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/LinkedBlockingQueue.html
