Friday, June 05, 2009

Testing multiple environments with JMeter

Motivation
Most projects have a number of environments through which the code moves. e.g. A Development environment, A Test environment, A User Acceptance test environment, A reference environment and finally the Production environment. Hence the same test script is to be targeted to multiple environments. In theory all of this is automated , and anything that succeeds/fails in one environment would succeed/fail consistently in all other environments. However environmental differences and human errors almost always cause one to hear "But it works on my machine"
Whats needed is a way to run the same test against different environments easily
Solution
An assumption we are going to make here is that these tests are automated and would be run from the command line, in our case using ANT.
The solution has to address two basic requirements
a. Parameterization of the various environments
b. You should still be able to run the test in GUI mode (when you are modifying/extending the test)
To implement this we will use JMeter properties[1] and use normal ANT [2],[3] features.

While writing the HTTP tests, add an HTTP Request Default element

If the Server name or IP is to be varied then enter
${__property(run.server,,yourdevserver.com)}
This will look for a property named run.server , but will use yourdevserver.com if no property is specified. For the third parameter, use the server against which you want to run the test while running in GUI mode.

Finally in your Build Script that you use to run Jmeter

<jmeter jmeterhome=".." testplan="${run.test.plan}"

resultlog="${report.dir}/${run.test.report}-${run.env}-${DSTAMP}${TSTAMP}.jtl">

<property name="jmeter.save.saveservice.output_format" value="xml"/>

<property name="run.server" value="${run.server}" />

</jmeter>


where the ANT property run.server can be varied to run this against different environment.

Sample

TODO attach ant and jmx file.


References
[1] JMeter Properties
[2] JMeter Ant task
[3] Ant Manual

6 comments:

Anonymous said...

I don't understand what you are trying to explain.Can you explain little bit clearly with some example please?

Deepak Shetty said...

Typical project lifecycle is you deploy your code to test.yoursite.com and test it out then to qa.yoursite.com for some business users to test then to www.yoursite.com your final website.
When you run a test you just want to say run it against test environment or run it against qa environment or run it against production environment without having to go and change your test (for e.g. if you specify it in HTTPRequestDefault then you need to go and modify the test)

Anonymous said...

Hi Deepak ,
Thanks for your response.I am new to JMeter but i am very familiar with Load runner.

For example,i have recorded the script in http://alpha.mytest.com .
Now i want to test the same script in Beta Environment.In Load runner i just parametrize the alpha portion.That is i create parameter as {server} (which has the value of alpha and beta) and substitute in
http://{Server}.mytest.com.
I did the same thing in JMeter but it throws an error "java.net.UnknownHostException:" along with other java exceptions.
while recording the script i used 'Store 1st sampler of each group only' option.
Also everyone is asking to use http request Defaults.Can you explain clearly with some screenshots please?
I understand that you are a genius in the field.Please explain this topic by putting some screenshots

Thanks in Advance.

Deepak Shetty said...

you need to look at basic JMeter documentation for variables and properties.

In jmeter the notation would be ${Server} if you wanted to use a variable called Server or if you want to read a property then the example is in the post ${__P(propertyname,,defaultvalue)} (A property is a value across threads which is what the server name would be - a variable is per thread so the appropriate mapping is a property)

The next thing is how do you specify the actual value of the variable/property and this blog post describes how to specify a default or pass the value from command line - when you run the test

Venky said...

Hi Deepak,

Is there a way to pass on the host and port info mentioned in the pom.xml to the jmeter test using ${__P(propertyname,,defaultvalue)}

I have a test and a prod profile of the format, xxxx.xxxx.com:8080

Deepak Shetty said...

Since you mention pom.xml I assume you are using maven? I dont use maven so I wouldnt know how to do it.

If you are asking about ANT and Jmeter - yes you can pass anything you want - the example shown does precisely that (passing host name) - you could pass port as well as a separate property