Thursday, January 02, 2014

Modifying JMeter scripts programatically

Problem : Is there any scripting or GUI way to massively change from WebService(SOAP)
Request (DEPRECATED) to SOAP/XML-RCP Request?

Solution : The JMeter script file is an actual XML file so any solution that manipulates XML can be used to solve the above problem.  As Michael Kay's excellent books on XSLT were staring at me , the way I implemented this was in XSLT.
The first thing we need to do was use what is called an identity transform
<xsl:template match="@*|node()">
  <xsl:apply-templates select="@*|node()" /> 

i.e. essentially copy the source XML to the destination. With that done we can now get to the problem at hand - which is intercept all the deprecated requests and transform them to the new one

In order to do so we can create a JMeter script which has both of these elements and look at what XML gets generated so that we know the source and destination XML we want
The deprecated XML
 <WebServiceSampler guiclass="WebServiceSamplerGui" testclass="WebServiceSampler" testname="WebService(SOAP) Request (DEPRECATED)" enabled="true">
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
            <collectionProp name="Arguments.arguments"/>
          <stringProp name="HTTPSampler.domain">server</stringProp>
          <stringProp name="HTTPSampler.port">port</stringProp>
          <stringProp name="HTTPSampler.protocol">http</stringProp>
          <stringProp name="HTTPSampler.path">path</stringProp>
          <stringProp name="WebserviceSampler.wsdl_url"></stringProp>
          <stringProp name="HTTPSampler.method">POST</stringProp>
          <stringProp name="Soap.Action">ws-soapaction</stringProp>
          <stringProp name="HTTPSamper.xml_data">ws-SOAP-DATA</stringProp>
          <stringProp name="WebServiceSampler.xml_data_file">C:\Users\dshetty\Downloads\apache-jmeter-2.9\bin\logkit.xml</stringProp>
          <stringProp name="WebServiceSampler.xml_path_loc"></stringProp>
          <stringProp name="WebserviceSampler.timeout"></stringProp>
          <stringProp name="WebServiceSampler.memory_cache">true</stringProp>
          <stringProp name="WebServiceSampler.read_response">false</stringProp>
          <stringProp name="WebServiceSampler.use_proxy">false</stringProp>
          <stringProp name="WebServiceSampler.proxy_host"></stringProp>
          <stringProp name="WebServiceSampler.proxy_port"></stringProp>
          <stringProp name="TestPlan.comments">comment</stringProp>

The non deprecated SOAP request
        <SoapSampler guiclass="SoapSamplerGui" testclass="SoapSampler" testname="SOAP/XML-RPC Request" enabled="true">
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
            <collectionProp name="Arguments.arguments"/>
          <stringProp name="SoapSampler.URL_DATA">url</stringProp>
          <stringProp name="HTTPSamper.xml_data">data</stringProp>
          <stringProp name="SoapSampler.xml_data_file">c:/test.xml</stringProp>
          <stringProp name="SoapSampler.SOAP_ACTION">soapaction</stringProp>
          <stringProp name="SoapSampler.SEND_SOAP_ACTION">true</stringProp>
          <boolProp name="HTTPSampler.use_keepalive">false</boolProp>
          <stringProp name="TestPlan.comments">comment</stringProp>

Knowing this , the template is easy
  <xsl:template match="WebServiceSampler ">
    <xsl:element name="SoapSampler">
        <xsl:attribute name="guiclass">SoapSamplerGui</xsl:attribute>
        <xsl:attribute name="testclass">SoapSampler</xsl:attribute>
            <xsl:when test="@testname ='WebService(SOAP) Request (DEPRECATED)' ">
                <xsl:attribute name="testname">SOAP/XML-RPC Request</xsl:attribute>
                <xsl:attribute name="testname"><xsl:value-of select="@testname"/></xsl:attribute>
        <xsl:attribute name="enabled"><xsl:value-of select="@enabled"/></xsl:attribute>
        <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
            <collectionProp name="Arguments.arguments"/>
        <stringProp name="TestPlan.comments"><xsl:value-of select="stringProp[@name='TestPlan.comments']/text()" /></stringProp>
        <stringProp name="SoapSampler.URL_DATA"><xsl:value-of select="stringProp[@name='HTTPSampler.protocol']/text()" />://<xsl:value-of select="stringProp[@name='HTTPSampler.domain']/text()" />:<xsl:value-of select="stringProp[@name='HTTPSampler.port']/text()" /><xsl:value-of select="stringProp[@name='HTTPSampler.path']/text()" /></stringProp>
        <stringProp name="HTTPSamper.xml_data"><xsl:value-of select="stringProp[@name='HTTPSamper.xml_data']/text()" /></stringProp>
        <stringProp name="SoapSampler.xml_data_file"><xsl:value-of select="stringProp[@name='WebServiceSampler.xml_data_file']/text()" /></stringProp>
        <stringProp name="SoapSampler.SOAP_ACTION"><xsl:value-of select="stringProp[@name='Soap.Action']/text()" /></stringProp>
        <stringProp name="SoapSampler.SEND_SOAP_ACTION"><xsl:value-of select="(stringProp[@name='Soap.Action']/text() != '')" /></stringProp>            
        <boolProp name="HTTPSampler.use_keepalive">false</boolProp>        

Where we just map the older elements into their newer forms (wherever possible). So for example we match an element named "WebServiceSampler" and replace it with "SoapSampler" . We transform the testname to either use the default if the deprecated element used the default or we use the name the user specified. We copy attributes and nested elements.
 You can then use ANT or any other way to style your input scripts into newer ones for one or many files
<target name="transformWS">

An alternate approach would have been to generate java code that could parse XML and remove/replace objects but the identity transform mechanism makes this technique hard to beat. Any Object binding mechanism would run into issues when JMeter changes it schema but the XSLT technique is limited to only having to worry about source and destination.
I've also always toyed with the idea of creating a functional suite of tests that user's would  run at will without being limited to the tests defined in the file and this is the approach I would use there as well - more to come.

Example -!892


Stephen said...

Hi Deepak,

It was great and exceedingly useful reading your blog in understanding JMeter.
I require your Expertise suggestion for one of my requirement.
I have a requirement to Automate functional Testing of an Enterprise Application which is built on ATG Platform.
I am required to provide an automation solution which is UI Independent (To overcome drawbacks of Locator Dependency). I was suggested JMeter as a solution to accomplish this by my Peers. I am successful in accomplishing certain functions like Login and Registration.

My ask is,

Is JMeter capable of handling execution of thousands of test cases on an enterprise level?
Is JMeter the right solution to be taken forward?

Your suggestions will greatly help in taking a critical decision.Thanks in Advance!!!

Deepak Shetty said...

>Is JMeter capable of handling execution of thousands of test cases on an enterprise level?

It sounds as if you want to combine unit tests + functional tests + load tests ?
In which case the answer is "maybe".
a. Reuse in JMeter is possible but is somewhat clunky . UI test cases usually have a bunch of steps that are repeated.
b. Other unit/functional test tools like say TestNG have quite a few features that are important to functional tests (e.g. the ability to mix and match test cases to create something like "basic" ,"advanced", "ordering" test suites that allow you to run the same tests in varying combinations.
They have the ability to express dependencies (e.g. execute Login before executing Order test case) in intutive and easy ways
Some of these are possible to do with coding discipline in JMeter , but its not out of the box support.
c. Some UI functional tests are best handled in tools like Selenium(especially AJAX) - Jmeter is not a browser - yadda, yadda
If you still want to use JMeter you'd have to figure out how to organize your various scripts - One large script doesnt work too well.
Jmeter does better when the steps for the test case are more or less the same but you have variants of data that are fed into the same scripts. It also lets you create functional scripts that can be very changed easily into load tests.

My personal philosophy is to use multiple tools - even if I end up duplicating some efforts. So likely in your case I'd probably not rely only on JMeter
To reiterate there are two ways to run HTML type of tests
a. You drive a browser (e.g. Selenium/QTP)
b. You simulate the HTTP packets (Jmeter, HTTPUnit etc)

Each technique has advantages and disadvantages and usually one solution doesn't fit all use cases .

>Is JMeter the right solution to be taken forward?
Thats not a question I can answer with the data you have given. The largest one I have done is about 200 tests - but note I am a dev , not a tester - but I will say I got better results than the official test team who used QTP/load runner

Another factor that is often ignored is who will be audience who will creates the tests for you and what their skill sets are. To use Jmeter effectively you must have basic programming expertise , preferably java (but if you know any language , its not that hard to pick up). If you have manual testers then are likely to prefer point and click record and replay tools rather than Jmeter.

Stephen said...

Thanks a lot for your valuable suggestions Deepak!!!