Monday, January 11, 2010

JMeter Graphs and ANT

A follow up to the previous posts, I've integrated the graph code with ANT so that the HTML reports provided by running JMeter from the command line and styling them can be extended to include the graphs.

The sample report that I generated is shown below
The custom report hyperlinks the titles of the normal summary tables to lead to graphs for the same.
To implement this we need to
a. Allow the graph code to be invoked from ANT. This can be done by writing a simple java class with a main method that passes parameters on the command line, or we could write a custom ant task. I wrote a custom ant task as a proof of concept. We need to customise the build script as well
b.Modify the XSLT to write out image anchors.

These steps are described below.
The custom ant Task
public class AggregateGraphTask extends Task {
private String outputDir;
private String outputFilePrefix;
private Boolean showThreshold = Boolean.TRUE;
private Double threshold = 500D;
private String jmeterResultFile;
private String jmeterHome;

public String getJmeterHome() {
return jmeterHome;
}

public void setJmeterHome(String jmeterHome) {
this.jmeterHome = jmeterHome;
}

public String getJmeterResultFile() {
return jmeterResultFile;
}

public void setJmeterResultFile(String jmeterResultFile) {
this.jmeterResultFile = jmeterResultFile;
}

public String getOutputDir() {
return outputDir;
}

public void setOutputDir(String outputDir) {
this.outputDir = outputDir;
}

public String getOutputFilePrefix() {
return outputFilePrefix;
}

public void setOutputFilePrefix(String outputFilePrefix) {
this.outputFilePrefix = outputFilePrefix;
}

public Boolean getShowThreshold() {
return showThreshold;
}

public void setShowThreshold(Boolean showThreshold) {
this.showThreshold = showThreshold;
}

public Double getThreshold() {
return threshold;
}

public void setThreshold(Double threshold) {
this.threshold = threshold;
}

@Override
public void execute() throws BuildException {
try {
GraphClient.init(jmeterHome);
String outputPrefix = outputDir + File.separator + outputFilePrefix;
if(Boolean.TRUE.equals(showThreshold)) {
GraphClient.writeAggregateChartWithThreshold(jmeterResultFile, outputPrefix, showThreshold, threshold);
} else {
GraphClient.writeAggregateChart(jmeterResultFile, outputPrefix) ;
}
} catch(Exception e) {
throw new BuildException(e);
}
}

}


This is a pretty straightforward class which calls our API's based on parameters passed to it. It declares fields for all the attributes it expects and then calls the Graph API's.

The ANT build.
I assume you have JMeter working from ANT, this assumes all the libraries needed for ANT and JMeter are in place

<project name="Jmeter" basedir="." default="runOfflineGraph">
<property name="lib.dir" value="${basedir}/lib"/>
<property name="report.dir" value="${basedir}/report"/>
<property name="styles.dir" value="${basedir}/styles"/>
<property name="export.dir" value="${basedir}/export"/>
<property environment="env"/>
<property name="jmeter.home.dir" value="${env.JMETER_HOME}"/>
<property name="jfreechart.home.dir" value="${env.JFREECHART_HOME}"/>

<path id="run.classpath">
<fileset dir="${jmeter.home.dir}" includes="**/*.jar"/>
<fileset dir="${jfreechart.home.dir}" includes="**/*.jar"/>
<fileset dir="${lib.dir}" includes="*.jar"/>
</path>

<target name="clean">
<delete dir="${report.dir}" />
<delete dir="${export.dir}" />
</target>

<target name="init">
<mkdir dir="${report.dir}" />
<mkdir dir="${export.dir}" />
</target>


<target name="runJMeter" depends="init">
<taskdef
name="jmeter"
classname="org.programmerplanet.ant.taskdefs.jmeter.JMeterTask"/>
<taskdef name="aggregatechart" classname="org.md.jmeter.ant.AggregateGraphTask" classpathref="run.classpath"/>

<tstamp />
<property name="uniqueTStamp" value="${DSTAMP}${TSTAMP}" />
<property name="imageNamePrefix" value="AggregateChartThreshold-${uniqueTStamp}" />
<property name="jmeter.result.fileName" value="${run.test.report}-${uniqueTStamp}" />
<property name="jmeter.result.file" value="${report.dir}/${jmeter.result.fileName}.jtl" />
<jmeter
jmeterhome="${jmeter.home.dir}"
testplan="${run.test.plan}"
resultlog="${jmeter.result.file}">
<property name="jmeter.save.saveservice.output_format" value="xml"/>
<property name="run.threadcount" value="${run.threadcount}" />
<property name="run.loopcount" value="${run.loopcount}" />
<property name="sample_variables" value="${sample_variables}" />
</jmeter>
<xslt
in="${jmeter.result.file}"
out="${report.dir}/${jmeter.result.fileName}.html"
style="${styles.dir}/${xsl.file}">
<param name="imageNamePrefix" expression="${imageNamePrefix}"/>


<aggregatechart jmeterHome="${jmeter.home.dir}" jmeterResultFile="${jmeter.result.file}" outputDir="${report.dir}"
outputFilePrefix="${imageNamePrefix}" showThreshold="true" threshold="200"/>

</target>


<target name="runOfflineGraph" depends="init">
<antcall target="runJMeter">
<param name="run.test.plan" value="OfflineGraphs.jmx"/>
<param name="run.test.report" value="OfflineGraph"/>
<param name="sample_variables" value=""/>
<param name="run.threadcount" value="1"/>
<param name="run.loopcount" value="5"/>
<param name="xsl.file" value="OfflineGraph.xsl" />
</antcall>
</target>
</project>
Important points are
a) we define a run.classpath which has everything we need at runtime to generate the graphs.
b) we have a taskdef aggregatechart for our custom task
c) we invoke the custom chart by passing it the parameters we need. These are closely linked with the previous steps in the build. The result jog from jmeter (${jmeter.result.file}) is passed as an input to the task. The image file names to be generated are important ${imageNamePrefix} as we need to reference this in the stylesheet. The directory to which the Graph code writes must be the same as the XSLT output (or atleast the XSLT and Graph code must be consistent in where the images are referenced from in the HTML)

The XSLT stylesheet
The changes here are pretty straightforward. I've copied extras/jmeter-results-report_21.xsl and renamed it to OfflineGraph.xsl.
The important changes are

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:param name="imageNamePrefix">AggregateChartThreshold</xsl:param>

We pass in a parameter that is to be used while generating the img tag in the HTML

            <xsl:call-template name="summary" />
<hr size="1" width="95%" align="left" />

<xsl:call-template name="pagelist" />
<hr size="1" width="95%" align="left" />

<xsl:call-template name="detail" />
<xsl:call-template name="graph-images" />

We call our custom template


            <th><a href="#AverageGraph">Average Time</a></th>
<th><a href="#MinimumGraph">Min Time</a></th>
<th><a href="#MaximumGraph">Max Time</a></th>

We change the titles to be anchors (named anchors that are next to the images)


<xsl:template name="output-image">
<xsl:param name="suffix" />
<xsl:element name="img">
<xsl:attribute name="src"><xsl:value-of select="$imageNamePrefix" />-<xsl:value-of select="$suffix" />.png</xsl:attribute>
</xsl:element>
</xsl:template>

<xsl:template name="graph-images">
Graphs
<br />
<b>Minimum</b>
<br />
<a name="MinimumGraph"></a>
<xsl:call-template name="output-image">
<xsl:with-param name="suffix">Min</xsl:with-param>
</xsl:call-template>
<br />
<b>Maximum</b>
<br />
<a name="MaximumGraph"></a>
<xsl:call-template name="output-image">
<xsl:with-param name="suffix">Max</xsl:with-param>
</xsl:call-template>
<br />
<b>Average</b>
<br />
<a name="AverageGraph"></a>
<xsl:call-template name="output-image">
<xsl:with-param name="suffix">Avg</xsl:with-param>
</xsl:call-template>
<br />
<b>Median</b>
<br />
<a name="MedianGraph"></a>
<xsl:call-template name="output-image">
<xsl:with-param name="suffix">Median</xsl:with-param>
</xsl:call-template>
<br />
<b>90 percentile</b>
<br />
<a name="NinetyPerGraph"></a>
<xsl:call-template name="output-image">
<xsl:with-param name="suffix">90</xsl:with-param>
</xsl:call-template>
</xsl:template>

Finally we output img tags. The knowledge of how the Aggregate Graph generates the suffix for the images is hardcoded into the stylesheet. The passed parameter is used to form the filename as well (this is used to allow multiple runs , so that the images don't overwrite previous ones)

Running the ANT build
This is the run.cmd I use (windows only)


set JAVA_HOME=C:\bea102\jdk150_11
set JMETER_HOME=C:\projects\R1-Portal-CMS\test\jakarta-jmeter-2.3.4
set JFREECHART_HOME=C:\work\java\jfreechart-1.0.13
set ANT_HOME=C:\work\java\apache-ant-1.7.1
set PATH=%JAVA_HOME%\bin;%ANT_HOME%\bin;%PATH%
set CLASSPATH=%JMETER_HOME%\extras\ant-jmeter-1.0.9.jar;%CLASSPATH%
ant %*
We set some environment properties that the build needs and run it.


Source Code is available here

31 comments:

Anonymous said...

Found your blog posts very informative - thanks for sharing!

Regards,
Anapurna

Anonymous said...

nice one
it helped me lot
Keep Continue..

TestEverything said...

Hi, for generating graph as you mentioned to write ant task.
Can you please tell how to write that task or any jar file. I have to put in ant/bin or where? As a tester i am not aware with ant task creation, can you mention the steps to do just like copy paste.It will be great help for me.
I have Ant/jFree chart/jmeter folders with Environment set
Thanks in advance
Waiting for ur reply

Deepak Shetty said...

The source code for the task is linked in the post. Once you create the jar note that the
<taskdef name="aggregatechart" ..
has a classpathref of run.classpath. Your jar should be present in ay of the paths referred to in run.classpath.

Look at the definition of run.classpath - it includes jars present in Jmeter, Jfreechart and a basedir/lib so you can just drop the jar there. (or you can also drop it into $ANT_HOME/lib

Anonymous said...

Thanks a Lot. Finally I did that and able to generate the report with graphs(min,max,average,90%).Now how can i generate the graph with throughput value?

Deepak Shetty said...

you'd have to write your own code for throughput. the sample included shows how to do it when you run the same test repeatedly for different number of threads - however you might be using the same test with varying the threads within the test. You'd have to write your own ant task or modify the existing one.

Anonymous said...

Hi Deepak,
Thanks for the post. this is very helpful. I was wondering what would we need to change to get the original JMeter graphs instead of these or get a different graph from jfreechart? I mean to get a different type of graph in the html what needs to be changed

Deepak Shetty said...

@Anonymous

http://theworkaholic.blogspot.com/2009/12/graphs-for-jmeter-parsing-jmeter-result.html explains how to get some graphs - it should have most of the logic you need to generate whatever graph you want.

If you arent a programmer then configure JMeter to output CSV and import into excel and draw charts

Anonymous said...

hi my test case is executed but i am not able to generate the graph results.Ony a icon is shown in this case in graph images.

Anonymous said...

Hi i am able to generate graphs.Since i ahd many URL requests , letters are not clear in x axis of images.its not able to accomodate. how to increase the x axis width of imagefile.I have modified the width in all visualizer classses but no use.

Anonymous said...

Hi Deepak. Please guide me how to have bigger graph images so that it can accomodate more URL labels. So that label will be clear rather blur.

Deepak Shetty said...

You need to look at the JFreeChart API's . In my source code there will be code like
ChartUtilities.writeChartAsPNG(fos, chart, WIDTH, HEIGHT);

and you can change the width/height to whatever you want .

Anonymous said...

HI Deepak. I tried the same thing earlier . but not worked. I tried once again carefully.it's working fine. Thanks for your support.Now i am able to get bigger images. How threshold will be calculated? Does average is nothing but threshold?

Deepak Shetty said...

Threshold is something you decide. Say you want all the pages to respond under 5 seconds then thats the value you are looking at (but its whatever you want for your app)

Anonymous said...

it seems it is set to some 220 seconds.How to change threshold value?

Anonymous said...

thanks deepak for such wonderful blog.I've a question for you though i tried to work it out but wasn't able to come to the solution.I'm new at jmeter and ant.My question for you is related to using multiple XSLT files to produce more than one result in ur ant build.How can i accomplish that task?.Any references or pointers would be very highly regarded.Thanks in advance for your help and congrats on such a wonderful blog.

Deepak Shetty said...

@Anonymous

You can use the XSLT tag as often as you want - The input JTL file remains the same, the output file and XSLT changes based on your requirement
e.g.




Deepak Shetty said...

Report 1
<xslt
in="${report.dir}/${run.test.report}-${run.env}-${DSTAMP}${TSTAMP}.jtl"
out="${report.dir}/${run.test.report}-${run.env}-${DSTAMP}${TSTAMP}_reporttype1.html"
style="${styles.dir}/${xsl1.file}"/>

Report 2
<xslt
in="${report.dir}/${run.test.report}-${run.env}-${DSTAMP}${TSTAMP}.jtl"
out="${report.dir}/${run.test.report}-${run.env}-${DSTAMP}${TSTAMP}_reporttype2.html"
style="${styles.dir}/${xsl2.file}"/>

Anonymous said...

Hi thanks for such a wonderful blog.Since this post is about using ant i want to ask a question.How do i save perfmon data using ant and display the result as html.Help will be appreciated.
Thanks again

Anonymous said...

Yeah actually I referred this one yesterday.so my question is where do i write and where do i save Custom ANT task.

i think i am getting prolong error bcoz i wrote ur code in build.xml
currently i run my jmeter scripts from Ant cmd line using build .xml .

also is there any additional jar i need for supporting Custom ANT task?

Anonymous said...

I found it i think. you wrote it in eclipse and saved it as .java file.
it looks like a very high level code
for begginers like me :).I wish i wld be like you.I will go through your code and will try to implement.

Stay tune bunch of question coming up from me.

is this blog the only way to reach or get help from you?

Deepak Shetty said...

@Anonymous
Yes this was written for developers - So using the eclipse workspace you have to export the code as a jar file - in order for ANT to pick up this file , the jar file needs to be in a classpath (the easiest way to do this is to copy it into ANTs lib directory)

Usually JMeter help can be got from emailing JMeter mailing lists - However since this is nothing directly related to jmeter , either the blog or my email is the only way to contact me

Unknown said...

Hi Deepak,

This post was really helpful

Sad part is that I do not have any development knowledge...
So I am unable to create jar file using the class provided..

I guess you might have generated the "AggregateGraphTask.jar" file... Can you share the jar file

Unknown said...

Tried as below:
Using - Apache-Ant-1.9.4
Created Build.xml as following:
---------------------------------------------------------------





















---------------------------------------------------------------
Created "AggregateGraphTask.java" with the code provided in the main post class.

Executed the ant build.

Getting below error:
******************************************************
D:\Apache-Ant-1.9.4>ant
Buildfile: D:\Apache-Ant-1.9.4\build.xml

compile:
[javac] D:\Apache-Ant-1.9.4\build.xml:14: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set t
o false for repeatable builds
[javac] Compiling 1 source file to D:\Apache-Ant-1.9.4\classes
[javac] D:\Apache-Ant-1.9.4\src\AggregateGraphTask.java:64: error: cannot find symbol
[javac] GraphClient.init(jmeterHome);
[javac] ^
[javac] symbol: variable GraphClient
[javac] location: class AggregateGraphTask
[javac] D:\Apache-Ant-1.9.4\src\AggregateGraphTask.java:65: error: cannot find symbol
[javac] String outputPrefix = outputDir + File.separator + outputFilePrefix;
[javac] ^
[javac] symbol: variable File
[javac] location: class AggregateGraphTask
[javac] D:\Apache-Ant-1.9.4\src\AggregateGraphTask.java:67: error: cannot find symbol
[javac] GraphClient.writeAggregateChartWithThreshold(jmeterResultFile, outputPrefix, showThreshold, threshold);
[javac] ^
[javac] symbol: variable GraphClient
[javac] location: class AggregateGraphTask
[javac] D:\Apache-Ant-1.9.4\src\AggregateGraphTask.java:69: error: cannot find symbol
[javac] GraphClient.writeAggregateChart(jmeterResultFile, outputPrefix) ;
[javac] ^
[javac] symbol: variable GraphClient
[javac] location: class AggregateGraphTask
[javac] 4 errors

BUILD FAILED
D:\Apache-Ant-1.9.4\build.xml:14: Compile failed; see the compiler error output for details.
******************************************************

Unknown said...

Sorry some how build xml is not displayed in comment

Deepak Shetty said...

@srini boy
The source code uploaded was an eclipse workspace (not an ANT build)
So you would have to download eclipse, open this workspace, import the java project and then just say export jar (I will need a day or so to do it since I dont have a setup available)
However if you are not a developer , you will run into many issues - It's probably better you do one of

a. Write the logs to CSV in JMeter and use excel (for smaller apps)
OR
b. use Jmeter-plugins
OR
c. Many examples on Internet that are more user friendly than my examples

Venkat Matta said...

Hi Deepak,

Thanks for your blog and very helpful also .

I am trying to run the jmeter tests in our remote machine by using ANT build.Please let me know what are the properties we have to add in the ant build

And i am trying to add jfreechat also .. i add the jar files to the classpath but i am unable to generate the charts.Could you please elaborate little bit.

Regards,
Venkat

Deepak Shetty said...

To run Jmeter from ANT you need
http://www.programmerplanet.org/projects/jmeter-ant-task/

You havent provided enough details to attempt an answer to your question

Unknown said...

Hi Deepak,

Great post.. I can get the table part in the html report, but I'm not able to get the charts in the report.
I downloaded your source code, exported to a jar file using Juno.. then placed the jar in JMeter's "extras" folder. (I faced a few issues with source code as well, error msg stating that "the jars(junit, itext etc.) were not there in the classpath").
Then made the necessary changes in the build.xml. But still the graphs are neither generated in the extras\report folder (.png images) nor getting embedded in the report html doc.

let me know if there's anything that I have missed..

Thanks,
Namrata

Deepak Shetty said...

@Namrata
>then placed the jar in JMeter's "extras" folder.
These charts are not part of JMeter runtime - The way this is supposed to work is, you run your tests and get the results as a Jmeter results file (.jtl) (also as part of the ANT build) - Then you process the result .jtl file with this code and generate the graphs.
So if you have created a .jar file with this code , it needs to go into a directory thats being referenced in ANT - like in the example I put it in my lib directory and used that







Note that Im also running my JMeter test first and then using the generated result file directly to create the graphs. if you arent doing that then you need to modify the ANT script to read the result file and pass it in.

Finally if it still doesnt work you will have to see each step and see whats failing
a) Did the Jmeter test run(OfflineGraphs.jmx in my example) and did it create the result file(OfflineGraph) ?
b) Did the xslt target run ?if so the HTML file should get generated
c) finally did the java target run ? If not what error got printed (run ant with verbose and debug flags)

Deepak Shetty said...

Looks like blogpsot swallowed the code snippet - The section that has classpath is
path id="run.classpath" in the build file (Also check all your java versions - dont compile with a higher version of java and run with a lower version)