Sunday, August 23, 2009

Performance Tuning tales

When you ask a J2EE guy something about performance tuning you'd probably get something that includes JVM tuning heap space, survivor ratio's , types of GC or you might get the don't use EJB , minimise the number of remote calls , or the use Hibernate , use Prepared Statements, use Cache etc etc. Interestingly enough for my current project we had implemented most of the above, tested it locally and seemed to get <4 second times for most pages. And we missed a big problem (hint it's a web based system) , can you guess?
The system responds in under 4 seconds when accessed locally. When accessed through a browser based in New York some (Can you guess?) requests take 20+ seconds. Luckily we did run external tests as soon as the production systems were available so we found this out a couple of months before the system was actually released.
In any case , in hindsight as soon as we knew the problem existed it was relatively easy to guess and verify(YSlow) the problems. We only had few small images so we knew the problem wasn't there.
a. Whats good for development isn't necessarily good for deployment. e.g. We do normally split CSS files , Javascript files and include them separately. Yes a browser will cache these files ( you do add Etags don't you?), but the first request and HTTPS will be slow if the browser must make these connections (normally at the most two at a time). - We used Yahoo and ANT to combine the CSS into one file and the Javascript into another (at build time) drastically reducing the response times
b. GZIP. Creating Gzipped versions of the files and dynamically gzipping all the content (a flag on the webserver) also brought down the times.
Just doing the above brought down the times to under 4 seconds even when the site was accessed remotely on limited bandwidth clients.
Moral of the story : Always test , never guess when it comes to performance tuning.

Wednesday, August 19, 2009

Testing

There are various kinds of test's that can be performed on a system , but it looks like most engineers don't agree on the definition of the test. For e.g. in my current project a 'unit test' tests out the end to end functionality but it must be performed by a developer i.e. any test that a developer performs is a unit test!
In any case this is my usage

Unit Test - Generally meant to represent that only a small subset of the code is under test. You might find a few fanatics who argue to the effect that anything that goes to the database isn't a unit test, anything that needs an external interface isn't a unit test etc etc. Ignore them. The key is that the test might not exercise the flow as it finally would. There might be mocked interfaces, hardcoded data. Generally written by developers and generally easier to automate. Unit tests are also low hanging fruit, and contrary to what most agile engineers will tell you, are pretty much useless expect to impress some manager with 'We have automated unit tests!' , 'We have continuous builds with JUnit reports!','We have 99% code coverage!'. The quality of these tests written is mostly poor (they are written by developers after all), the test data provided rarely covers boundary conditions, invalid data, exceptional conditions. Yes Yes I know you should have high code coverage. It's quite easy to game the high code coverage (which is what happens when metrics are imposed by management) but even if it wasn't, the code coverage is only as good as the code and the test.
e.g. function int add(int one, int two) {
return 4;
}
Test function with add(2,2), add(-1,5) , 100% code coverage , automated unit tests all green , mission accomplished!?
i.e. If the developer doesn't account for the boundary conditions in code, he isn't likely to test out those scenarios either.

Integration Test - Generally tests out the interface between multiple systems, though the data might be faked. If there are only two types of test that you can carry out on your system then this is one of them.
These tests are extremely hard to write (or atleast make it repeatable), and are very very useful. The earlier you can have these tests up and running on your system, the better the quality of your system. These tests are hard to write not because of technical problems(which exist) but because of people issues. Different systems are normally run by different teams (sometimes even external to your organisation) have their own schedules , develop at their own pace and make their own assumptions and are notoriously non co-operative. Technical limitations are normally due to non repeatable time sensitive data which either needs the data reset to a known state or data created from scratch.

Functional Test (End to End test). - Tests out functionality from a user's perspective. This is the second type of test that must be run and the earlier you can run these tests in your system the better the quality of the system is. Normally nothing is faked , actual data is used for the tests. When these tests are performed by a business user / stake holder these become Acceptance tests. These tests are extremely important and difficult to fully automate. e.g. These would also include UI tests (this page doesnt work on my browser!).

Performance/Load - Functional tests run in parallel (normally a good mixture) with varying concurrent users. Easy to do if the functional tests can be automated. In most cases difficult to simulate (especially if the system is already live, easier to do the first time). Depending on the duration you run this is also by different names. Some organizations refer to long running tests as smoke tests. (used to smoke out memory leaks) whereas some organisations use smoke tests to refer to some important functional tests that are run to check that no major errors exist.

System Tests - Used to check that all systems are up and running. Not really a test category by itself , but useful to abort test runs.