JMeter and SLA's

One of the current issues on our site is that while we profiled and performance and load tested the important pages before we went live, we haven't done it for subsequent builds and releases. There are various excuses for this (lack of time, lack of representative environments, restrictions on the actions that may be performed because the site is live), none of them really justified. However no matter how much you test the site before hand, the site may still malfunction, perhaps transiently on production. For e.g. we sometimes got timeout errors between 6:00 to 10:00 a.m. (it was eventually determined to be a Database index compacting job that was creating trouble). The problem is that we had to be reactive, look at the logs, there was a timeout, run around like headless chickens because the site was working fine now, no access to the environment to see whats happening etc. Now we could have configured logs to automatically notify us when there are errors but this would only work if there was a timeout (in our 60 seconds for any remote operation). If a page that normally takes 2 seconds to load took anything under 60 seconds we would not see errors.
In previous projects , we had OVIS, which I believe is expensive, but my current project has no such commerical tool. Open source tools all seem to solve parts of the problem , but there didn't seem to be any tool that did everything I wanted.
Briefly
a. Flexible schemes to measure response times. We needed to be able to simulate accessing stand alone urls, login flows, checkout flows, search flows.
b. Ability to store the data and view trend graphs
c. Ability to run the tests on a schedule
d. Ability to specify thresholds for each page (again with a fair degree of flexibility) and mark responses as failed
e. Flexible notification schemes

The choice of technologies I used were based more on things I wanted to learn or refresh rather than the best there is, so keep in mind this is more of a toy than I would have liked
a. JMeter for response times. I'm not really interested in loading the site, nor do I want exact browser render times, I'm just looking for ballpark numbers and deviations, especially after builds or at odd hours.
b. Hudson for scheduling Jmeter builds. I chose hudson because I haven't used it before.
c. STAX for parsing JTL. I chose stax because I wanted to be able to parse large files, and i already know SAX , but I've never used STAX.
d. Tomcat with JSP + Spring. I've loved Spring JDBC ever since I've used it (take that Hibernate, JPA, JDO, EJB). It removes all the redundant code while not sacrificing the power of SQL , and there is no learning curve beyond Spring. I chose JSP over any of the MVC framework because of shortage of time. While people may insist how their preferred framework saves them tons of time , this only applies in the long run
e. JQuery for all the javascript stuff
f. JFreeChart for the chart related functionality. I've used this before and found it to be a solid library.
g. Derby for the database, this is something I have not used before, I wanted a reasonably stable database , non embedded.

Most of the things Ive written aren't really reusable, in addition this was quick and dirty, so don't expect this to work for you
a. The JMeter Script
b. Parsing the JMeter Script and loading it into the database
c. Scheduling JMeter to run in Hudson
d. Writing a UI around this
e. Allowing administrators to configure thresholds and notifications
f. Notifications