More straws

http://jeffsutherland.com/scrum/2006/11/scrum-supports-cmmi-level-5.html
http://jeffsutherland.com/scrum/2006/11/is-cmmi-worth-doing.html
From the inventor of scrum
(Disclaimer i am working in a CMM Level 5 company and its possible indeed probable i am biased)
So the agile management folks see value in CMM level 5 principles. After all what could be incorrect about
Level 1 - Uncertainty. Success depends on individual effort.
Level 2 - Awakening. Basic project management practices are established.
Level 3 - Enlightenment. Standard process throughout organization.
Level 4 - Wisdom. Detailed metrics are collected and evaluated.
Level 5 - Certainty. Continuous process improvement via metrics feedback.
Wouldnt everyone want to be level 5 , certain their project succeeds?
But isnt agile about people above process?
Doesnt agile tacitly imply that if the people arent good no process is going to save anything?
Certainly there isnt anything in agile that forbids collection of metrics and forbids improvement. Indeed doesnt the automation of everything in sight actually allow you to do this.
But but but and herein lies the problem, the practice of collecting metrics is so perverted (see the timesheets which have a task called filling in timesheet) that almost always means that people will fudge it.
It also looks like agile management consultants have discovered there is money to be made by coaching people to be agile scrum cmmi pcmm bs577 compliant. Or whatever.
So will the agilisits be enlightened? Does a standard process have more value than a tweaked one? Whatever happened to the these are guidelines and must be changed for your project, use whatever works for you?

Agile no more

Its amazing to see how developer friendly practices are perverted into management friendly practices as soon as a pack of managers are let loose on them.Right so lets begin at the beginning Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan (http://www.agilemanifesto.org)While we could quible over whether we collaborate with customers or negotiate with them , its fairly reasonable to assume that no software engineer is going to disagree with the spirit of what was said here.All of the above principles are developer friendly. No more follow that CMM level 5 process, No more according to my project plan we should have finished 53.23% percent of project. are we there yet?But look agile has crossed the chasm and what do we haveAgile consultants who have the "you must tailor agile to meet your needs" that means you need an agile coach or a scrum master whatnot because evidently if you tried to do it yourself your project would fail because you would do agile wrong.Oh look agile projects do need management. And it must be something new otherwise how would we consultants ever earn money?Heres a new management technique. Lets call it Scrum. We cant obviously have a position called Project Manager , lets call it scrum master. Lets not call it a requirements document, lets call it a product backlog. Lets not call it an iteration , lets call it a sprint. Oh yes how about burndown charts, yep sounds like something people would pay money to be taught what it is. Oh and wait we need to know what our teammates are upto every minute. Lets ask for status updates every day. Oh and we need to have data on the minutest of things. Hmm we cant call it a timesheet but we'll make them fill up the hours for every minute task and we'll make them constantly reestimate each task.Ok so thats an exaggeration.But think about it. Say the functionality on a website was show orders for a user. What would a developer say (lets assume the overall architecture and software and platforms have been decided). Mostly something like ok it would take about a week for me to do it. If you fell behind you 'd sit up for 2 hours more if you were ahead you'd quit and go home early. In about a week you'd be done if you were reasonably competent. Developer friendly? Id think so. if there were problem you'd escalate as soon as you knew of it. If the problem was technical you'd ask your peers at lunch or tea.But thats not what's being practised.1 week as a coarse estimate wont do any more. You have to provide finer tasks so that you know if you are in red. You have to track at an hour basis for the tasks. You have to give an update every day. You have to work uniformly for 8 hours every day because supposedly that increases productivity. Bah. Anyone who's coded knows that if the problem is interesting and you've started on it , you wont stop unless you've solved the problem Or atleast its that way for some people.Not all people are alike.And so we come to the original manifesto. For the agile projectsIs it individual and interactions over processes and tools ? Probably not. look around the debates in agile . To pair program or not to (process) , to TDD or not to (process), unit test all code (process), daily standup calls(process), use wiki for collaboration (tool), use junit(tool) etc etc
Working software over comprehensive documentation ? Product/Sprint Backlogs, Burn down charts, User stories (all documentation)
Customer collaboration over contract negotiation ? Ambiguous at best. And certainly i have yet to see any agile consultants who havent negotiated contracts.
Responding to change over following a plan? No more and no less than before, perhaps evidenced by the fact that the manifesto is signed in 2001 and 5 years hence there has been no change to it. Lessons learnt in the 5 years have definitely not caused the manifesto to change. Rather any failures are met with the standard "you arent doing it right"
Perhaps the problem is in the first item in the process. Individuals. And individuals are flawed and biased.

More Agile

Burst out laughing when I read
“Soon all of our singletons were composite, we could use the factory method on our abstract factory to get the prototype of the builder which would make a proxy for the decorated adapter that acted as a facade for the visitor which took us to see all of our singletons”- http://jroller.com/page/rolsen?entry=turn_it_up_to_11
Since it reminds me of a certain project I was in where the chief designers had obviously come out of a design patterns course and so there were *Factory aplenty which then lead to the creation of a class named FactoryFactory and the clients of this API who ofcourse had their own factory came up with FactoryFactoryFactory. Ok the last class is fictional , but the one before that is an actual class.
And so yes the latest rage is Agile and Scrums and what nots. I remember reading Kent Beck’s book on extreme programming about 5 years ago when agile had not yet caught up here in India, agreeing vehemently on the accept change and disagreeing just as vehemently on pair programming (some people interpret this as side by side programming, a practice I wholeheartedly endorse , and not just for technical reasons, its how I met my wife to be :-) )
As the years went by and I saw more and more of the strict waterfall model , the im not going to start design till requirements are signed of, im not going to start development till design is signed off, I realized the agilest were right , waterfall like practices are no way to run a project. Agile was the way to go. And though we didn’t practice all the various XP practices or Scrum or whatever, most projects I worked on were pretty much flexible, with changes being accepted after the date for code freeze (which magically thawed)
Oh just to clarify , we are a service/project oriented organization , providing services to various clients. The client I work with now has decided to go Agile in a major way and to impress the client agile training is now mandatory for everyone in our group in our organization. And ive kept up with some of the articles that the agilists keep publishing. And so the inevitable conclusion. There is no practice in the world that people cant screw up.
So lets see where do we start.
First the proponents of agile itself. While they make the right noises (Each process must be tailored to what works for you), the answers given most of the time is ,(to quote cedric beust) “if you aren’t getting benefits from agile , you aren’t doing it right.”
Second the insistence by some on absolutely silly practices. Like Code the test first before you write the code. I cant and don’t want to code tests that don’t bloody well compile , I have to define the interfaces and classes implementing those first atleast! Though of course testing is good , automated tests are better, unit tests are ok, mock tests are ok, Integration tests are worth their value in gold. But try getting some agilist to talk about tests other than unit tests.
Then we have the silly rules that follow from the above. You must have 90% coverage in your tests cases, otherwise code is rejected. Heres a thought experiment. Write a function doubleIt , return the value 4 always and write a test case which passes in the value 2. You have voila 100% coverage. Beat that. No need to write more tests. If you impose silly rules , developers will work around them.
And then we have the other problems. Daily status calls, not exceeding 15 mins. Must be at the start of the day. Ok cool. Umm the delivery head is US based, the customer in UK and the dev team in India. Whats the start of the day? Oh agile doesn’t work so well for distributed teams! Well what do I do then go back to waterfall? Stupid.
And the ofcourse the famous if you aren’t doing practice x, you aren’t agile.
I think I prefer the time when in a project of duration of 1 month you could relax for 15 days and then work double for the next 15 instead of work uniformly for the 30 days.
Though people may say that quality in the latter case is better, I don’t think that’s necessary.
I don’t think agile works in the real world. Distributed teams are a fact of life. Multi vendor projects with conflicting interests are a fact of life too. I suppose agile would work with a closely knit team. But that’s fast becoming an exception in the areas we work.
And with new processes and positions (like agile coach) the Agile methodology is becoming as rigid as the other processes.
And a final thought whats remained the rule in software , good people can make any process work, incompetents can make any process fail.

Agile

The other day I got a ppt from a friend who got it from someone else about a project in my company which is a “success” because how they implemented “agile methodologies” and how it helped and the practices they followed with general patting of backs and lessons learnt and improvements still to be made.
It’s a 75 slide power point.
Get the picture?
Or lets see the Extreme programming principles. On one hand embrace change and on the other you must follow x,y,z to have a successful project(with no guarantees of success). What happens if you don’t pair program or do test driven development or whatever else is the latest breakthough in software engg? Is it that you don’t embrace change?
I think agile is best described by uncle bob , “never be blocked”. And once you accept that, the agile manifestos, the programming principles the slide guides aren’t all that necessary. True they might help, but the problem is any rigid process (agile extreme or otherwise) causes blocks.
Write junits for everything. Unit test are really low value for money. Integration tests are very hard( especially if it’s a distributed system, developed by multiple disparate teams )over space and time. All the wonderful stubs and mocks so loved by the TDD guys fail when the stub and mocks are replaced by the actual system. Which isn’t to say stubs aren’t good, but they give you a false sense of security. And can hide flaws. For e.g. There was a defined web service interface. It wasn’t ready we mocked it. All was well. Till we ran it with real data. Turns out we get stuff like ##YOUSHOULDNTSEETHIS## the xml structure is the same , the web service has the same interface but surprisingly none of us thought to write stubs and mocks that returned elements with junk values.Or what to do in that case or even how to identify them.
You can and should violate principles if the need arises. That’s agile. Embrace change. a

The Fallacies of Design

The Fallacies of Design
While reading the various UML/Pattern/Architecture/OO/XP/TDD books and literature, I think most authors forget some of the basic maladies that plague software projects. Or perhaps because there are no solutions for the problems, it is best to ignore them. So far there have been very few insurmountable problems as regards to the technologies involved in projects that I have worked on. They have always been workarounds , tactical pathworks and other hacks. But there are the other issues which can derail a project.
So here are the various fallacies I have seen

1. There is a best solution that solves your problems.


2. Systems that you need to integrate/interact with can be changed to implement the best possible solution


3. People within the project will cooperate with you. People outside the project will cooperate with you


4. The People defining the requirements know what they want. Once a requirement is signed of by said rule, it wont change


5. Budget is infinite, Time is infinite


6. Managers can manage, Designers can design, Analysts can analyze and coders can code.


7. The solution to any problem is a pattern/platform/approach/methodology


8. If the test cases pass and code coverage is > x% then there wont be problems in the wild


9. People will always tell the truth


10. Murphys law does not apply to software.

None of these fallacies are technical. They aren’t meant to be. Its rarely the technology that derails a project.

Design Patterns v/s Experience

While reading through TestNG and comparing it with JUnit, it is obvious that TestNG is far simpler to use and at first glance looks to be designed better (or has more features). Note I haven’t written a single TestNG test yet.
And I remembered reading , when people complained about the limitations of JUnit that JUnit was designed as a framework that could be extended. And how JUnit had used all sorts of design patterns.
Which leads us to Design Patterns v/s Experience question. I do not suggest these two are mutually exclusive. The comparison of JUnit and TestNG is not strictly fair, and neither is this a feature by feature comparison, nor an indictment of design pattern approaches. Its simply that having JUnit (and its varied extensions around) allowed Cedric to create a better framework. Like not having to implement a class or interface for your tests. The different ways of running setup (as opposed to the design pattern used by Gamma and co.) . And while a catalog of Design Patterns exist, how do you quantify or catalog the Experience that allows you to select the correct Design Pattern?
Experience also teaches you when design patterns need not be used. (oh yes there are cases, but I leave that for a separate entry).
And if you had to choose a candidate with Design patterns on his CV(All candidates know factory,singleton,value object, dao and session façade) or one who seemed to have the experience, choose the guy with experience (This has nothing to do with 6 yrs experience and no DP in CV!)

Youth And Responsibility

Its obvious to anyone who thinks about such things that the only hope for the future are the youth. The big picture aside, lets look at youth only in the context of the workplace. Its also painfully evident that no one trusts the young trainees just out of the trainee program.
We are of course polite. They dont have enough experience we say, we cant risk it - what if they screw up on production?. We(the experienced people, the guys who have seen it all) can't spend all our time reviewing what the trainees are delivering. In other words it is easy to find rational arguments which prevent us from delegating responsibility/tasks to the young 'uns.
The plain truth is that we don't trust them. Or we trust that they will screw up.
Which is a sorry state of affairs.
I need to break out of this mental block . And I think I'm still someone who trusts the younger people around him a lot more than others.
But it has to improve

The Anarchist Management Model

One of the things that keeps occupying my thoughts these days is how much that we learn from life can be applied to our work.
One of the major problems in work can be summarised as Management. Most non managers would not hesitate in calling Management evil , and only perhaps were we really feeling charitable would we call Management a necessary evil. But Managment persists as a practice , even worse you have younger generations(am i really this old?) growing up wanting to be managers. You have people joining the project stating that they want a management role.
As such I'm against all sorts of management in principle. I think this has to do with the fact that as a form of governance I prefer Anarchy. And currently Management is a pseudo democratic dictatorship. You dont really have a say on who is your manager (hence a sort of dictatorship) but perhaps results /feedback can get the manager removed.
Id like to see a project where people take responsibility for their actions. Divide the work amongst themselves based on their competencies. Set targets for themselves and meet them. Resolve their problems without needing outside intervention.
Could it Happen ?

Culture and Design

Does the culture we have been brought up with affect the design work we do?
Case In point.
One of the clients of my company is spending large amounts of money to attain homogenity in the technology/platforms/systems it uses to avoid the integration problems that arise with heterogenous systems.
My personal preference has always been to develop systems independently using whatever is best for that system. And then figure out ways to integrate them.
Both approaches have their shortcomings and advantages of course.

Im wondering if the fact that diversity that is so much a part of Indian life has anything to do with my preference. If so i'm grateful for it.

Interviews

One of the things that is part of my my workstack is obliging Recruitment whenever they want some J2EE interviews conducted.
It is a highly painful experience.
On one side we have the questions that we are given to ask by Theoreticians who seem to be detached from real world programming (what is the output of i++ + ++i or how many overloaded constructors does the File object have - note these arent really questions we are supposed to ask , but you get the idea) .
On the other side we have the candidates. It looks like they come prepared to answer questions like the above. And because I always begin with what are the technical details of your project and what did you work on, 9 times out of 10 the candidate looks back with a blank stare. Or provides some inane details like There is a login screen , the user enters his password and is taken to his homepage. The End. I am not alone in this regard. Other people from my company have expressed similar sentiments. Perhaps the reason is no good candidates are applying to my company :-) .
This actually leaves only one way out if we want competent resources. Hire people out of college. Train them and then mentor them.
It remains to be seen how successful this will be

Loyalty

Do you remember the time you were in school? The time when your school was the bestest there was , when all the people in your school were smarter than any other school , where the grounds were better to play in than in anyone else's school, where the teachers were the best and kindest, where the principals were stern but what the heck you liked them anyway.
I do.
One of the things you learnt in school was loyalty and that loyalty begets loyalty. And there was a thing called Honour.Do you remember the time the teacher asked who did whatever and everyone knew but no one said a word.? Atleast my school(and with that i include the campus , the teachers and my fellow students) did try to teach me that. But then i did tell you it was the best school.
One of the first things that struck me in the induction speech when i joined by company (5.5 years back) was the thing someone from senior management said. He said You should not be loyal to the company. He said the company does not even want your loyalty. It will do what is best for itself and you should do what is best for yourself. He went on to refer to the practice in my company of giving long service awards at the end of 3 years and joked about it.
The Achieving excellence session went on to teach you new keywords WIIFM WIIFU (whats in it for me , whats in it for us)
At that time i would rather listen to If(Rudyard Kipling)
If you can talk with crowds and keep your virtue,
Or walk with kings - nor lose the common touch;
I resolved that this was a case of keeping my virtue while walking with the crowd and thefact that the company doesnt care for my loyalty does not in any way prevent me from being loyal.
I suspect that i should have bowed to experience instead of indulging in youthful idealism.
So much so that the fact that i have stuck with my company for 5 and a half years is now a liability. Supposedly IT HR people feel that if a software engineer sticks in the same company for more than two years there is something wrong with him - lack of ambition, initiative whatever.
So why am i still in my company