Monday, 29 November 2010

When is Agile not an Agile?

These days the darling buzz word in software industry is 'Agile'. "Are you agile? You're still using Waterfall, V-Model? That's soooo 2010!"

This post is not about explaining what Agile is. If you want to know what it is, point your browser to the Agile Manifesto. What I want to share is what I believe to be some people's misconception of what Agile is and when to recognise when it isn't.

As a contractor, I had the opportunity to work in different client sites and I have seen how some organisations had more success with Scrum (one of the most commonly used Agile methods) compared to others, and that was a surprising discovery because the very mechanism of Agile projects is supposed to ensure higher probabliltiy of the software being delivered and of certain quality.

So, why do some Agile projects did not live up to the expectations of Agile methodology? In most cases that I've seen, it's because they lost sight of the key aspects of the Agile manifesto. So, I'll re-iterate some of these here:
  • Early and continuous delivery of valuable software.
  • Deliver working software frequently, from a couple of weeks to a couple of months
  • Working software is the primary measure of progress.
To combine all of these into a single statement, it would read thus: "Early and continuous delivery of working software frequently is the primary measure of progress."

So, let's define what is 'working software' and how Agile teams sometimes forget this little fact. A working software is one which is deploy-able into production to be used by real users.

In a truly Agile house, to be able to develop working software continuously is not an issue but in organisations where you are transitioning from the traditional methods of development into Agile then there is a risk that by simply following the mechanics of Scrum you could mistakenly believe that a potentially deliverable working software will come out the other end, just because you're following Scum.

Let's illustrate this with an example. In a traditional software project, a typical system being built can be usually broken down into 3 logical components; the user-facing front-end component (which could either be a desktop or web-based application, or an interface to an external system), the central processing engine component (holding the business logic and business processes) and the back-end component (mainly databases and/or interfaces to other external systems). There would be 3 different teams building each of the components and this would be followed by Integration Test phase, System Test phase and User Acceptance Test phase, prior to deployment. And, it is only when the project is in the System Testing phase that one gets the visibility as to how the system works as a whole and only after UAT that the software reaches the 'potentially deliverable working software' state.

The mistake that some people make is when changing from the traditional development to Agile methods, they simply change their ways of working to accommodate the Agile process, i.e. if we were to take the above example, they would still have 3 separate teams building the 3 components of the system, with each team working in an Agile manner, falsely believing that just because they are using the tools and processes of Scrum they are now doing Agile.

Once the components are built, they still need to go through the integration testing phase, system testing phase and UAT phase. What they have forgotten is the fact that the software that each Agile team is building forms just the component of a complete system, and therefore, there is no complete working software at anytime during the development, which is in a deploy-able state to production. If this happens then they have missed the most important value that Agile methodologies brings.

Saturday, 21 August 2010

What is good quality software?

As software engineers we all strive to build good quality software but along the way, things didn't go as planned and we inevitably end up releasing software that is far from our original intended quality goal.

But what is good quality software (GQS)? The answer to this question is so obvious that there appear to be little point in writing a blog for it, but is the answer really as simple as we think it is?

On the surface, the answer seems obvious. A GQS is one that just works. But is that all? As a professional software tester, we need a little more elaboration on the definition of  GQS, in order for us to be able to measure the software which we are testing against some quality benchmark that we set, so that we can say with certainly if the software has indeed achieved what is deemed to be an acceptable level of quality.

From my experience, GQS is not made up of not just one but several measurements. But let's start with the most obvious that even we testers use in the most simplest case: a GQS is one which is free from bugs.

To build a piece of software which is completely free from bugs is an Utopian ideal, but in the real work, the best we can do is to test the software to best of our abilities within the time we are given to test it. As Testers, this is where 90% of all our efforts are spent in our working lives; finding bugs and retesting bugs once they are fixed by the Developers. Even if we were given an infinite amount of time and infinite resources to test a piece of software and prove once and for all that a piece of software is 100% free of bugs, does it make it a good quality software?

The answer, as you might expect, is not quite. So, what else? The second measurement for GQS is does it do what the user want? This also seems an obvious statement to make but many seasoned developers and testers would testify that after spending months (or sometimes years) developing a system and tested it to the hilt and deployed it to production and only to find that the system doesn't do what the user wanted. A perfectly working software which doesn't do what the user want is not GQS.

Finally, the last measure of GQS is somewhat subjective, and it's how easy and 'enjoyable' the software to use. Ease of use is difficult to quantify since what seems easy to one user might not be so easy to another. Ease of use depends on familiarity of concepts and past experience. For example, someone who has used Microsoft Windows all his life would find the Apple Mac OS X difficult to use because the entire concept of the desktop interface is very different, and yet the Mac OS X is generally regarded as being the most user-friendly of the Desktop interfaces. So, taking into considering the user's past experience on similar interfaces, the application still has to be easy to use, even to someone who has used similar application before.

Tuesday, 18 May 2010

Money for Nothing and Change for Free --- Using Scrum for fixed time/cost project

This is my understanding of the example Jeff Sutherland gave at the Scrum Master course that I attended recently when one of the delegates asked him if Scrum is suitable for a fixed time/cost project.

His answer was, naturally Yes. What he said was that you should get a complete product backlog from the client and then you would estimate the work for the entire backlog and say to your client that the total points (either function points or story points) in the product backlog would be, say 450 points. Then in your contract, you would write that you will deliver a product with the features, which are 450 points worth in total.

Furthermore, you would add a few more clauses in the contract, which says the following:

  1. the client will prioritise the requirements so that the most important or most risky features are listed first.
  2. the client is able to change their requirements without penalty during the project, provided that the client reviews the existing feature list (or backlog) and remove or de-prioritise one or more features (or stories) having the same points as the new requirements, to ensure that the total points of the project remains constant. This is the 'Change for Free' process.
  3. the client is able to terminate the contract before the agreed end date, if you manage to deliver the product ahead of time or when the client deems the product is sufficiently developed for their needs, and it's not worth their time or money to spend any more. In this case, the client agrees to pay you 50% of the fee that you would have received if the project carries on to its full term. For example, if the project is 9 months long and the cost is $100,000, then if the client decides to terminate the project in 6 months, they need only to pay you 50% of the $33,000 remainder. This is what he calls 'Money for Nothing' principle.

To achieve this, you would naturally develop the product by adding features in the order provided by the client, i.e. the most important ones first, and you will provide demos (or reviews) of the product to the client at regular intervals (i.e. at the end of each sprint).

By developing using SCRUM with the most important/valuable features first, what would tend to happen, according to J. Sutherland, is that the client, in many cases, after seeing the product demos over the course of several sprints they may come to the conclusion that they don't need the 200 or so features (which equated to 450 story points) that they originally spec'ed out, but instead they only needed 80 or so of the most important features that they asked for and which you have already implemented, and that the remaining 120 features are of low priority and not worth the money they would have to spend.

In this case, you are effectively saying, "OK, if you want to terminate the contract, just give us 50% of what you still owe us and we'll go our own separate ways." This will save them 'some' money for not having the spend the full remainder that is due at the end of the contract, PLUS, you get half of what you would have got for not having to do anything.

Monday, 15 February 2010

Seminar notes available to download

The PDF file of the seminar 'Developing Quality Software Through Formal Testing' presented at Myanmar Info Tech premise of MICT Park on 22 January 2010 is now available to download from here.

Friday, 12 February 2010

Testing without documentation

Any seasoned tester would have at least one story to tell when he/she has to test a system or a piece of software without any documentation, and those who have done so would also tell you the many problems they faced as a consequence of having to do so.

But testing a piece of software without documentation isn't something that's going to go away, regardless of how loud we, the testers, complain or shout about it. The sad reality of life is that this has happened in the past, it is still happening now and will, undoubtedly, continue to happen in the future.

Rather than discarding this undesirable task of testing without documentation as one of life's little unavoidable evils, we should learn to cope with this situation so that when we are faced with this situation again, we would at least know how to do the best job we can, rather than complaining that it makes our lives hard.

So, what do we do when we are faced with the situation of having to test an application with nothing to compare it with?


You would use Exploratory Testing to discover what the application does, and treating purely as a Black Box.
You would record the behaviour that we observed, in as much detail as possible. This means the following:
  • running end-to-end scenarios of the application, on the first instance
  • for each interface/dialog, trying out all controls (e.g. text boxes, check boxes, lists, drop-down boxes, radio buttons) and noting their behaviours
  • Making a list of all input data type/class and how they are processed by the system and the outputs generated.
Once you have completed exploring the system and noting the behaviour you would present these to the developers, analysts and stakeholders, i.e. end-users, and determine if the observed behaviour is what is being designed or for requested by the users.

Any discrepancies identified between the system behaviour and what the user expected or what the analyst designed can then be deemed as defects.

Thursday, 11 February 2010

Software Testing book from M$

It's interesting that the biggest software company in the world which has released the most buggy of Operating System that has ever being thrusted upon millions of innocent users, the Windows Vista,  they have the tenacity to release a book entitled "How We Test Software At Microsoft", available from Amazon.


I'll let the authors off the hook slightly by saying that they may not have been responsible for the said Vista, but could have been working on one of many different products that Microsoft did produced successfully, e.g. Halo video games.

But still if you want to find out how the people at Microsoft test their software then it's a good book to get.