- Any activity which doesn't ultimately lead to finding bugs or stopping bugs from appearing in the code in the first place is a waste of time - Cem Kaner
- I will not fake a test project - James Bach
But, today, I had to fake a test project and I felt like I was abused professionally.
I work in a team which developed an piece of software that is to be embedded inside a commercial electronic device and as a Scrum team we have been developing and doing mainly exploratory testing during the sprints. Now that our software has reached the stage of maturity and being readied for commercial release, more and more people outside our team (but within the company) are demanding to know the quality of our software.
We duly provided them a report stating the kind of testing we did, the area of functionality covered, a list of all outstanding issues (no Major or Critical ones) and our professional opinion on the quality of the product. Sadly, this report wasn't good enough, since for them, the measure of quality is based solely on game of numbers; how many tests ran, how many passed, how many failed, percentage not run, pass rate, fail rate, etc.
Today, we received an email from an external team who has the power to decide which software gets to be included in the commercial device demanded that we provide them with these test result figures or else our software will be dropped.
Not wanting to waste many months' worth of our team effort over the test results, I decided to give them what they wanted. I had an old and out of date spreadsheet which has around 85 test cases which checked for the basic features of our software, but had none of the complex test scenarios (that we discovered and used in our exploratory testing) which led us to finding all the critical bugs which we have fixed and retested.
And I spent the rest of the afternoon and early evening dutifully going through all of those out of date, incomplete test cases and marked each one of them as Passed or Failed and at the end we arrived at these magic numbers:
- Test run: 100%
- Passed: 95%
- Failed: 5%
and then I resubmitted our test report with these figures.
And on the way home, I thought about what I did today and I felt used. I felt abused. I felt humiliated. I felt violated. I felt dirty and unclean. No matter how much I washed myself with soap in the shower, I still felt dirty inside and this feeling will stay with me for some time.
1 comment:
Yeah - but you still got paid. Thats what you go to work for.
Post a Comment