Subscribe by Email

Thursday, October 17, 2013

What happens when you find a serious defect right before you release ??

The end game of a software development schedule can be very critical. It is the timeframe when you are hoping that there are no critical problems that pop up - given that the time involved in turning around for solving critical problems is less, and the amount of tension that such a problem causes to everyone in the team can be enough to give a coronary to everybody involved. Once we had a defect come up 2 days before we were supposed to release the product, and the complications were very bad (in terms of decision making - we had to decide whether the defect needed to be taken, we had to get somebody very safe to diagnose that defect, we had to evaluate the fix to see that there was nothing else that could get broken, we had to then roll out the fix into the product and test the hell out of the fix to ensure that there was nothing that was getting broken). All of this caused a huge amount of tension, and we had management on our heads, wanting to know the progress, and more worryingly, why this defect was not caught before, and whether we were confident that we had done enough testing to ensure that we had caught all other such defects such as this one.
Typically, when you reach a situation like this, you need to ensure that you are thinking through all the options. It is all right to brazen it out and hope that everything will go well and say that you are fine with releasing the product. But, without having done a proper analysis, that would not be the correct option. If you want to get the product released in this haste, then you might be reaching a situation where user find serious defects after the product has been released, and that is something that no one wants. Such a situation, if it happens more than once, can cause loss of user confidence in the product, and to some extent in the organization, and have a lot of serious consequences. However, in most cases, I know teams tend to brazen it out even if they see much more problems later.
But, on the other hand, you cannot suddenly decide that you are willing to take a delay on the product release date to get some extra confidence of the testing (which would have been shaken due to the recent serious defect found). This sounds good, but there are costs involved with such a decision. A product delay causes a loss in revenue, also can cause some customer confidence problems if the organization suddenly has to announce a delay in release, and cause a huge impact on the team morale as well because of management involvement. However, it may be less of an impact than if the product is released and the customers find many problems.
So how do you make such a decision ? Well, that is the million dollar question. And there are no easy answers. To a large extent, whatever decision is made has a number of risks, but it is important to get genuine feedback from the testing team about what they feel, especially from the test managers (and this needs to be done in environment where there are fewer recriminations). Finally, the team manager needs to own the decision and be able to justify this in front of management.

No comments:

Facebook activity