Subscribe by Email


Showing posts with label Defect process. Show all posts
Showing posts with label Defect process. Show all posts

Thursday, March 19, 2015

Trying to solve defects where the flow is not repetitive .. (contd..)

In the previous post (Trying to solve defects where the flow is not repetitive ..), we talked about defects that are not easily reproducible, and some points around that. In this post, we will talk about such defects with more points. Handling defects that are not reproducible is one of the biggest pain points of the development cycle, since you know that even though the defect is not easily reproducible, it will come back to bite you at some point or the other. So, you cannot ignore those defects, although after trying to reproduce a defect like this for hours, it is tempting to just forget those defects. It needs a strong policy to ensure that the team does not overlook such defects.

- The team needs to have a policy about handling defects that ensures that the team does not overlook some of these defects. The policy needs to define how the defect needs to be handled, how it is tracked, how many times does the team needs to re-test those scenarios, whether such defects need to be handed over to other testing personnel, and so on. These policies need to be ensured as a part of the system, so that people know exactly how to handle such defects.
- These kind of non-reproducible defects do not follow any easy pattern, but from time to time, groups of developers and testers do figure out steps on how to figure out how to reproduce some of these defects. I remember specific cases where a developer wrote an app that would be run along with the application during the testing process, and helped in around 10% of the cases of non-reproducible defects, and that was an amazing feat, since it helped save a lot of time and helped in resolving defects that were otherwise difficult to fix.
- Statistics for these kind of defects is very important, and during the development process rather than at the end. What these statistics for non-reproducible defects do is to determine for the team management about how many these kind of defects are being found, as well as how many are being resolved. Every team needs to specify a number beyond which action needs to be taken. For example, during a specific 3 week period, we found that there was a spike in the number of non-reproducible defects that were being logged. This in turn resulted in more time being spent by the testers on trying to reproducible and hence resolve these defects. A solution needed to be found for this - and we spent some more time on focusing superior development effort on these defects. Eventually, this extra effort helped, since it was a new version of an external component that was causing problems to the application and resulting in many such non-reproducible defects.

As a result, even though it is difficult to visualize handling such non-reproducible defects, they are a reality and teams need to have a process of handling such defects.


Sunday, March 15, 2015

Trying to solve defects where the flow is not repetitive ..

If the title of the post did not make sense, then let me explain. What I refer to is the case where a defect has been logged (let's not discuss the severity of the defect right away), but the flow to replicate the defect is not easy. Anyhow, it is easy when a defect can be easily replicated, when the steps are easy. However, for anybody who has gone through the whole process of defect detection, analysis and resolution, finding an easy method of replicating the bug steps can almost be counted as lucky. There can be numerous cases where the steps for replicating a defect is not so easy.
Consider the case where a defect is caused due to a particular variable loading a specific value. This value has not been loaded during the current workflow, but gets loaded during a step in an unrelated workflow. Now, depending on whether the user has gone through this unrelated workflow previously, the defect will show up or will not show up. One might think that the person would be able to replicate this defect easily, but this is easy to think post-facto once the defect process has been understood. For the tester, there is no way of easily knowing that the defect is getting caused due to the operation of an unrelated workflow; it is only the skilled tester who tries the workflow inside the product multiple times who can try and figure that the defect is happening only in those cases where the unrelated workflow has been used. These kind of stories are common (not the majority of cases, but common enough that testers and developers need to be aware of these issues).
What does one do in these kind of cases ? Well, such cases are handled different by different teams, but let me suggest a few points for these:
- It is important to understand that every defect is important, and whether the defect is fixed or left unresolved needs to be decided in a determined manner. For a customer, the defect can be problematic, especially when the person used the unrelated workflow and wonders why the team left such a defect not fixed.
- Another variable that decides what needs to be done is the severity of the defect. For defects that were severe, it is not easy to decide to close the defect as unresolved. For defects that are minor, the bug decision committees would need to decide whether these minor defects need to be fixed, or can be deferred; that the team was comfortable with the minor inconvenience that such defects would cause to the customers.
- It is not easy for the developers to fix defects that the testing team cannot easily replicate, but that does not mean that they should not be fixed. There will be a larger number of customers than the testing cases within the development team, and one can be sure that some of the defects that could not be replicated will show up and cause inconvenience to the customer,
- There are methods that the development team can use to try and make it easier to find a solution to the defect. For example, it might be possible for the testing team to use special development versions of the software where the values of key variables can be recorded during the testing process, and this would help in trying to figure out exactly what is going on; and there are other development steps that can be done to try and get more detail on what exactly is causing these defects.
- Teams can setup processes on how to treat such defects; letting the tester spend some adhoc time on these defects; or give these defects to a different tester in order to get a fresh set of eyes; or ensure that the developer has some time for these defects in order to try and get some sort of solutions for atleast some of these defects.

Read more about this in the next post (Trying to solve defects where the flow is not repetitive .. (contd..))


Monday, April 22, 2013

When is a defect not a defect and should be classified as a feature ?

This is an interesting question that causes a large amount of debate in many teams during their development process. A defect is something that needs to be corrected (in terms of the discussion in software teams) - and this is without getting into the exact semantics of the definition of a defect. So, in the regular process, a tester finds a problem that needs to be corrected and is assigned to a member of the development team to fix. The development team member analyses the defect, finds the required change, and then makes the necessary fix, the fix is rolled into the product and can then be tested by the tester (the same one or somebody else) and if it is fixed, the defect is marked as verified and can be closed.
This works most of the time, but there are cases when the process does not work like this. For example, there are times when the defect that is marked in the system, when analysed, is found to have a big change. It could be that the fix for the defect causes a change in a code section or file that is used in a number of areas of the application. When this happens, it is not so easy to make a change, since there can be a large impact. The impact analysis of such a change needs to be more wide-spread, looking at the change that needs to be done in the different areas, the change needs to be thoroughly reviewed, and then once the change is made, a thorough testing needs to happen. However, this is not the case that I am talking about, where the defect is still a defect and not a feature.
The feature change is a different kind of defect. This is when a defect is logged which talks about some change in the workflow. It could be possible that when a defect is logged, the defect was logged on the basis of an interpretation of the tester, or the defect is actually tracking a specific point of the product where the feature definition was missing or flawed. Consider a case where a team has defined a new feature, where all the workflows that were written as a part of the product development were done, and then it was discovered during testing that the specific workflow of one area was not done as it should have been.
So what to do ? Well, whenever such a defect is written, it should be categorized as a feature rather than a defect. A defect means that the focus is more on the fixing of the defect, while if you are looking at the problem from a feature perspective, there will be more attention paid. The product manager, the designer, the feature team which was looking at the feature, all of them should be brought in, and they need to look at what the nature of the change should be, whether this impacts more than just the workflows before and after, and so on. It is also possible that if this is at a late enough stage, this group makes a decision that the feature will be left as it is, or the team recommends the change which is then made. Doing it as a feature will also ensure that other teams that need to look at feature changes such as documentation and localization will be informed as a matter of process. In the end, the treatment of such a change may be finally be done through the defect process, but this is done only after the change has had a thorough look at it.


Facebook activity