Defect Management: Dividing overall defect reports into separate functional areas - Part 3

This is a series of posts where I look at the creation of a defect status report, in a way that it provides enhanced information to the team and its managers and helps them to make decisions. In the previous post (Count of new incoming defects), I talked about adding more parameters to the defect report that help in information that can let the team know whether the number of defects getting added on a daily basis will enable the team to reach their defect milestones by the required time and date. This kind of data, coming in on a regular daily cycle, helps the team to decide whether their current defect pattern is on the desired path, above it, or below it, and accordingly make the required decisions.
This post will see more details added to the defect report that provides a lot of useful information to the team. The last post talked about the flow of incoming defects that move to the ToFix state. However, there is another type of information that is relevant. Defects that are present in the system are not only owned by the development team. In addition to the defects in core code, there may be defects that are present in the components used in the application. These are defects that are not attributable to the development team, but to the vendors or other groups that provide the components. A number of teams that I know typically track these defects in the defect database, but distinct from the defects with the core team.
The nature of defects that are against external components is different from those in the core code. Even though to the customer it does not matter whether the defect is within the core code or in an external component, the amount of effort required in terms of coordination and communication is entirely different from the other defects that are with the core developmental team. If a defect is with a component that is not owned by the team, the timeline for fixing of the defect may take longer and need persuasion; or there may be a lot of back and forth between the tester and the external component team to study the situation in which the defect occurs (which also includes sending the environment in which the defect occurred to the external vendor - and this has its own cost and restrictions, since if the team is working on a new version of the software, there would NDA issues and IP issues related to sending details of the environment to the external component team), and so on. Another concern could be that that even if such a defect is resolved, it might need a new version of the component, which has its own extra cost about testing the component on its own to check whether it is fine or there are other issues with the same.
As a result, it needs to separate out the incoming defects about whether they belong to the core team or whether they are attributable to people outside the team; and if the proportion of such defects that are outside the core team is increasing, it is a matter of concern to the team, since resolving such defects typically takes much more effort and time.

Defect Management: Dividing overall defect reports into separate functional areas - Part 2

Part 1 (Dividing defect reports into ToFix, ToTest and ToDefer counts) of this post talked about the importance of Defect Management in a software project, and then got into some details about the regular sending out of a report on Total defects, with these defects having been broken down into ToFix, ToTest and ToDefer stats, maintained on a graph over a period of time with daily updates, so that the team and the managers can figure out whether the team is on progress to resolve these bugs.
This post continues on this line, talking about additional items that can be added to this defect chart and metrics to provide more information to the team and determine whether it is on the right track or not. Are all these metrics important ? There is a lot of comments about not over-burdening people with too many statistics, and there are more comments about letting people do their work rather than sending so many statistics that they stop looking at these stats. However, it is also true that the role of the team managers is to look at the broader situation in terms of project status, and the defect management is an important part of this. It is true that the team members should not be burdened with these figures, but for the team managers, it is critical to look at such data.
So, the team looks at the ongoing figures for defects in terms of ToFix over a period of days and tries to determine whether the team is on the right track or not. So what else should you be capturing ? Another metric that can now be added to such a report is about the number of defects that are still incoming. There are primarily 2 ways in which defects can be added to the count of developers:
- New defects that are logged against the development team and which add to their count and to the overall ToFix count
- Defects that have been rejected by the testing team after they have been marked fixed by the developer but there is a problem in the fix (this can vary a lot among different teams and even in a team - a developer could be fixing defects with hardly any returns and there could be another developer who is under pressure and many of whose defects are returned because of some problems). So, whether to determine this kind of statistic and calculate metrics for such a case determines of whether the team is seeing such kind of returns for the defect management.
Once you have these kind of defect counts, it helps in determining the current status of defects and see whether the team is on the right track. So, you have a total count of open ToFix defects, and there is a decline in such a count needed to hit the deadlines. However, for getting to such a deadline, you need the number of incoming defects to be also fitting into this strategy. If there are a large number of incoming defects, then the team will not be easily able to determine whether their ToFix defect count is decreasing by the amount they want to hit their targets, and this then needs a change to the strategy to determine whether the team will get there or not.

Defect Management: Dividing overall defect reports into separate functional areas - Part 1

Handling defects is one of the major efforts that plays an integral role in handling a project schedule and making it successful. I have known multiple teams where the team did not have a good running estimate of their defect count and the defect estimation over the remaining period of time left in the schedule; as a result, when the team was closer to the final stages of the schedule, they found that they had too many defects that made the remaining part of the schedule very tight - which meant that if they were to do an accurate reckoning of their status, they would need to either defer more defects and maybe end up with a product that is lower in quality; or the product would need to extend their timeline / schedule, which has a huge implication  for the team and many other teams that are involved in the release schedule of the product.
How do you avoid this ? The first paragraph of this post points out a huge problem, but the answer cannot be handled in a single post; it can be handled by a single cheesy phrase but which does not provide any solutions - "You need to do Defect Management". Now, let us get down to the meat of this post - this post just takes a specific aspect of defect management - sending a split of the defect counts as per the different areas. This in turn provides a better view of the defect picture to the team and helps in the process of overall defect management.
We wanted to have a system whereby we could track the counts for each of the separate functional areas and yet have the management team have access to these data on an ongoing basis. These also helped the separate functional teams do a targeting of the counts of the defects of their respective functional areas and work towards reducing this count. So, we took the overall data for defects for the product (open defects) and split these into the following areas:
Open defects:
ToFix (these are primarily defects owned by the development team, although there could be defects that are carried by other team - such as where there are defects with components supplied by external teams)
ToTest (these are primarily defects owned by the testing team, although since anybody can file a defect within the team, there may be people other than the testing team who own a defect)
ToDefer (the exact terminology of these defects can be different across organizations; but these are typically defects that are with a defect review committee for evaluation. These can be significant defects that need evaluation by the committee before they are to be fixed, or these can be defects that are not worthy of fixing but the team wants the committee to take a final call, and so on).
What is the purpose of sending out separate stats on a regular basis ? These data, when plotted on a regular graph over a period of time provides a large amount of information. The team and the managers, although they are in the thick of things, sometimes need to see such kind of aggregate information to take a good decision. For example, if the team is in the second part of the cycle and close to the timeline, and yet the ToFix graphs do not show a declining trend, then this is something to worry about. When such a stage happens, I have seen the development team manager doing a major discussion with the team to figure out how to reduce these counts and figure out what is happening. In extreme cases, I have seen the team actually take a hard look at these defect counts and then make a recommendation for extending the schedule (which is not a simple step to take).

What are different tips to estimate testing time?

Time is a very important factor when it comes to the success of any matter. Therefore, timing plays a great role in the successful completion of a software project. It would not be wrong to say that the time estimation like other aspects of the software engineering forms an equally important part of the whole software development cycle.

BENEFITS OF KEEPING TIME ESTIMATION

- Keeping time estimation before the start of the software project keeps the whole development cycle on track.
- This doesn’t let your time get wasted.
- Since, there is a time limit; you have to complete the project within that time period.
- Furthermore, if you complete your projects on time, your clients will be impressed and your reputation will build up which in turn will fetch you more projects.
- An experienced software developer might be able to make better time estimations as compared to the one who is fresh in the industry.
- One who has worked up on various different software projects certainly must be having an idea of the time that will be taken up by the testing process.

TIPS FOR ESTIMATING TESTING TIME ARE:
Testing time cannot be estimated blindly. It should be done accurately and it should be realistic.

1. BUFFER TIME
- Your time estimation should involve some buffer time.
- But, keep in mind that it should be realistic.
- The role of the buffer is to help in case you have an unexpected delay in the software testing process.
- This buffer time accounts for the lost time.
- Apart from providing time for coping up with delays, a buffer also helps in providing the maximum coverage for the testing processes.

2. TIME TAKEN BY BUG CYCLE
- Never forget that this time estimation also includes the time that will be taken up by the bug cycle.
- You may estimate some time for a cycle, but remember that the actual cycle can very well require much more time.
- This problem should be avoided.
- As we all know that the testing process depends on the structure and design of the program.
- The more good the structure and design is, less will it take time.
- If the structure of the program itself is not good then more and more time will be required to fix the subsequent problems and this leads to the over run of the time estimation.

3. INCLUDE UNEXPECTED LEAVES
- The estimated testing time should also have a place for the unexpected leaves.
- Some members of the software development may require a leave until the completion of the project.
- This will help to keep your testing time estimation realistic.

4. AVAILABILITY OF RESOURCES
- You should keep in mind the availability of the resources for the time period within which you have to complete your project.
- If in case you get short of any of the resources you can update your testing time estimations accordingly.
- This is another measure to keep your time estimation realistic.

5. COMPARISON BETWEEN OLDER & NEWER VERSION OF SOFTWARE
- You can sometimes compare this software version with its older version for the test outputs.
- This will save your precious time.
- This is termed as parallel testing.
- Based on the testing time estimation of the older version you can decide time estimation for the upcoming version.

6.COUNT YOUR MISTAKE & REVIEW
- It is a universal fact that everybody makes mistakes.
- So, there is possibility that you may make some mistake while estimating the testing time.
- So don’t forget to review it once and make any changes if required.
- Always keep in mind that changing testing time estimations can have a bad effect on your reputation.
- So don’t make changes unless very much required.

7. COUNT YOUR EXPERIENCE
- You can very well employ your past experience to make wise time estimations.

8. EVALUATE YOUR TEAM EFFICIENCY
Know the work efficiency of your team members.

What is a bug and bug life cycle? What are guidelines for deciding severity of bugs?

A bug is defined as a defect or some abnormal behavior of software. Testing plays an important part in the removal of bug. Bug has to travel the whole bug life cycle until it is closed. The cycle includes following stages:
- New
When the bug is posted for first time and not yet approved.
- Open
When tester approves that bug is genuine.
- Assign
Bug is assigned to the developer.
- Test
After fixing the bug, it is assigned to testing team to re-test it.
- Deferred
When the bug is changed to deferred state, the bug is expected to be fixed in next releases.
- Rejected
If the developer feels that the bug is not genuine, he can reject the bug.
- Duplicate
If bug is repeated twice or two bugs gives the same concept, then one bug is labeled duplicate.
- Verified
Once the bug is fixed, it is verified that no bug is present and status is changed to verified.
- Reopened
In this stage, the bug traverses the bug cycle once again because the bug still exists.
- Closed
If the bug is fixed and does not exist, the tester changes the status to closed.

SEVERITY AND PRIORITY OF THE BUG HAS TO FOLLOW GUIDELINES:
- Critical bug prevents further testing of the product under test. No work around is possible for such bugs.
- Major bug is in which defect does not function as expected or cause other functionality to fail.
- Medium or average bug in which defects do not conform to standards and conventions.
- Minor or low bugs do not affect the functionality of the system.

To write bug description, follow these guidelines:
- Be specific.
- Use present tense.
- No unnecessary words.
- No exclamation points.
- Do not use all CAPS.
- Mention steps.