Determining the urgency of defects

Determining the Urgency of Resolving Defects in Software Development

Why Defect Resolution is Critical

Software defects can impact user experience, security, and overall functionality. Knowing when to prioritize defect resolution is essential for maintaining software quality and reliability.

Factors That Determine Defect Urgency

Not all defects require immediate attention. Some can wait for a later release, while others demand urgent resolution. Here are the primary factors that influence defect urgency:

1. Severity of the Defect

• Critical defects: Issues that cause system crashes, data loss, or security breaches must be fixed immediately.
• Major defects: Problems that significantly affect functionality but have workarounds can be scheduled for the next release.
• Minor defects: Cosmetic issues or minor inconveniences can be resolved in future iterations.

2. Impact on Users

The number of users affected by a defect is a key consideration. If a bug impacts a large user base or a critical feature, fixing it should be a top priority.

3. Business Implications

Some defects may lead to revenue loss or compliance violations. In such cases, resolving the issue quickly is crucial to maintaining business integrity.

4. Frequency of Occurrence

Bugs that occur frequently require more immediate attention compared to rare issues. High-frequency defects can indicate deeper systemic problems that need to be addressed.

5. Security Risks

Security vulnerabilities should always be addressed urgently to prevent potential data breaches or cyber-attacks.

6. Dependencies on Other Features

If a defect affects multiple components or prevents other features from functioning, resolving it becomes more urgent.

How to Prioritize Defect Resolution

Using a structured approach to defect prioritization ensures that resources are allocated efficiently. Consider the following strategies:

1. Categorization Using a Defect Matrix

Developers can use a defect matrix to classify issues based on severity and impact, helping teams prioritize effectively.

2. Utilizing Agile Methodologies

Frameworks like Scrum and Kanban help teams assess defects dynamically and allocate sprint resources accordingly.

3. Stakeholder Involvement

Engaging product owners and end-users ensures that defect prioritization aligns with business and customer needs.

Final Thoughts

Determining defect urgency is essential for maintaining software quality, user satisfaction, and security. By leveraging structured prioritization methods, teams can optimize resources and enhance software reliability effectively.

Not every defect should be fixed

During the course of a software development project, whether this be an updated version of a product or a one-off software project, the key currencies of the project are primarily features and defects (features and defects are what the teams end up doing most of the time during the development cycle, and in fact, at some intense points of the cycle, the team will be doing mostly defect finding, fixing, releasing fixes and re-test (in the Waterfall cycle), once the feature is released, it is bug fixing and more bug fixing.
In the software team, these are busy times. In a number of such development cycles, the number of defects that there are there in the product are such that it seems tough to think that all of these will be fixed in the given time cycle. This cycle of defect fixing primarily involves the developer and the tester. Some of these would be simple defects, where the feature as defined is not working and needs to be fixed by the developer. However, there will be other cases where there is some ambiguity in terms of how the feature is working and whether it is as per the definition - it could be that the feature definition is not complete to cover all the cases or that there is some disagreement regarding the way that the feature was supposed to be working vs. how it was written.
The biggest problem could be that the resolution of such issues happened in a path not intended to happen, something that the product manager had not intended. Now, it is not necessary that it happens this way, but one has to prepare for such an eventuality, or to be more clear, to define a process so that such a thing does not happen. The actual process is something that needs to be defined for each team, since what works for one team may not work for another team. For example, I once worked with a team that asked that all defects be triaged by a bug committee which decided whether it needed to be fixed or not, and if so, what would be the proper method of fixing (although the actual fix is something that the developer and tester could decide). Why this process may not work for all teams is because of the quantum of defects that may come in, overwhelming the defect review committee and causing a backlog. Other teams may find such a process too severe, trusting the developers and testers that they will not automatically make suspect fixes, checking with a defect committee or with the Product Manager before actually making a feature change or refinement.
However, it is essential that this be talked about and decided with the management and with the team before proceeding, else there is a chance that feature changes may happen just at the developer and tester level.  

Able to report defects in an agreed format

During the course of a software development project, one of the most critical workflows is the defect workflow. The software coding team releases features and code to the testing team, which tests these features against their test cases and if there are defects, these are typically logged in a defect tracking system where their progress can be monitored and they can be tracked to closure (either with the defect being closed and an upgraded feature released, or with the defect having been closed as not to be fixed or not even being a defect at all).
However, this is an area that leads to a lot of dispute. There can be significant discussions and disputes between  the coding team and the testing team over what the severity and priority of a defect can mean, and from my experience, what I have seen tells me that even if one were to define a sort of standard for these terms across the organization, individual teams still need to work out their own precise definition of what these terms mean. Even more critical is the fact that individuals coders and testers also understand these terms and even though these can be subjective criteria, they also have developed a level of understanding with their counterparts in the different teams so that even though there may be some dispute over these terms when applied to a specific defect, the individuals can work it out.
Even though I stated some easy solutions in the above paras, there are many complications that come  about during the course of a software development project. For example, there can be senior coders who have a lot of heft and hence can speak with a lot of authority to members of the testing team. I remember a case where a senior developer called a new tester and asked him to explain the defect he had raised - it was marked as a very high severity and the developer felt that it was a side case and should not have been marked as a very high severity. This discussion ended with a conclusion, but there have been other cases where the tester felt that they were right and resented the fact that the developer used his / her seniority to try and talk them down. These issues can become serious if they happen many times, and it may become necessary for a defect review committee or the respective team leads/  managers to resolve these kind of issues. Because human nature being what it is, there  will be teams where you will have some individuals who get into these sort of disputes and they need to be resolved quickly.
For the above case, I remember one team which took a more drastic approach. They had set up an defect review committee that met once every few hours and every new defect that was created had to be reviewed by the committee before it could be taken up for any action. Without trying to criticize, it did seem odd because it meant the senior members who were part of the committee had to spend their time even on trivial defects that could be in most cased discussed and resolved between the developer and the tester.
Another problem that seemed to be happening at regular intervals was when a new member would come into the team, whether through new hiring or through a transfer from another team. People from another team could sometimes cause more challenges since they would have their own conceptions of the defect workflow and would find it hard to understand why this team may have a different version of the same. In these cases, some amount of hand holding by a more senior member of the team would really help. 
These cases can go on and on, but the basic idea is that there needs to be a spirit of discussion and cooperation between team members that will help to understand these workflows and follow them in a manner that reduces disputes.

Trying to solve defects where the flow is not repetitive .. (contd..)

In the previous post (Trying to solve defects where the flow is not repetitive ..), we talked about defects that are not easily reproducible, and some points around that. In this post, we will talk about such defects with more points. Handling defects that are not reproducible is one of the biggest pain points of the development cycle, since you know that even though the defect is not easily reproducible, it will come back to bite you at some point or the other. So, you cannot ignore those defects, although after trying to reproduce a defect like this for hours, it is tempting to just forget those defects. It needs a strong policy to ensure that the team does not overlook such defects.

- The team needs to have a policy about handling defects that ensures that the team does not overlook some of these defects. The policy needs to define how the defect needs to be handled, how it is tracked, how many times does the team needs to re-test those scenarios, whether such defects need to be handed over to other testing personnel, and so on. These policies need to be ensured as a part of the system, so that people know exactly how to handle such defects.
- These kind of non-reproducible defects do not follow any easy pattern, but from time to time, groups of developers and testers do figure out steps on how to figure out how to reproduce some of these defects. I remember specific cases where a developer wrote an app that would be run along with the application during the testing process, and helped in around 10% of the cases of non-reproducible defects, and that was an amazing feat, since it helped save a lot of time and helped in resolving defects that were otherwise difficult to fix.
- Statistics for these kind of defects is very important, and during the development process rather than at the end. What these statistics for non-reproducible defects do is to determine for the team management about how many these kind of defects are being found, as well as how many are being resolved. Every team needs to specify a number beyond which action needs to be taken. For example, during a specific 3 week period, we found that there was a spike in the number of non-reproducible defects that were being logged. This in turn resulted in more time being spent by the testers on trying to reproducible and hence resolve these defects. A solution needed to be found for this - and we spent some more time on focusing superior development effort on these defects. Eventually, this extra effort helped, since it was a new version of an external component that was causing problems to the application and resulting in many such non-reproducible defects.

As a result, even though it is difficult to visualize handling such non-reproducible defects, they are a reality and teams need to have a process of handling such defects.

Trying to solve defects where the flow is not repetitive ..

If the title of the post did not make sense, then let me explain. What I refer to is the case where a defect has been logged (let's not discuss the severity of the defect right away), but the flow to replicate the defect is not easy. Anyhow, it is easy when a defect can be easily replicated, when the steps are easy. However, for anybody who has gone through the whole process of defect detection, analysis and resolution, finding an easy method of replicating the bug steps can almost be counted as lucky. There can be numerous cases where the steps for replicating a defect is not so easy.
Consider the case where a defect is caused due to a particular variable loading a specific value. This value has not been loaded during the current workflow, but gets loaded during a step in an unrelated workflow. Now, depending on whether the user has gone through this unrelated workflow previously, the defect will show up or will not show up. One might think that the person would be able to replicate this defect easily, but this is easy to think post-facto once the defect process has been understood. For the tester, there is no way of easily knowing that the defect is getting caused due to the operation of an unrelated workflow; it is only the skilled tester who tries the workflow inside the product multiple times who can try and figure that the defect is happening only in those cases where the unrelated workflow has been used. These kind of stories are common (not the majority of cases, but common enough that testers and developers need to be aware of these issues).
What does one do in these kind of cases ? Well, such cases are handled different by different teams, but let me suggest a few points for these:
- It is important to understand that every defect is important, and whether the defect is fixed or left unresolved needs to be decided in a determined manner. For a customer, the defect can be problematic, especially when the person used the unrelated workflow and wonders why the team left such a defect not fixed.
- Another variable that decides what needs to be done is the severity of the defect. For defects that were severe, it is not easy to decide to close the defect as unresolved. For defects that are minor, the bug decision committees would need to decide whether these minor defects need to be fixed, or can be deferred; that the team was comfortable with the minor inconvenience that such defects would cause to the customers.
- It is not easy for the developers to fix defects that the testing team cannot easily replicate, but that does not mean that they should not be fixed. There will be a larger number of customers than the testing cases within the development team, and one can be sure that some of the defects that could not be replicated will show up and cause inconvenience to the customer,
- There are methods that the development team can use to try and make it easier to find a solution to the defect. For example, it might be possible for the testing team to use special development versions of the software where the values of key variables can be recorded during the testing process, and this would help in trying to figure out exactly what is going on; and there are other development steps that can be done to try and get more detail on what exactly is causing these defects.
- Teams can setup processes on how to treat such defects; letting the tester spend some adhoc time on these defects; or give these defects to a different tester in order to get a fresh set of eyes; or ensure that the developer has some time for these defects in order to try and get some sort of solutions for atleast some of these defects.

Read more about this in the next post (Trying to solve defects where the flow is not repetitive .. (contd..))

Emergency defect fixing: Giving local fixes for quick verification

During the process of defect fixing and verification, there is a standard process whereby a build process is created and defect fixes checked into this build. What this does is to ensure that every day (the builds typically come every day) the defects which were fixed the previous day are available for testing in a proper installer which can be used by the testing team similar to the product that is available to the customer.
This process works pretty well, as long as everybody involved knows the process well, and there are people in the process who have responsibilities for the different parts of the process (for example, somebody who ensures that the build stability systems are in place, others who do a quick smoke testing to ensure that the build is usable), and so on.
However, such a system cannot protect against a case where a defect has not been fixed, either fully, or partially. In the normal case of operations, it is normal to have defects that are not fixed and are rejected by the testing team, or some part of the defect not being fixed well, and a new defect being filed for the same.
When does this process not work ? Consider the case where the product development process is nearing the end of the schedule. In such a case, the defects that are to be fixed are restricted, and only those defects that are being allocated for fixing are passed onto the development team and the list given to the testing team for verification. However, the cost of a failed defect fix can be fairly high.
A defect fix that has failed would mean that the build for that day is not ready for use, and this can be very expensive.
When such parts of the schedules have been reached, there is the need for much closer interaction between the specific developer and the tester(s) for that defect fix. When the developer has made the fix, he/she would work with the tester and provide the fix in a local build, made on the developer's machine, which could be quickly tested by the tester to ensure that the fix has been made to the satisfaction of both the developer and the tester. This goes a long way to help that the build that comes the next day is usable and important fixes are not failing.
There are some problems that can still happen in this process. The local build may not be incorporating changes made by the other developers, and this can cause a dependency problem that may still cause the defect fix to fail. However, the chance of this happening is low (or can be monitored by the developer to reduce the failure rate) and goes a long way to ensure that the development process has reduced risk near the end of the cycle.
However, this requires close collaboration between the tester and the developer, and is not really required to be done in the regular part of the development cycle, since there is an overhead involved in the process.

What happens when you find a serious defect right before you release ??

The end game of a software development schedule can be very critical. It is the timeframe when you are hoping that there are no critical problems that pop up - given that the time involved in turning around for solving critical problems is less, and the amount of tension that such a problem causes to everyone in the team can be enough to give a coronary to everybody involved. Once we had a defect come up 2 days before we were supposed to release the product, and the complications were very bad (in terms of decision making - we had to decide whether the defect needed to be taken, we had to get somebody very safe to diagnose that defect, we had to evaluate the fix to see that there was nothing else that could get broken, we had to then roll out the fix into the product and test the hell out of the fix to ensure that there was nothing that was getting broken). All of this caused a huge amount of tension, and we had management on our heads, wanting to know the progress, and more worryingly, why this defect was not caught before, and whether we were confident that we had done enough testing to ensure that we had caught all other such defects such as this one.
Typically, when you reach a situation like this, you need to ensure that you are thinking through all the options. It is all right to brazen it out and hope that everything will go well and say that you are fine with releasing the product. But, without having done a proper analysis, that would not be the correct option. If you want to get the product released in this haste, then you might be reaching a situation where user find serious defects after the product has been released, and that is something that no one wants. Such a situation, if it happens more than once, can cause loss of user confidence in the product, and to some extent in the organization, and have a lot of serious consequences. However, in most cases, I know teams tend to brazen it out even if they see much more problems later.
But, on the other hand, you cannot suddenly decide that you are willing to take a delay on the product release date to get some extra confidence of the testing (which would have been shaken due to the recent serious defect found). This sounds good, but there are costs involved with such a decision. A product delay causes a loss in revenue, also can cause some customer confidence problems if the organization suddenly has to announce a delay in release, and cause a huge impact on the team morale as well because of management involvement. However, it may be less of an impact than if the product is released and the customers find many problems.
So how do you make such a decision ? Well, that is the million dollar question. And there are no easy answers. To a large extent, whatever decision is made has a number of risks, but it is important to get genuine feedback from the testing team about what they feel, especially from the test managers (and this needs to be done in environment where there are fewer recriminations). Finally, the team manager needs to own the decision and be able to justify this in front of management.

Defect Management: Dividing overall defect reports into separate functional areas - Part 3

This is a series of posts where I look at the creation of a defect status report, in a way that it provides enhanced information to the team and its managers and helps them to make decisions. In the previous post (Count of new incoming defects), I talked about adding more parameters to the defect report that help in information that can let the team know whether the number of defects getting added on a daily basis will enable the team to reach their defect milestones by the required time and date. This kind of data, coming in on a regular daily cycle, helps the team to decide whether their current defect pattern is on the desired path, above it, or below it, and accordingly make the required decisions.
This post will see more details added to the defect report that provides a lot of useful information to the team. The last post talked about the flow of incoming defects that move to the ToFix state. However, there is another type of information that is relevant. Defects that are present in the system are not only owned by the development team. In addition to the defects in core code, there may be defects that are present in the components used in the application. These are defects that are not attributable to the development team, but to the vendors or other groups that provide the components. A number of teams that I know typically track these defects in the defect database, but distinct from the defects with the core team.
The nature of defects that are against external components is different from those in the core code. Even though to the customer it does not matter whether the defect is within the core code or in an external component, the amount of effort required in terms of coordination and communication is entirely different from the other defects that are with the core developmental team. If a defect is with a component that is not owned by the team, the timeline for fixing of the defect may take longer and need persuasion; or there may be a lot of back and forth between the tester and the external component team to study the situation in which the defect occurs (which also includes sending the environment in which the defect occurred to the external vendor - and this has its own cost and restrictions, since if the team is working on a new version of the software, there would NDA issues and IP issues related to sending details of the environment to the external component team), and so on. Another concern could be that that even if such a defect is resolved, it might need a new version of the component, which has its own extra cost about testing the component on its own to check whether it is fine or there are other issues with the same.
As a result, it needs to separate out the incoming defects about whether they belong to the core team or whether they are attributable to people outside the team; and if the proportion of such defects that are outside the core team is increasing, it is a matter of concern to the team, since resolving such defects typically takes much more effort and time.

Defect Management: Dividing overall defect reports into separate functional areas - Part 2

Part 1 (Dividing defect reports into ToFix, ToTest and ToDefer counts) of this post talked about the importance of Defect Management in a software project, and then got into some details about the regular sending out of a report on Total defects, with these defects having been broken down into ToFix, ToTest and ToDefer stats, maintained on a graph over a period of time with daily updates, so that the team and the managers can figure out whether the team is on progress to resolve these bugs.
This post continues on this line, talking about additional items that can be added to this defect chart and metrics to provide more information to the team and determine whether it is on the right track or not. Are all these metrics important ? There is a lot of comments about not over-burdening people with too many statistics, and there are more comments about letting people do their work rather than sending so many statistics that they stop looking at these stats. However, it is also true that the role of the team managers is to look at the broader situation in terms of project status, and the defect management is an important part of this. It is true that the team members should not be burdened with these figures, but for the team managers, it is critical to look at such data.
So, the team looks at the ongoing figures for defects in terms of ToFix over a period of days and tries to determine whether the team is on the right track or not. So what else should you be capturing ? Another metric that can now be added to such a report is about the number of defects that are still incoming. There are primarily 2 ways in which defects can be added to the count of developers:
- New defects that are logged against the development team and which add to their count and to the overall ToFix count
- Defects that have been rejected by the testing team after they have been marked fixed by the developer but there is a problem in the fix (this can vary a lot among different teams and even in a team - a developer could be fixing defects with hardly any returns and there could be another developer who is under pressure and many of whose defects are returned because of some problems). So, whether to determine this kind of statistic and calculate metrics for such a case determines of whether the team is seeing such kind of returns for the defect management.
Once you have these kind of defect counts, it helps in determining the current status of defects and see whether the team is on the right track. So, you have a total count of open ToFix defects, and there is a decline in such a count needed to hit the deadlines. However, for getting to such a deadline, you need the number of incoming defects to be also fitting into this strategy. If there are a large number of incoming defects, then the team will not be easily able to determine whether their ToFix defect count is decreasing by the amount they want to hit their targets, and this then needs a change to the strategy to determine whether the team will get there or not.

Defect Management: Dividing overall defect reports into separate functional areas - Part 1

Handling defects is one of the major efforts that plays an integral role in handling a project schedule and making it successful. I have known multiple teams where the team did not have a good running estimate of their defect count and the defect estimation over the remaining period of time left in the schedule; as a result, when the team was closer to the final stages of the schedule, they found that they had too many defects that made the remaining part of the schedule very tight - which meant that if they were to do an accurate reckoning of their status, they would need to either defer more defects and maybe end up with a product that is lower in quality; or the product would need to extend their timeline / schedule, which has a huge implication  for the team and many other teams that are involved in the release schedule of the product.
How do you avoid this ? The first paragraph of this post points out a huge problem, but the answer cannot be handled in a single post; it can be handled by a single cheesy phrase but which does not provide any solutions - "You need to do Defect Management". Now, let us get down to the meat of this post - this post just takes a specific aspect of defect management - sending a split of the defect counts as per the different areas. This in turn provides a better view of the defect picture to the team and helps in the process of overall defect management.
We wanted to have a system whereby we could track the counts for each of the separate functional areas and yet have the management team have access to these data on an ongoing basis. These also helped the separate functional teams do a targeting of the counts of the defects of their respective functional areas and work towards reducing this count. So, we took the overall data for defects for the product (open defects) and split these into the following areas:
Open defects:
ToFix (these are primarily defects owned by the development team, although there could be defects that are carried by other team - such as where there are defects with components supplied by external teams)
ToTest (these are primarily defects owned by the testing team, although since anybody can file a defect within the team, there may be people other than the testing team who own a defect)
ToDefer (the exact terminology of these defects can be different across organizations; but these are typically defects that are with a defect review committee for evaluation. These can be significant defects that need evaluation by the committee before they are to be fixed, or these can be defects that are not worthy of fixing but the team wants the committee to take a final call, and so on).
What is the purpose of sending out separate stats on a regular basis ? These data, when plotted on a regular graph over a period of time provides a large amount of information. The team and the managers, although they are in the thick of things, sometimes need to see such kind of aggregate information to take a good decision. For example, if the team is in the second part of the cycle and close to the timeline, and yet the ToFix graphs do not show a declining trend, then this is something to worry about. When such a stage happens, I have seen the development team manager doing a major discussion with the team to figure out how to reduce these counts and figure out what is happening. In extreme cases, I have seen the team actually take a hard look at these defect counts and then make a recommendation for extending the schedule (which is not a simple step to take).

Working with external teams for getting them to resolve their defects

When we do our planning for the next release of the software, we normally consider a number of risks, seeking to define a risk database that tabulates various known issues that could cause problems to the schedule and the smooth progress of the project. One lesser known risk that sometimes does not make it to the list of known risks relates to defects with people who are outside the core development team. For example, you may have a component that is being used in the product, and there is a defect against that component. Such defects have a higher risk than defects that are with the core team, even if the defect is of the same severity. There are many reasons for these defects having a higher risk. Some of these are:
- The outside team may not follow the same schedule or the priority as the product team.
- In fact, supporting the product team may be a lower priority for the component team, since they may have been given a task of providing support to other teams. This can happen a lot in organizations where there are a number of teams, and a strategic product will get much more support. For example, if the MS Access team and the MS Office team report defects in a component used by both teams, who do you think will get the priority support. In such cases, there will need to be more discussions, escalations, and other such measures before there is support granted.
- There is a much higher degree of coordination and communication needed when interaction with a developer who is outside the core development team
- The same defect may be treated as expected behavior by another team that is using the same component. You may be having a doubt about why a defect can be treated as a feature, but in some cases, a particular workflow behavior may be seen as a defect by one team, and desired behavior by another team. In such cases, the situation may be such that the defect may never be fixed.
- When the defect is with a component that is an open source component similar, there is no guarantee that such a defect may be fixed.

Given some of these reasons above, defects with developers who are outside the core development team are a much higher risk than most teams tend to visualize. We once had a hair raising case where the component we were integrating had a defect, but somehow the developer working on it was not able to replicate the defect. This was close to an important pre-release milestone, and we needed to get some fixing done on this defect, but this back and forth discussion and coordination at a remote location meant that it took around 2 weeks to get the defect fixed and a new component integrated, and this was a big problem for us.
What do you do when you have such dependencies on extended teams ? Well, you can never really get away from the defects, but atleast have processes setup to reduce the time period and coordination cost for these defects. What should you do ?
- Have an understanding with the remote team about the level of testing at their end. If necessary, send them test cases and ask them to provide completed test cases before they provide the component. This helps in reducing the chances of defects passing through their end.
- Setup a process where they can received updated copies of your software so that they can test with such latest software
- Ensure that the external team totally understands your defect management system and processes and they have been provided permissions for the software.
- Have a protocol with them about exchanging information with them about defects and if there are queries on these defects
- Have a regular meeting with them, this help in ensuring that any issues from either end are resolved quickly
- Ensure that any serious defects are communicated to them at a higher priority. This also includes defects that need to be fixed on priority.
- Define an escalation matrix with them, so that if there are cases where you are dis-satisfied or need quick action, what is the process to follow.

All of these will not mean that you will not have external dependencies on defects with external team members, but the problems associated with these defects will be reduced.

When is a defect not a defect and should be classified as a feature ?

This is an interesting question that causes a large amount of debate in many teams during their development process. A defect is something that needs to be corrected (in terms of the discussion in software teams) - and this is without getting into the exact semantics of the definition of a defect. So, in the regular process, a tester finds a problem that needs to be corrected and is assigned to a member of the development team to fix. The development team member analyses the defect, finds the required change, and then makes the necessary fix, the fix is rolled into the product and can then be tested by the tester (the same one or somebody else) and if it is fixed, the defect is marked as verified and can be closed.
This works most of the time, but there are cases when the process does not work like this. For example, there are times when the defect that is marked in the system, when analysed, is found to have a big change. It could be that the fix for the defect causes a change in a code section or file that is used in a number of areas of the application. When this happens, it is not so easy to make a change, since there can be a large impact. The impact analysis of such a change needs to be more wide-spread, looking at the change that needs to be done in the different areas, the change needs to be thoroughly reviewed, and then once the change is made, a thorough testing needs to happen. However, this is not the case that I am talking about, where the defect is still a defect and not a feature.
The feature change is a different kind of defect. This is when a defect is logged which talks about some change in the workflow. It could be possible that when a defect is logged, the defect was logged on the basis of an interpretation of the tester, or the defect is actually tracking a specific point of the product where the feature definition was missing or flawed. Consider a case where a team has defined a new feature, where all the workflows that were written as a part of the product development were done, and then it was discovered during testing that the specific workflow of one area was not done as it should have been.
So what to do ? Well, whenever such a defect is written, it should be categorized as a feature rather than a defect. A defect means that the focus is more on the fixing of the defect, while if you are looking at the problem from a feature perspective, there will be more attention paid. The product manager, the designer, the feature team which was looking at the feature, all of them should be brought in, and they need to look at what the nature of the change should be, whether this impacts more than just the workflows before and after, and so on. It is also possible that if this is at a late enough stage, this group makes a decision that the feature will be left as it is, or the team recommends the change which is then made. Doing it as a feature will also ensure that other teams that need to look at feature changes such as documentation and localization will be informed as a matter of process. In the end, the treatment of such a change may be finally be done through the defect process, but this is done only after the change has had a thorough look at it.

What are merits and demerits of ad hoc test approach?

Ad hoc testing as the name suggests is the testing that is carried out without any formal planning and formal documentation though it can be used for scientific experimental studies.

- The ad hoc testing is meant to be carried out only once until and unless a defect is found.
- It would not be wrong if we call the least formal methodology of testing a software application or system.
- It has always been under criticism because of its uneven structure and also because the defects discovered through this kind of testing are difficult to reproduce due to a lack of written test cases.

However, there is still a plus point of ad hoc testing as important defects or bugs can be found quickly and easily without much efforts and pre planning.

Principle of Ad-hoc testing
The ad hoc testing is based on the principle of improvisation i.e., the software testers seek to find the errors and bugs by any means that seems appropriate to him/ her or we can say tests are performed according to the convenience of the tester.

Ad hoc testing can be appropriately called a primitive version of error guessing which itself is an un- sophisticated version of exploratory testing.

- Formal testing is carried out without any test cases and it proves to be a great help in deciding the duration and scopes for the other types of testing that have to be carried out upon the software system or application.

- This also helps the tester to learn more and more about the software system or application prior carrying out the other kinds of testing.

- It can be called as the least formal method of testing.

- The quick discovery of defects is the best use of ad hoc testing.

- Only reading the specifications and requirements doesn’t give a good sense of how the program or the software system or the application actually works and behaves.

- Even the documentation of the testings does not provide the feel of the software system or application.

- Ad hoc testing can be effectively used to find discrepancies in the testing strategies of the tester and the relations between the sub- systems of the software can be explored that would have been left un- exposed.

- In a way we can say that it serves a tool for checking the quality of our testing methodology.

- It helps in finding the missing cases and so these can be added to the list of test cases.

- Discovering new test cases with ad hoc testing is an indication that root cause analysis should also be performed.

While carrying out the ad hoc testing you should ask yourself and your team members “what other tests should we need to carry out in order to supplement this and make testing more effective?”

SOME IMPORTANT FACTS
- Defects or errors found during ad hoc testing provide examples of forgotten test cases.

- Determining the priority for carrying out other testing can also be thought as another use of ad hoc testing.

- For example, if a program allows the user to carry out some task and the ad hoc testing shows that this function works well, then the formal testing of this functionality might be postponed until the areas having problems are completed.

- On the other hand, if the ad hoc testing discovers any problem, then the formal testing for this functionality can be given highest priority.

- It is concerned with application testing without taking into considerations any rules and regulations.

- For carrying out ad hoc testing effectively the tester should have strong knowledge of the software system or application. Ad hoc testing is performed mainly to save time.

Software Six Sigma for Software Engineering

Software Six Sigma is a strategy to enhance and sustain continuous improvements in software development process and quality management. It uses data and statistical analysis to measure and improve company's performance by eliminating defects in manufacturing and service related processes.

ATTRIBUTES OF SIX SIGMA

- genuine metric data.
- accurate planning.
- real time analysis and decision support by the use of statistical tools.
- high quality product.
- software improvement costs and benefits.

STEPS IN SIX SIGMA METHODOLOGY

- Customer requirements are defined, project goals via well defined methods.
- Quality performance is determined by measuring existing process and its output.
- Analyzing the defect metrics.
- Process improvement is done by eliminating the root causes of defects.
- Process control to ensure changes made in future will not introduce the cause of defects again.
These steps are referred to as DMAIC(define, measure,analyze,improve and control) method.

- Design the process to avoid the root causes of defects and to meet customer requirements.
- Verify the process model will avoid defects and meet customer requirements.
This variation is called DMADV(define, measure, analyze, design, and verify) method.

What is a defect and what is defect management ?

Defects determine the effectiveness of the testing what we do. If there are no defects, it directly implies that we do not have our job. There are two points worth considering here, either the developer is so strong that there are no defects arising out, or the test engineer is weak. In many situations, the second is proving correct. This implies that we lack the knack.

For a test engineer, a defect is:
- any deviation from specification.
- anything that causes user dissatisfaction.
- incorrect output.
- software does not do what it intended to do.

Software is said to have bug if it features deviates from specifications.
Software is said to have defect if it has unwanted side effects.
Software is said to have error if it gives incorrect output.

Categories of Defects
All software defects can be broadly categorized into the below mentioned types:
- errors of commission : something wrong is done.
- errors of omission : something left out by accident.
- errors of clarity and ambiguity : different interpretations.
- errors of speed and capacity.

Types of defects that can be identified in different software applications are conceptual bugs, coding bugs, integration bugs, user interface bugs, functionality, communication, command structure, missing commands, performance, output, error handling errors, boundary-related errors, calculation errors, initial and later states, control flow errors, errors in handling data, race condition errors, load conditions errors, hardware errors, source and version control errors, documentation errors and testing errors.

Statistical Quality Assurance - Overview

Statistical quality assurance reflects a growing trend throughout industry to become more quantitative about quality. And what does Statistical quality assurance means. Well, those big word imply the following series of steps steps that form the process:

- Information about software defects is collected and categorized.
- An attempt is made to trace each defect to its underlying cause.
- Using the Pareto principle (80 percent of the defects can be traced to 20 percent,
and isolate the 20 percent).
- Once the vital few causes have been identified, the defects are corrected.

Causes of errors:
- incomplete or erroneous specification (IES).
- misinterpretation of customer communication (MCC).
- intentional deviation from specification (IDS).
- violation of programming standards (VPS).
- error in data representation (EDR).
- inconsistent module interface (IMI).
- error in design logic (EDL).
- incomplete or erroneous testing (IET).
- inaccurate or incomplete documentation (IID).
- error in programming language translation of design (PLT).
- ambiguous or inconsistent human-computer interface (HCI).
- miscellaneous (MIS).

In conjunction with the collection of defect information, software developers can calculate an error index (EI) for each major step in the software engineering process. After analysis, design, coding, testing, and release, the following data are collected:
Ei = the total no. of errors uncovered during the ith step in the process.
Si = the no. of serious errors.
Mi = the no. of moderate errors.
Ti = the no. of minor errors.
PS = the size of the product at the ith step.
At each step in the software engineering process, a phase index (PI i ) is computed:
PI i = ws (Si/Ei) + wm(Mi/Ei) + wt(Ti/Ei)
Error index (EI) can be computed as follows:
EI = (PI 1 + 2 PI 2 + 3 PI 3 + iPI I)/PS

Weakness of Function Point Analysis

Function Point Analysis is seen as a very important and useful technique for requirements estimation, and for numerous other benefits (see previous post for more details). However, even such a famous method has its detractors, with a number of people / studies pointing out issues with the technique. Here are some of these issues / weaknesses / problems:

- FPA is seen as not being fully suited for object oriented work with an objection that the core of the technique, function points cannot be reasonably counted for object-oriented requirements specifications. The problems are that several constructs of object oriented specifications representation can be interpreted in various ways in the spirit of FPA, depending on the context.
- Function point counts are affected by project size; ideally Function Points should not be affected by project size since they measure each function, but this does not work out in actual practise
- Function Point Counting techniques have been found to be not easy to apply to systems with very complex internal processing or massively distributed systems
- Difficult to define logical files from physical files
- The validity of the weights that were present in the initial technique that the founder of FPA, Albrecht, set up as well as the consistency of their application has been challenged
- Different companies do calculate function points slightly different (actually depending on the process and people who do the actual Function Counts), making intercompany comparisons questionable and negating one of the core benefits of having standardised Function Counts
- There is a conflict in the usage of FPA for project size purposes, this conflict being with another standard measure of counting - The number of lines of code is the traditional way of gauging application size and is claimed to be still relevant because it measures what software developers actually do, that is, write lines of code. At best it can be used along with Function Counts.
- Doing FPA means that you are depending on converting back the available information to actually do essentially the same thing as requirements specification, and should end up with the same types of errors. This process is touted as a big error prone area.
- Function points, and many other software metrics, have been criticized as adding little value relative to the cost and complexity of the effort which are major factors in decision making
- The effort in computing function points has some base errors inherent because much of the variance in software cost estimates are not considered (such as business changes, scope changes, unplanned resource constraints or reprioritizations, etc.).
- Function points don't solve the problems of team variation, programming tool variation, type of application, etc.
- FP was originally designed to be applied to business information systems applications. So, the data dimension was emphasized and as a result, FPA was inadequate for many engineering and embedded systems.
- Another problem, this one dealing with the technical process of FPA comes up when assessing the size of a system in unadjusted function points (UFPs). The
classification of all system component types as simple, average and complex is not sufficient for all needs.
- Counting FP's is a major factor, requiring the presence of a skilled counter. Many companies get this work done by people not having the desired skill level (this happens for other tools as well, but counting correct FP's is critical to the whole system)
Inspite of these problems, FPA is a very useful tool, and probably a very good fitment for doing estimation.

Defect Management

What is a defect?

For a test engineer, a defect is:
• Any deviation from a given specification.
• Anything that causes user dissatisfaction
• Incorrect output
• Software is not behaving the way it should behave.

Difference between bug, defect and error:-
• Software is said to have a bug if the features deviate from the specifications.
• Software is said to have a defect if it is having some unwanted side effects.
• Software is said to have an error if it is giving an incorrect output.

DEFECT TAXONOMIES

All software defects can be broadly categorized into the below mentioned types:
• Errors of Commission : when something wrong is done.
• Errors of Omission : when something is left out by accident.
• Errors of Clarity & Ambiguity : when there are different interpretations.
• Errors of speed & capacity.
• Conceptual or Design bugs.
• Coding bugs
• Integration bugs.
• User interface errors.
• Functional errors.
• Communication errors.
• Missing commands
• Performance
• Output
• Error handling errors.
• Boundary related errors
• Calculation errors
• Control flow errors.
• Race condition errors.
• Load condition errors.
• Hardware errors.
• Source & Version control errors.
• Documentation errors.
• Testing errors.