Determining the urgency of defects

Determining the Urgency of Resolving Defects in Software Development

Why Defect Resolution is Critical

Software defects can impact user experience, security, and overall functionality. Knowing when to prioritize defect resolution is essential for maintaining software quality and reliability.

Factors That Determine Defect Urgency

Not all defects require immediate attention. Some can wait for a later release, while others demand urgent resolution. Here are the primary factors that influence defect urgency:

1. Severity of the Defect

• Critical defects: Issues that cause system crashes, data loss, or security breaches must be fixed immediately.
• Major defects: Problems that significantly affect functionality but have workarounds can be scheduled for the next release.
• Minor defects: Cosmetic issues or minor inconveniences can be resolved in future iterations.

2. Impact on Users

The number of users affected by a defect is a key consideration. If a bug impacts a large user base or a critical feature, fixing it should be a top priority.

3. Business Implications

Some defects may lead to revenue loss or compliance violations. In such cases, resolving the issue quickly is crucial to maintaining business integrity.

4. Frequency of Occurrence

Bugs that occur frequently require more immediate attention compared to rare issues. High-frequency defects can indicate deeper systemic problems that need to be addressed.

5. Security Risks

Security vulnerabilities should always be addressed urgently to prevent potential data breaches or cyber-attacks.

6. Dependencies on Other Features

If a defect affects multiple components or prevents other features from functioning, resolving it becomes more urgent.

How to Prioritize Defect Resolution

Using a structured approach to defect prioritization ensures that resources are allocated efficiently. Consider the following strategies:

1. Categorization Using a Defect Matrix

Developers can use a defect matrix to classify issues based on severity and impact, helping teams prioritize effectively.

2. Utilizing Agile Methodologies

Frameworks like Scrum and Kanban help teams assess defects dynamically and allocate sprint resources accordingly.

3. Stakeholder Involvement

Engaging product owners and end-users ensures that defect prioritization aligns with business and customer needs.

Final Thoughts

Determining defect urgency is essential for maintaining software quality, user satisfaction, and security. By leveraging structured prioritization methods, teams can optimize resources and enhance software reliability effectively.

Trying to solve defects where the flow is not repetitive .. (contd..)

In the previous post (Trying to solve defects where the flow is not repetitive ..), we talked about defects that are not easily reproducible, and some points around that. In this post, we will talk about such defects with more points. Handling defects that are not reproducible is one of the biggest pain points of the development cycle, since you know that even though the defect is not easily reproducible, it will come back to bite you at some point or the other. So, you cannot ignore those defects, although after trying to reproduce a defect like this for hours, it is tempting to just forget those defects. It needs a strong policy to ensure that the team does not overlook such defects.

- The team needs to have a policy about handling defects that ensures that the team does not overlook some of these defects. The policy needs to define how the defect needs to be handled, how it is tracked, how many times does the team needs to re-test those scenarios, whether such defects need to be handed over to other testing personnel, and so on. These policies need to be ensured as a part of the system, so that people know exactly how to handle such defects.
- These kind of non-reproducible defects do not follow any easy pattern, but from time to time, groups of developers and testers do figure out steps on how to figure out how to reproduce some of these defects. I remember specific cases where a developer wrote an app that would be run along with the application during the testing process, and helped in around 10% of the cases of non-reproducible defects, and that was an amazing feat, since it helped save a lot of time and helped in resolving defects that were otherwise difficult to fix.
- Statistics for these kind of defects is very important, and during the development process rather than at the end. What these statistics for non-reproducible defects do is to determine for the team management about how many these kind of defects are being found, as well as how many are being resolved. Every team needs to specify a number beyond which action needs to be taken. For example, during a specific 3 week period, we found that there was a spike in the number of non-reproducible defects that were being logged. This in turn resulted in more time being spent by the testers on trying to reproducible and hence resolve these defects. A solution needed to be found for this - and we spent some more time on focusing superior development effort on these defects. Eventually, this extra effort helped, since it was a new version of an external component that was causing problems to the application and resulting in many such non-reproducible defects.

As a result, even though it is difficult to visualize handling such non-reproducible defects, they are a reality and teams need to have a process of handling such defects.

Trying to solve defects where the flow is not repetitive ..

If the title of the post did not make sense, then let me explain. What I refer to is the case where a defect has been logged (let's not discuss the severity of the defect right away), but the flow to replicate the defect is not easy. Anyhow, it is easy when a defect can be easily replicated, when the steps are easy. However, for anybody who has gone through the whole process of defect detection, analysis and resolution, finding an easy method of replicating the bug steps can almost be counted as lucky. There can be numerous cases where the steps for replicating a defect is not so easy.
Consider the case where a defect is caused due to a particular variable loading a specific value. This value has not been loaded during the current workflow, but gets loaded during a step in an unrelated workflow. Now, depending on whether the user has gone through this unrelated workflow previously, the defect will show up or will not show up. One might think that the person would be able to replicate this defect easily, but this is easy to think post-facto once the defect process has been understood. For the tester, there is no way of easily knowing that the defect is getting caused due to the operation of an unrelated workflow; it is only the skilled tester who tries the workflow inside the product multiple times who can try and figure that the defect is happening only in those cases where the unrelated workflow has been used. These kind of stories are common (not the majority of cases, but common enough that testers and developers need to be aware of these issues).
What does one do in these kind of cases ? Well, such cases are handled different by different teams, but let me suggest a few points for these:
- It is important to understand that every defect is important, and whether the defect is fixed or left unresolved needs to be decided in a determined manner. For a customer, the defect can be problematic, especially when the person used the unrelated workflow and wonders why the team left such a defect not fixed.
- Another variable that decides what needs to be done is the severity of the defect. For defects that were severe, it is not easy to decide to close the defect as unresolved. For defects that are minor, the bug decision committees would need to decide whether these minor defects need to be fixed, or can be deferred; that the team was comfortable with the minor inconvenience that such defects would cause to the customers.
- It is not easy for the developers to fix defects that the testing team cannot easily replicate, but that does not mean that they should not be fixed. There will be a larger number of customers than the testing cases within the development team, and one can be sure that some of the defects that could not be replicated will show up and cause inconvenience to the customer,
- There are methods that the development team can use to try and make it easier to find a solution to the defect. For example, it might be possible for the testing team to use special development versions of the software where the values of key variables can be recorded during the testing process, and this would help in trying to figure out exactly what is going on; and there are other development steps that can be done to try and get more detail on what exactly is causing these defects.
- Teams can setup processes on how to treat such defects; letting the tester spend some adhoc time on these defects; or give these defects to a different tester in order to get a fresh set of eyes; or ensure that the developer has some time for these defects in order to try and get some sort of solutions for atleast some of these defects.

Read more about this in the next post (Trying to solve defects where the flow is not repetitive .. (contd..))

Supporting a previous version of the product - level of support

Suppose you are part of an application that has released several versions. What do you do about support for previous versions ? There is a strong temptation to set it such that most of the effort is focused towards creating great new versions for the ongoing and future versions, and if there is any support for the previous versions, a lot of that support will be focused on how to move the users to the latest version rather than really supporting them while they are using the previous version. After all, the money is to be made when users move to the latest version, and the more the support for the previous version, the more the cost of doing so.
However, this philosophy has underdone some changes in the last many years. No organization will directly accept that they do not provide good support to uses of previous versions, but at the same time, there are sizable costs associated with supporting users who are on previous versions. What are some of these costs ?

- One of these costs involve releasing patches and updates for these previous versions. When software has been developed and released, you would expect that there would be no real defects in software after it has been released, and hence there would not be the need for software patches on a regular basis. However, this is far from true. Once a software has been released, defects that were not found during the testing phase will pop up, either found by the team in the new version (but which also existed in the previous version), or found by users and affecting some of their workflows. Some of these defects would be serious enough that they need to be fixed - and in today's world where users can share problems and report lack of support by the organization on various social forum, it can be critical to continue to interact with users. As a result, the organization needs to evaluate such issues and be seen to be responsive.
And then there is the overall environment. There can be updates issued by the operating system or updates to other files that can cause problems. Once for example, an update issued by a popular anti-virus software caused some files in the application to get frozen, and that crippled a section of the software (which had been released 2 versions back). Like many such problems, these were reported by users who came across these issues (typically the users are able to see the problem, but to figure out the problem, there is a need to have some intense technical research done by the support team, sometimes backed up by the product team). If the product team allows such problems to continue on user forums without some obvious research or effort undertaken, it can lead to user dis-satisfaction. The organization could pretend that these were for previous versions of software and such problems are fixed in the latest versions, but such a defense is not easy to undertake and few users are convinced about such a reason. In fact, such a defense tends to put off users, who would think that they would also not be supported when such a problem is faced by them later on.

Read more about this complex topic in the next post on this series (TBD).

What is the concept of flow control?

- Flow control is an important concept in the field of data communications. 

- This process involves management of the data transmission rate between two communicating nodes. 

- Flow control is important to avoid a slow receiver from being outrun by a fast sender. 

- Using flow control, a mechanism is designed for the receiver using which it can control its speed of transmission.

- This prevents the receiving node from getting overwhelmed with traffic from the node that is transmitting.

- Do not confuse yourself with congestion control and flow control. Both are different concepts. 

- Congestion control comes in to play when in actual there is a problem of network congestion for controlling the data flow. 

On the other hand the mechanism of flow control can be classified in the following two ways:

1. The feedback is sent to the sending node by the receiving node.
2. The feedback is not sent to the sending node by the receiving node.

- The sending computer might tend to send the data at a faster rate than what can be received and processed by the other computer. 

- This is why we require flow control. 

- This situation arises when the traffic load is too much up on the receiving computer when compared to the computer that is sending the data. 

- It can also arise when the processing power of the receiving computer is slower than the processing power of the one that is sending the data.

Stop and Wait Flow Control Technique 

- This is the simplest type of the flow control technique. 

- Here, when the receiver is ready to start receiving data from the sender, the message is broken down in to a number of frames. 

- The sending system then waits for a specific time to get an acknowledgement or ACK from the receiver after sending each frame. 

- The purpose of the acknowledgement signal is to make sure that the frame has been received properly. 

- If during the transmission a packet or frame gets lost, then it has to be re-transmitted. 

- We call this process as the automatic repeat request or ARQ. 

- This technique has a problem which is that it is capable of transmitting only one frame in one go. 

- This makes the transmission channel very inefficient. 

- Therefore, until and unless the sender gets an acknowledgement it will not proceed further for transmitting another packet. 

- Both the transmission channel and the sender are left un-utilized during this period. 

- Simplicity of this method is its biggest advantage. 

- Disadvantage is the inefficiency resulting because of this simplicity. 

- Waiting state of the sender creates inefficiency. 

- This happens usually when the transmission delay is shorter than the propagation delay. 

- Sending longer transmissions is another cause for inefficiencies. 

- Also, it increases the chance for the errors to creep in this protocol. 

- In short messages, it is quite easy to detect the errors early. 

- By breaking down one big message in to various separate smaller frames, the inefficiency increases. 

- This is so because these pieces altogether take a long to be transmitted.

Sliding window Flow Control Technique 

- This is another method of flow control where permission is given to the sender by the receiver for continuously transmitting data until a window is filled up. 

- Once the window is full, sender stops transmission until a larger window is advertised. 

- This method can be utilized in a better way if the size of the buffer is kept limited. 

- During the transmission, space for say n frames is allocated to the buffer. 

- This means n frames can be accepted by the receiver without having to wait for ACK. 

- After n frames an ACK is sent consisting of the sequence number of the next frame that has to be sent. 

Defect Management: Dividing overall defect reports into separate functional areas - Part 3

This is a series of posts where I look at the creation of a defect status report, in a way that it provides enhanced information to the team and its managers and helps them to make decisions. In the previous post (Count of new incoming defects), I talked about adding more parameters to the defect report that help in information that can let the team know whether the number of defects getting added on a daily basis will enable the team to reach their defect milestones by the required time and date. This kind of data, coming in on a regular daily cycle, helps the team to decide whether their current defect pattern is on the desired path, above it, or below it, and accordingly make the required decisions.
This post will see more details added to the defect report that provides a lot of useful information to the team. The last post talked about the flow of incoming defects that move to the ToFix state. However, there is another type of information that is relevant. Defects that are present in the system are not only owned by the development team. In addition to the defects in core code, there may be defects that are present in the components used in the application. These are defects that are not attributable to the development team, but to the vendors or other groups that provide the components. A number of teams that I know typically track these defects in the defect database, but distinct from the defects with the core team.
The nature of defects that are against external components is different from those in the core code. Even though to the customer it does not matter whether the defect is within the core code or in an external component, the amount of effort required in terms of coordination and communication is entirely different from the other defects that are with the core developmental team. If a defect is with a component that is not owned by the team, the timeline for fixing of the defect may take longer and need persuasion; or there may be a lot of back and forth between the tester and the external component team to study the situation in which the defect occurs (which also includes sending the environment in which the defect occurred to the external vendor - and this has its own cost and restrictions, since if the team is working on a new version of the software, there would NDA issues and IP issues related to sending details of the environment to the external component team), and so on. Another concern could be that that even if such a defect is resolved, it might need a new version of the component, which has its own extra cost about testing the component on its own to check whether it is fine or there are other issues with the same.
As a result, it needs to separate out the incoming defects about whether they belong to the core team or whether they are attributable to people outside the team; and if the proportion of such defects that are outside the core team is increasing, it is a matter of concern to the team, since resolving such defects typically takes much more effort and time.

Defect Management: Dividing overall defect reports into separate functional areas - Part 2

Part 1 (Dividing defect reports into ToFix, ToTest and ToDefer counts) of this post talked about the importance of Defect Management in a software project, and then got into some details about the regular sending out of a report on Total defects, with these defects having been broken down into ToFix, ToTest and ToDefer stats, maintained on a graph over a period of time with daily updates, so that the team and the managers can figure out whether the team is on progress to resolve these bugs.
This post continues on this line, talking about additional items that can be added to this defect chart and metrics to provide more information to the team and determine whether it is on the right track or not. Are all these metrics important ? There is a lot of comments about not over-burdening people with too many statistics, and there are more comments about letting people do their work rather than sending so many statistics that they stop looking at these stats. However, it is also true that the role of the team managers is to look at the broader situation in terms of project status, and the defect management is an important part of this. It is true that the team members should not be burdened with these figures, but for the team managers, it is critical to look at such data.
So, the team looks at the ongoing figures for defects in terms of ToFix over a period of days and tries to determine whether the team is on the right track or not. So what else should you be capturing ? Another metric that can now be added to such a report is about the number of defects that are still incoming. There are primarily 2 ways in which defects can be added to the count of developers:
- New defects that are logged against the development team and which add to their count and to the overall ToFix count
- Defects that have been rejected by the testing team after they have been marked fixed by the developer but there is a problem in the fix (this can vary a lot among different teams and even in a team - a developer could be fixing defects with hardly any returns and there could be another developer who is under pressure and many of whose defects are returned because of some problems). So, whether to determine this kind of statistic and calculate metrics for such a case determines of whether the team is seeing such kind of returns for the defect management.
Once you have these kind of defect counts, it helps in determining the current status of defects and see whether the team is on the right track. So, you have a total count of open ToFix defects, and there is a decline in such a count needed to hit the deadlines. However, for getting to such a deadline, you need the number of incoming defects to be also fitting into this strategy. If there are a large number of incoming defects, then the team will not be easily able to determine whether their ToFix defect count is decreasing by the amount they want to hit their targets, and this then needs a change to the strategy to determine whether the team will get there or not.

Defect Management: Dividing overall defect reports into separate functional areas - Part 1

Handling defects is one of the major efforts that plays an integral role in handling a project schedule and making it successful. I have known multiple teams where the team did not have a good running estimate of their defect count and the defect estimation over the remaining period of time left in the schedule; as a result, when the team was closer to the final stages of the schedule, they found that they had too many defects that made the remaining part of the schedule very tight - which meant that if they were to do an accurate reckoning of their status, they would need to either defer more defects and maybe end up with a product that is lower in quality; or the product would need to extend their timeline / schedule, which has a huge implication  for the team and many other teams that are involved in the release schedule of the product.
How do you avoid this ? The first paragraph of this post points out a huge problem, but the answer cannot be handled in a single post; it can be handled by a single cheesy phrase but which does not provide any solutions - "You need to do Defect Management". Now, let us get down to the meat of this post - this post just takes a specific aspect of defect management - sending a split of the defect counts as per the different areas. This in turn provides a better view of the defect picture to the team and helps in the process of overall defect management.
We wanted to have a system whereby we could track the counts for each of the separate functional areas and yet have the management team have access to these data on an ongoing basis. These also helped the separate functional teams do a targeting of the counts of the defects of their respective functional areas and work towards reducing this count. So, we took the overall data for defects for the product (open defects) and split these into the following areas:
Open defects:
ToFix (these are primarily defects owned by the development team, although there could be defects that are carried by other team - such as where there are defects with components supplied by external teams)
ToTest (these are primarily defects owned by the testing team, although since anybody can file a defect within the team, there may be people other than the testing team who own a defect)
ToDefer (the exact terminology of these defects can be different across organizations; but these are typically defects that are with a defect review committee for evaluation. These can be significant defects that need evaluation by the committee before they are to be fixed, or these can be defects that are not worthy of fixing but the team wants the committee to take a final call, and so on).
What is the purpose of sending out separate stats on a regular basis ? These data, when plotted on a regular graph over a period of time provides a large amount of information. The team and the managers, although they are in the thick of things, sometimes need to see such kind of aggregate information to take a good decision. For example, if the team is in the second part of the cycle and close to the timeline, and yet the ToFix graphs do not show a declining trend, then this is something to worry about. When such a stage happens, I have seen the development team manager doing a major discussion with the team to figure out how to reduce these counts and figure out what is happening. In extreme cases, I have seen the team actually take a hard look at these defect counts and then make a recommendation for extending the schedule (which is not a simple step to take).

Defect handling - Planning how to find bug bugs earlier in the cycle

In every cycle of software development, whether this be product development or working on a project, one of the key items is about finding defects. There can be small defects or big defects, but the plan is always to try to fix these defects. Smaller defects can sometimes be more easy to handle, since they have a lesser impact on the customers, and in several cases, it would even be easy to defer some of these bugs if they are low severity and there is a squeeze on time and resources. However, larger bugs, those that have an impact on functionality or workflows are harder, and so are the ones that are complex or need more time to fix. These are the sort of bugs / defects that a team would find hard to defer or leave alone, and they can be critical to fix.
One of the biggest problems with such defects is the time period in which many of these defects are found. Defects which have a high impact are typically found in the latter part of the cycle, primarily because a lot of the functionality comes to be ready in the latter half of the cycle. If there are a number of features in development, the earlier parts of the cycle see the development of these features. As time progresses, these features start getting into a good shape and the integration points of these features start getting worked on. This is the time when the workflows of the product start coming into shape, and that is when the testing team will be able to check the integration of these features into one another.
The testing effort at this stage is able to detect workflow problems, as well as design flaws where the type and amount of information flowing from one feature to the other may have issues, and not happening as per design. These defects take more time to analyse, and may also need teams from the different feature areas to collaborate to figure out the defects. As a result, the time involved to fix these defects is more, and this gets problematic when the number of such defects found is more than expected, which means that the time that the team has to fix issues may be less. Further, the later that such defects are found, the greater the risk that the fix for such defects can cause other problems; because of such risks, defects found later are at a greater risk of not being fixed.
What do you do ? Some of the issues may be more problematic to solve. Feature development work focuses on the specific features in the earlier part of the cycle, with the focus shifting to integration only later in the cycle, so changing the timelines for this may be more difficult. But, it is possible to do studies to estimate the amount of bugs that will be found (far easier of this is just a new version of the product being developed and there is historic data), and then plan for more time for the same. At the same time, a number of problems typically are found out if there is inadequate time spent during the design and architecture phase and teams should ensure that they are spending the right amount of time on these activities. Further, as the feature is being developed, workflow or integration related flows should be tested, even if integration has not been completed. An example of this can be done is to prepare a software harness which will allow the input and output of data from the various features even if the integration has not been done. Doing this ensures that a number of the defects that are found post integration can be found earlier in the cycle, and this saved a lot of time and effort.

What is meant by Statistical Usage Testing?

Statistical usage testing is the testing process that is aimed at the fitness of the software system or application.

The test cases chosen for carrying out statistical usage testing mostly consist of the usage scenarios and so the testing has been named as statistical usage testing. Software quality is ensured by the extensive testing but that has to be quite efficient. Testing expenditures covers about 20 – 25 percent of the overall cost of the software project. In order to reduce the testing efforts, deploy the available testing tools since they can create automated tests. But usually what happens is that the important tests require manual intervention with the tester requiring thinking about the usage as well behavior of the software. This is just the repetition of the tasks that were done during the requirements analysis phase.

About Statistical Usage Testing

- A usage model forms the basis for the creation of tests in statistical usage testing.

- Usage model is actually a directed usage graph more like a state machine and it consists of various states and transitions. 

- Every transition state has a probability associated with it regarding the traversal of the transition when the system would be in a state that marks the beginning of the transition arc. 

- Therefore, the sum of the probabilities of outgoing transitions sum up to unity for every state.

- Every transition can be associated with an event and more with parameters that are known to trigger the particular transition. 

- Such event associated transitions can be further related to certain conditions called the guard conditions. 

- These conditions imply that the transition occurs only if the value of the event parameter satisfies the condition.

- For assigning probabilities to the transitions, 3 approaches have been defined as follows:

1. Uninformed approach: In this approach, same probability is assigned to the exit arcs of a state.
2. Informed approach: In this approach, a sample of user event sequences for calculating suitable properties. The sample is captured from either an earlier version of the software or its prototype.
3. Intended approach: This approach is used for shifting the focus of the test to certain state transitions and for modeling the hypothetical users.

- According to a property termed as the marcov property, the actual state is what on which the transition probabilities are dependent. 

- However, they are independent of the history again by the property. 

- This implies that the probabilities must be fixed numbers. 

- A system based up on this property is termed as a marcov chain and it requires conclusion of some analytical descriptions. 

- Usage distribution is one among such descriptions. 

- It gives for every state its steady–state probability i.e., appearance rate that is expected.

- All the states are associated to one or the other part of the software system or application and the part of the software that attracts more attention from the tests is shown by the usage distribution. 

- Some other important descriptions are:

1. Expected test case length
2. Number of test cases required for the verification of the desired reliability of the software system or application.

- The idea of the usage model generation can be extended by handling guard conditions and enabling the non–deterministic behavior of the system depending on the state of the system’s data. 

- All this helps towards the application of the statistical usage testing to systems over a wide range. 

- The use cases are defined by the top–level structure of the unified modeling language (UML). 

What kinds of functions are used by Cleanroom Software Engineering approach?

Harlan Mills and his colleagues namely Linger, Poore, Dyer in the year of 1980 developed a software process that could promise building zero error software at IBM. This process is now popularly known as the Cleanroom software engineering. The process was named in accordance with an analogy with the manufacturing process of the semiconductors. 

The Clean room software engineering process makes use of the statistical process and its control features. The software systems and applications thus produced have certified software reliability. The productivity is also increased as the software has no defects at delivery. 

Below mentioned are some key features of the Cleanroom software engineering process:

1. Usage scenarios
2. Incremental development
3. Incremental release
4. Statistical modeling
5. Separate development
6. Acceptance testing
7. No unit testing
8. No debugging
9. Formal reviews with verification conditions

Basic technologies used by the CSE approach are:

1. Incremental development
2. Box structured specifications
3. Statistical usage testing
4. Function theoretic verification

- The incremental development phase of the CSE involves overlapping of the incremental development and from beginning of specification to the end of the test execution it takes around 12 – 18 weeks.

- Partitioning of the increments is critical as well as difficult. 

- Formal specification of the CSE process involves the following:

1. Box structured Designing: Three types of boxes are identified namely black box, state box and clear box.
2. Verification properties of the structures and
3. Program functions: These are one kind of functions that are used by the clean room approach.

- State boxes are the description of the state of the system in terms of data structures such as sequences, sets, lists, records, relations and maps. 

- Further, they include specification of operations and state in-variants.

- Each and every operation that is carried out needs to take care of the invariant. 

- The syntax errors present in a constructed program in clean-room are checked by a parser but is not run by the developer.

- A team review is responsible for performing verification which is driven by a number of verification conditions. 

- Productivity is increased by 3–5 times in the verification process as compared to the debugging process. 

- Proving the program is always an option with the developers but it calls for a lot of math intensive work.

- As an alternate to this, clean room software engineering approach prefers to use a team code inspection in terms of two things namely:

1. Program functions and
2. Verification conditions

- After this, an informal review is carried out which confirms whether all conditions have been satisfied or not. 

- Program functions are nothing but functions describing the prime program’s function.

- Functional verification steps are:

1.    Specifying the program by post and pre-conditions.

2.    Parsing the program in to prime numbers.

3.    Determining the program functions for SESE’s.

4.    Defining verification conditions.

5.    Inspection of all the verification conditions.

- Program functions also define the conditions under which a program can be executed legally. Such program functions are called pre-conditions.

- Program functions can even express the effect the program execution is having up on the state of the system. Such program functions are called the post conditions.

- Programs are mostly expressed on terms of the input arguments, instance variables and return values of the program. 

- However, they cannot be expressed by local program variables. 

- The concept of nested blocks is supported by a number of modern programming languages and structured programs always require well nesting. 

- The process determining SESE’s also involves parsing rather than just program functions.

What are requirements for Cleanroom Software Engineering? What is the need for CSE?

The process got its name from the term cleanroom which is part of the process that fabricates the semiconductors. The basic ideology behind the cleanroom software engineering process is that it is more focused up on avoiding the defects rather than removing them.  It makes use of a combination software quality methods and formal methods. Cleanroom software engineering has shifted the individual craftsmanship to peer reviewed process, from sequential development to incremental one, from informal designing to discipline specification and designing, from informal coverage to statistical usage testing and so on.

Cleanroom software engineering is actually an integration of the following three practices:

1. Program verification
2. Software engineering modeling
3. Statistical software quality assurance

The correctness of the design specifications is verified via mathematically based proof solutions. The role of the statistical usage testing here is to dig out the high impact errors. 

The following the stages through the incremental development:

1.   Establishing the requirements

2.   Formal specifications

3.   Development of the increment

4.   Delivery of the software

5.   Requirements change request

Why Cleanroom Software Enginnering is required?

- Cleanroom software engineering is required to develop software systems and application with zero defects. 

- Though it takes a lot of time to be implemented but the payoff is quite high and both quality and productivity are increased. 

- The actual cleanroom process begins after the formal specification has been done. 

- The focus then shifts towards building a more explicit design. 

- Next, the design is verified as per the specifications. 

- Two things namely statistical and mathematical reasoning are combined and used together by the cleanroom approach for test generation and testing as well. 

- Cleanroom software engineering has proven to be the first practical attempt for developing the software with statistical quality control and delivering the software product with a known MTTF.

Requirements of Cleanroom Software Engineering

- The key requirements of cleanroom approach are nothing but specifications, requirements and verification methods.

- These formal specifications and verifications are used for developing software of high quality and with a minimum of errors. 

- Statistical usage testing is also required for the evaluation of the reliability of the software product. 

Cleanroom software engineering is followed by a number of benefits:

1. Zero failures: This is the objective itself and the software is developed with a minimum number of errors if zero not possible.
2. Short development cycles: These are the resultant of the incremental process. The rework is avoided and therefore new teams get a chance to experience productivity increase that has increased by two folds.
3. Longer product life: The product life is kept longer with the help of a usage model and investments detailed specifications.

- It is also a fact that the cleanroom techniques are not much used since some people believe them to be too mathematical, theoretical, radical etc. to be used in the real development processes. 

- Also, these techniques rely heavily on the statistical quality control and correctness verification rather than relying on unit testing. 

- This means that there is a large deviation from the traditional software development approach. 

- Proving the program is always there in the option list but it requires a lot of intensive sophisticate mathematical work. 

- Therefore, in place of this, cleanroom uses an alternative.

- A team code inspection is structured in terms of verification conditions and program functions.

-  Then an informal review is carried out which confirms whether or not all the verification conditions have been satisfied. 

What is Cleanroom Software Engineering?

Cleanroom software engineering is one of the fastest emerging software development processes and has been designed for the production of the software systems and applications with a reliability of certificate level. The credit for the development of this process goes to Harlan Mills and a couple of his colleagues among which one was Alan Hevner at the IBM Corporation. 

What is Cleanroom Software Engineering?

- The cleanroom software engineering is focused on defect prevention rather than their effective removal. 

- The process was named so since the word cleanroom evoked the sense of cleanrooms that are used by the electronics industry for preventing the defects from entering the semiconductors during the fabrication process. 

- The first time when the cleanroom process was used was in the late 80s. 

- This process began to be used for military demonstration process in the early 1990s. 

Principles of Cleanroom Approach

Cleanroom process has its own principles which we have discussed below:

1. Development of software systems and applications based up on formal methods: Box structure method is what that is used by the cleanroom development for specifying and designing a software product. Later, team review is used for carrying out verification of the design i.e., whether it has been correctly implemented or not.
2. Statistical quality control through incremental implementation: An iterative approach is followed in the cleanroom software engineering process i.e., the software system is evolved through increments in which the implemented functionality gradually increases. Pre–established standards are used for measuring the quality of all the increments for making verification that the process is making acceptable process. In case the process fails to meet the quality standards, testing of the current increment is stopped and the process is returned to the designing phase.
3. Statistically sound testing: Software testing in cleanroom development process is carried as a disguise of a statistical experiment. A subset that represents software’s i/p and o/p trajectories is selected and then subjected to testing. The sample so obtained is  then considered for statistical analyzation so as to get an estimation  of the software’s reliability and level of confidence.

Features of Cleanroom Software Engineering

Software products developed using the cleanroom software engineering process have zero defects at the delivery time. Below mentioned are some of the characteristics features of the cleanroom software engineering:

1. Statistical modeling
2. Usage scenarios
3. Incremental development and release
4. Separate acceptance testing
5. No requirement of unit testing and debugging
6. Formal reviews with verification conditions

The defects rate was recorded as follows:

1. <3 .5=".5" and="and" delivered="delivered" kloc="kloc" o:p="o:p" per="per">

2.7 Per KLOC between first execution and first delivery.

Basic technologies thus used can be listed as:

1. Incremental development: Each increment is carried out from end – to – end and in some cases there is overlapping development of the increments. This whole process takes around 12 – 18 weeks and partitioning though being critical proves to be difficult.
2. Function – theoretical verification: A parser may check the constructed program for syntax errors but it cannot be executed by the program developer. Verification conditions drive the team review for verification. Verification is improved by 3- 5 times than debugging. Formal inspections also fall under this category.
3. Formal specifications: This further includes:

a)    Box structured specification: It includes 3 types of boxes namely:

Ø  Black box

Ø  State box

Ø  Clear box

b)    Verification properties

c)    Program functions

4. Statistical usage testing: It helps in implementing cost effective orientation and process control. It provides a stratification mechanism to deal with situations that are critical. 

What are main features of TestComplete?

Many of the software applications are being written as web–based applications that can be run in a browser. The measure of effectiveness with which these applications are tested varies from organization to organization. 

- Test complete automated testing tool offers the answer to this demand. 

- For tests such as regression tests, responsiveness can be generated only through automated testing tools as such. 

- Automated testing is the way to provide many benefits including repeat-ability and speed of the test execution. 

- Test automation is known to induce long term efficiency in a software system or application.

- Developers also get a rapid feedback and can carry out unlimited iterations of the tests.

- Reporting gets customized and finding defects that were missed during the manual testing becomes easy.

-  However, automation always does not prove to be advantageous.

Features of TestComplete

Test complete  testing tool comes with certain features which we shall state now:

1. Keyword testing: This tool comes with a keyword–driven test editor that is built– in and consists of many keyword operations corresponding to the appropriate automated testing actions.
2. Test record and play back: This tool records the key actions which are required to play back the test. All the actions other than the required ones are then discarded.
3. Full featured script editor: This is another built – in editor using which the test scripts can be written manually. This editor comes with some special plug – ins that provide further assistance.
4. Script debugging features: This feature lets you stop before every statement that can be executed so that you can keep a track of what is going on and make changes accordingly.
5. Access to properties and methods of the objects: The names of all the visible elements can be read by this tool including internal elements of the following applications:

a)   Delphi

b)   C++builder

c)   .net

d)   WPF

e)   Java

f)    Visual basic etc.

Also this tool enables the access to values through test scripts so that they can be verified and used in the tests.

6. Unicode support: Tool has a Unicode character set support which enables the user to test the applications that are non – ASCII and use characters such as Hebrew, greek, Arabic, katakana and so on.
7. Issue–tracking support: This tool comes with issue tracking templates which can be deployed for the creation as well as modification of the items that reside in issue – tracking systems. The tool currently provides support for the following:

a)   Microsoft visual studio 2010,2008 and 2005 team system

b)   Bugzilla

c)   Automated QA AQdev team

8. Open architecture (COM based): An open API, COM interface forms the basis for the test complete’s engine. This makes this tool independent of the source – language and enables it to read the debugger info. It can use this info during run time via debug info agent of test complete.
9. Test visualizer: This feature of test complete lets you take screen shots of the test recording as well as play back thus allowing you to make comparisons among the actual as well as the expected screens during the run time.
10. Support for plug–ins: This feature allows the third party vendors to connect with test complete via their software systems and applications.