What is meant by defect leakage?

Defects can be defined as any discrepancies that can disrupt the functioning of a software system or application and often they are known to give nightmares and headaches to the software testers.

WHAT ARE DEFECTS?
- Defects are responsible for the production of incorrect or unexpected results and abnormal behavior of the software program.
- Defects are a consequence of the mistakes made by the software developers.

It is very difficult to set straight a buggy program. There are so many concepts that revolve around these defects. One of such concepts is defect leakage and we will be discussing it here in this article.

DEFECT LEAKAGE
- You can imagine water leaking out of a container having holes in it. We can take the defect leakage in the same context and define it.
- The defect leakage can be thought of as a phenomenon in which the defects leak out in the entire software system or application.
- It is due to the presence of the means for the escape of the defects.
- No present software testing methodology is so efficient that it can discover or dig out all the errors and bugs.
- Even after many times of rigorous testing still many bugs remain in the software system or application hidden and undiscovered.
- These defects are discovered later during the latter stages of the software development life cycle.
- Such defects are said constitute the defect leakage factor.
- Tracking the defect leakage is very time consuming process as well as effort consuming process.

APPROACHES FOR FINDING DEFECT LEAKAGE

APPROACH #1:
- The commonly followed approach for finding the defect leakage is the matrix method.
- The defect leakage can be tracked down using a standard matrix provided you can spare much of your time for following this approach.
- You will require to analyze all the defects and determine where actually the defect would have been discovered when those stages were in progress.
- Defect leakage indicates a lot about your programming efficiency and testing skills.
- If a lot of gap exists in your findings then it indicates that your testing and programming efficiency is quite low.
- In some cases it happens that you find defect leakage in the unit level which you could have discovered then and there itself and if not there at least that could have been caught at the level of the integration testing.
- The gap here that we are talking about is nothing but the difference between the actual defect logging time and ideal defect logging time.
- Defect leakages add to greater headaches than the usual defects so try avoiding them as far as possible by following a proper testing strategy and an effective test plan.
- All you need to do is to keep all your processes systematic and organized.
- If you are having a lack of time and need a fast completion of your testing work, then there is a greater probability that you may leave many bugs and errors here and there and later have problems with the defect leakages.

APPROACH #2:
- Another approach that you can follow is using a defect tracking automated system.
- This system files up a detection phase whenever a bug is logged.
- This bug is then rectified in the injection phase.
- Following this approach you can easily define the bug slippage also.
- You can also define the areas in which improvement is needed.
- This approach requires appropriate categorization of the defect injection phase.
- The categorization depends on the perception of the tester.
- A leaked defect is the one that could not be discovered during the usual testing but showed up during the production of the software.
- The documentation for the defect leakage is not produced by the quality assurance people.
- The reason why a defect was missed during the testing is often probed by the program managers.

Basis Path Testing - A white Box testing technique.

Basis Path Testing uses the algorithmic flow of the program to design tests. Test cases that are written to test the basis set execute every statement in the program at least once.

- A flow graph should be drawn only when the logical structure of a component is complex. The flow graph allows you to trace program paths more readily. When compound conditions are encountered in a procedural design, the generation of a flow graph becomes slightly more complicated.

- Independent program path is a path which has at least one new set of processing statements or a new condition. An independent path should move along one edge that has not been traversed before. To know how many paths to look for, cyclomatic complexity is a useful metric for predicting those modules that are likely to be error prone. It can be used for test planning as well as test case design. Cyclomatic complexity provides the upper bound on the number of test cases that will be required to guarantee that every statement in the program has been executed at least one time.

- There are few steps to derive test cases or the basis set.
A flow graph is drawn using the design or code.
The cyclomatic complexity of flow graph is evaluated.
A basis set of linear independent path is determined.
Test cases are prepared forcing the execution of each path in basis set.

- Graph matrix is a square matrix whose size is equal to the number of nodes on flow graph. It is a tabular representation of a flow graph. Adding link weight to the each matrix entry, graph matrix can become a powerful tool to evaluate program control structure during testing.

Requirements Traceability Matrix (RTM) - A tool for managing requirements

The Requirements Traceability Matrix is a tool for managing requirements. It is not only used in requirements engineering but through out the software engineering process. The white box approach refers to the development of system requirements without
considering the technical implementation of those requirements.

Components of Requirements Traceability Matrix

- RTM ID: A unique identification number for a specific requirement.
- Requirements: This is a structured sentence describing the requirements in a "shall" format.
- Notes: These are additional notes of requirements.
- Requestor: This is the person who requested the requirement.
- Date: This is the date and time the requirement was requested by requestor.
- Priority: This is the priority given to the requirements.

While the above elements are the initial RTM, one can customize the matrix to tie the requirements with other work products or documents. It can be done by:

- Relative to RTM ID: It specifies if the current RTM is directly or indirectly related to other requirements.
- Statement of Work (SOW): This is a description that relates the requirements directly back into a specific paragraph within the Statement of Work. It is a document that specify the nature of the engagement of the development team with their clients.

The components and the use of the RTM should be adapted to the needs of the process, project and product. It grows as the project moves along.

What comprises Test Ware Development : Test Strategy Continued...

Test ware development is the key role of the testing team. Test ware comprises of:

Test Strategy

Before starting any testing activities, the team lead will have to think a lot and arrive at a strategy.The following areas are addressed in the test strategy document:
- Test Groups: From the list of requirements, we can identify related areas, whose functionality is similar. These areas are the test groups. We need to identify the test groups based on the functionality aspect.

- Test Priorities: Among test cases, we need to establish priorities. While testing software projects, certain test cases will be treated as the most important ones and if they fail, the product cannot be released. Some other test cases may be treated like cosmetic and if they fail, we can release the product without much compromise on the functionality. This priority levels must be clearly stated.

- Test Status Collections and Reporting:
When test cases are executed, the test leader and the project manager must know where exactly we stand in terms of testing activities. To know where we stand, the inputs from the individual testers must come to the test leader. This will include, what test cases are executed, how long it took, how many test cases passed and how many failed etc. Also, how often we collect the status is to be clearly mentioned.

- Test Records Maintenance: When the test cases are executed, we need to keep track of the execution details like when it is executed, who did it, how long it took, what is the result etc. This data must be available to the test leader and the project manager, along with all the team members, in a central location.This may be stored in a specific directory in a central server and the document must say clearly about the locations and the directories.

- Requirements Traceability Matrix: Ideally, each software developed must satisfy the set of requirements completely. So, right from design, each requirement must be addressed in every single document in the software process. The documents include the HLD, LLD, source codes, unit test cases, integration test cases and the system test cases. In this matrix, the rows will have the requirements. For every document, there will be a separate column. So, in every cell, we need to state what section in HLD addressed in every single document, all the individual cells must have valid section ids or names filled in.

- Test Summary: The senior management may like to have a test summary on a weekly or monthly basis. If the project is very critical, they may need it on a daily basis also. It addresses what kind of test summary reports will be produced for the senior management along with the frequency.

Requirements Management

Requirements management involves communication between the project team members and stakeholders, and adjustment to requirements changes throughout the course of the project. To prevent one class of requirements from overriding another, constant communication among members of the development team is critical. For example, in software development for internal applications, the business has such strong needs that it may ignore user requirements, or believe that in creating use cases, the user requirements are being taken care of. The purpose of requirements management is to assure the organization documents, verifies and meets the needs and expectations of its customers and internal or external stakeholders.

Requirements traceability is concerned with documenting the life of a requirement. It should be possible to trace back to the origin of each requirement and every change made to the requirement should therefore be documented in order to achieve traceability. Even the use of the requirement after the implemented features have been deployed and used should be traceable.

The purpose of the Requirements Traceability Matrix is to help ensure the object of the requirements conforms to the requirements by associating each requirement with the object via the traceability matrix.
A traceability matrix is used to verify that all stated and derived requirements are allocated to system components and other deliverables.

- Features Traceability Table : It shows how requirements relate to important customer observable system/product features.
- Source Traceability Table : It identifies the source of each requirement.
- Dependency Traceability Table : It indicates how requirements are related to one another.
- Subsystem Traceability Table : it categorizes requirements by the subsystem that they govern.
- Interface Traceability Table : It shows how requirements relate to both internal and external system interfaces.