What is the key difference between preventative and reactive approaches to testing?

There are two main approaches to testing namely:

1. Preventative approach to testing and
2. Reactive approach to testing

What is Preventive Approach to testing?

- Preventative approach to testing is analytical in nature but reactive is rather heuristic.

- The designing of the preventative tests takes place after the production of the software. 

- On the other hand, the reactive tests are designed quite early i.e., in response to the feed back or review comments given by the customers.

- But both of these tests are designed as early as possible. 

- On an overall basis, preventative tests are created earlier than the reactive tests. 

- The preventative testing approach is based up on the philosophy that the quality of the software system or application under testing can actually be improved by testing if it is done early enough during the software development life cycle.

- However, one specification is to be followed for implement this approach to testing which is that test cases should be created for the validation of the requirements before the code is written.

- Preventive approach is followed because it can be applied to all the phases of the project and not just to the code. It also reduces cost of correcting the underlying faults. 

What is Reactive approach to testing?

- Reactive approach to testing is considered to be a performance testing activity in the field of performance management. 

- The developers often do not think about the performance of the software system or application that they are developing during the early stages of the SDLC or software development life cycle. 

- More often, the performance quotient is neglected till the testing of the system is complete. 

- This is known fact that the performance of a software system or application revolves around its architecture. 

- Designing an effective architecture is a high cost activity and in some of the cases the whole system is trashed because of the presence of the huge deviations in performance factors. 

- However, waiting for problems related to the performance to surface and then dealing with them is not always the best option. 

- So, performance testing is considered to be a reactive approach to testing since the system does not gathers much importance during the early stages of the life cycle. 

- Reactive approach to testing is more of a kind of a “fix it later” approach which is less effective than the preventative approach.

- Even quality control is a reactive approach to testing. 

- Every step of the production development cycle involves quality control. 

- It is not that the developers are simply sitting idle and noting down all the areas where potential issues are suspected after the final outcome.

Importance of both the approaches

- Actually preventative and reactive approaches are considered to be two “test strategies” which are used to define the objectives of the software’s testing and how they are to be achieved. 

- These two approaches also act as a determining factor in the cost and amount of effort that is to be invested in the testing of the software system or application. 

- In preventative approach the testers are involved right from the beginning of the project. 

- The specification and preparation i.e., test planning and test design is also carried out as soon as possible. 

- Preventative approach involves activities like:

1. Reviews
2. Static analysis

- Contrary to the preventative approach, in reactive approach testers are involved late in a project.

- The test planning and designing is started after the development of the project has ended. 

- Reactive approach involves techniques like exploratory testing.

Explain Test plan, Test Strategy and Test Scenario?

The concept of software testing seems like an amalgam of similar terms, all mixed up in to each other. We all get confused sometimes between the different but similar sounding terms. 

For greater understanding of the software testing, one needs to know exactly what all the inclusive terms mean and how and where they are used or implemented. This piece of writing is dedicated to the cause of removal of such confusions. In this article we are going to clear three major concepts of software testing namely test plan, test strategy and test scenarios.

1. Test Plan

- It does the same as a normal plan would do i.e., detail out a systematic approach to the accomplishment of a certain task.

- For the test plan the task is the successful completion of a particular software testing. 

- A test plan gives a detailed approach of all the processes or activities to be undertaken during the testing.

- A test plan marks the work flow of the software testing mechanism. 

- A test plan states the strategy to be followed to make the software testing successful.

- It provides a means to check whether or not the software system or application meets all the requirements and specifications as mentioned by the client or the customer in the documentation. 

- The test plan is usually created by the testers.

- A typical test plan includes the following fields:

      (a)  Compliance test or design verification: performed on the product samples while the actual product is in development stage.

   (b)  Production test or manufacturing: performed during the assemblage process of the product for assuring quality and performance.       

      (c)  Commissioning test or acceptance: performed while the product is being installed.

      (d)  Repair test or servicing: performed during the product’s service life.

      (e)   Regression test: performed on an existing version of the product to ensure the working of the functionalities.

               

- The level of the design of the test plan depends on the complexity level of the software system or application. 

- For more complex systems high level test plans are created. 

- Apart from the above mentioned tests, a typical test plan must cover about the test methodologies, test coverage and test responsibilities. - It also states what all requirements are to be verified in the testing.

2. Test Strategy

- Test strategy forms an essential part of the test plan. 

- To put it simply we can say that the test strategy is nothing but a superficial sketch or outline of the actual test plan of the software development cycle. 

- A test strategy is developed just to inform the project developers about the approach that is to be followed during the testing. 

- The key components of a typical test strategy are objectives of the testing, test methods, test duration, resource requirements and test environment. 

- Test strategies give the description of the risk mitigation, tests to be carried out and not to forget the entry and exit criteria for the tests. 

- For each level of software testing, an individual test strategy is created.

3. Test Scenario

- Test scenarios form an essential component of the scenario testing. 

- These test scenarios provide a hypothetical measure to help the tester out of some complex problem encountered during the testing. 

- The main characteristics of a typical scenario are its complexity, credibility, motivational story with a reasonable outcome. 

- The scenarios are usually lengthy as compared to the length of the test cases probably because of more number of steps are present in a test scenario. 2 or more scenarios together make up a scenario file. 

What comprises Test Ware Development : Test Strategy Continued...

Test ware development is the key role of the testing team. Test ware comprises of:

Test Strategy

Before starting any testing activities, the team lead will have to think a lot and arrive at a strategy.The following areas are addressed in the test strategy document:
- Test Groups: From the list of requirements, we can identify related areas, whose functionality is similar. These areas are the test groups. We need to identify the test groups based on the functionality aspect.

- Test Priorities: Among test cases, we need to establish priorities. While testing software projects, certain test cases will be treated as the most important ones and if they fail, the product cannot be released. Some other test cases may be treated like cosmetic and if they fail, we can release the product without much compromise on the functionality. This priority levels must be clearly stated.

- Test Status Collections and Reporting:
When test cases are executed, the test leader and the project manager must know where exactly we stand in terms of testing activities. To know where we stand, the inputs from the individual testers must come to the test leader. This will include, what test cases are executed, how long it took, how many test cases passed and how many failed etc. Also, how often we collect the status is to be clearly mentioned.

- Test Records Maintenance: When the test cases are executed, we need to keep track of the execution details like when it is executed, who did it, how long it took, what is the result etc. This data must be available to the test leader and the project manager, along with all the team members, in a central location.This may be stored in a specific directory in a central server and the document must say clearly about the locations and the directories.

- Requirements Traceability Matrix: Ideally, each software developed must satisfy the set of requirements completely. So, right from design, each requirement must be addressed in every single document in the software process. The documents include the HLD, LLD, source codes, unit test cases, integration test cases and the system test cases. In this matrix, the rows will have the requirements. For every document, there will be a separate column. So, in every cell, we need to state what section in HLD addressed in every single document, all the individual cells must have valid section ids or names filled in.

- Test Summary: The senior management may like to have a test summary on a weekly or monthly basis. If the project is very critical, they may need it on a daily basis also. It addresses what kind of test summary reports will be produced for the senior management along with the frequency.

What comprises Test Ware Development : Test Strategy

Test ware development is the key role of the testing team. Test ware comprises of:

Test Strategy

Before starting any testing activities, the team lead will have to think a lot and arrive at a strategy. This will describe the approach, which is to be adopted for carrying out test activities including the planning activities. This is a formal document and the very first document regarding the testing area and is prepared at a very early stage in software development life cycle. This document must provide generic test approach as well as specific details regarding the project.

The following areas are addressed in the test strategy document:
- Test Levels: The test strategy must talk about what are the test levels that will be carried out for that particular project. Unit, Integration and system testing will be carried out in all projects. But many times, integration and system testing may be combined.

- Roles and Responsibilities: The roles and responsibilities of test leader, individual testers, project manager are to be clearly defined at a project level. the review and approval mechanism must be stated here for test plans and other test documents. Also, we have to state who reviews the test cases, test records and who approved them.The documents may go through a series of reviews or multiple approvals and all these are mentioned in this section.

- Testing tools: Any testing tools which are to be used in different test levels must be clearly identified. This includes justifications for the tools being used in that particular level also.

- Risks and Mitigation: Any risks that will affect the testing process must be listed along with the mitigation. By documenting the risks in this document, we can anticipate the occurrence of it well ahead of time and then we can proactively prevent it from occurring. Sample risks are dependency of completion of coding, which is done by sub-contractors, capability of testing tools etc.

- Regression Test Approach: When a particular problem is identified, the programs will be debugged and the fix will be done to the program. To make sure that the fix works, the program will be tested again for that criteria. Regression test will make sure that one fix does not create some other problems in that program or in any other interface. So, a set of related test cases may have to be repeated again, to make sure that nothing else is affected by a particular fix. how this is going to be carried out is elaborated in this section.

What is the possible test approach for simulation system ?

A simulation system's primary responsibility is to replicate the behavior of the real system as accurately as possible. Therefore, a good place to start creating a test plan would be to understand the behavior of the real system.

- Subjective Testing:
It mainly depends on an expert's opinion. An expert is a person who is proficient and experienced in the system under test. Conducting the test involves test runs of the simulation by the expert and then the expert evaluates and validates the results based on some criteria. Advantage of this approach is that it can test those conditions which cannot be tested objectively. Disadvantage is that the evaluation of the system is based on the expert's opinion which may differ from expert to expert.
- Objective Testing:
It is mainly used in the systems where the data can be recorded while the simulation is running. This testing technique relies on the application of statistical and automated methods to the data collected.
Statistical methods are used to provide an insight into the accuracy of the simulation. These methods include hypothesis testing, data plots, principle component analysis and cluster analysis.
Automated testing requires a knowledge base of valid outcomes for various runs of simulation. The knowledge base is created by domain experts of the simulation system being tested. The data collected in various test runs is compared against this knowledge base to automatically validate the system under test. An advantage of this kind of testing is that the system can continually be regression tested as it is being developed.

Risk Based Testing and the strategy behind risk based testing

Risk analysis is applicable on the level of system, subsystem and individual function or module. Testing is used in software development to reduce risks associated with a system. Risk-based testing (RBT) is a type of software testing that prioritizes the features and functions to be tested based on the risk they represent.
Risk-based testing is a skill. It’s not easy to know the ways that a product might fail, determine how important the failures would be if they occurred, and then develop and execute tests to discover whether the product indeed fails in those ways.
The main input into risk based testing is the business requirements supplied by the customer of a software application or system which outlines all of the features which must be present and explain how they should work, how each process should function and what the software should do.

Test managers prioritize tests to fit in with the project’s schedule and the test resources available.A risk based approach to testing takes a much deeper look at the real underlying needs of the project and what really matters to the end-customer.
Risk based testing is about carefully analysing each requirement and each test to ensure that the most important areas of the system and at the same time, those areas which are more likely to experience a failure receive the most attention from the test team. When risk based testing is deployed, every requirement must be rated for likelihood of failure and the impact of failure.
By analyzing the risk of a failure occuring with a specific component or feature and also the impact of failure if that component or feature failed in a real-life situation, project resources can be more efficiently allocated to focus on testing what really matters in the limited time available.
A risk based testing (RBT) approach can help save time and reduce costs on your testing project. Risk based testing enables the test manager to make an informed choice when allocating test resources on a project.

Selecting a test strategy for regression testing

Regression testing is selective retesting of the system; executed with an objective to ensure the bug fixes work and those bug fixes have not caused any un-intended effects in the system.
The selection of test cases for regression testing depends on:
- Requires knowledge on the bug fixes and how it affect the system.
- Includes the area of frequent defects.
- Includes the area which has undergone many/recent code changes.
- Includes the area which is highly visible to the users.
- Includes the core features of the product which are mandatory requirements of the customer.
Selection of test cases for regression testing depends more on the criticality of bug fixes than the criticality of the defect itself.
Do not focus on the test cases that are bound to fail and those test cases which has no or less relevance to the bug fixes. Select more positive test cases than negative test cases for final regression test cycle as this may create some confusion and unexpected heat. It is also recommended that the regular test cycles before regression testing should have right mix of both positive and negative test cases. Negative test cases are those test cases which are introduced newly with intent to break the system.

Good approach is to plan and act for regression testing from the beginning of project before the test cycles. One of the ideas is to classify the test cases into various Priorities based on importance and customer usage.

For an Effective Regression Testing :
- Create a regression test plan identifying focus areas, strategy, test entry and exit criteria. It can also outline Testing Prerequisites, Responsibilities, etc.
- Create test cases.
- Defect tracking.