How can we reduce the risk by using a test plan?

A test plan apart from just serving the purpose of a guide for the software testing life cycle also plays a great role in the assessment of the risks associated with the development of that particular software system or application and also provides the alternatives or builds for overcoming those factors which are the root cause of the risk.

The preparation of the test plan must be based on all the risks identified during the process of risk assessment. This article is focussed up on the relation that exists between the risk and the test plan and how a risk based test plan can help reducing the overall risk.

How a risk based test plan reduce the overall risk?

- Such a test plan is effective only when a test oriented risk analysis is carried out during the software system or application development life cycle.

- With the advancing and sophisticated technology the complexity of the software systems and applications is also increasing and so does it lengthens the testing process which is quite exhaustive.

- The software testing methodologies therefore should be quite selective and should also be chosen keeping in mid the time limit and the budget of the project.

- It is often stressed by many testers that the testing should be based on the risk which is not possible until and unless the testers are well equipped with the knowledge of the risk.

- It is a wise decision of concentrating the testing more on the area which is at higher risk compared to other parts of the software system or application.

- Many researchers have made a rigorous research on the subject of risk based testing and have stated that a software system or application requires apart from the understanding of software testing, the knowledge of risk and its analysis also.

Types of Risks
There are two types of risks namely:

1. Forward Risks
These are the risks associated with the operation of the system and one of the major cause of the software failure.

2. Backward Risks
These are the risks associated with the development issues like those mentioned below:
(a) Inappropriate design
(b) Casual programming
(c) Incorrect specifications
(d) Inadequate management

Risk is thought of as a function of two components as described below:

1. The probability of occurrence of an undesirable event that is defined and
2. The degree of the severity of the consequences if the event of the system failure does occur.

In most of the cases, the consequences following the failure of the software system are related to the purpose of the software and therefore reducing the risk is not always an option.

Thus, this leads us to the conclusion that the risk must be reduced by reducing the probability of the failure of the software system which in turn can be achieved only with an efficient test plan.

Now you must be wondering can this actually happen? Yes of course.

- The software testing reduces the risks by digging out many of the bugs.

- But, the actual risk reduction is based up on the implementation of the corrected code and functions.

- The failure of software system is not systematic and hence cannot be predicted by just checking out the history.

- So for the cases like these, only the estimation of the potential consequences of the failure works via the risk based testing that acts as a single factor analysis in
such cases.

- For some it may seem like a trivial process but, it is not so and calls for a thorough examination of the system failure.

- The single factor analysis though does not produce a very correct estimate; it does help the testers in focussing their testing on the code that is buggier than the other units.

What is the entry and exit criterion for system testing?

Scientifically, the term system testing is looked up as the testing of both the components of the system i.e., software and hardware. It has been categorized as one of the software testing techniques under the category of black box testing, and so this eliminates the need of any knowledge regarding the internal structure and design of the source code.

The system testing is carried out on an integrated, complete and finished software system or artefact because after the completion of the system only, it can be checked for its ability to cooperate according to the specified conditions and requirements.

This article states the entry and exit criteria for a software system or application to undergo system testing.

About System Testing

- The system testing is all aimed at the detection of all the discrepancies, defects and constraints lurking in the software system or application.

- System testing takes up the job for dealing with the inconsistencies and flaws that are present in the system software which is in turn constituted up of integrated software and hardware components.

- System testing is carried out in concern for the detection of defects within present within the assemblages i.e., inter- assemblages representing the software system or application as a whole entity.

- You don’t have to worry if you already have your software incorporated successfully with the appropriate hardware system, it can be still served as input to the test.

- The software system itself is an integrated part of any other software or hardware system and has dealt with the system integration testing successfully can also be considered as an input for the system testing.

- Being very non similar to the integration testing and unit testing, the system testing takes the whole software system or application as one unit or module for testing.

- We can say that in a way the system testing is a means to explore the functionality of a software system.

- For carrying out system testing there are two options: either you carry it out before you software system or application is assembled or after the system has been finished and completed.

- While carrying out the system testing, you should always make sure you are strictly following the defined systematic procedures.

- Only specifically designed test cases should be used for performing system testing.

- System testing holds the responsibility for dealing with the basic and important contexts namely:
1. Functional requirement specification (FRS) and
2. System requirement specification (SRS).

- System testing does not only deal with the testing of the design of the software system but, also its features as expected by the customers and behaviour.

Entry criteria for System Testing

1. All the units must have passed the unit testing and that too with successful grades.

2. All the modules or units must have successfully passed the integration testing with good grades.

3. Their should be a resemblance between the surrounding environment and the production environment.

4. The following steps should have been followed during the system testing:

- Creation of the system test plan.
- Creation of the test cases.
- Creation of the scripts for building the environment.

Exit Criteria for System Testing

1. All the requirements specifications mentioned in the documentation have been met.

2. The issues have the severity level of 1 and 2 have been resolved.

3. The application is now working as per the expectations of the users.

4. ll the issues related to the critical defects must have been fixed and closed.

5. All the necessary documentations (like quick start guide, final publications draft etc) have been reviewed.

Explain empirical vs defined & prescriptive process?

To make any process successful it has to be controlled in a way that it achieves its predefined goals or we can say it should be guided along the path of success. This fact holds good for any type of process in this world and so does for the processes in the field of software engineering.

In the field of software engineering, two approaches have been identified for keeping a control over the development and other related processes namely:


1. The empirical process control method and
2. The defined process control method.


In this article the above mentioned two approaches have been compared so that you get a better understanding of both the programs. So let us see how these two approaches control the processes.

The Empirical Process Control Method

- The empirical process control model was defined to exercise or control the process via following some frequent adaptations as well as frequent inspections.

- These inspections and the adaptations both are meant for the generation of the unpredictable and unrepeatable outcomes or results and can be thought of as being imperfectly designed.

- Since for the past many years the various software development methodologies have been known to be controlled by the latter approach i.e., the defined process control method.

- But we all know that every time a certain same output or outcome cannot be expected from the software development processes.

- Therefore, most of the agile software development methodologies are controlled by the empirical process control model and the most famous example being that of the “scrum” agile software development methodology.

- The term empirical process control model itself justifies as the term empirical means the information acquired by the means of experimentation and observation.

- Here the information is achieved by means of inspection and adaptations which serve as a means for the observation and experimentation.

- The process of the empirical control is constituted of a continuous cycle of adaptation of the process as per the requirements and inspection of the process for correct working.

- The empirical control process model has three pillars as we can make out from the definition of the process control model without which it cannot be called as the empirical process control method:
1. Transparency
2. Inspection and
3. Adaptation.

- The first pillar i.e., transparency indicates that the outcomes of the affects of the empirical process control model and the aspects affecting the outcomes should be visible to the programmers and developers who are responsible for controlling the whole process.

- The second pillar i.e., the inspection indicates that all the aspects of the control process should be monitored quite frequently to enable the fast and early detection of the unacceptable variances.

- The third pillar i.e., the adaptation indicates the adjustment of one or more aspects as required of the control process if the software system or application being processed is observed to lie outside the acceptable limits implying that the result will also be unacceptable.

- The defined process control model approach is adopted when the underlying mechanisms of the software system or application are well understood by the programmers and the developers.

The Defined Process Control Model

- The defined process control model can be thought of as a theoretical approach.

- When a well defined set of inputs is given, it is obvious that the same outcomes will be generated every time the program executes.

- With the well understood technologies and stable requirements, one can very well predict a whole software project.

- Even nowadays the empirical process control model holds as the essence of the agile software development processes.

- Empirical process holds good for the complex development processes which encounter difficulty in the production of repetitive outcomes.

What is meant by adaptive and predictive planning?

Learning is an important aspect of any development process be it of any field. Learning can be classified in to many types, but in this article only two types have been discussed namely:
- Adaptive learning and
- Predictive learning

Adaptive Planning or Learning

- Adaptive learning is considered to be a computer driven educational method in which the computers are the interactive teaching devices rather than having human teachers do the teaching.

- The presentation of the educational material is adapted by the computers according to the weaknesses of the students which are determined by the responses of the students to the questions asked by the computer.

- Here the whole learning process is motivated by the idea of using electronic education for incorporating the interactive values to a student that would have been provided by an actual human tutor or teacher.

- This technology encompasses various aspects taken from the fields like education, psychology, computer science and so on.

- Adaptive learning was evolved because it is not possible to achieve tailored learning with non adaptive and traditional approaches.

- The learner is transformed from the passive receptor of the information to the collaborator of the educational processes.

- The primary application of the adaptive learning is in basically the following two fields which have been designed as both web applications and the desk top applications:
1. Education and
2. Business training.

Adaptive learning is also known by several other names like:
1. Computer based learning
2. Adaptive educational hyper- media
3. Intelligent tutoring systems
4. Adaptive instructions
5. Computer based pedagogical agents

Models or Components of Adaptive learning

The whole process of adaptive learning has been divided in to some separate models or components as mentioned below:

1. Student Model
- This model keeps a track of the student and learns about him.
- This model makes use of algorithms that have been researched for over 20 years.
- The CAT (computer adaptive testing) makes use of the simplest means for the determination of the students’ skill.
- Nowadays, the students’ models make use of richer algorithms for providing a more extensive diagnosis of the weaknesses of the students.
- This it does by linking the questions and the concepts and using ability levels to define the strengths and weaknesses.

2. Instructional model
- This model is actually responsible for conveying the information.
- It makes use of the best technological methods for educational purposes like multimedia presentations along with the expert teacher advice.
- When the students make mistakes, the model provides them with useful hints.
- These hints can be question specific.

3. Instructional environment
This provides an interface for the system and human interaction.

4. Expert model
- This model is responsible for teaching the students using the stored information which is to be taught.
- This may include solutions for question sets, lessons and tutorials.
- Some very sophisticate expert models may use expert methodologies for the illustration of the solution of the questions.
- In some of the adaptive learning systems, the qualities of an expert model may be acquired by an instructional model.

Predictive Planning or Learning

- The predictive learning involves machine learning i.e., to say an agent has to build a model of its own environment type by carrying out various actions in several circumstances.

- The knowledge of the effects of the actions tried out is used for turning the models in to planning operators.

- This is done so that the agent is able to act purposefully in that environment.

- We can say that the predictive learning is all about learning with a minimum of the mental structure that exists already.

- Some say that this kind of learning has been inspired by the Piaget’s account of the construction of knowledge of the world by interacting with it.

What is the entry and exit criterion for user acceptance testing?

The acceptance is quite an important choice for the clients or the customers. It plays a very important role when it comes to the addressal of the issues related to the acceptance of the software system or application by the client or the customer.

Like any other testing, the acceptance testing also has some of its pre defined entry and exit criteria that a software system or application needs to satisfy before it can undergo the acceptance testing process.

This article is focussed up on those entry and exit criteria only but first let us take a glimpse of what acceptance testing really is so that it becomes easy for us to understand the entry and exit criterion for the acceptance testing.

About User acceptance Testing

- For the field of software engineering, this kind of testing has been termed as the user acceptance testing since it is carried out in order to obtain confirmation from the user or the client that the developed software system or application meets its specified and agreed up on requirements and specifications.

- This confirmation is provided by the SME or the subject matter expert who is the owner of the software system or application under testing after carrying out several trials and reviews.

- The user acceptance is therefore one of the final software testing methodologies that is carried out before handing over the software system or application to its owner.

- The user acceptance testing is preferably carried out via the users of that software system or application which are in the contact of the client or mentioned in the users requirements specification document.

- As many as the formal tests required are created by the test developer or designer based on the different levels of the severity of the errors and flaws.

- Typically, for an ideal acceptance testing the test designer should handle the creation of the formal system and integration test cases for the same software system or application.

- The user acceptance testing serves as a means of final verification of the well functioning of the software system or application by creating the real world conditions for the its usage as it will used by the customer and required business function under process.

- The system needs to perform as intended because then only it can be subjected to its reasonable extrapolation in the process of product at the same level of the stability and reliability.

- Unlike other software testing methodologies, the test cases of the user acceptance testing do not serve to identify the simple problems, errors and show stopper defects (system crashes, failures and hangs etc).

- It is so because all such defects are fixed by the testers and developers in the earlier stages of the software testing life cycle.

- There is another reason for this testing to be performed which is to give confidence and assurance to the client or the customer that the system will perform well in the production phase.

- Some contractual or legal requirements are also signed at the end of the acceptance testing.

Entry Criterion for User Acceptance Testing

1. The transition meeting of the integration testing must be signed off.
2. The functional requirements and the business requirements have been met and verified in the integration testing.
3. The test cases for the user acceptance testing are ready to be executed.
4. The test environment for the UAT should be ready.
5. Required access of the resources for testing should be granted.
6. All the critical bugs have been previously addressed.
7. The reports of the previous testing should be handed over to the client.

Exit Criterion for User Acceptance Testing
1. No defects are found.
2. Defects with the medium priority are found.
3. There is no hindrance in the business process.
4. The UAT meeting is signed off.

What is the entry and exit criterion for unit testing?

First let us brief up ourselves with what the concepts of the unit testing and then we shall proceed further in discussing about the entry and exit criterion for unit testing.

About Unit Testing

- Unit testing is a self explanatory term.

- It involves the testing of the smallest units or modules of the software system or application in order to determine whether or not they are working properly and in the desired manner.

- Since this testing methodology is employed for testing the smallest individual unit of the software system or application, hence it got the name “unit testing”.

- If the program or application has been developed using procedural programming oriented language then the entire module is treated as units in the unit testing or else in general, the individual procedures and functions are the units.

- For the applications and softwares in which objected oriented programming has been implemented, the entire application’s interface.

- The test cases for the unit testing are created by the white box testers since they have an in depth knowledge of the software system or application.

- Sine the units are tested in isolation with the other units and independently, hence the test cases created for unit testing are also independent in nature.

Unit Test Case

- A unit test case can be assisted in unit testing by using substitutes like those mentioned below:
1. Mock objects
2. Methods stubs
3. Test harnesses
4. Fakes and so on.

- The test cases for the unit testing are both written and executed by the software developers.

- This ensures that the source code is at par with the design and architecture of the software system or application and works as accordingly as specified.

- The test cases can be implemented in two ways i.e., manually or through the use of automation tools as a part of build automation.

Entry Criterion for Unit Testing

- The functional specifications requirements of the software system or application under test need to be frozen.

- The technical design specifications need to complete and approved. They should have been released.

- The system specifications document also should be complete, approved and released.

- No issue should be pending in the query issue register regarding the requirements under the unit testing.

What else unit testing involves?

- Apart from just testing the modules, the unit testing involves the verification of the requirement specifications in regard with the finalized design and functional specifications requirements.

- For harnessing the maximum benefits of the unit testing, all of the requirements specifications should be signed off.

- Unit testing is said to be complete when the source code is complete and that too according to the specifications.

- All the design specifications should meet the design standards and the entire unused variable, code files should have been removed from them.

- The code documentation including the commenting and AOT documentation must also be complete and approved.

- To put it simple the code should be in such a state so as to be released to the customers or the clients.

Exit Criterion for Unit Testing

- All the units have successfully passed the unit test.

- The code is complete according to the requirements.

- No elements and features are missing.

- The possible errors and warnings have been resolved. This criterion is optional since most of the times it’s not possible to remove all of the errors.

- Code optimization for all the three tiers has been done.

- There is no error in the performance optimization.

- All label files have been created.

- All unused files have been removed.

- Redundant code is removed.

What is the entry and exit criterion for production verification testing?

Some entry and exit criteria have been predefined for all the software testing methodologies and in the same way some criterion have been defined for the production verification testing also.

For a software system or application to process from one phase of the software testing life cycle to the other one, it has to satisfy all the exit criteria of the previous software testing methodology and entry criteria of the software testing methodology that it is about to undergo.

So this article states the entry and exit criteria for the production verification testing and before that we have given a discussion about the production verification testing.

About Production Verification Testing

- Production verification is also an important part of the software testing life cycle like all the other software testing methodologies and is carried out after the successful completion of the user acceptance testing phase.

- The entire production verification testing deals with the simulation of the cutover of the whole production process as close to the true value as possible.

- Production verification testing also serves as an opportunity for the conduction of a full real like full dress rehearsal of the changes in the business requirements if any.

- The production verification is not like the parallel testing since there is a difference of the goal.

- The goal of the production verification testing is to verify that the data is being processed properly by the software system or application rather than comparing the results of the data handling of the new software system software or application with the current one as in the case of parallel testing.

- This software testing methodology has been designed for the verification of the following aspects:
a) Proper running of any batch processes against the actual data values of the production process.
b) Business process flows
c) Proper functioning of the data entry functions

Here we list some of the entry and exit criteria of the production verification testing:

Entry Criteria for Production Verification Testing

- For the production verification testing to commence it is important that the documentation of the previous testings is produced and the issues and faults that were discovered then are fixed and closed.

- If there is a final opportunity for the determination of whether or not the software system or application is ready for the release, it is the production verification testing.

- The completion of the User acceptance testing is over and has been approved by all the involved parties.

- The documentation of the known defects is ready.

- The documentation of the migration package has been completed, reviewed and approved by all the parties and without fail by the production systems manager.

- For the production verification testing, the testers need to remove or uninstall the software system or application from the testing environment and re-install it again as it will be installed in the case of the production implementation.

Exit Criteria for Production Verification Testing

- The processing of the migration package is complete.

- Apart from just the simulation of the actual production cut over, the real business activities are also simulated during the phase of the production verification testing.

- Since it is the full rehearsal of the production phase and business activities, it should serve the purpose of the identification of the unexpected changes or anomalies presiding in the existing processes as a result of the production of the new software system or application which is currently under the test.

- The installation testing has been performed and its documentation is ready and signed off.

- The documentation of the mock testing has been approved and reviewed.

- A record of the system changes has been prepared and approved.