What are Test Case Documents and what is the general format of test cases?

The test cases will have a generic format as below:
- Test Case ID : The test case id must be unique across the application.
- Test case description : The test case description should be very brief.
- Test Prerequisite : The test pre-requisite clearly describes what should be present in the system, before the test executes.
- Test Inputs : The test input is nothing but the test data that is prepared to be fed to the system.
- Test Steps : The test steps are the step-by-step instructions on how to carry out the test.
- Expected Results : The expected results are the ones that say what the system must give as output or how the system must react based on the test steps.
- Actual results : The actual results are the ones that say outputs of the action for the given inputs or how the system reacts for the given inputs.
- Pass/Fail : If the expected and actual results are same then test id Pass otherwise Fail.

The test cases are classified into positive and negative test cases.Positive test cases are designed to prove that the system accepts the valid inputs and then process them correctly. Suitable techniques to design the positive test cases are specification derived tests. The negative test cases are designed to prove that the system rejects invalid inputs and does not process them. Suitable techniques to design the negative test cases are error guessing, boundary value analysis, internal boundary value testing and state transition testing. The test cases details must be very clearly specified, so that a new person can go through the test cases step by step and is able to execute it.
In an online shopping application, at the user interface level, the client request the web server to display the product details by giving email id and username. The web server processes the request and will give the response. For this application, we design the unit, integration and system test cases.

What comprises Test Ware Development : Test Plan - Acceptance Test Plan (ATP)

The client at their place performs the acceptance testing. It will be very similar to the system test performed by the software development unit. Since the client is the one who decides the format and testing methods as part of acceptance testing, there is no specific clue on the way they will carry out the testing. But it will not differ much from the system testing. Assume that all the rules, which are applicable to the system test, can be implemented to acceptance testing also.

Since this is just one level of testing done by the client for the overall product, it may include test cases including the unit and integration test level details.

Test Plan Outline

- BACKGROUND: This item summarizes the functions of the application system and the tests to be performed.
- INTRODUCTION
- ASSUMPTIONS: Indicates any anticipated assumptions which will be made while testing the application.
- TEST ITEMS: List each of the items(programs) to be tested.
- FEATURES TO BE TESTED: List each of the features(functions or requirements) which will be tested or demonstrated by the test.
- FEATURES NOT TO BE TESTED: explicitly lists each feature, function, or requirement which would not be tested and why not.
- APPROACH: Describe the data flows and test philosophy. This section also mentions all the approaches which will be followed at the various stages of the test execution.
- ITEM PASS/FAIL CRITERIA: Itemized list of expected output and tolerances.
- SUSPENSION/RESUMPTION CRITERIA: Must the test run from start to finish? Under what circumstances it may be resumed in the middle? Establish check-points in long tests.
- TEST DELIVERABLES: What, besides software, will be delivered? It includes test report and test software.
- TESTING TASKS: It includes functional and administrative tasks.
- ENVIRONMENTAL NEEDS: It includes security clearance, office space and equipment and hardware/software requirements.
- RESPONSIBILITIES: It includes what are the tasks in section 10? What does the user do?
- STAFFING AND TRAINING
- SCHEDULE
- RESOURCES
- RISKS AND CONTINGENCIES
- APPROVALS

The schedule details of the various test pass such as unit tests, integration tests, system tests should be clearly mentioned along with estimated efforts.

What comprises Test Ware Development : Test Plan - System test Plan

The system test plan is the overall plan carrying out the system test level activities. In the system test, apart from testing the functional aspects of the system, there are some special testing activities carried out, such as stress testing etc. The following are the sections present in system test plan:

- What is to be tested?
This section defines the scope of system testing, very specific to the project. Normally, the system testing is based on the requirements. All the requirements are to be verified in the scope of the system testing. This covers the functionality of the product. Apart from this, what special testing is performed are also stated here.

- Functional groups and the sequence
The requirements can be grouped in terms of the functionality. Based on this, there may be priorities also among the functional groups. For example, in a banking application, anything related to customer accounts can be grouped into one area, anything related to inter-branch transactions may be grouped into one area etc.

- Special Testing Methods
This covers the different special tests like load/volume testing, stress testing, interoperability testing etc. These testing are to be done based on the nature of the product and it is not mandatory that every one of these special tests must be performed for every product.

Apart from above sections, the following sections are also addressed:
- System Testing Tools
- Priority of functional groups
- Naming Convention for test cases
- Status reporting mechanism
- Regression test approach
- ETVX criteria
- Build/Refresh Criteria

What comprises Test Ware Development : Test Plan - Integration Test Plan

The integration test plan is the overall plan for carrying out the activities in the integration test level, which contains the following sections:

- What is to be tested?
This section clearly specifies the kinds of interfaces fall under the scope of testing internal, external interfaces, with request and response is to be explained. This need not go deep in terms of technical; details but the general approach how the interfaces are triggered.

- Sequence of Integration
When there are multiple modules present in an application, the sequence in which they are to be integrated will be specified in this section. In this, the dependencies between the modules play a vital role. If a unit B has to be executed, it may need the data that is fed by unit A and unit X. In this case, the units A and X have to be integrated and then using the data, the unit B has to be tested. This has to be stated to the whole set of units in the program. Given this correctly, the testing activities will lead to the product, slowly building the product, unit by unit and then integrating them.

- List of modules and interface functions
There may be N number of units in the application but the units that are going to communicate with each other, alone are tested in this phase. If the units are designed in such a way that they are manually independent, then the interfaces do not come into picture.This is almost impossible in any system, as the units have to communicate to other units, in order to get different types of functionalities executed. In this section, we need to list the units and for what purpose it talks to the others needs to be mentioned. This will not go into technical aspects, but at a higher level, this has to be explained in plain English.

Apart from above sections, it also includes:
- Integration Testing Tools
- Priority of Program Interfaces
- Naming Convention for test cases
- Status reporting mechanism
- Regression test approach
- ETVX criteria
- Build/Refresh criteria.

What comprises Test Ware Development : Test Plan - Unit Test Plan

The test strategy identifies multiple test levels, which are going to be performed for the project. Activities at each level must be planned well in advance and it has to be formally documented. Based on the individual plans only, the individual test levels are carried out.
The plans are to be prepared by experienced people only. In all test plans, the (ETVX) Entry-Task-Validation-Exit criteria are to be mentioned. Entry means the entry point to that phase. Task is the activity that is performed. Validation is the way in which the progress and correctness and compliance are verified for that phase. Exit tells the completion criteria of that phase, after the validation is done.

ETVX is a modeling technique for developing worldly and atomic level models. It is a task based model where the details of each task are explicitly defined in a specification table against each phase i.e. Entry, Exit, Task, Feedback In, Feedback Out, and measures.
There are two type of cells, unit cells and implementation cells. The implementation cells are basically unit cells containing the further tasks. A purpose is also stated and the viewer of the model may also be defined e.g. to management or customer.

Types of Test Plan

Unit Test Plan (UTP)
The unit test plan is the overall plan to carry out the unit test activities. The lead tester prepares it and it will be distributed to the individual tester, which contains the following sections:

- What is to be tested?
The unit test plan must clearly specify the scope of unit testing. In this, normally the basic input/output of the units along with their basic functionality will be tested. In this case, mostly the input units will be tested for the format, alignment, accuracy and the totals.

- Sequence of testing
The sequence of test activities that are to be carried out in this phase are to be listed in this section. This includes, whether to execute positive test cases first or negative test cases first, to execute test cases based on the priority, to execute test cases based on test groups etc.

- Basic functionality of units
The independent functionalities of the units are tested which excludes any communication between the unit and other units. The interface part is out of scope of this test level.

Apart from these, the following sections are also addressed:
- Unit testing tools
- Priority of program units
- Naming convention for test cases
- Status reporting mechanism
- Regression test approach
- ETVX criteria

What comprises Test Ware Development : Test Strategy Continued...

Test ware development is the key role of the testing team. Test ware comprises of:

Test Strategy

Before starting any testing activities, the team lead will have to think a lot and arrive at a strategy.The following areas are addressed in the test strategy document:
- Test Groups: From the list of requirements, we can identify related areas, whose functionality is similar. These areas are the test groups. We need to identify the test groups based on the functionality aspect.

- Test Priorities: Among test cases, we need to establish priorities. While testing software projects, certain test cases will be treated as the most important ones and if they fail, the product cannot be released. Some other test cases may be treated like cosmetic and if they fail, we can release the product without much compromise on the functionality. This priority levels must be clearly stated.

- Test Status Collections and Reporting:
When test cases are executed, the test leader and the project manager must know where exactly we stand in terms of testing activities. To know where we stand, the inputs from the individual testers must come to the test leader. This will include, what test cases are executed, how long it took, how many test cases passed and how many failed etc. Also, how often we collect the status is to be clearly mentioned.

- Test Records Maintenance: When the test cases are executed, we need to keep track of the execution details like when it is executed, who did it, how long it took, what is the result etc. This data must be available to the test leader and the project manager, along with all the team members, in a central location.This may be stored in a specific directory in a central server and the document must say clearly about the locations and the directories.

- Requirements Traceability Matrix: Ideally, each software developed must satisfy the set of requirements completely. So, right from design, each requirement must be addressed in every single document in the software process. The documents include the HLD, LLD, source codes, unit test cases, integration test cases and the system test cases. In this matrix, the rows will have the requirements. For every document, there will be a separate column. So, in every cell, we need to state what section in HLD addressed in every single document, all the individual cells must have valid section ids or names filled in.

- Test Summary: The senior management may like to have a test summary on a weekly or monthly basis. If the project is very critical, they may need it on a daily basis also. It addresses what kind of test summary reports will be produced for the senior management along with the frequency.

What comprises Test Ware Development : Test Strategy

Test ware development is the key role of the testing team. Test ware comprises of:

Test Strategy

Before starting any testing activities, the team lead will have to think a lot and arrive at a strategy. This will describe the approach, which is to be adopted for carrying out test activities including the planning activities. This is a formal document and the very first document regarding the testing area and is prepared at a very early stage in software development life cycle. This document must provide generic test approach as well as specific details regarding the project.

The following areas are addressed in the test strategy document:
- Test Levels: The test strategy must talk about what are the test levels that will be carried out for that particular project. Unit, Integration and system testing will be carried out in all projects. But many times, integration and system testing may be combined.

- Roles and Responsibilities: The roles and responsibilities of test leader, individual testers, project manager are to be clearly defined at a project level. the review and approval mechanism must be stated here for test plans and other test documents. Also, we have to state who reviews the test cases, test records and who approved them.The documents may go through a series of reviews or multiple approvals and all these are mentioned in this section.

- Testing tools: Any testing tools which are to be used in different test levels must be clearly identified. This includes justifications for the tools being used in that particular level also.

- Risks and Mitigation: Any risks that will affect the testing process must be listed along with the mitigation. By documenting the risks in this document, we can anticipate the occurrence of it well ahead of time and then we can proactively prevent it from occurring. Sample risks are dependency of completion of coding, which is done by sub-contractors, capability of testing tools etc.

- Regression Test Approach: When a particular problem is identified, the programs will be debugged and the fix will be done to the program. To make sure that the fix works, the program will be tested again for that criteria. Regression test will make sure that one fix does not create some other problems in that program or in any other interface. So, a set of related test cases may have to be repeated again, to make sure that nothing else is affected by a particular fix. how this is going to be carried out is elaborated in this section.