Verification Strategies - Reviews - Technical Reviews and Requirement Review

Technical reviews confirm that product conforms to specifications, adheres to regulations, standards, guidelines, plans, changes are properly implemented, changes affect only those system areas identified by the change specification.
The main objectives of technical reviews are as follows:
- Ensure that the software confirms to the organization standards.
- Ensure that any changes in the development procedures are implemented as per the organization pre-defined standards.

In technical reviews, the following software products are reviewed:
- Software requirements specification.
- Software design description.
- Software test documentation.
- Software user documentation.
- Installation procedure.
- Release notes.
The participants of the review play the roles of decision-maker, review leader, recorder, technical staff.

Requirement Review : A process or meeting during which the requirements for a system, hardware item or software item are presented to project personnel, managers, users, customers, or other interested parties for comment or approval. Types include system requirements review, software requirements review. Product management leads the requirement review. Members from every affected department participates in the review.

Input Criteria: Software requirement specification is the essential document for the review. A checklist can be used for the review.
Exit Criteria: It includes the filled and completed checklist with the reviewers comments and suggestions and the re-verification whether they are incorporated in the documents.

What is Verification and different strategies of Verification - Reviews and Management Reviews

Verification is the process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase.
Verification process helps in detecting defects early and preventing their leakage downstream. Thus, the higher cost of later detection and re-work is eliminated.

The different strategies in Verification are:

Review

A process or meeting during which a work product or set of work products is presented to project personnel, managers, users, customers, or other interested parties for comment or approval. The main goal of reviews is to find defects. reviews are a good compliment for testing to help assure quality. Few purposes of SQA reviews are assuring the quality of deliverable's before the project moves to the next stage. Once a deliverable has been reviewed, revised as required, and approved, it can be used as a basis for the next stage in the life cycle.
Types of reviews include management reviews, technical reviews, inspections, walk-through and audit.

- Management Reviews :
These reviews are performed by those directly responsible for the system in order to monitor progress, determine status of plans and schedules, confirm requirements and their system allocation. The main objectives of management reviews are:
* Validate from a management perspective that the project is making progress according to the project plan.
* Ensure that the deliverables are ready for management attention.
* Resolve issues that require management attention.
* Identify any project bottlenecks.
* Keeping the project in control.
Support decisions are made during such reviews include corrective actions i.e. changes in the allocation of resources or changes to the scope of the project.
In management reviews, audit reports, contingency plans, installation plans, risk management plans, software q/a are reviewed.
The participants of the review play the roles of decision maker, review laeder, recorder, management staff and technical staff.

Different testing activities in Programming/Construction and Operations and Maintenance Phase

The main testing points in this phase are:
- Check the code for consistency with design
The areas to check include modular structure, module interfaces, data structures, functions, algorithms and I/O handling.

- Perform the testing process in an organized and systematic manner with test runs dated, annotated and saved.
A plan or schedule can be used as a checklist to help the programmer organize testing efforts. If errors are found and changes are made to the program, all tests involving the erroneous segment must be re-run and recorded.

- Asks some challenges for assistance
Some independent party, other than the programmer of the specific part of the code should analyze the development product at each phase. The programmer should explain the product to the party who will then question the logic and search for errors with a checklist to guide the search. This is needed to locate errors the programmer has overlooked.

- Use available tools
The programmer should be familiar with various compilers and interpreters available on the system for the implementation language being used because they differ in their error analysis and code generation capabilities.

- Apply stress to the program
Testing should exercise and stress the program structure, the data structures, the internal functions and the externally visible functions or functionality. Both valid and invalid data should be included in the test set.

- Test one at a time
Pieces of code, individual modules and small collections of modules should be exercised separately before they are integrated into the total program, one by one. Errors are easier to isolate when the number of potential interactions should be kept small. Instrumentation-insertion of the some code into the program solely to measure various program characteristics can be useful here.

- Measure testing coverage/ When should testing stop?
If errors are still found every time the program is executed, testing should continue. Because errors tend to cluster, modules appearing particularly error-prone require special scrutiny. The metrics used to measure testing thoroughness include statement testing, branch testing and path testing. Statement testing is the coverage metric most frequently used as it is relatively simple to implement.

Testing Activities in Operations and Maintenance Phase

Correctness, modifications and extensions are bound to occur even for small programs and testing is required every time there is a change. Testing during maintenance is termed regression testing. The test set, test plan, and the test results for the original program should exist. Modifications must be made to accommodate the program changes, and then all portions of the program affected by the modifications must be re-tested. After regression testing is complete, the program and test documentation must be updated to reflect the changes.

Different Testing activities in Design phase

The design document aids in programming, communication, and error analysis and test data generation. The requirements statement and the design document should together give the problem and the organization of the solution i.e.what the program will do and how it will be done.
The design document should contain:
- Principal data structures.
- Functions, algorithms, heuristics or special techniques used for processing.
- The program organization, how it will be modularized and categorized into external and internal interfaces.
- Any additional information.

The testing activities should consist of:
- Analysis of design to check its completeness and consistency
The total process should be analysed to determine that no steps or special cases have been overlooked. Internal interfaces, I/O handling and data structures should specially be checked for inconsistencies.

- Analysis of design to check whether it satisfies the requirements
Check whether both requirements and design documents contain the same form, format, units used for input and output and also that all the functions listed in the requirement document have been included in the design document. Selected test data which is generated during the requirement analysis phase should be manually simulated to determine whether the design will yield the expected values.

- Generation of test data based on the design
The tests generated should cover the structure as well as the internal functions of the design like the data structures, algorithm, functions, heuristics and general program structure etc. Standard extreme and special values should be included and expected output should be recorded in the test data.

- Re-examination and refinement of the test data set generated at the requirements analysis phase.

The first two steps should also be performed by some colleague and not only by the designer or developer.

Software Localization - some details in terms of how the process work - Part 4

The previous post in this series (Part 3 of localization testing) talked about the time period when to start the process of doing the internationalization of the software, including getting the list of strings. In this post, I will talk more about the processes involved when you are trying to internationalize the code, more in terms of trying to ensuring maximum efficiency and cost reduction.
Most companies do not have the requisite talent for localizing the software, in terms of people who know the various languages in which the software needs to be localized, and for many, hiring and keeping people who know all this can be a task that they would rather outsource, and there are a number of companies that handle all this translation and localization work. However, before we get into this, there are more details that need to be explained.
Consider the case when your software needs to be internationalized. What is the process ? In the previous posts, we have talked about when to start the process, and how it is technically done, but not more details once the required strings are available for translation.
Well, during the translation process, the following is required:
- Line up resources in each desired language who can convert the various strings and return them back for incorporation
- For accuracy purposes, it is desired that these translations be reviewed through a formal review process. So, first the strings are generated for translation, these are translated, and then another language expert does the review
- Once the review is done, these are then incorporated and built into the software
- The software is now ready for internationalization testing, and there are 2 levels of testing required
- The first is functional testing where it is confirmed that the software works fine in the various desired languages. A language expert is not needed; in most cases, the tester takes the English language version and tests the other language keeping the English one in mind (except when the test cases specify a different need in the language version)
- Next, a language expert is needed to verify that the translations appear properly in the various sections of the application, the translations fit the context, and that there are no grammatical mistakes, etc. It is desirable to have a native language speaker do this level of testing.
-

Testing activities in Requirements Analysis phase

The following test activities should be performed during this stage :
- Invest in analysis at the beginning of the project.
Having a clear, concise and formal statement of the requirements facilitates programming, communication, error analysis and the test data generation. The requirements statement should record the following information and decisions :
+ Program function - what the program must do?
+ The form, format, data types and units for input.
+ The form, format, data types and units for output.
+ How exceptions, errors and deviations are to be handled?
+ For scientific computations, the numerical method or at least the required accuracy of the solution.
+ The hardware and software environment required or assumed.
Deciding the above issues is one of the activities related to testing that should be performed during this stage.

- Start developing the test set at the requirements analysis phase
Data should be generated which can be used to determine whether the requirements have been met. To do this, the input domain should be partitioned into classes of values that the program will treat in a similar manner and for each class a representative element should be included in the test data. In addition, following should also be included in the data set: boundary values, any non-extreme input values that would require special handling. The output domain should be treated similarly. Invalid input requires the same analysis as valid input.

- The correctness, consistency and completeness of the requirements should also be analyzed.
Consider whether the correct problem is being solved, check for conflicts and inconsistencies among the requirements and consider the possibility of missing cases.

Categories of Heuristics of Software Testing

There is a set of characteristics that lead to a testable software.
- Operability: The better it works, more efficiently it can be tested.The system should have few bugs or no bugs that should block the execution of tests and the product should evolve in functional stages.

- Observability: What we see is what we test. It includes a distinct output should be generated for each input, the current and past system states and variables should be visible during testing, all factors affecting the output should be visible, incorrect output should be easily identified, source code should be easily accessible and internal errors should be automatically detected and reported.

- Controllability: The better we control the software, the more the testing process can be automated and optimized. Check that all outputs can be generated and code can be executed through some combination of input. Check that the software and hardware states can be controlled directly by the test engineer. Check that inputs and outputs formats are consistent and structured. Check that the test can be conveniently specified, automated and reproduced.

- Decomposability: By controlling the scope of testing, we can quickly isolate problems and perform effective and efficient testing. The software system should be built from independent modules which can be tested independently.

- Simplicity: The less there is to test, the more quickly we can test it. the points to consider in this regard are functional, structural and code simplicity.

- Stability: The fewer the changes, the fewer are the disruptions to testing. The changes to software should be infrequent, controlled and not invalidating existing tests. The software should be able to recover well from failures.

- Understandability: The more information we will have, the smarter we will test. The testers should be able to understand well the design, changes to the design and the dependencies between internal, external and shared components.

- Suitability: The more we know about the intended use of the software, the better we can organize our testing to find important bugs.

The above heuristics can be used by a software engineer to develop a software configuration which is convenient to test and verify.