Methods of black box testing - Graph based testing methods and Equivalence Partitioning

Black Box Testing is testing without knowledge of the internal workings of the item being tested. There are four black box testing methods:

Graph-based Testing Methods
Basic idea: A "cause" is an input condition, and an "effect" is a specific sequence of computations to be performed. A cause-effect graph is basically a directed graph that describes the logical combinations of causes and their relationship to the effects to be produced.
- Black-box methods based on the nature of the relationships (links) among the program objects (nodes), test cases are designed to traverse the entire graph.
- A graph is created between the objects and the relationships.
- From the graph, each object relationship is identified and test cases written accordingly to discover the errors.

Equivalence Partitioning
This method divides the input domain of a program into classes of data from which test cases can be derived. It reduces the number of test cases. The guidelines of Equivalence Partitioning are :
- If an input condition specifies a range, one valid and two invalid equivalence classes are defined.
- If an input condition requires a specific value, then one valid and two invalid equivalence classes are defined.
- If an input condition specifies a member of a set, then one valid and one invalid equivalence class are defined.
- If an input condition is boolean, then one valid and one invalid equivalence class are defined.

Methods of black box testing - Boundary Value Analysis

Black Box Testing is testing without knowledge of the internal workings of the item being tested. There are four black box testing methods. Graph based testing method and Equivalence Partitioning have already been discussed.

Boundary Value Analysis
Boundary Value Analysis determines the effectiveness of test cases for a given scenario. Boundary Value Analysis (BVA) is a test Functional Testing technique where the extreme boundary values are chosen. Boundary values include maximum, minimum, just inside/outside boundaries, typical values, and error values. Boundary value analysis complements and can be used in conjunction with equivalence partitioning.
For boundary value analysis, the following guidelines should be used:
- For input ranges bounded by a and b, test cases should include values a and b and just above and just below a and b respectively.
- If an input condition specifies a number of values, test cases should be developed to exercise the minimum and maximum numbers and values just above and below these limits.

Advantages of Boundary Value Analysis
- Robustness Testing – Boundary Value Analysis plus values that go beyond the limits.
- Min – 1, Min, Min +1, Nom, Max -1, Max, Max +1.
- Forces attention to exception handling.

Limitations of Boundary Value Analysis
Boundary value testing is efficient only for variables of fixed values i.e boundary.

What are different types of black box testing ?

The base of the Black box testing strategy lies in the selection of appropriate data as per functionality and testing it against the functional specifications in order to check for normal and abnormal behavior of the system. These testing types are again divided in two groups:

Testing in which user plays a role of tester.

- Functional Testing : The testing of the software is done against the functional requirements.
- Load testing : It is the process of subjecting a computer, peripheral, server, network or application to a work level approaching the limits of its specifications.
- Stress Testing : The process of determining the ability of a computer, network, program or device to maintain a certain level of effectiveness under unfavorable conditions.
- Ad-hoc testing : Testing carried out informally; no formal test preparation takes place, no recognized test design technique is used, there are no expectations for results and randomness guides the test execution activity.
- Smoke Testing : It is done in order to check if the application is ready for further major testing and is working properly without failing up to least expected level.
- Recovery Testing : Testing aimed at verifying the system's ability to recover from varying degrees of failure.
- Volume Testing : Huge amount of data is processed through the application in order to check the extreme limitations of the system.
- Usability Testing : This testing is done if User Interface of the application stands an important consideration and needs to be specific for the specific type of user.

User is not required.

- Alpha Testing : Testing of a software product or system conducted at the developer's site by the end user.
- Beta Testing : The pre-testing of hardware or software products with selected typical customers to discover inadequate features or possible product enhancements before it is released to the general public. Testing of a rerelease of a software product conducted by customers.
- User Acceptance Testing : The end users who will be using the applications test the application before ‘accepting’ the application. This type of testing gives the end users the confidence that the application being delivered to them meets their requirements.

Black Box Testing - Techniques, Advantages, Disadvantages

Black-box testing alludes to tests that are conducted at the software interface. Black-box tests are used to demonstrate that input is properly accepted and output is correctly produced, and that the integrity of external information is maintained. There is no knowledge of the test object's internal structure. This method of test design is applicable to all levels of software testing: unit, integration, functional testing, system and acceptance. Black box testing tends to be applied during later stages of testing.
Main focus in black box testing is on functionality of the system as a whole. The term ‘behavioral testing’ is also used for black box testing.
To implement black box testing strategy, the tester should go through the requirements specifications and a user should know how the system should behave in response to the particular action.
Black-box testing attempts to find errors in the categories like incorrect or missing functions, interface errors, errors in data structures or external database access, behavior or performance errors, and initialization and termination errors.

Advantages of Black Box Testing
- Tester can be non-technical.
- Used to verify contradictions in actual system and the specifications.
- Test cases can be designed as soon as the functional specifications are complete

Disadvantages of Black Box Testing
- The test inputs needs to be from large sample space.
- It is difficult to identify all possible inputs in limited testing time. So writing test cases is slow and difficult.
- Chances of having unidentified paths during this testing.

Static and Dynamic Analysis in Testing and their differences.

Static analysis involves going through the code in order to find out any possible defect in the code. Dynamic analysis involves executing the code and analyzing the output.

- Static testing is about prevention.
- In static testing, the software is not actually used.
- It is generally not detailed testing, but checks mainly for the sanity of the code, algorithm, or document. It is primarily syntax checking of the code or and manually reading of the code.
- This testing is used by the developer who writes the code in isolation.
- Out of Verification and Validation, it is the verification portion.
- Static testing methodologies include code reviews, inspection and walkthroughs.
- Dynamic testing is about cure.
- The source code is actually compiled and run.
- It examines the physical response from the system.
- Out of Verification and Validation, it is the validation portion.
- Dynamic testing methodologies include unit testing, integration testing, system testing and acceptance testing.
- Static testing is many times more cost-effective than dynamic testing.
- Static testing achieves 100 statement coverage in a relatively short time while dynamic testing often often achieves less than 50 statement coverage.
- Static testing can be done before compilation while dynamic testing can take place only after compilation and linking.
- Static check is more profitable than the dynamics of the static check because tests are made at the initial stage.

Why is static testing more effective?
Static testing gives you comprehensive diagnostics for your code. It warns you about:
- syntax errors.
- code that will be hard to maintain.
- code that will be hard to test.
- code that does not conform to your coding standards.
- non-portable usage.
- ANSI violations.

Branch Coverage or Decision Coverage in White Box Testing

Code coverage analysis is the process of finding areas of a program not exercised by a set of test cases, creating additional test cases to increase coverage, and determining a quantitative measure of code coverage, which is an indirect measure of quality.
A Decision is a program point at which the control flow has two or more alternative routes. A decision coverage is the percentage if the decision outcomes that have been exercised by a test suite.
Branch coverage testing helps in validating of all the branches in the code and making sure that no branching leads to abnormal behavior of the application. It is a better practice as compared to statement coverage. It goes deeper into the code as compared to statement coverage.
It states whether the the boolean expressions are tested in control structures. It ensures for having adequate number of test cases for every program to ensure execution of every decision or branch at least once.

What are the advantages of branch coverage?
- To validate that all the branches in the code are reached.
- To ensure that no branches lead to any abnormality of the program’s operation.
- It eliminate problems that occur with statement coverage testing.

What are disadvantages of branch coverage?
- There may be other condition that can be used for decision making.
- This metric ignores branches within boolean expressions which occur due to short-circuit operators.

Statement Coverage Testing in White Box Testing

The purpose of white box testing is to make sure that functionality is proper and the information on the code coverage. It tests the internal structure of the software. It is also known as structural testing, glass testing and clear box testing.

Statement coverage is the most basic form of code coverage. A statement is covered if it is executed. Note that a statement does not necessarily correspond to a line of code. Multiple statements on a single line can confuse issues - the reporting if nothing else.

- In this type of testing the code is executed in such a manner that every statement of the application is executed at least once.
- It helps in assuring that all the statements execute without any side effect.
- Statement coverage criteria call for having adequate number of test cases for the program to ensure execution of every statement at least once.
- In spite of achieving 100% statement coverage, there is every likelihood of having many undetected bugs.
- A coverage report indicating 100% statement coverage will mislead the manager to feel happy with a false temptation of terminating further testing which can lead to release a defective code into mass production.
- We can not view 100% statement coverage sufficient to build a reasonable amount of confidence on the perfect behavior of the application.
- Since 100% statement coverage tends to become expensive, the developers chose a better testing technique called branch coverage.