How much testing is relevant in an agile scenario?

Testing is as relevant in an agile scenario if not more than a traditional software development scenario. Testing is the headlight of the agile project showing where the project is standing now and the direction it is headed. Testing provides the required and relevant information to the teams to take informed and precise decisions. The testers in agile frameworks get involved in much more than finding software bugs, anything that can bug the potential user is a issue for them but testers do not make the final call, it is the entire team that discusses over it and takes a decision over a potential issues.
A firm belief of agile practitioners is that any testing approach does not assure quality, it's the team that does or does not do it, so there is a heavy emphasis on the skill and attitude of the people involved.
Agile testing is not a game of gotcha, it's about finding ways to set goals rather than focus on mistakes.
Among the agile methodologies, XP i.e. Extreme Programming components are:
- Test First Programming
- Pair Programming
- Short iterations and release.
- Re-factoring
- User Stories
- Acceptance Testing

Test-First Programming

- Developers write unit tests before coding. It has been noted that this kind of approach motivates the coding, speeds coding and also improves design results in better designs.
- It supports a practice called re-factoring.
- Agile practitioners prefer tests to text for describing system behavior. Tests are more precise than human language and they are also a lot more likely to be updated when the design changes. How many times have you seen design documents that no longer accurately described the current workings of the software? Out-of-date design documents look pretty much like up-to-date documents. Out-of-date tests fall.
- Many open source tools like xUnit have been developed to support this methodology.

Methods of black box testing - Boundary Value Analysis

Black Box Testing is testing without knowledge of the internal workings of the item being tested. There are four black box testing methods. Graph based testing method and Equivalence Partitioning have already been discussed.

Boundary Value Analysis
Boundary Value Analysis determines the effectiveness of test cases for a given scenario. Boundary Value Analysis (BVA) is a test Functional Testing technique where the extreme boundary values are chosen. Boundary values include maximum, minimum, just inside/outside boundaries, typical values, and error values. Boundary value analysis complements and can be used in conjunction with equivalence partitioning.
For boundary value analysis, the following guidelines should be used:
- For input ranges bounded by a and b, test cases should include values a and b and just above and just below a and b respectively.
- If an input condition specifies a number of values, test cases should be developed to exercise the minimum and maximum numbers and values just above and below these limits.

Advantages of Boundary Value Analysis
- Robustness Testing – Boundary Value Analysis plus values that go beyond the limits.
- Min – 1, Min, Min +1, Nom, Max -1, Max, Max +1.
- Forces attention to exception handling.

Limitations of Boundary Value Analysis
Boundary value testing is efficient only for variables of fixed values i.e boundary.

Statement Coverage Testing in White Box Testing

The purpose of white box testing is to make sure that functionality is proper and the information on the code coverage. It tests the internal structure of the software. It is also known as structural testing, glass testing and clear box testing.

Statement coverage is the most basic form of code coverage. A statement is covered if it is executed. Note that a statement does not necessarily correspond to a line of code. Multiple statements on a single line can confuse issues - the reporting if nothing else.

- In this type of testing the code is executed in such a manner that every statement of the application is executed at least once.
- It helps in assuring that all the statements execute without any side effect.
- Statement coverage criteria call for having adequate number of test cases for the program to ensure execution of every statement at least once.
- In spite of achieving 100% statement coverage, there is every likelihood of having many undetected bugs.
- A coverage report indicating 100% statement coverage will mislead the manager to feel happy with a false temptation of terminating further testing which can lead to release a defective code into mass production.
- We can not view 100% statement coverage sufficient to build a reasonable amount of confidence on the perfect behavior of the application.
- Since 100% statement coverage tends to become expensive, the developers chose a better testing technique called branch coverage.

Testing Approaches: Functional Testing versus Structural Testing and Mutation testing

In functional testing, functionality of the module is tested and structure is not considered. Test cases based on specifications and internals of modules are not considered. This type of testing is also known as black box testing. The objective here is to ensure that `correct results'' are obtained when `good inputs'' are applied to the various parts of the design, and when `bad inputs'' are applied the design operates in a predictable manner. Functional testing can therefore be considered as concentrating on checking that the data paths operate correctly.

Structural testing is used to test the implementation of the program. Structural testing, also known as white box or open box testing, is normally applied to sequential HDL code and concentrates on checking that all executable statements within each module. Structural testing involves :
- Statement coverage : This ensures that each and every statement is tested. Software tools called profilers are used to carry this statement coverage.
- Branch coverage : Each and every condition is taken, and inputs are given in such a way that each branch is executed at least once.
- Path Coverage : To test loops, this is required. For example the statement
for(i=o;i<=100;i++) is executed 101 times.

Mutation testing is required to ensure that the software does not fail. It is a good debugging mechanism. The program is modified slightly to obtain mutants of the program. Different mutants are tested with the same test cases. If the mutant fails, and the actual program works correctly, confidence is gained in the program and test cases are considered good. To produce mutants, mutation operators are defined.
- Constant replacement
- Variable replacement
- Arithmetic operator replacement
- Relational operator replacement
- Goto label replacement