What is Orthogonal Array testing? - An explanation

There are a number of black box testing techniques; the one that is being discussed in this post is the orthogonal array testing. This technique provides a systematic as well as statistical strategy for testing software. The number of inputs to the system in this technique is small but large enough for testing each and every possible input as in exhaustive testing. This technique has proved to be quite helpful in discovering errors that show a faulty logic in the software systems. Orthogonal arrays can be applied in various testing types such as following:
- User interface or UI testing
- System testing
- Regression testing
- Configuration testing
- Performance testing and so on.

The permutations for the factor levels that consist of only one treatment have to be chosen in an uncorrelated way, so that every single treatment gives you a piece of information that is different from the others. The advantage of organizing testing in such a way is that a minimum number of experiments are required for gathering same information. Orthogonality is a property exhibited by the orthogonal vectors. Properties exhibited by the orthogonal vectors are mentioned below:
- Information conveyed by each vector is different from the information by other vectors in the sequence. That is, as we mentioned information conveyed by each treatment is unique to it. This is important otherwise there will be redundancy.
- It is easy to separate these signals on a linear addition.
- All the vectors are statistically independent of each other which means that there is no correlation between them.
- When these individual components are added linearly, the result is an arithmetic sum.

Suppose a system is having 3 parameters each of which has 3 values. We require total 27 test cases for testing all of the parameter combinations which is quite time consuming. So we use an orthogonal array for selecting a combination subset from these combinations. As a result of using the orthogonal array testing, maximization of the test coverage area is possible. At the same time this minimizes the number of test cases that have to be considered for testing. The pair that is selected is assumed to have the maximum number of defects. The technique works based up on this assumption. These many combinations are sufficient for catching the fault. The interaction of the input parameters between themselves has also to be considered. The array is said to be orthogonal because the occurrence of all pair wise combinations is once. The results of the test cases are assessed as follows:
- Single mode faults
- Double mode faults
- Multimode faults

Below mentioned are the major benefits of using this technique:
- The testing cycle time is reduced.
- The analysis process gets simpler
- Test cases are balanced which means that the defect isolation and performance assessments are straightforward.
- Saves up on costs when compared to the pair-wise testing. The coverage to all the defects can only be provided by testing all the combinations that are possible. But our schedule and budget often do not permit this. Therefore we are forced to select only a sample of the combinations from the test domain. Orthogonal array testing is a means for generating samples that provide high coverage for the validation of test domain effectively. This has made the technique particularly useful in the integration testing and testing of the configurable options. Software testers often face a dilemma during selection of the test cases. The quality of the software cannot be tested but only the defects can be detected. And the exhaustive testing is difficult even in the small systems. 

Potential quality problems when using Open Source components in your software product - Reduced level of control

Most software applications use components or services from teams that are external to them - these could be teams within the organization, or these could be from outside the organization. Many applications even use components that are built by Open source processes. These components can be very useful, but one needs to be mindful of the impact of using external components, especially if you are in the role of a project manager or a program manager. If a critical component is late or is of bad quality, it have a critical impact on the schedule of your project.
I remember a case where we had a component that was important, and we had just taken delivery of a new version of the component (it was available on the web page of the Open source project and the notes mentioned that this fixed a defect that was critical for us). This was incorporated in a development version of the product, which was sent out to beta testers after a quick application of some test cases. Unfortunately, in this case, Murphy's Law hit us bad. The quality of the external component was bad, it destroyed some of the test data that customers were using. Now, although we would warn the customers that they should not use their live data in Beta testing, the Beta customers had a certain expectation in terms of the quality of Beta software that they were using, and the release was less than that. We got chewed out by senior management that one time, although there was little that we could do.
This gets very tricky when you use Open Source software, since there is a reduced level of control in using such components. However, in most cases, you will be forced to use atleast some amount of open source software. Many components are available as Open Source, which are otherwise available for very expensive sums. Software companies do the trade-off between the cost of a controlled component from outside and the problems that may happen when you use an Open Source component, and invariably, better use can be made of the money, and you will end up using the Open Source software. This is even more so when the same component is being used by other groups within the company, as well as by other product in the same space.
One compromise that can be reached is about deciding which version of the Open Source component to be used, and deciding the amount of testing that needs to be done before and after integration of the component.   So, depending on the phrase of the project, if there is no real dependency such as Beta testing or other such delivery, you could take the latest version of the Open Source software and test the hell out of it, and then decide whether to proceed with the component. However, if you are still in a state where you have some dependencies, then it would be wiser to take an earlier version of the component, which has been well tested and is good for use. If there is a bug fix that is there in the latest version of the software, then you would need to make a decision about the software to use.
Overall, the stand that you need to take is no easy call - there are a lot of benefits of using an Open source component, but you need to review the attendant benefits and problems, and then take a call.

What are the qualities of a good tester?

The quality of the software testing depends up on the skills of the tester since the whole software testing process revolves on the instructions of the tester. Even a single wrong decision can cause the whole testing process go wrong.

A tester makes a decision based on his testing skills and capabilities. So a tester has to be very careful while taking any decisions regarding the software testing process.

ATTRIBUTES OF A GOOD TESTER

1. GOOD TECHNICAL & PRACTICAL KNOWLEDGE
- It is not necessary that a good tester might be an experienced one.
- A fresher having good technical and practical knowledge about the software testing methodologies and strategies can also perform effective software testing. - Though one learns from experience, it is often seen that there are many testers who even after having much experience are not able to carry out the software testing process efficiently in a well mannered way.
- They often keep bumping into the problems.

2. KNOWLEDGE ABOUT TESTING PROCESS
- A good tester keeps the overview of the whole testing process in mind while drafting out the test plan.
- He/ she include all the aspects to be tested in the test plan.

3. FAULT FINDING
- He insists on finding the faults earlier in the software system or application since discovering the bugs and errors in the earlier phases of the development helps a great deal.
- It takes less effort to fix a bug or an error in the initial stage of the development when compared to the efforts required in the later stages of development.

4. KNOWLEDGE ABOUT COMPUTER PROGRAMMING
- Testers know computer programming well.
- They are capable of maintaining good credibility with the fellow programmers.
- They have senior computer programmers working over them.

5. KNOWLEDGE ABOUT APPLICATION SOFTWARE
- They know the application software under testing very well.
- He/ she should be able to exploit the application software like an end user.
- Thinking like a user open ups new ways in which the application can be used.

6. SMARTNESS
- Good testers are really smart people and know how to exploit the application.
- Their intelligence is at par with the programmers.

7. NOTICING THINGS
- Good testers tend to notice those minute and little things that do not come to the notice of the other people.
- They keep a sharp eye on the behavior of the program.
- Even a little unusual happening can be an indication of a bug or error.

8. HANDLING CHAOS
- Obviously when a team comprising of people at different designations there will be a lot of chaos.
- Handling such chaos depends on the individual character.
- A good tester is expected to have good tolerance capacity or we can say he/ she should have patience.

9. HANDLING CRITICISM
- Testers have to suffer a lot of criticism from the programmers.
- A tester having good sense of humor can easily get through such a trap.

10. TENACITY
- Another social skill called tenacity is found in good testers.
- Tenacity is the ability of an individual to compromise with the opinions of the others.
- A good tester knows how to socially smart and is a good diplomat.

11. KEEPING TRACK
- A good tester keeps a track of his testing.
- He/ she keeps his observations organized.

12. SKEPTICAL NATURE
- A good tester is a bit skeptical.
- He/ she do not takes anything for granted and questions every thing suspicious for bugs and errors.

13. TOUGH
- A good tester is tough since he/ she has to suffer a lot of criticism and questions of the programmers.

14. SEARCHING NEW WAYS
- They are always searching for new ways of testing an application.
- They are technology hungry. Testing is quite a laborious task.

15. HONESTY
- Finally a good tester is honest with his duty and doesn’t follow any corruption.

What is the difference between functional and non-functional testing ?

Functional testing is a kind of testing whose cases of tests are based on the specific conditions of the component of the software that is under test. Inputs are fed to the function and the resulting output is examined. In this type of testing, the internal structure of the software source code or the program is not known. Functional testing can be considered as a kind of black box testing. Functional testing checks a program against specifications of the program and design of the documents. This kind of testing involves basically 5 steps.
The first being the identification of the functions expected to be performed by the software.
The second step involves the creation of data for input according to the specifications of the functions.
In the third step the output after processing of data is determined according to the specifications of the functions.
The fourth step deals with the execution of the cases of the tests.
The actual output is compared with the expected output in the last and final fifth step of the functional testing.

Functional testing forms a crucial part in software processes. What all does functional testing do ?
Functional testing is used to check performance.
Functional testing tests the graphical user interface (GUI) of the program.
Functional testing requires checking the whole program from one end to the other end.
This helps to ensure performance and quality.
It helps in developing programs faster.
Errors also result during functional testing which should be corrected with supporting unit tests.
Later they should be checked once more and verified with a functional test.
Functionality tests have got a great benefit. Many correct tests means the program is correct and working.
Functional testing increases the probability of automatically checking the program.
Functional testing is used to validate the whole input output conversion process.

To make functional testing more effective a process called “continuous integration relentless testing” should be implied. Functional testing is done according to business requirements as stated by client’s specifications. The various aspects of functional testing are listed below:
1. Smoke testing
2. Sanity testing
3. Unit testing
4. Top down testing
5. Bottom up testing
6. Interface testing
7. Usability testing
8. Regression testing
9. Alpha and beta testing
10. System testing
11. Pre user acceptance testing
12. White box testing
13. Black box testing
14. User acceptance testing
15. Globalization testing
16. Localization testing
In contrast to functional testing, the non- functional testing of a software application tests it for non-functional features and requirements. Sometimes there is an overlap of scope of various non functional tests amidst many non- functional requirements. Therefore, in such cases the names of the non functional tests are interchanged to avoid confusion. There are various aspects of non functional testing which have been listed below:
1. Documentation testing
2. Compliance testing
3. Baseline testing
4. Compatibility testing
5. Recovery testing
6. Localization testing
7. Performance testing
8. Endurance testing
9. Load testing
10. Internalization testing
11. Volume testing
12. Stress testing
13. Usability testing
14. Resilience testing
15. Scalability testing and
16. Security testing
In addition to the above mentioned aspects, the non- functionality testing also covers the following additional features:
1. Load testing
2. ergonomics testing
3. migration testing
4. penetration testing
5. data conversion testing
6. operational testing
7. network security testing
8. system security testing
9. installation testing
There’s a basic difference in both types of testing which is that the functional testing tells one what software does whereas non functional testing shows one how well the software executes.