What is a Test Harness?

Test harness is a rarely heard concept in the field of software testing. 

What is a Test Harness?

- A test harness is known by other name “automated test framework”. 

- A test harness can be defined simply as a collection of test data and softwares that have been configured in order to test a unit of a software program.

- The program is run under various conditions. 

- Under each condition the behavior as well as working of the program is observed and the outcomes are reported. 

Phases of Test Harness Process

The whole process of test harness is completed in two individual phases:

1.        Test execution engine and

2.        Test script repository

How is Test Harness carried out?

- Test harness process cannot be carried out without the automation of the tests. 

- The automated tests than can themselves call the concerned functions with the required parameters and execute them.

- The actual results are then compared to the expected results. 

- The process of test harness acts as a hook for the already developed code which is highly testable and can be checked out using an automation framework or test harness.  

What should a Test Harness do?

The below mentioned are the things that a test harness must do:

1. Run specific tests in order to allow optimization.

2. Orchestrate an environment during the run time.

3. Analyze the results

Certain objectives have been defined for a test harness:

1. Automation of the whole testing process.

2. Execution of the specified test cases.

3. Report the outcomes.

Advantages of a Test Harness

Test harness is quite an advantageous process and some of its advantages have been stated below:

1.  It increases productivity by automating the whole test process.

2.  It increases the probability of the occurrence of the regression testing.

3.  It increases the quality of the components of the software system and application.

4.  It helps ensure the similarity of the subsequent as well as the previous test cases.

5.  It helps in running the tests at any time whenever the testing staff is not available.

6. It effectively executes the test scripts including the conditions that are otherwise unexecutable since they are difficult to be simulated.

How Test Harness facilitate testing at Integration level?

- Test harness has also been developed to facilitate the testing at the integration level i.e., the integration testing. 

- The test stubs tested by a test harness are components of an application that is currently under development and during testing they are replaced by the working components of the developed application in the top down design. 

- Test harness serves as an external aid to the software system or application under testing by simulating functionalities and features which are not present in the immediate test environment.

- In other words, the test harness helps in providing substitutes in case any functionality is found to be missing during the test. 

- The same test harness when kept outside from the source code of the software system or application, it can be used for multiple projects again and again. 

- It forms a deliverable part of the project. 

- The test harness is not provided with any knowledge of test cases, test suites and test reports since it has the capability of simulating the functionality.

- The information on these aspects is fed to the test harness via associated automated testing tools and a testing framework. 

- A test harness can also have a graphical user interface for the ease of operation, logging and scripting of the test cases.

- A new test harness is written for each run time and language since it becomes very much difficult to write a test harness which will work across all the languages and run times.  

- Test harness generates the application which is required to run the tests by providing the required code, files, and test cases and so on.

Automation testing - Can sometimes not work inspite of all efforts, or take much more time ..

Doing an automation project can be sometimes fairly easy, it is just a simple matter of creating an objectives document, creating an execution plan, budgeting for the resources, making sure that the people committed to doing this are on the job, and that the plan is doing well. In a majority of cases, the project runs as per the schedule and the automation project gets completed.
However, this need not happen every time. Like any other project, there can be cases where the project runs into a lot of trouble or where there is something new being attempted, and the going gets rough. I have a personal experience of the same.
We had a project where we running into a shortage of resources for a testing case, and the case seemed to one where the testing was of a repetitive nature; hence, the project seemed to be one such where if we could setup an automation project, it would meet all the mission objectives. So, we got a person who was an automation expert, worked out all the objectives, worked out an execution plan along with a schedule, and started work.
We soon ran into problems that would disturb projects where the work is of a sort that was not done before. The project involved a situation where a large DVD was needed to be tested regularly. Each DVD that was manufactured had a small customization for individual clients, with the changes between the individual DVD's being very small. However, each DVD was needed to be tested thoroughly since each DVD was sent to individual clients, and it was not acceptable that any of the clients received a non-functioning DVD. The DVD needed to be put on a fresh machine, virus tested, the customization as well as the functionality of the application needed to be tested. The attempt was to create an image where many of the testing situations (including a virus check) were inserted, so that the test cases could be run automatically with a single resource just coordinating the work, and noting any errors if produced.
However, we ran into errors on many of the steps, since there did not seem to be anybody in the company who had done these sort of automation work before, and research on the internet did not seem to work either.

What is Automated Testing Framework (AUT) ?

The organization needs a testing framework that provides a end to end approach for implementing testing activities.It is necessary to provide a flexible, scalable and cost effective ‘on-demand’ testing approach for all non-functional and functional testing needs and setup the overall testing process. It increases the long-term success of a project.Also, testing activities are easier to manage.

The automated tools require the test engineer to understand a scripting language (VB Script, Java Script, etc.) to write their automated test cases. These tools usually have the ability to create the scripts using record and playback, but this does not always write the most efficient scripting code and is not as re-usable and maintainable.
Automated Testing Framework is a collection of libraries and utilities designed to ease unattended application testing in the hands of developers and end users of a specific piece of software. An AUT is :
- set of abstract concepts,
- processes,
- procedures and,
- environment in which automated tests will be designed, created and implemented.

Automated Testing Framework will achieve the fastest time to value by implementing automation on a foundation of proven processes, technology and knowledge and to reduce total automation costs, increase the quality, focus on quality.