What is data driven automation testing? What are data driven automation frameworks and data driven scripts?

What is Data Driven Automation Testing

- Data driven automation testing or DDT as the term itself explains is a type of testing that involves tests which are exclusively driven by data. 

- A set of conditions is directly fed to the application as input and the output obtained is verified.

- The process which involves soft coded test environment controls and settings is also verifiable. 

- Input is supplied in the form of a row of the table by the tester in its utter simplest form. 

- This table consists of the values corresponding to the partition or boundary input spaces.

- According to the control methodology of data driven testing the test configuration is read from the data base. 

- This is one king of software testing for which you have so many methodologies to implement it. 

- All the methodologies call for different efforts for creation and subsequent maintenance and hence these methodologies are known to co -exist.

- The basic benefit one can get from the data driven testing is the ease with which the additional inputs can be added to the table on the discovery of the new partitions in it or addition to the SUT or system under test or software product. 

- One aspect of the data driven testing that makes it quite popular in the field of test automation is the cost. 

- Though being quite cheap when it comes to automated testing, the data driven testing can soar high on the cost if carried out manually. 

Methodology of Data Driven Testing

- Data driven testing can be considered to be a testing involving the creation of test scripts to be executed along with the data sets corresponding to them in a frame work. 

- A re-usable test logic is offered by the testing frame work that helps a lot in the reduction of the maintenance part of the software system or application and brings about improvement in the test coverage. 

- For an application, there can be one or more than one central data sources where all this stuff such as input and result data values can be stored. 

- In data driven testing the actual organization and format can be made specific to implementation. 

- Both input and output verification values are included in the test data.

- The facility of harvesting data from a running system has been provided via a ‘sniffer’ or a purpose – built custom tool for the automation environments which are quite advance or we can say mature. 

- The play back of the harvested data is performed by the frame work of data driven testing thus resulting in a automated regression testing tool that is quite powerful. 

- There are certain tasks that are to be carried out during the data driven testing and are coded in the test scripts:

1. Navigation through the program
2. Performing read operations on the data sources
3. Logging of the test status
4. Logging of the test information and so on.

- Anything that carries a potential to vary or to change variably (for example, end points, locations, test data, environment and so on.) are segregated from the test scripts or logic. 

- These elements are then placed in what is called an external asset. 

- This external asset comprises a test data set or configuration. 

- The logic which the test scripts execute is operated by the data values. 

What is keyword driven testing framework and methodology?

Keyword driven testing is one of the testing methodologies which is gaining popularity in the area of the test automation. Well, in this article, we are going to discuss regarding the frame work and methodology of the keyword driven testing. 

What is Keyword Driven Testing?

- The key word driven testing methodology is actually dependent on frame work. 

- It depends on such a frame work that calls for the development of the key words and data tables.

- But there is one condition which is that it should be independent of the test automation tool which is to be used for their execution. 

- Another condition is that it should also be independent of the test script code which is used to drive the application under test or AUT as well as the data. 

- In this testing methodology, the functionality of the application under test or AUT need to be documented in a table. 

- Also, the functionality needs to be stated as in step by step instructions for each and every test. 

- This then actually becomes a frame work approach for creating automated regression tests. 

- The frame work approach thus obtained proves to be successful in achieving many benefits of the automated testing when compared to the traditional methodology for capture and playback of the custom scripts. 

- Whenever automated testing is required, two sets of requirements come in to existence namely:

1. Business Requirements: This category included the following requirements:

a) Maintainability: The maintenance of the tests should be easy as well as there should be a minimum need for updating scripts with changes made to the application and newer versions.

b) Selective testing: Less requirement of in depth testing. Also it should be easy to select test scenarios for a much targeted regression.

c) Recovery: It should be possible for the test suite to resume testing from the point where it failed or crashed at a later point of time. The need for re- testing the functionality should be eliminated.

d) Scalability: For gaining the time efficiency the execution of the tests from different machines at the same time should be possible.

e) Inter environment portability: All the environments must support the test scripts without making nay changes to them.

f)   Usability: Minimal amount of training must be required by employees to learn to execute the automated suite. There should be no manual intervention in the execution of the scripts.

g) Reporting: The status of each test script must be known easily.

2. Technical requirements: This category includes:

a) Data abstraction: There should complete isolation of the test data for the execution of the scripts from the actual script code.

b) Separation of concerns: Concerns at each layer must be addressed by the distinct layers of the architecture.

c)  Portability

Methodology of the keyword driven testing lies in two different phases namely:

1. Planning phase and
2. The implementation phase

The planning phase involves the analyzation of the application and determination of operations and objects to be used in the business processes. The requirement of customized keywords is determined for the following purpose:

1. Additional functionality
2. Business level clarity
3. Maximizing efficiency
4. Maintainability

The second phase i.e., the implementation phase operates around a collection of objects or references usually known as repository which saves the info for unique identification of the objects. This phase also involves the development and documentation of the keywords of the business level in function libraries. Development of customized functions is required for the creation of the functional libraries. This is all done for the application under test or AUT.

The methodology of the keyword driven testing calls for more advance planning and time investment.

What is QTP? Which environments are supported by QTP?

QTP or quick test professional is another very useful testing tool from HP (Hewett Packard) like others such as winrunner, test director and so on. 

Well, the quick test profession has been designed especially for testing methodologies that are quite rigorous in nature such as functional testing and regression testing. 

What is Quick Test Professional?

- Quick test professional is a sole creation of HP software division unlike the other softwares that were actually developed by mercury interactive and hired by HP. 

- Although the code for this testing suite was written by mercury interactive itself.

- The most stable release of the quick test professional till now has been the 11.0 version. 

- Quick test professional can only be operated on Microsoft windows operating system. 

- The quick test professional or QTP is categorized under the category of test automation tools since it facilitates the automation of the functional and regression tests. 

- The quick test professional forms a part of the HP’s quality center tool suite and can be very well used in the enterprise quality assurance. 

- The license of this testing suite is proprietary. 

-The following aspects are most supportable by quick test professional:

1. Key word interfaces
2. Scripting interfaces

- The quick test professional like the other testing suites by HP sports a quite good graphical user interface. 

- The quick test professional makes use of the VBScript i.e., the visual basic scripting edition scripting language for making the specifications of a test procedure. 

- Apart from this, the visual basic scripting language is used for the manipulation of the objects as well as controls of the software system or application that is currently under test. 

- The quick test professional carries out the regression testing and functional testing by means of a user interface which may be either a web interface or some native graphical user interface. 

- The working principle of the quick test professional is that is works through the identification of the several GUI objects present in the graphical user interface or some web page. 

- It can also perform the following functions:

1. Performing desired operations which may consist of user events like mouse click or generating some input from key board.
2. Capturing object properties such as handler ID, object name and so on.

- The visual basic scripting language is used by the quick test professional only for accomplishing short and simple tasks.

- For performing complex and big tasks, it is required that you modify the VBScript that lies underneath the code.

What environments are supported by QTP?

Regarding the environments, quick test professional is known to support most of the environments since it is quite advance as compared to its old counterpart- the winrunnner. 

There are so many releases of quick test professional as mentioned below:

1. 5.5- first release in the year of 2001
2. 6.0 – released in 2002
3. 6.5 – released in 2003
4. 7.0 – this version was never released.
5. 8.0 – released in 2004
6. 8.2 – released in 2005
7. 9.0 – released in 2006
8. 9.1 – released in 2007
9. 9.2 – released in 2007
10. 9.5 – released in 2008
11. 10.0 – released in 2009
12. 11.0 – released in 2011

Different versions of quick test professional are known to support different number of environments based up on its capabilities and features.

However, the latest quick test professional version 11.0 is known to support the following:

1. Web
2. Java core
3. Java advanced
4. .net
5. WPF
6. SAP
7. Oracle
8. Siebel
9. Delphi
10. People soft
11. Stingray 1
12. Power builder
13. Flex
14. Terminal emulator
15. Mainframe terminal emulators
16. Windows mobile
17. Web services
18. Visual age Small talk
19. Silverlight

How does agile teams work?

Agile teams work together as a team but include roles filled by specific individuals.
- First is the product owner, who is responsible for the product vision and for prioritizing features the team will work on.
- Next is the customer, who is the person paying for the project or purchasing the software once it is available.
- Users, developers, and managers are other roles on an agile project.

How does agile team work?

- Agile teams work in short, time-boxed iterations that deliver a working product by the end of each iteration.
- The features developed in these iterations are selected based on the priority to the business.
- This ensures that the most important features are developed first.
- User stories are a common way for agile teams to express user needs.
- Agile teams understand that a plan can rapidly become out of date. Because of this, they adapt their plans as appropriate.

What kind of planning are used for agile teams?

Agile teams use three levels of planning:
- Release planning : The release plan looks ahead for the duration of the release - typically, three to six months.
- Iteration planning : The iteration plan looks ahead only the duration of one iteration - typically, two to four weeks.
- Daily planning : A daily plan is the result of team member commitments made to each other in a daily stand-up meeting.

During release planning, the whole team identifies a way of meeting the conditions of satisfaction for the release, which includes scope, schedule, and resources. To achieve this, the product owner may need to relax one or more of her conditions of satisfaction.
A similar process occurs during iteration planning, when the conditions of satisfaction are the new features that will be implemented and the high level test cases that demonstrate the features were implemented correctly.

What skills needed to be a good software test automator?

If you have studied the history of the test automation process you might have observed that there have been so many failures which have now become the lessons for the entire testing community. Because of the lessons learnt it was decided, that the scripts that will have to be developed must be reusable. 

In order make most of any testing methodology, it is required that it must be made manageable as well as reusable. 

There are certain things that a tester should keep in mind to be a good tester:

1. Test automation is not a sideline but rather a full time effort.
2. Test frame work and test design should be treated as separate individual entities an not as the same one.
3. The test frame work should be independent of the software system or application that is under testing.
4. It should be easy to expand, perpetuate and maintain the test frame work.
5. The test design as well as the test strategy should be independent of the frame work.
6. The test strategy should be so effective that is should be capable of removing the complexities related to the test frame work from the testers.

Skills needed to be a good software test automator

In this article we have focussed up on the skills that are needed to be a good software test automator. Some other skills are:

1. Good logic for programming code of the scripts.
2. The automator must have good analytical skills.
3. The automator must have adequate knowledge about the testing tools that are to be used in the automation process.
4. He/ she should have the habit if thinking out of the box.
5. Critical in nature
6. A good automator can think well from the user’s point of view.
7. A good automator keeps an eye on the details of the test automation process.
8. The knowledge about domain is a must.
9. Good judgement skills.
10. Good code writing skills.

Qualities of a good Test Automator

- A good test automator is the one who implements a test strategy that supports development of the intuitive tests and can be executed both via automated tests and manually.

- The test strategy that a good test automator uses will allow the tests to highlight the steps to be performed.

- A good test automator develops a frame work to harness the benefits form the implementation of the key word drive testing scripts and tradition testing scripts. 

- A good automator implements a test automation frame work in such a way that it is completely independent of the software application that is under question. 

- A good automator fully documents and publishes the test automation frame work.

- One thing that should always be kept in mind is that the testers are only testers and not programmers. 

- Good testers and automators are those who have both the testing as well as programming skills. 

- Most of the testers are domain experts only having zero or very little technical skills that are an immediate requirement for the software testing. 

- Many testers also have a habit of splitting the time between testing and development phases and so they do not require to learn a complex scripting language. 

Describe the concept of phase containment?

In this article we have focussed on an important concept namely phase containment.

Process of Phase Containment

- The process of phase containment deals with the removal of the defects and bugs present in a software system or application while it is still under its SDLC or software development life cycle. - The process of phase containment prefers the early removal of the bugs and defects. 

- It is named so because this process is all about containing faults in one specific phase of the software development life cycle before they get enough time to escape out and affect the software development in the successive phases of the software development life cycle. 

"There are two types of error. One type of the errors are the one which were introduced in the preceding phase of software development and now have accumulated in the current phase and the second types of error are the one which have been introduced in the current phase of software development itself. But the former kinds of errors are called defects and not probably errors". 

- The concept of the phase containment is promoted whenever this concept is related to the organization’s profitability and cost.

- But in order to relate the concept with the organization’s cost and profitability, the identification of the errors and defects that escaped from the previous phases of the software development life cycle and found their place in the successive phases of the software development. 

- Another thing that is required is the determination of the average costs of the defects and errors that were caught in the later phases of software development. 

- It becomes difficult to sort out errors and faults once the software product is out in the market as proven by some research. 

Methodologies to gain control of software product

- So many technologies and methodologies have been developed today to gain control over the quality of the software product.

- They are:

1. Static analysis: This activity involves the analyzation of the program code with the purpose of formatting the errors prevailing in the software system and specific coding.
2. Unit testing: This activity involves the developer leveraging his/ her knowledge for breaking the program code.
3. Code reviews: This activity involves taking the steps to ensure the security of the software system or application and better accountability.
4. Code complete criteria: This step involves providing consistent hand off to the development team.

Metrics used in Phase Containment Process

- The phase containment process makes use of the phase containment metrics.

- These phase containment metrics serve the purpose of making sure whether the developers are on the track or this process is on the track i.e., the process whether is working as desired for the company and organization or not.

- Commonly three types of metrics that are used in the process of phase containment namely:

1. Trailing metric: The purpose of this metric is to find out the downstream impact of the process of the phase containment.
2. Adoption metric: This phase containment metric is intended for making sure that whether or not the software systems developers are adhering the to standards of the phase containment process.
3. Effectiveness metric: This type of phase containment matrix is used to make sure that the phase containment process is working out well or not and how the developers are maintaining it.

This process of phase containment is used to make sure that the all the aspects of the quality assurance are incorporated in to all the phases of the software development life cycle process.

What are the main attributes of test automation?

Test automation is nowadays counted among the best software development aids that we have today. It is a great testing aid that seeks to resolve the drudgery of the testers. Test automation has become so common nowadays that every one of us knows quite a lot about it.

What does Test Automation do?

- Test automation actually controls the execution of the test cases by using an automated process. 

- Also, the comparison of the actual and the predicted outcome takes place in the same and "take it easy" manner.  

- One thing that should be noted is that only those manual processes can be automated which have deployed a formalized testing process. 

- In the test automation process, the user actions are recorded and the checkpoints are introduced in to the testing script so that playing it back from a particular point is made easy when you have to perform regression tests. 

- The manual tests that passed can be automated and is mostly used in the testing methodologies like:

1. Data driven testing
2. Regression testing
3. Performance testing and so on.

Attributes of Test Automation

By attributes, we mean the characteristics of the test automation process. It has got quite a lot of positive attributes. So let us what all attributes we can state for test automation:

1. Repeatability:

Automation supports repeated execution of a test case several times again and again. When you need to execute the same test case again and again the test automation comes as a big win for you.

2. Programmability: 

The test cases can be programmed to be executed on a number of different machines whereas the manual tests have to be executed in a sequential way one after the other.

3. Reliability: 

You can rely on test automation for repeat-ability and accuracy.

4. Impact

5. Criticality

6. Maintainability: 

The maintenance is just that you have to update the test automation suites for every new release of the software system or application.

7. Flexibility: 

With test automation it becomes quite easy for you to run automated tests that frequently change according to the regressions at particular time intervals and also the tests can be run in mainstream scenarios.

8. Efficiency: 

The ratio of total cost to the required effort need is reasonable.

9. Portability: 

The automated tests can be executed in different environments unlike manual test cases.

10.Robustness: 

The automated tests are quite effective for the software systems or applications that are quite unstable or change rapidly.

11. Usability: 

The automated tests can be used to a large extent by different types of users.

12. Speed

13. Accuracy

14. Relentlessness: 

The automated test generator as well as the process never gets tired generating and running test cases.

There are some tests like exploratory testing and usability which cannot be automated because of their free style. Testers involved in a testing team support diverse software systems and applications that are quite unrelated to each other. 

- If the projects that demands implementation of a unique testing strategy, then the testers involved in the whole will appear more to be a hindrance rather than being a help. 

- The testers will require tie to get adjusted to the new environment which unfortunately they cannot get and thus lowering the level of their productivity. 

- We cannot waste time on re- engineering the development frame works but we can surely harness test automation power for every new application that we need to develop. 

- The application can be continuously improved since the test automation has got so many positive attributes that can surely benefit the whole testing process. 

What are the advantages and disadvantages of random testing?

Ever heard about random testing? It would not be so shocking if your answer is no since this software testing methodology is rarely used. We have dedicated this entire article regarding the discussion about random testing, it advantages as well as disadvantages. 

Random testing is actually a kind of functional testing. It is used in the conditions when the time taken for writing and running tests is quite long or the problem is too complex and hence it is not possible for every test case to be executed.

Advantages and Disadvantages of Random Testing

- One of the advantage of random testing is that you can rely on the assertions in the program code.

- There is one more advantage of random testing which is that you can make inferences regarding the reliability of the application in production if you have a selection of random tests that have been generated by reference. 

- One of the big disadvantages of the random testing is that you need to know when a test fails as there is no self indication given by the system. 

- To carry out random testing you require an oracle. 

"By oracle we mean you need to throw random inputs at the software system or application code and that too from multiple possible threads. And if no error or fault occurs, then you can make sure that your software system or application is working well."

- Another disadvantage of random testing is that very often you may come across some situations in which you will have two distinct implementations of the same specifications namely:

1. The golden model and
2. The implementation.

- In such cases the test is declared pass if and only if both of the implementations agree to a defined accuracy. 
- When you decide to carry out a random testing you first need to make sure that the tests that you are going to use are sufficiently random and they cover overall functionality of the software system or application.

-  Another disadvantage is that the random testing is not efficient than the directed testing. But the advantage here is that the time needed for generating test cases for random testing is quite less than creating a set of directed tests. 

- Once you have programmed your random test generator, it can work 24 hours a day generating whole lot of new tests. 

- Often a conflict arises in the minds of the testers whether they should choose between random testing or functional testing. 

- Here, it becomes necessary to know about the number of defects a technique can dig out. 

- However, the random testing proves to be useful even in the situations in which many defects are not discovered per time interval since this testing can work without any manual intervention. 

- Usually, the above mentioned testing processes i.e., random testing and functional testing are found together in combination rather than alone. 

- The usage also depends on the software system or application that is under testing. 

- The test cases used in such combinational testing have been termed as directed random tests since the tests cases can be classified on the basis of their randomness or functionality. 

- Out of all the tests, very few are 100 percent random and usually they are not so of the interesting kind. 

- Whenever you have a random test that consists of quite a big number of random elements (that are mutually constrained), then it becomes difficult to avoid thrashing which also accounts as a disadvantage. 

- Certain languages have been developed for defining the test cases for random testing like E and Vera etc. 

What is meant by data driven automation?

Test automation is perhaps one of the best testing methodologies and this testing methodology is exploited usually by one of the either ways as mentioned below:

1. Code driven testing and

2. Graphical user interface testing

In this article we are going to discuss about the data driven automation. Data driven automation is a self justifying term. You can easily make out that data driven automation is one in which the data is obtained from the external files.

About Data Driven Automation

Now what is that which makes it so special with the testers? 

-  In this type of automation, you can easily change the input data without going down.

-  Data driven automation in a way regulates easy maintenance of the scripts.

- Data driven automation is all about how the software system or application performs with multiple sets of the input data. 

- While carrying out data driven automation, it is necessary that you link the data to the test scripts with which the concerned data is being driven. 

- This whole process of linking the data to the test scripts is called “parameterization of the tests”. 

- The data to be fed as input is recorded in a data table. 

- This operation can be done either manually or it can also be automated by using some data driver wizard. 

- Data driven automation is like picking up data from a data table (which has been parameterized) or query data base and writing down these parameters in some other file so that the software can track it back. 

- As we approach a complex functionality whose testing requires more test cases, the work becomes quite laborious as the testing becomes more and more complicated. 

- For multiple sets of test cases as well as data you can execute one common test in order to figure out which data will pass the test and which will not. 

- The automation script is created for just one input and is used for the rest of the similar inputs. - Data driven automation today has become critical and strategic necessity. 

- In the past, the software systems and applications were quite simple but today we have explosive web deployments and software systems and applications that require rigorous software testing. 

Now the question arises, “how to test such software systems and applications?” there are two answers to this:

1. Either we increase the number of people employed in manual testing or
2. We migrate to some greater level of test automation.

Data driven automation is often abbreviated to DDA. As the size and the complexity of the functionality grows, it becomes more and more laborious to test the functionality since the number of test cases also grows. 

In most of the tools like Winrunner, the data cannot be obtained from excel or notepad and so therefore it has to be taken from the data driven tables. 

What are different tools used for smoke testing?

The success of a software testing methodology depends a lot on the quality and type of tools that are being used to carry out that particular test. 

In this article we have discussed about the tools that are used for smoke testing. One can find so many tools around that are available for automating the smoke tests. 

Functions performed by these tools

Using these tools a lot of functions can be performed few of which have been mentioned below:

1. Using these smoke testing tools one can easily and quickly write the test cases and scripts as well as automate them and guess what? These tools do not require any programming.
2. With these smoke testing tools, the validation of all the builds can be carried out easily before any changes are introduced in to them.
3. Nothing comes as handy as these smoke testing tools when one needs to stabilize the whole build process.
4. The available tools make it easy the verification of a build to determine its readiness for further full scale testing.
5. The smoke testing tools help in conducting lightening fast tests determining whether or not the major functionalities are intact and still working as expected.
6. The tools help in reducing the drudgery of the testers as well as the developers by helping in the early detection of the problems and errors.
7. These tools help a great deal in cutting down the overall development costs by significant margins providing the team with sufficient time period for developing the software system or application.

Tools for Smoke Testing

- So many tools are available using which you can perfect your tests like advanced web recorder, object recorder, editor and so on. 

- The tools provide so many actions to be performed for editing the tests if it is required to make any changes in the software system or application. 

- Such an approach to testing i.e., using tools allows flexibility between the test and testers and eliminates the need of creating new tests. 

- There are tools called “work flow designers” that come with an intuitive graphical representation and using which you can easily create as well as manage the work flows of high level of the entire suite of your smoke tests. 

- There are testing tools available that help you introduce check points in the application so that at any point your software application functionality can be further verified. 

- There is one tool available for testing methodologies that is perhaps the most sought after testing tool. 

- This tool lets you convert a test script in to an executable code and then run it on multiple systems without installing testing application on each and every machine. 

- This feature of this tool facilitates an automation that never existed before. 

- Any of the test scripts can be run and with any inputs operating from any data base or application software and almost in every kind of environment. 

- Some reporting and log tools have been introduced which possess advance reporting features like:

1. Time lines
2. Report customization tools that are quite easy to use.
3. Report dash board
4. Audit trail capability (this feature provides you with a complete data base of all the events that took place in the automated testing environment.)
5. Visual logs

- With a tool having all such features a tester can easily analyze the reports with visual aids and fix them quickly.

- These tools provide the testers with a more complete view of the smoke test runs. 

- Though smoke testing can be carried out manually also it is advisable to use automated test tools which initiate tests by a similar process that generates the build itself.