Test Driven Development - The Process

Continuing from the previous post on the basics of TDD (link), this post continues with the topic of Test Driven Development.

What is the process of Test Driven development?
For the test driven development to be implemented on some software artifact, its units should be kept small. By units here we mean a group or class of functions related to each other. Sometimes these units might also be referred to as the modules. These are a couple of the benefits of using small units:
> The debugging effort is reduced – Upon detection of test failures; it becomes easy to track down the faults when you have smaller units.
> Tests are often self – documenting: Readability and understandability is increased by virtue of small tests.

TDD can be converted into ATDD (acceptance test driven development) by mixing TDD with more advanced practices. The criteria that the customer specifies are converted into acceptance tests which are then used for driving the UTDD (unit TDD) process in traditional manner. With this process it is ensured that there is an automated mechanism which can be used by the customers for deciding whether their requirements have been met or not. The ATDD provides the development team with a fixed target i.e., the acceptance tests which keeps them steadily focused on the requirements of the customers. Now let us examine the TDD cycle. It's test driven development cycle consists of the following phases:

1. Adding a test: The beginning of a new feature is marked by writing a test which must fail because of being written before the implementation of the feature. If this test succeeds then either the test is defective or the feature has already been added to the software. Before writing a test, the requirements and specifications must be fully understood by the developer by means of user stories and use case stories. This step puts the focus of the developer on the requirements before he/ she begins writing the code that makes a subtle yet importance difference.
2. Running the tests and checking if they fail: This step does the step of validating the correct working of the test harness. The test itself is also tested in this process ruling out the possibility that the new test will pass always. This results in increase in confidence in software.
3. Allowing the test to pass by writing some code: the code written at this step is not perfect as proved by later tests but it is eventually improved. The code is written only to pass the test.
4. Running the tests: If all the tests pass, the programmer can be sure of the code that the requirements are being met.
5. Refactoring the code: Now the code has to be refactored as required. This also involves placing the code in its logical place and removing any redundant code. It has to be made sure that the function and variable names represent properly their current values. Any misinterpreted constructs should be clarified. After this the code should be re – run to be sure that the refactored code has not changed the other functionalities.
6. Repeat: Another test is taken to test the next functionality. The steps should be kept small. If the new code does not satisfy this test or if it fails, the changes made should be undone instead of excessive debugging. Maintaining revertible checkpoints becomes easy if continuous integration is used. If external libraries are being used, then it is necessary that the size of the increments should be as small as the library itself unless the library is not sufficient or it has bugs. Between each test run, there can be maximum 10 edits. This cycle goes on until all the functionalities have been tested.

What is Test Driven development?

The test driven development (or TDD in short) is a type of software development process that emphasizes on the use of repetitive development cycles. These development cycles are quite short when compared to the cycles followed in other development processes. Initially an automated test case i.e., an initially failing one is written by the developer defining a new function or improvement that is required. Next a code is produced (as small and as efficient as possible) for passing the test. Lastly, this code refactored so as to make it standard compliant. This technique was discovered by Kent Beck in the year of 2003. He stated that this software development technique is an inspiration for confidence and produces simple designs.
There is a similarity between extreme programming’s concepts of test – first programming and this test – driven development process. However, lately TDD has developed a general interest for its own. The concept behind the test driven development is also applied to the debugging and improvement of the legacy code that has been produced using some older methods. There are a number of aspects in which test – driven development can be used. For instance, TDD principles such as YAGNI (you aren’t gonna need it) and KISS (keep it simple stupid) are used in general context too. Since the focus is on producing only the required amount of code necessary to pass the test, the code that is developed is much clear and clean than what is developed by other methods. Another famous principle of TDD is “fake it till you make it”. For achieving some concepts including advanced design (for example, design pattern), the tests that would produce this design have to be written. The code may be simpler than the actual target code but would still pass all the tests.  At first, you might seem to disagree with it but, on the positive side it allows the developer to focus on the important things.

The first thing that should be done is to write the tests even before the functionality that has to be tested. This approach has a number of benefits:
- It helps in ensuring that the application is test ready making it easy on the developers on how to test it from outset. It eliminates the scene of worrying later.
- It makes sure that for every feature there is a test.
- This approach facilitates an early understanding of the product requirements. Thus, ensuring the test code’s effectiveness. Also the product quality can be focused up on continually.

On the other hand, the feature – first code pattern in development organizations may push the developer to the next feature which results in negligence in full testing. Also the compilation for the first test might not even take place because the classes and functions it requires might not exist then. But, even then the first tests are treated as executable specifications.
The test cases are first failed for ensuring that the test is working and is able to catch the fault or error. Once this is established, we can proceed with the implementation of the underlying functionality. This strategy is called the “test driven development mantra”. Also known as the red/green refactor where red stands for failing and green stands for passing. In TDD we repeat the process of adding the failing test cases and then passing them and again refactoring. When at each stage the programmer receives the expected results, his mental code model keeps getting stronger and reinforced. Further this boosts his productivity and instills in him a sense of self - confidence. This is the development style that TDD follows.

What is Orthogonal Array testing? - An explanation

There are a number of black box testing techniques; the one that is being discussed in this post is the orthogonal array testing. This technique provides a systematic as well as statistical strategy for testing software. The number of inputs to the system in this technique is small but large enough for testing each and every possible input as in exhaustive testing. This technique has proved to be quite helpful in discovering errors that show a faulty logic in the software systems. Orthogonal arrays can be applied in various testing types such as following:
- User interface or UI testing
- System testing
- Regression testing
- Configuration testing
- Performance testing and so on.

The permutations for the factor levels that consist of only one treatment have to be chosen in an uncorrelated way, so that every single treatment gives you a piece of information that is different from the others. The advantage of organizing testing in such a way is that a minimum number of experiments are required for gathering same information. Orthogonality is a property exhibited by the orthogonal vectors. Properties exhibited by the orthogonal vectors are mentioned below:
- Information conveyed by each vector is different from the information by other vectors in the sequence. That is, as we mentioned information conveyed by each treatment is unique to it. This is important otherwise there will be redundancy.
- It is easy to separate these signals on a linear addition.
- All the vectors are statistically independent of each other which means that there is no correlation between them.
- When these individual components are added linearly, the result is an arithmetic sum.

Suppose a system is having 3 parameters each of which has 3 values. We require total 27 test cases for testing all of the parameter combinations which is quite time consuming. So we use an orthogonal array for selecting a combination subset from these combinations. As a result of using the orthogonal array testing, maximization of the test coverage area is possible. At the same time this minimizes the number of test cases that have to be considered for testing. The pair that is selected is assumed to have the maximum number of defects. The technique works based up on this assumption. These many combinations are sufficient for catching the fault. The interaction of the input parameters between themselves has also to be considered. The array is said to be orthogonal because the occurrence of all pair wise combinations is once. The results of the test cases are assessed as follows:
- Single mode faults
- Double mode faults
- Multimode faults

Below mentioned are the major benefits of using this technique:
- The testing cycle time is reduced.
- The analysis process gets simpler
- Test cases are balanced which means that the defect isolation and performance assessments are straightforward.
- Saves up on costs when compared to the pair-wise testing. The coverage to all the defects can only be provided by testing all the combinations that are possible. But our schedule and budget often do not permit this. Therefore we are forced to select only a sample of the combinations from the test domain. Orthogonal array testing is a means for generating samples that provide high coverage for the validation of test domain effectively. This has made the technique particularly useful in the integration testing and testing of the configurable options. Software testers often face a dilemma during selection of the test cases. The quality of the software cannot be tested but only the defects can be detected. And the exhaustive testing is difficult even in the small systems. 

Some advantages and disadvantages of white box testing

White box testing can be applied at the following levels in software testing process:
- Unit testing
- Integration testing
- System testing

However, commonly it is carried out at the unit level. White box testing at this level is used for exploiting the paths through the program units, and for testing paths between different units, and on the system level for testing paths between the various sub-systems. In this article we discuss about the advantages and disadvantages of using this testing methodology.

Advantages: This testing methodology ranks among the widely used testing methodologies, with an increasing number of people using it, although it does require technical knowledge and capabilities. We have three major advantages from it.
First, in this technique it is beneficial to have knowledge about the source code unlike the other methodologies where the tester should not know much about the source code. This helps in thoroughly testing the application.
Second, this methodology has made it possible to optimize the code by helping in uncovering the hidden errors and removing them.
Third, it provides an opportunity for introspection.
Another plus point is that the testing can be started with just one developed unit at hand. We don’t have to wait for the whole program to be ready with GUI. Majority of the paths are covered in white box testing.

Disadvantages: Just as white box testing has advantages; it has its own minus points too.
The first major disadvantage is that it gets very complex once you start with it. This happens so because every path in the program has to be tested and so all the paths have to be identified which can become very time consuming and difficult indeed. For this the programmer as well as the tester must have a great deal of knowledge in this level of detail.
Second major disadvantage is that white box testing consumes too much time. In every project it is not possible to testing each and every single path. It is certain that some of the paths will go unnoticed. This is why the white box test cases are very complex and can only be implemented with a thorough knowledge of the application and code. Maintenance of the test scripts proves to be a burden as the complexity increases and changes have to be made in the implementation.

This testing methodology requires many testing tools which might not be available at hand instantly. The work of a mechanic is analogous to white box testing i.e., the programmer examines the source code as to see why it is not working.
First the tester or the programmer must analyze the source code with the help of comprehensive software documentation, code and the samples.
Second, the tester needs to think for the ways or methods by which he can disrupt the normal functioning of the application or what are the input factors that can cause the program to go awry? Now based up on these assessments, the white box testing techniques can be implemented. The assessments have to be made carefully for white box testing to be successful. In simple words white box testing is just a means for the verification of the source code.
The logic and the structure of the code must be known to the person who is testing it. Logical decisions are implemented. Each logical decision tests a different path. In white box testing programmers can only substitute for testers i.e., they only can test the application. If other testers are hired, they will take some time in understanding the source code of the program or need a high degree of technical knowledge.

What are main features of TestComplete?

Many of the software applications are being written as web–based applications that can be run in a browser. The measure of effectiveness with which these applications are tested varies from organization to organization. 

- Test complete automated testing tool offers the answer to this demand. 

- For tests such as regression tests, responsiveness can be generated only through automated testing tools as such. 

- Automated testing is the way to provide many benefits including repeat-ability and speed of the test execution. 

- Test automation is known to induce long term efficiency in a software system or application.

- Developers also get a rapid feedback and can carry out unlimited iterations of the tests.

- Reporting gets customized and finding defects that were missed during the manual testing becomes easy.

-  However, automation always does not prove to be advantageous.

Features of TestComplete

Test complete  testing tool comes with certain features which we shall state now:

1. Keyword testing: This tool comes with a keyword–driven test editor that is built– in and consists of many keyword operations corresponding to the appropriate automated testing actions.
2. Test record and play back: This tool records the key actions which are required to play back the test. All the actions other than the required ones are then discarded.
3. Full featured script editor: This is another built – in editor using which the test scripts can be written manually. This editor comes with some special plug – ins that provide further assistance.
4. Script debugging features: This feature lets you stop before every statement that can be executed so that you can keep a track of what is going on and make changes accordingly.
5. Access to properties and methods of the objects: The names of all the visible elements can be read by this tool including internal elements of the following applications:

a)   Delphi

b)   C++builder

c)   .net

d)   WPF

e)   Java

f)    Visual basic etc.

Also this tool enables the access to values through test scripts so that they can be verified and used in the tests.

6. Unicode support: Tool has a Unicode character set support which enables the user to test the applications that are non – ASCII and use characters such as Hebrew, greek, Arabic, katakana and so on.
7. Issue–tracking support: This tool comes with issue tracking templates which can be deployed for the creation as well as modification of the items that reside in issue – tracking systems. The tool currently provides support for the following:

a)   Microsoft visual studio 2010,2008 and 2005 team system

b)   Bugzilla

c)   Automated QA AQdev team

8. Open architecture (COM based): An open API, COM interface forms the basis for the test complete’s engine. This makes this tool independent of the source – language and enables it to read the debugger info. It can use this info during run time via debug info agent of test complete.
9. Test visualizer: This feature of test complete lets you take screen shots of the test recording as well as play back thus allowing you to make comparisons among the actual as well as the expected screens during the run time.
10. Support for plug–ins: This feature allows the third party vendors to connect with test complete via their software systems and applications. 

What is a QTP batch testing tool?

Under the context of quick test professional the term given for the execution or running of multiple tests or a series of tests at the same time is “batch testing” or sometimes it is also called “batch execution”. 

However, this batch testing cannot be performed normally like the other applications, a special tool is needed. Quick test professional provides such a tool and it is called “batch testing tool” or the “test batch runner”. 

The batch testing process follows the below mentioned process:

1. Creation of individual tests.
2. Execution of the individual tests created in the first step.
3. Formation of batches using the batch testing tool.
4. Execution of the batch tests on tests from batch testing tool.
5. Observation of result via test result viewer.

Here one thing to be noted is that the provision of viewing batch wise result is not available in quick testing professional for the batch testing sessions. 

Creating a Test Batch

- For creating a test batch, you need to hog down to the tools menu in the menu bar of the quick test professional window.

- Then select test batch runner, then batch, then add option, then specify the path of the test to be added, then save it with appropriate extension like MTB or module test batch and  finally close the test batch runner. 

Execution of Test Batch

- For the execution of a test batch you again need to open the batch testing tool i.e., the “test batch runner” by following the same sequence of steps as discussed for the creation of a test batch. 

- But instead of the add option you need to select the run option. 

- When you do this, the test batch runner is launched and the tests are executed one by one. 

Here if you don’t want to use test batch runner or if you are facing issues with it, you can go for batch testing using other mercury products by checking the “allow other mercury products” check box and clicking OK. 

- In some cases it may be required to run only some specific tests and not all.

- This is also possible in quick test professional via test batch runner. 

- For doing this, you just need to open the test batch runner like always and go to option “open test batch”, then select or deselect from the lists of the tests and go for run test batch. 

- The selected tests are kept in waiting mode till they are all executed one by one. 

- Another mercury product that serves the same purpose as the test batch runner is the “multi test manager”. 

- After the execution of the scripts is complete via test batch runner you can open up that particular test script, then go to automation, then results option and here you have the results. 

- When your test batch is set to batch mode, the entire results cannot be viewed at a time. 

- This whole program can be scheduled if you want it to be scheduled by going to the scheduled tasks option in the control panel. 

- There are some known issues with the quick test professional’s batch testing tool like consolidated test results cannot be viewed and if by chance the batch test execution stops in the mid way, the batch cannot be executed again. 

- The batch runner window should be closed and opened again. 

- Whenever a test is completed, the QTP window comes in the front and the AUT window goes back, the next is opened and executed. 

- For multi test manager the same steps are followed as for the quick test professional. 

What is Object Spy? How to Use it?

Object spy is one of the most important tools in winrunner as well as in quick test professional. In this article we are going to discuss what role object spy serves in quick test professional and how it is used in the same. 

What is an Object Spy?

- With the help of object spy the basic structure of any of the test objects can be known. 

- The best thing about object spy in quick test professional is that the structure of a particular object can be viewed in the tree format. 

- All this makes understanding of the test objects much easier. 

- In addition to this use, the object spy can also aid us in viewing the test object and run time properties as well as methods of all the objects present in the software system or application. 

How the object spy can be used in quick test professional?<>

- For launching object spy just go to tools menu and click on the object spy tab. 

- Clicking on this tab will open up a dialog box named “object spy”.

- In that dialog box you need to select the application software whose object methods and properties you wish to see. 

- After this in the object spy window you need to select the pointer image. 

- All the objects present in that particular application will be listed and from these you can select the one whose object methods and properties you wish to view. 

- There are two particular things that can be viewed using the object spy as mentioned below:

1. Properties of a specific object and
2. Methods applicable for that object.

- Many different properties of the selected objects can be viewed. 

- This viewing of the object methods and properties is facilitated by the pointing hand mechanism of the object spy. 

- As this pointer is hovered above the objects, their corresponding methods and properties are displayed in the window of the object spy dialog box. 

- The details of the object may also include the hierarchy tree of the test objects. 

- The object spy also displays the syntax for the object methods at your command. 

- Different properties can be viewed in different environments. 

- Objects such as following can be viewed using the object spy in quick test professional:

1. Dialog
2. Static
3. Active X
4. Edit and so on.

- The object spy can be considered to be the same as the GUI spy that comes with the winrunner. 

- The object spy contributes a lot while writing descriptive programming. 

- In object, the properties of the objects are displayed along with their corresponding values. 

- Object spy is such a feature provided by the quick test professional with which the total information regarding the objects can be obtained.

- There are 3 tabs in the object spy dialog box namely:

1. Properties tab
2. Methods tab and
3. Navigation tab

- Clicking on the properties tab all the properties of the objects along with the values and hierarchy tree are displayed. 

- Clicking on the methods tab all the methods of the test objects and run time objects are displayed. - Clicking on the navigation tab, the tool bar is displayed. 

- Object spy just like the gui spy of the winrunner is an in built feature. 

- For opening the object spy apart from just clicking on the object spy option on the tool bar there are two more ways:

1. Go to the object repository window and double click the object spy icon.
2. After opening the object repository navigate to the tools and select the object spy option. 

Can you test database using WinRunner? What are all the different DB that WinRunner can support?

- The testing of the databases has been made possible using the winrunner with the help of the data base record check points. 

- These runtime data base record check points can be added to the test scripts and data being processed in to information in the application software during the execution of the test can be compared with the corresponding record present in the data base that you want to test. 

- With the help of these check points, it is possible to compare the contents of the data bases of the different versions of the application software.

- Whenever a data base check point is created, a query is defined in the data base of the application and the values contained in the result set are tested by the data base check point. 

- The query can be defined in any one of the following ways:

1. By using Microsoft query (it can installed from Microsoft office’s custom installation).
2. Manually by defining an ODBC query i.e., by creating an equivalent SQL statement.
3. By using data junction for creating a conversion file.

- The data base check point is said to fail when no match is found between the comparison and success criteria that has been specified for that particular check point. 

- A successful run-time data base record check point can be defined as the one where one or more than one matches were found. 

- One major characteristic of these check points is that they can be used in loop as well. 

- Results for all the iterations of the loop are recorded as separate entities. 

- The run time data base record checkpoints can be added to the test in order to make comparisons between the current values in the data base and the information that is being displayed in the application software while a test is in the process of execution.

- The run time record checkpoint wizard can be used for the following purposes:

1. Defining the query.
2. Identification of the application controls containing relevant information.
3. Defining the success criteria of the individual check points.

- While testing your data base you may land in some situations where you may require comparing the data files of different formats. 

- Following the below stated steps for creating a run time data base record check point manually:

1. Record the application up till where you want the data to be verified on the screen.
2. Calculate the expected values of the corresponding records in the data base.
3. Add the expected values to an edit field.
4. Using GUI map editor teach the winrunner about the controls of the application and the edit fields of the calculated values.
5. Add TSL statements to the test script in order to calculate the expected data base values extract the values and write the extracted values to the corresponding edit fields.

- Before you run your tests do make sure that all the applications with the edit fields containing the calculated values are open. 

- Earlier we mentioned the term “ODBC” in the article, it is actually a technology which enables the computer to connect to a data base so that the information can be retrieved by the winrunner easily. 

- Those who developed winrunner did not knew what kind of data base the AUT uses so it becomes impossible for the winrunner to know the way to connect to the data base of your application. 

- To overcome this problem, the ODBC was developed which can be taught how to connect to the data base. 

- Thus, it is because of ODBC that almost all the data bases are supportable by the winrunner.

How to choose a black box or a white box test?

White box testing is concerned only with testing the software product; it cannot guarantee that the complete specification has been implemented. Black box testing is concerned only with testing the specification; it cannot guarantee that all parts of the implementation have been tested. Thus, black box testing is testing against the specification and will discover faults of omission, indicating that part of the specification has not been fulfilled. White box testing is testing against the implementation and will discover faults of commission, indicating that part of the implementation is faulty. In order to completely test a software product both black and white box testing are required.

White box testing is much more expensive in terms of resources and time as compared to black box testing. It requires the source code to be produced before the tests can be planned and is much more laborious in the determination of suitable input data and the determination if the software is correct or incorrect. It is advised to start test planning with a black box testing approach as soon as the specification is available. White box tests are to be planned as soon as the low level design (LLD) is complete. The Low Level Design will address all the algorithms and coding style. The paths should then be checked against the black box test plan and any additional required test cases should be determined and applied.

The consequences of test failure at requirements stage are very expensive. A failure of a test case may result in a change, which requires all black box testing to be repeated and the re-determination of the white box paths. The cheaper option is to regard the process of testing as one of the quality assurance rather than quality control. The intention is that sufficient quality is put into all previous design and production stages so that it can be expected that testing being relied upon to discover any faults in the software, as in case of quality control.

Manual Testing and what are its drawbacks/limitations.

Software testing is carried out manually in many organizations. The test team generates various test cases and manually tests each and every feature after the product is reached its completion stage. If defect is found, It is rectified and again the software is tested using the test cases. There are few drawbacks or limitations of manual testing.
- Manual testing is error prone because test engineers become bored when testing has to be done repeatedly. To test a we site, each and every link has to be checked.
- Manual testing is very time consuming. The same set of operations need to be done repeatedly.
- Regression testing has to be done to ensure that changes in one portion of the software have no ill effects on other portions of the software.
- To do performance testing, many resources are required, both computer and people. On the other hand, automated tools facilitate running the software from a single client machine.
- Even to manage the testing process is complicated as the testing has to be planned, bugs have to be tracked and reliability analysis has to be performed.

Manual testing is time consuming, error prone and requires lot of infrastructure and manpower, these limitations can be overcome if the testing process is automated. The testing tools reduce manual testing to a large extent and the testing can be done done automatically.

Testing Approaches: Functional Testing versus Structural Testing and Mutation testing

In functional testing, functionality of the module is tested and structure is not considered. Test cases based on specifications and internals of modules are not considered. This type of testing is also known as black box testing. The objective here is to ensure that `correct results'' are obtained when `good inputs'' are applied to the various parts of the design, and when `bad inputs'' are applied the design operates in a predictable manner. Functional testing can therefore be considered as concentrating on checking that the data paths operate correctly.

Structural testing is used to test the implementation of the program. Structural testing, also known as white box or open box testing, is normally applied to sequential HDL code and concentrates on checking that all executable statements within each module. Structural testing involves :
- Statement coverage : This ensures that each and every statement is tested. Software tools called profilers are used to carry this statement coverage.
- Branch coverage : Each and every condition is taken, and inputs are given in such a way that each branch is executed at least once.
- Path Coverage : To test loops, this is required. For example the statement
for(i=o;i<=100;i++) is executed 101 times.

Mutation testing is required to ensure that the software does not fail. It is a good debugging mechanism. The program is modified slightly to obtain mutants of the program. Different mutants are tested with the same test cases. If the mutant fails, and the actual program works correctly, confidence is gained in the program and test cases are considered good. To produce mutants, mutation operators are defined.
- Constant replacement
- Variable replacement
- Arithmetic operator replacement
- Relational operator replacement
- Goto label replacement

Test Automation Framework: What is the Hybrid Test Automation Framework

In previous articles, we learned about the following Automation frameworks:
- Data Driven Testing (link)
- Test Library Architecture Framework (link)
- Keyword-driven/table-driven testing (link)
- Test script modularity (link)
This post covers the 5th such test automation framework, called Hybrid Test Automation Framework. Just like the name suggests, this is what people would ask for, why can't you combine the benefits and strong points of the above test automation frameworks and try to remove their weaknesses, and that is what you get. This is called the Hybrid Test Automation Framework. It is one of the most successful automation frameworks, and is also one of the frameworks that other frameworks eventually mature into.
So, since the keyword driven architecture has some powerful benefits in the form of libraries and utilities, this framework allows the data driven scripts to take advantage of these libraries and utilities; and also make these data driven scripts smaller and reduce the risk of their failure. This framework has many utilities that allow the conversion of currently used scripts into the equivalent of keyword driven (when needed).
Some of the properties of the Hybrid Test Automation Framework are: Core Data Driven Engine, the Component Functions, the Support Libraries, and the Application Map (App Map).

Test Automation Frameworks: What is Keyword-driven/table-driven testing (including definition)

For a few of the last posts, we have been looking at more details about test automation, including the benefits of automation, some scenarios in which we should not use automation, and the scenarios in which we should use automation. In addition, we started discussions around the use of test automation frameworks and how they prove to be more beneficial than just creation test cases as you do recording of your test scenarios. In this post, let us consider one of the test automation frameworks, based on "Keyword-driven/table-driven testing".
'Keyword driven testing' and 'table driven testing' seem like 2 different sets of words, but they actually are used to refer to the same method. They denote an application independent framework, which requires the development of data tables and keywords. These data tables and keywords are independent of the test automation took being used, and also independent of the test scripts used to drive the application that is being tested. These keyword driven tests look very similar to the manual test cases. When keyword driven tests are being used, a table is used to document the functionality being used, and this functionality is also mapped through step by step instructions for each test. The entire testing process is driven based on data.

Benefits of keyword driven testing:
- If the tester needs to be quickly on the job, extensive training on the tool can be done later, but at the instant of testing, the tester needs to know the keywords, and the format of the test plan.
- The Scripting language can be written by somebody who has expertise on the scripting language, and this activity can happen earlier to the test plan. The tester does not have to be bothered about the scripting process.
- A spreadsheet format can be used for writing the detail test plan.

Some problems with this technique:
- If there are a large number of keywords, then the tester needs to learn all these, and this initial effort can take time. Once done, then this is no longer a constraint.
- You need people skilled in using the Scripting language of the tool being used.

Scenarios in which to use automated testing - when to use automated versus manual testing

In a previous post (What is Automated Testing), I had talked about what is automated testing, and what are the benefits of automated testings. In this post, I will explain about some of the scenarios in which the use of automated testing is more likely (and beneficial). This is also part of the discussion about where to use manual testing versus where to use automated testing.
- There is a tremendous benefit to using automated software where the work is repetitive and follow a certain pre-defined set of cases. Consider the case where a team is working on a set of features for a new version of the product. The team needs to also ensure that earlier released features continue to work and are not impacted by any changes in code in other parts of the application. This can be done through the process of automating the test cases for these features, so that the testing happens automatically every time a new build is released.
- When a large number of test cases have to be carried out in a fixed time frame, then these can be done through automation. Automation will typically execute a large number of test cases in a much shorter time frame than manual testing.
- Allows load testing. When you need to simulate situations where there are a large number of users load testing the system, automation can help to a great deal in such cases. For applications that are seeking to evaluate their performance under stress conditions, automation is necessary.
- Testing with a large number of different inputs. Suppose there is a software that takes inputs from the user, and produces an output. In such cases, the testing for the input cases would need to consider a range of different inputs, including acceptable values and a range of unacceptable values. These are done far better through automation testing, since doing this manually involves a lot of effort.
- Automated testing can be tweaked to do a number of tests that would be difficult for people otherwise. For example, during the course of a normal product development cycle, thousands of files can be touched, and it is impossible to test all the changes on a regular basis, especially for stuff such as security errors (and there can be many types of checks that the development team would want to do)
And there can be more benefits, would love to hear more from people who use automated testing ..

Benefits of automated testing processes and systems, including scenarios where automated testing is done

Before we start with details about what are the benefits of automated testing, it is worth an explanation about what automated testing is, and then move on from their.
What is automated testing ?
Automated testing is the process whereby manual testing that is in place is automated (with the assumption that these manual testing systems exist, or are in progress enough that these can then be automated. In more detail, automation can be explained as the use of strategies and tools (along with processes) that help or eliminate human efforts and intervention; one of the main benefits is to ensure that repetitive tasks that are necessary but involve boring and regular tasks can be handled through automation.
What are some of the benefits of automation ?
- Regular and boring tasks can be removed from the necessity of humans doing it. One perfect example is the use of a series of testing during the process of software development whenever a new build or a fix is made. Such testing is required to ensure that the build is safe to use (internally it means that a number of tests that were required to be done by a person can now be done by the automation system). One side benefit is that such automation can be scheduled; so if a build comes in at 6 AM, there is no need for somebody to be there to carry out the series of tests. The automation software can handle doing the testing at that time.
- The testing is reliable. Given that the testing steps are being carried out by software, this testing will always follow the same pattern, without any variation
- Load testing. Setting up automation is the first part of setting up load testing, since being able to run a series of tests in order to load testing depends on being able to run the tests through an automation framework
- Scriptable or programmable. Automation tools allow you to customize the automation testing process, thus giving you a huge degree of flexibility to control what all you can do
- Use again and again. Once you create an automation framework, you can use it again and again. If the user interface changes, a small amount of re-work would be enough to run the automation tests
- Speed. Automation tests will run pretty fast, much faster than a human would be able to do the testing
- Comprehensive testing abilities. Organizations that start using automation testing frameworks eventually move onto building complex automation frameworks that cover a large portion of their functionality
- Improves the human testing abilities. Releasing humans testers from testing mundane and repetitive tasks can help them to do more value added testing, adding to product quality.