What are different Web Functional/Regression Test Tools?

As functional and regression testing is important for many software systems, the same way it is important for the web applications to undergo the functional and regression testing. At present we have a number of tools available for web functional/ regression testing.  

In this article we shall discuss many such tools available:

1. ManageEngine QEngine: This tool is for functional testing and load testing of the web applications. This tool enables you to carry out GUI testing in minutes and below are some of its features:

Ø Portability: With this feature you can record scripts in windows and play them in linux without any need of recreation.

Ø  Scripting capabilities: These are simplified script creation, keyword driven testing, data – driven testing, object repository, Unicode support.

Ø  Playback options: It includes playback synchronization, chain scripts, multiple playback options etc.

Ø  Validation and verification: The tool comes with a rich library of built – in functions for constructing function calls for requirements such as dynamic property handling, database handling, and screen handling and so on.

Ø   AJAX testing

Ø  Reporting capabilities: The tool provides you with clear and powerful reports indicating the status of the execution of the test.

2. SeleniumHQ: This tool has a number of smaller projects that combined to create a testing environment to suit your needs:

Ø  Selenium IDE: This one’s an add – on for firefox and can be used for replaying the tests in the same.

Ø  Selenium remote control: Web browsers can be controlled with this client/ server system located on either a local host or a remote one.

Ø  Selenium grid: This is the same as the previous one but can handle multiple servers at a time.

Ø  Selenium core: Testing system based on JavaScript.

Ø Further specific selenium projects have been developed for ruby, eclipse and rails.

3. Rapise: It was developed by Inflectra inc. This tool has an extensible architecture and cross browser testing capabilities. It supports various versions of Mozilla Firefox, chrome, MS internet explorer and so on. The tool comes with built – in support for AJAX, YUI, GWT, AIR, Silverlight, flash/ flex etc. one can use this tool for carrying out keyword as well as data driven testing through excel spreadsheets. The tool identifies the objects based up on CSS and Xpath. For bitmaps, the tool comes with a built – in OCR (optical character recognition). It uses JavaScript for scripting purposes and therefore also has a JavaScript editor.
4. funcUnit: This is an open source web application testing framework. The API is based up on JQuery. Almost all the modern browsers are supported on Linux and MAC. Selenium can also be used for executing the tests. It can simulate various user input events clicking, typing, and dragging the mouse and so one. 

5. qUnit: Any JavaScript code that is generic can be tested by this tool. It is somewhat similar to the JUnit but operates up on JavaScript features. 

6. EnvJS: This is a simulated browser environment and an open source tool whose code has been written in javascript. 

7. QF – test: It has been developed by quality first software  as tool for cross  - platform testing and cross browser automation of the web applications. It can be used to test web applications based up on html, ajax, GWT, exTJS, richfaces, Qooxdoo, java and so on. The tool has got some small scale capabilities for test management, intuitive user interface, extensive documentation, capture/ playback mechanism, component recognition and so on. It can be used for handling both custom and complex GUI objects. It has got customizable reporting and integrated test debugger system. 

8. Cloud testing service: This enables the cloud capabilities to be utilized while web testing. It has been developed by the cloud testing limited. Here, web functionality can be recorded via selenium IDE and a web browser. The scripts can be uploaded to the cloud testing website.

What are two methods provided by TestPartner for functional test development?

TestPartner is quite a successful GUI software testing tool. Micro focus developed this product with the intention of providing means for functional testing and GUI testing to the software developers and testers who have hard deadlines to work up on. This testing tool helps these software developers and testers to achieve more of testing in less time. 

Two primary methods  are provided by the TestPartner for functional test development:

1. Code oriented development

2. Visual storyboard – based development environment

About Code Oriented Development

- Code oriented development uses the Microsft VBA (visual basic for applications). 

- The major benefit of including this in the development environment is that using it, users can access the rich IDE (integrated development environment). 

- Access to IDE is important since it contains all the core visual basic application libraries plus the capabilities that are inheritable in VBA such as error handling, ability to reference public libraries, debugging and so on.

- A library of VBA – based functionality is provided by the Compuware TestPartner and has been specifically built to be used in the functional test automation.

About Visual Storyboard

- Visual storyboard – based development environment interface is also known as the “visual navigator”.

- It is used extensively for developing the functional test automation. 

- The following things are provided to the user through this capability:

Ø  A screen preview i.e., the screen shot of window of application under test.

Ø A list of test automation steps. These are the steps are the steps that were performed while testing against the screen.

Ø  A story board providing a series of test steps that have been performed against a number of screen previews.

- There is another advantage of this approach which is that the users can access the test automation functionality.

- In addition to this they get the ability to add variabalization, different test logics and so on. 

Advantages and Benefits of TestPartner

- Version 6.3.0 is the latest in the series of Testpartner. 

- Testpartner lets you accelerate the testing process at the same time delivering high quality software products on time. 

- The functional test automation through Testpartner is quite easy and cost effective. 

- Testpartner proves to be a powerful tool for functional test automation. 

- The tool is quite easy to understand and use. 

- Users get intuitive insights through the test analysis and visual test creation. 

- Further, there is no requirement of code which it makes usable even for the non – technical business users. 

- Accelerated testing means that the testers and developers can keep pace with the development and quality is no longer seen as a bottleneck. 

- With the visual and storyboard oriented approach of the Testpartner, such business processes can be captured that can empower the experts for creating the sophisticated test cases without being able to write the script code. 

- The analyzation of the test results is simplified by the Testpartner through its visual storyboard. 

- With this story board, it becomes easy to check where the test actually failed and where the modification is required. 

- Testpartner is well equipped with modular structures and powerful extensions. 

- With these, the technical users can access all the debugging and scripting capabilities of VBA for creating advanced test cases that are repeatable.

- Integration of the test cases that have been written in VBA or visually is possible in Testpartner.

- Tests created by the Testpartner are quick in adapting to the changes made to the application. 

- The test results can be used in incorporating the maintenance changes, thus accelerating the maintenance process. 

- The built – in verification capability of the Testpartner lets you add steps for testing the results.

Explain TestPartner - a GUI Software Testing Tool?

- TestPartner was developed by Micro Focus as a software testing tool especially for testing the graphical user interface (GUI) of the software systems and applications. 

- This testing tool was developed with the intention of empowering the software developers to functionally automate the GUIs of the AUTs. 

- The primary goal was to make the team capable of performing more application testing in a given time period as compared to the manual testing that could be performed in same amount of time. 

- TestPartner offers two basic methods for functional testing:

1. Code oriented development and
2. Visual story board based development

- Recently, the TestPartner version 6.3.0 has been released. 

- It is great product for accelerating the testing process and delivering high quality applications well before the deadline.

- IT managers who have the responsibility of ensuring the quality of the application need to deal with several business challenges that are critical such as:

Ø  Verification of the functionality of the functions i.e., whether or not they are working as expected and meet the business requirements.

Ø  Testing of the multiple heterogeneous technologies simultaneously.

Ø  Keeping up with the fast paced release schedules that are ever changing.

Ø  Striking a balance between the application quality and the associated risks.

- TestPartner provides such functional testing capabilities that very well accelerate the testing process and enables the timely successful delivery of the software.

- This it is known to do at a low cost but with no compromise in the quality. 

- TestPartner achieves this through its unique tiered testing approach. 

- The main feature of this approach is that following it can make it possible for the testers and developers to achieve more in the given time through effective collaboration.

- This software product is even beneficial for the non – technical users. 

- The visual and the contextual approach of the TestPartner lets the non – technical users to be a part of the testing process. 

- For the expert software developers and testers it provides the access to microsoft’s visual basic for the AUTs. 

- This gives them power to cope with the most complex testing problems. 

- TestPartner maintains a special flexibility and focus on the usability of the application allowing the developers to produce more positive results with minimal requirement of training.

- It helps leverage the experienced as well as the non – technical testers to test the applications more efficiently. 

- It encourages the collaboration throughout the development cycle thus achieving greater project effectiveness. 

- Using TestPartner one can even automate the regression testing process, thus quickening the acceptance of the regular updates such as services packs and patches. 

- The documentation produced by the TestPartner is quite clear and free of any ambiguity. - This further accelerates the resolution and identification of various issues related to the application under testing. 

- It makes easy for you to understand the test results so that you feel confident in releasing the software. 

- These test results can be can be used for maintaining the tests, providing better ROI. 

- One can improve the quality of the application leading to lower overall development cost. 

Quality efforts can be accelerated for a number of platforms such as:

1. Web
2. .net
3. Java
4. SAP
5. Oracle
6. Other windows – based distributed applications.

What kind of testing engines are supported by Fitnesse?

Fitnesse testing tool is typically a testing method designed for providing a highly usable interface around the frame work called FIT. Therefore, it is mostly intended to support the agile style of regression testing and acceptance testing. In this style of regression and acceptance testing, all the functional testers collaborate together with the developers in order to come up with a good testing suite. 

- Fitnesse testing revolves around the idea of the black – box testing. 

- The system under test is taken to be a black box. 

- Output generated in response to the already defined inputs is used to perform tests against the considered black box.

- The job assigned to the functional tester is to design the tests in function specific sense and use fitnesse testing tool to express them. 

- On the other side a software developer is assigned the job of making a connection between the SUT and the Fitnesse tool so that the execution of the tests can take place and the two outputs could be compared. 

- The basic idea behind this whole testing process is to develop a forced collaboration among the developers and the testers so that the mutual understanding of the system’s requirements can be improved.

- Four components constitute the testing process through fitnesse namely:

1. A wiki page
2. A testing engine
3. A test fixture
4. System under test

Types of Testing Engines supported by FitNesse

- A piece of java code is used to establish a link between the SUT and the generic testing engine.

- Testing engine is responsible for carrying out most of the mapping and invoking the fixture methods. 

- Two engines are majorly supported by the fitnesse testing tool:

FIT Testing Engine 

- This engine serves more than just a testing engine. 

- It is more like a testing frame work in itself that makes use of a combination of functionality for invoking tests; carry out an interpretation of the wiki pages and generation of the output pages. 

- This is the testing engine around which the testing engine was built originally to serve initially as a user interface.

- And this is the story behind the name of the tool ‘fitnesse’. 

- This testing engine combines a number of responsibilities in to one unlike others that divide one responsibility in to many smaller ones.

- But software developers need to pay price for this since in this an FIT engine requires an inheritance from the base classes of the FIT frame work.

- However, java counterpart can experience some sort of difficulty since one chance of class inheritance of a developer is claimed by the frame work. 

- This also coveys a fact that the test fixtures are actually heavy weight constructs in nature. 

- Because of these considerations only, it was decided to adopt an alternative to FIT testing engine called SLIM testing engine.

SLIM Testing Engine

- Simple list invocation method is abbreviated to SLIM and serves as the best alternative to FIT testing engine. 

- It does not focuses on the combination of the wiki – based testing elements rather it emphasizes up on the invocation of the testing fixtures. 

- Unlike the FIT testing engine, it is invoked under the remote control of the Fitnesse wiki engine and then runs as an individual server. 

- Interpreting a wiki page and generating its result is now considered to be a part of the wiki testing engine. 

- Further the light weighted fixtures are allowed in the SLIM testing engine, which are nothing but simple POJOs. 

- One does not need to extend the fixtures or put any other frame work classes in to use. 

How is test execution done by Fitnesse Testing Tool?

FitNesse testing tool based on the integration testing frame work is now widely used for acceptance testing more than unit testing since it has the ability to facilitate a description of the function in a detailed manner. 

In this article, we shall discuss about how the execution of the tests is carried out using the Fitnesse testing tool. 

The testing process using the fitnesse testing tool involves 4 major components for every test:

1. Wiki page that is used to express the test in the form of a decision table.
2. A testing engine for interpreting this wiki page.
3. A test fixture that is invoked by the engine and itself invokes the SUT or system under test in turn.
4. SUT(system under test) that is currently under test.

- Out of the above 4 mentioned components two are produced by the software development team namely the fixture and the wiki page. 

- The team produces the system under test but it is not considered from the point of view of the black box test.

- Decision tables which express a test are included in the wiki page.

- The test fixture is the link that is made between the SUT and the generic testing through a piece of code written in java. 

- Mapping that is done between the fixture and the wiki page is an example of simple convert – to – camel case mapping. 

- Such kind of mapping is applicable to almost all the headings and is used for the identification of the name of the class of the fixture and its methods as well. 

- Whenever a heading ending is encountered, its value is to be considered to be read from the fixture while values from the other headers are considered as input for the fixture.

- The column order of the table i.e., left to right order is followed for calling the methods of the fixtures. 

- Testing engine is the component that actually carries out the mapping process. 

- It also invokes majority of the fixture methods. 

What kind of engines are supported by Fitnesse Testing Tool?

Two types of engines are supported by the fitnesse testing tool:

The FIT Engine: 

- This is more like a frame work rather than just being an engine. 

- It carried out the following purposes:

a)   It combines the functionality for invoking the tests,

b)   Interpretation of the wiki pages,

c)   Generation of the output pages.

- This engine was named so because the tool originally developed around this. - This engine works by combining the responsibilities rather by separating them.

The SLIM Engine: 

- SLIM stands for simple list invocation method and is used as an alternative to the FIT engine. 

- This engine implements the slim protocol. 

- Like FIT, SLIM does not combine all of the elements of the wiki – based testing, rather its focus is on invocation of the fixture. 

- This engine works as a separate server that is invoked by the fitnesse wiki engine remotely. 

- Simple POJOs like light–weighted fixtures are allowed by this engine. 

- These fixtures neither use any frame work classes nor extend it. 

- This leads to a simplified design thus allowing the designer to focus up on the call of the SUT properly and in as simple way as possible.

- This also keeps the ways for inheritance open, thus allowing the fixtures to be developed whenever necessary. 

- Inputs and expected output are coupled together to form the tests described in Fitnesse. 

- These couplings are considered to be a kind of variation of the tables. 

- A number of such variations are supported by the Fitnesse testing tool.

Explain FitNesse testing tool? What are principles of Fitnesse?

- FitNesse is an automated testing tool that has been developed to serve as a wiki and a web server for the development of software systems and applications. 

- This testing tool is entirely based up on the frame work for integrated testing that has been developed by Ward Cunningham. 

- It has been designed to support acceptance testing more than unit testing. 

- It comes with the feature of facilitating the description of the system functions in detail.

- With FitNesse testing tool, the users using a developed system can enter the input when it has been specially formatted i.e., non – programmers are able to access this format. 

- FitNesse tool interprets this input and automatically creates all the tests. 

- The system then execute these tests and returns the output to the users. 

- The main advantage of following this approach is that a very fast feedback can be obtained from the users. 

- Support in the form of classes called ‘fixtures’ is provided by the developer of the SUT i.e., system under testing. 

- The credit of writing the code for the fitNesse tool goes to Robert C. Martin and his colleagues in java language. 

- Since the program was developed in java, therefore initially it supported only java but now over the time it has got versions in a number of languages such as Python, C++, Delphi, ruby, C# and so on. 

Principles of FitNesse Testing Tool

This software works on certain principles which we shall discuss now:

FitNesse as a testing method: 

- Originally, it was designed as an interface using the fit frame work and it proved to be highly usable. 

- As such it is known to support the regression tests and black box acceptance tests in an agile style.

- This style of testing involves all the functional testers working in collaboration with the software developers in a software development project in an effort of developing a testing suite. 

- FitNesse testing revolves around the notion of black box testing.

- This involves considering the system to be a black box and testing it in the terms of the output that is automatically generated by the tool in accordance with the given inputs. 

- The responsibility of the functional tester is to design tests in a sense of functionality and expressing the same in the fitNesse tool. 

- On the other hand, the responsibility of the software developer is to connect the tool to the SUT so that tests can be executed and the actual output can be compared to the expected one. 

- The idea that drives this tool is forcing the functional testers and software developers to come up with a common language for an improved collaboration which will eventually lead to an improved mutual understanding of the SUT.

Fitnesse as a testing tool:

- Fitnesse defines the tests as inputs and outputs coupled together. 

- These inputs and outputs coupled together are expressed as variations of a decision table. 

- It supports a number of variations that range from tables that execute queries to tables for literal decision and to tables that express the testing scripts. 

- A free form table is the most generic variation that the designers can interpret in any way they like. 

- However, some sort of table is always used to express the tests. 

- The primary focus of FitNesse is on the easy creation of the tests, thus allowing the testers to maintain a high quality for the tests rather focusing on how the tests are to be executed. 

- Three factors are involved in the creation of the tests through fitNesse:

a)   Easy creation of the tables.

b)   Easy translation of the tables in to calls to SUT.

c)   Maintaining flexibility in the documentation of the tests. 

What is meant by Statistical Usage Testing?

Statistical usage testing is the testing process that is aimed at the fitness of the software system or application.

The test cases chosen for carrying out statistical usage testing mostly consist of the usage scenarios and so the testing has been named as statistical usage testing. Software quality is ensured by the extensive testing but that has to be quite efficient. Testing expenditures covers about 20 – 25 percent of the overall cost of the software project. In order to reduce the testing efforts, deploy the available testing tools since they can create automated tests. But usually what happens is that the important tests require manual intervention with the tester requiring thinking about the usage as well behavior of the software. This is just the repetition of the tasks that were done during the requirements analysis phase.

About Statistical Usage Testing

- A usage model forms the basis for the creation of tests in statistical usage testing.

- Usage model is actually a directed usage graph more like a state machine and it consists of various states and transitions. 

- Every transition state has a probability associated with it regarding the traversal of the transition when the system would be in a state that marks the beginning of the transition arc. 

- Therefore, the sum of the probabilities of outgoing transitions sum up to unity for every state.

- Every transition can be associated with an event and more with parameters that are known to trigger the particular transition. 

- Such event associated transitions can be further related to certain conditions called the guard conditions. 

- These conditions imply that the transition occurs only if the value of the event parameter satisfies the condition.

- For assigning probabilities to the transitions, 3 approaches have been defined as follows:

1. Uninformed approach: In this approach, same probability is assigned to the exit arcs of a state.
2. Informed approach: In this approach, a sample of user event sequences for calculating suitable properties. The sample is captured from either an earlier version of the software or its prototype.
3. Intended approach: This approach is used for shifting the focus of the test to certain state transitions and for modeling the hypothetical users.

- According to a property termed as the marcov property, the actual state is what on which the transition probabilities are dependent. 

- However, they are independent of the history again by the property. 

- This implies that the probabilities must be fixed numbers. 

- A system based up on this property is termed as a marcov chain and it requires conclusion of some analytical descriptions. 

- Usage distribution is one among such descriptions. 

- It gives for every state its steady–state probability i.e., appearance rate that is expected.

- All the states are associated to one or the other part of the software system or application and the part of the software that attracts more attention from the tests is shown by the usage distribution. 

- Some other important descriptions are:

1. Expected test case length
2. Number of test cases required for the verification of the desired reliability of the software system or application.

- The idea of the usage model generation can be extended by handling guard conditions and enabling the non–deterministic behavior of the system depending on the state of the system’s data. 

- All this helps towards the application of the statistical usage testing to systems over a wide range. 

- The use cases are defined by the top–level structure of the unified modeling language (UML). 

Explain Cucumber Testing Tool?

A need was felt for a tool that could create and run automated acceptance tests that were developed in so called BDD or behavior driven development style. Thus, cucumber testing tool came in to existence. 

About Cucumber Testing Tool

- The software for this tool has been developed in ruby programming language. 

- However, the cucumber testing tool does work for the projects beyond the ruby level. 

- For some ruby, cucumber testing tool is like a bridge to go to the target language. 

- For other, gherkin parser still continues to be relevant while they use the target language for the implementation purpose.

- There are certain feature documentations that are created in business – facing text and these can also be executed with the help of cucumber testing tool. 

- Features might be defined using one or more than one scenarios. 

- These are executed using the ruby definition of the test implicit comprising of few steps. 

- For the ruby tool box, the cucumber testing tool proved to be the most popular testing frame work in the month of January 2012. 

- The leading frame work was RSpec. 

- In the same year in the month of April, the cucumber testing tool was moved to a category of the acceptance test frameworks. 

- A survey was carried out which revealed that cucumber testing tool stands 5^th in the ranking of the popular testing frameworks being used by the startup companies.

- Cucumber is just an implementation of the behavior driven development based up on ruby.

- Using it, the user can describe how the system behaves with some natural language comprising of some specific keywords. 

- Firstly, a feature file is created explaining a feature of the software system or application as well as certain scenarios pertaining to a number of different test situations. 

- Secondly, step definitions are written in ruby programming language that explains the purpose of that particular step. 

- Writing those definitions in ruby language provides flexibility in the execution of the test cases. 

- The testing tool comes with definitions for an acceptance testing tool called webrat. 

- This tool can be used for simulation of the browser not having javascript support. 

- It also provides option for making a combination with water that is a web automation frame work and is used for implementing the web automation tests based up on web browser.

- Using cucumber, with this combination counts as a traditional way of using it but anyways it works quite well. 

- In testing using cucumber, a feature is tested against a number of test scenarios that might describe several operations. 

- The step definitions part needs to be explained so that the tool knows what it has to do next. 

- In some of the cases, water is used with water – web driver gem and Rspec gem. 

- Constants can be defined as a reference to the browser and for web.

- Hash PAGES is not required in every case but proves to be beneficial when there are a number of pages involved. 

- The tests thus developed can be executed either when the full web site is ready or during the implementation phase itself. 

- Even if you decide to go with the second option, you will have to run the test one more time when the next release goes out. 

- You have a third option i.e., for using the test as a stand – alone set for the web site. 

- Another plus point of the cucumber testing tool is that it displays in quite a formatted way that displays steps that were both successful and failure. 

- Green color is used for displaying successful steps and red is used for the failed ones. 

- All that has been described for the cucumber implies for the water web testing frame work also. 

What kinds of functions are used by Cleanroom Software Engineering approach?

Harlan Mills and his colleagues namely Linger, Poore, Dyer in the year of 1980 developed a software process that could promise building zero error software at IBM. This process is now popularly known as the Cleanroom software engineering. The process was named in accordance with an analogy with the manufacturing process of the semiconductors. 

The Clean room software engineering process makes use of the statistical process and its control features. The software systems and applications thus produced have certified software reliability. The productivity is also increased as the software has no defects at delivery. 

Below mentioned are some key features of the Cleanroom software engineering process:

1. Usage scenarios
2. Incremental development
3. Incremental release
4. Statistical modeling
5. Separate development
6. Acceptance testing
7. No unit testing
8. No debugging
9. Formal reviews with verification conditions

Basic technologies used by the CSE approach are:

1. Incremental development
2. Box structured specifications
3. Statistical usage testing
4. Function theoretic verification

- The incremental development phase of the CSE involves overlapping of the incremental development and from beginning of specification to the end of the test execution it takes around 12 – 18 weeks.

- Partitioning of the increments is critical as well as difficult. 

- Formal specification of the CSE process involves the following:

1. Box structured Designing: Three types of boxes are identified namely black box, state box and clear box.
2. Verification properties of the structures and
3. Program functions: These are one kind of functions that are used by the clean room approach.

- State boxes are the description of the state of the system in terms of data structures such as sequences, sets, lists, records, relations and maps. 

- Further, they include specification of operations and state in-variants.

- Each and every operation that is carried out needs to take care of the invariant. 

- The syntax errors present in a constructed program in clean-room are checked by a parser but is not run by the developer.

- A team review is responsible for performing verification which is driven by a number of verification conditions. 

- Productivity is increased by 3–5 times in the verification process as compared to the debugging process. 

- Proving the program is always an option with the developers but it calls for a lot of math intensive work.

- As an alternate to this, clean room software engineering approach prefers to use a team code inspection in terms of two things namely:

1. Program functions and
2. Verification conditions

- After this, an informal review is carried out which confirms whether all conditions have been satisfied or not. 

- Program functions are nothing but functions describing the prime program’s function.

- Functional verification steps are:

1.    Specifying the program by post and pre-conditions.

2.    Parsing the program in to prime numbers.

3.    Determining the program functions for SESE’s.

4.    Defining verification conditions.

5.    Inspection of all the verification conditions.

- Program functions also define the conditions under which a program can be executed legally. Such program functions are called pre-conditions.

- Program functions can even express the effect the program execution is having up on the state of the system. Such program functions are called the post conditions.

- Programs are mostly expressed on terms of the input arguments, instance variables and return values of the program. 

- However, they cannot be expressed by local program variables. 

- The concept of nested blocks is supported by a number of modern programming languages and structured programs always require well nesting. 

- The process determining SESE’s also involves parsing rather than just program functions.