How can you handle exceptions in QTP?

Exception handling is one of the major features of any test automation and testing software suite. The same is with the HP’s quick test professional which is also a test automation and software testing suite. 

How exceptions are handled in QTP?

- The exception handling thing is managed by the quick test professional by means of some recovery scenarios. 

- The basic goal of exception handling in quick test professional is to have the tests running even if some unexpected failure is encountered. 

- There is some memory space associated with the applications that are tested. 

- This memory space is hooked by the HP quick test professional and this in turn gives rise to some exceptions which may cause the quick test professional to falter, terminate and become unrecoverable. 

- The recovery scenarios used by the quick test professional come in built with the whole software package. 

- But it is not always beneficial to rely on the recovery scenarios rather it is always better if you can handle the exceptions and errors yourself. 

- A recovery scenario manager is also available which has an in – built wizard using which you yourself can define you very own recovery scenarios. 

- This wizard can be accessed by going to the tools menu and then selecting the option “recovery scenario manager”. 

- These recovery scenarios deal in three steps as stated below:

1. Triggered events
2. Recovery steps and
3. Post recovery test run.

Some may think that using the recovery scenarios is the only option for handling the exception in quick test professional but it is not so.  

- Another option is which involves the use of descriptive programming.

- This approach is better than the former option since using it your application can gain more visibility and robustness. 

- Though, the recovery manager can be used for many scenarios but some of the real time scenarios cannot be handled using it. 

- In such cases descriptive programming is the alternative. 

- Third option will be make use of the exception handling capabilities of the visual basic script like the following:

1. Err object
2. On error resume next
3. On error go to 0 statements and so on.

- The last two can be used at the script level. 

- The in–built recovery scenario of the quick test professional supports only 4 exceptions:

1. Object state exceptions
2. Pop up exceptions
3. Application crash exceptions and
4. Test run error exceptions

- A simple example is when you play back the recorded script, the AUT screen is minimized and a run time error is generated as “object not visible”.

- A “test run handler” can be used for this. 

- Four trigger events have been defined during which the recovery manager is supposed to be used:

1. When  a pop up window appears while an application is open for the test run,
2. When the value or the state of a property of an object changes,
3. When a step in the test run becomes unsuccessful or fails terribly, and
4. When the open application itself fails while the test is in progress.

- All of the above mentioned 4 triggers are nothing but exceptions. 

- The quick test professional can be instructed accordingly regarding how to recover from an unexpected event or failure that occurred during the test run in the application development environment. 

- Separate individual files can be created for different scenarios as per the requirements. 

- If you search the web you can find an advance qtp script that can attach as well as enable the recovery scenario file when the test run is in progress. 

What are the different quality attributes in software architecture?

Software is of no use without good quality. Incorporating high quality in software is very difficult. The term “quality” is itself very hard to be defined. We can only measure quality using certain attributes. These attributes are called software quality attributes.

WHAT IS THE IDEA BEHIND QUALITY ATTRIBUTES?
1. Quality attributes of a software system or application define its behavior.
2. The quality attributes give a certain idea of how much suitable is the software for a purpose or environment.
3. It is an essential component of software architecture.
4. Identification of quality attributes is necessary as it helps the system to be built according to the intended behavior.

Certain scenarios are used to measure the quality attributes.
1. The scenarios are very helpful in helping the program developer to make decisions that are qualitative in nature and related to the program behavior.
2. Writing good scenario statements is very important in identification of quality attributes.
3. Although the scenarios help in the identification of attributes, they don’t provide an effective way to achieve those attributes that has to be figured out separately.

Architectural methodologies come to help at this point.
1. There are several architectural methods possible to achieve a particular quality attribute.
2. The software architect should select the right method and use it according to the needs of the software.
3. He should justify the selection of a method and provide documentation on it.

Over the years, some basic quality attributes have been identified and recorded in a list. The quality attributes have been classified in to 4 categories:

1.Run-time system qualities
These attributes are measured when the software is in execution mode.
- Functionality is the ability via which the system completes the assigned task.
- Performance is the collective measure of resource utilization, the response time, and the through put of the system.
- Security is the ability of the system to protect itself from unauthorized access and modifications.
- Availability is the time period for which the system is available to use.
- Inter-operability is the ability of a system to run in cooperation with another system.
- Usability includes other attributes like affect, control, help, and efficiency.

2.Non runtime system quality attributes
These attributes cannot be measured when the software is in execution mode.
- Portability is the ability of a system to perform well in varying environments.
- Modifiability is the adaptive ability of a system by virtue of which it is able to adapt to the changes.
- Integratibility is the ability of the components of the system to work in coordination with each other.
- Reusability is the measure of extent to which the existing features can be used in new software.
- Testability is the measure of the extent to which the software can be tested.

3.Business quality attributes
These attributes affect the other quality attributes.
- Marketability is the value of the system in the market.
- Cost is the money value of the software.
- Schedule is the expected utilization of the system.
- Appropriateness

4.Architecture quality attributes
These are specific attributes concerning the software architecture.
- Correctness is said to account for over all satisfaction of the system requirements.
- Conceptual integrity is the ability of the components of a system to integrate with each other.

5.Domain specific quality attributes
These are specific attributes concerning the business domain.
- Calibrability is the ability of a software system by virtue of which it recalibrates itself according to a specific range of work.
- Sensitivity is the measure of response of the software system to the measuring of a quality attribute.

The software architecture is greatly influenced by the quality attributes of a system. A measure taken in favor of one quality attribute often goes against another.

What are different aspects of distributed testing?

We have heard a lot about different kinds of testing such as regression testing, scalability testing, web testing, unit testing, visual testing, and performance testing and so on. But do you know what is distributed testing? Ever heard about it? No? Then this piece of writing is certainly for you!

This type of testing usually receives very less coverage and that’s why most of the people are not familiar with it. Here I have attempted to explain what is meant by distributed testing and how it compares with its non distributive counterpart.

Non distributed testing can be defined as the tests that run or execute only on a single computer and usually do not involve any kind of interaction with the other computer systems. I used the word “usually” here because there exist some tests that are executed from a host machine to test the target device which holds an embedded or a real time operating system. Non distributed test cases can be configured very easily.

Non distributed testing is further divided into two sub categories namely local testing and remote testing. They have been discussed in details below:

- Local Testing
This kind of testing involves running the test cases on a local computer system. The tests used are called local tests. For performing local test you don’t have to be connected to a network connection.

- Remote Testing
This kind of testing requires a network connection so that you can run a test on a remote computer system to which you don’t have local access. This is very comfortable since you can work right from your desk and you also get the results right on your desk. Remote tests can be performed on several computer systems at a time. The best about thing about remote testing is that no matter how many software systems are under the distributed testing, there is no interference between the processors of different CPUs.

Now that you have got the idea of how non distributed testing is like, it will be easy for you to understand distributed testing is like.
- A distributed test case consists of many parts that interact with each other.
- Each part of the test case is executed on different computer system.
- The interaction of the different parts of the distributed testing sets it apart from non distributed testing.
- If you notice the testing is all about testing the interaction between different computer systems.
- All of the test cases being processed on different processors have a common aim irrespective of the system on which they are performed.
- Distributed testing is not to be confused with simultaneous testing since in simultaneous testing there is no interaction between the different tests.
- Platform proves to be one of the great challenges to the distributed testing.
- The testing environment should be capable of working on all the platforms involved effectively.
- After setting up of your testing environment, you need to make a test plan or say that you need to describe how you want to carry out distributed testing.
- This can be done via a test scenario.
- A test scenario lists all the test cases and also describes how they are to carried out on the computer systems.
- The description of the test cases is provided in the form of a formal directive.
- Test scenario is an effective way to describe test cases.
- For distributed testing we use distributed directives and for non distributed testing we use remote directives.

What is Test Driven Development Methodology and its benefits?

Test-driven Development(TDD)is a method of developing software that adopts a test
first approach plus refactoring. The test-first approach is a programming technique, involving analysis, design, coding and testing all together. Refactoring is used to improve the design of the code without changing the observable behavior of the software, it changes the internal structure. It is cleaning up the bad code. In test-driven development, each new test provides feedback into the design process which helps in driving design decisions.

WHAT ARE DIFFERENT BENEFITS OF TEST DRIVEN DEVELOPMENT

- It allows simple incremental development.
- It involves a simpler development process.
- It provides constant regression testing.
- It improves communication.
- It improves understanding of required software behavior.
- It centralizes knowledge.
- It improves software design.

WHAT ARE DIFFERENT STEPS IN TEST DRIVEN DEVELOPMENT

- Write a test that defines how the software should behave.
- Make the test run as easily and quickly as possible.
- Clean up the code.

After integration testing, the system is tested as a whole for functionality and fitness of use. It includes performance testing, stress testing, security testing, recovery testing.
In object oriented, test requirements are derived from analysis artifacts or work products that use UML such as the class diagrams,sequence diagrams and collaboration diagrams. Use case diagram is a good source to derive test cases. They represent high functionality. Use cases are not independent. They have sequential dependencies. A scenario is an instance of a use case, or a complete path through the use case.

What are Test Case Documents and how to design good test cases?

Designing good test cases is a complex art. The complexity comes from three sources:
- Test cases help us discover information. Different types of tests are more effective for different classes of information.
- Test cases can be good in a variety of ways. No test case will be good in all of them.
- People tend to create test cases according to certain testing styles, such as domain testing or risk based testing. Good domain tests are different from good risk based tests.
A test case specifies the pretest state of the IUT and its environment, the test inputs or conditions, and the expected result. The expected result specifies what the IUT should produce from the test inputs. The specification includes messages generated by the IUT, exceptions, returned values, and resultant state of the IUT and its environment. Test cases may also specify initial and resulting conditions for other objects that constitute the IUT and its environment.

A scenario is a hypothetical story, used to help a person think through a complex problem or system.

Characteristics of Good Scenarios

A scenario test has five key characteristics:
a story that is motivating, credible, complex and easy to evaluate.The primary objective of test case design is to derive a set of tests that have the highest attitude of discovering defects in the software. Test cases are designed based on the analysis of requirements. use cases, and technical specifications, and they should be developed in parallel with the software development effort.

A test case describes a set of actions to be performed and the results that are expected. A test case should target specific functionality or aim to exercise a valid path through a use case. This should include invalid user actions and illegal inputs that are not necessarily listed in the use case. A test case is described depends on several factors, e.g. the number of test cases, the frequency with which they change, the level of automation employed, the skill of the testers, the selected testing methodology, staff turnover, and risk.

How much testing is relevant in an agile scenario?

Testing is as relevant in an agile scenario if not more than a traditional software development scenario. Testing is the headlight of the agile project showing where the project is standing now and the direction it is headed. Testing provides the required and relevant information to the teams to take informed and precise decisions. The testers in agile frameworks get involved in much more than finding software bugs, anything that can bug the potential user is a issue for them but testers do not make the final call, it is the entire team that discusses over it and takes a decision over a potential issues.
A firm belief of agile practitioners is that any testing approach does not assure quality, it's the team that does or does not do it, so there is a heavy emphasis on the skill and attitude of the people involved.
Agile testing is not a game of gotcha, it's about finding ways to set goals rather than focus on mistakes.
Among the agile methodologies, XP i.e. Extreme Programming components are:
- Test First Programming
- Pair Programming
- Short iterations and release.
- Re-factoring
- User Stories
- Acceptance Testing

Test-First Programming

- Developers write unit tests before coding. It has been noted that this kind of approach motivates the coding, speeds coding and also improves design results in better designs.
- It supports a practice called re-factoring.
- Agile practitioners prefer tests to text for describing system behavior. Tests are more precise than human language and they are also a lot more likely to be updated when the design changes. How many times have you seen design documents that no longer accurately described the current workings of the software? Out-of-date design documents look pretty much like up-to-date documents. Out-of-date tests fall.
- Many open source tools like xUnit have been developed to support this methodology.

Understanding Scenario Based Testing

Scenario based tests (SBT) are best suited when you need to tests need to concentrate on the functionality of the application than anything else.
Suppose, you are testing an application which is quite old and it is a banking application. This application has been built based on the requirements of the organization for various banking purposes. Now, this application will have continuous upgrades in the working.
Let us assume that the application is undergoing only functional changes and not the user interface changes. The test cases should be updated for every release. Over a period of time, maintaining the test ware becomes a major set back. The Scenario based tests would help you there.
As per the requirements, the base functionality is stable and there are no user interface changes. There are only changes with respect to the business functionality. As per the requirements and the situation, it is clearly understood that only regression tests need to be run continuously as a part of testing phase. Over a period of time, the individual test cases would become difficult to manage. This is the situation where we use scenarios for testing.
To derive scenarios, the following can be used as a basis:
- From the requirements, list out all the functionality of the application.
- Using a graph notation, draw depictions of various transactions which pass through various functionality of the application.
- Convert these depictions into scenarios.
- Run the scenarios when performing the testing.

Scenario based tests are not only for legacy application testing, but for any application which requires you to concentrate more on the functional requirements. I f you can plan out a perfect test strategy, then the scenario based tests can be used for any application testing for any requirements. Scenario based tests will be a good choice with a combination of various test types and techniques when you are testing projects which adopt UML (Unified Modeling Language) based development strategies.

What are different terminologies used in LoadRunner ?

Application performance testing requirements are divided into scenarios using LoadRunner.
- A scenario defines the events that occur during each testing sessions.
- A scenario defines and controls the number of users to emulate, the actions that they perform, and the machines on which they run their emulations.

Vusers
LoadRunner works by creating virtual users who take the place of real users operating client software. LoadRunner works by creating virtual users who take the place of real users operating client software. Vusers emulate the actions of human users working with your application. A scenario can contain tens, hundreds, or even thousands of
Vusers.

Vusers Scripts
The actions that a Vuser performs during the scenario are described in a
Vuser script. When you run a scenario, each Vuser executes a Vuser script. Vuser scripts include functions that measure and record the performance of the server during the scenario.

Transactions
Transactions are defined to measure the performance of the server. Transactions measure the time that it takes for the server to respond to tasks submitted by Vusers.

Rendezvous Points
Rendezvous points are inserted into Vuser scripts to emulate heavy user load on the server. Rendezvous points instruct multiple Vusers to perform tasks at exactly the same time.

Controller
LoadRunner Controller is used to manage and maintain your scenarios. Using the Controller, you control all the Vusers in a scenario from a single workstation.

Hosts
The LoadRunner Controller distributes each Vuser in the scenario to a host when the scenario is executed. The host is the machine that executes the Vuser script, enabling the Vuser to emulate the actions of a human user.

Performance Analysis
Vuser scripts include functions that measure and record system performance during load-testing sessions. During a scenario run, you can monitor the network and server resources. Following a scenario run, you can view performance analysis data in reports and graphs.

Scenarios in which to use automated testing - when to use automated versus manual testing

In a previous post (What is Automated Testing), I had talked about what is automated testing, and what are the benefits of automated testings. In this post, I will explain about some of the scenarios in which the use of automated testing is more likely (and beneficial). This is also part of the discussion about where to use manual testing versus where to use automated testing.
- There is a tremendous benefit to using automated software where the work is repetitive and follow a certain pre-defined set of cases. Consider the case where a team is working on a set of features for a new version of the product. The team needs to also ensure that earlier released features continue to work and are not impacted by any changes in code in other parts of the application. This can be done through the process of automating the test cases for these features, so that the testing happens automatically every time a new build is released.
- When a large number of test cases have to be carried out in a fixed time frame, then these can be done through automation. Automation will typically execute a large number of test cases in a much shorter time frame than manual testing.
- Allows load testing. When you need to simulate situations where there are a large number of users load testing the system, automation can help to a great deal in such cases. For applications that are seeking to evaluate their performance under stress conditions, automation is necessary.
- Testing with a large number of different inputs. Suppose there is a software that takes inputs from the user, and produces an output. In such cases, the testing for the input cases would need to consider a range of different inputs, including acceptable values and a range of unacceptable values. These are done far better through automation testing, since doing this manually involves a lot of effort.
- Automated testing can be tweaked to do a number of tests that would be difficult for people otherwise. For example, during the course of a normal product development cycle, thousands of files can be touched, and it is impossible to test all the changes on a regular basis, especially for stuff such as security errors (and there can be many types of checks that the development team would want to do)
And there can be more benefits, would love to hear more from people who use automated testing ..