What is meant by Statistical Usage Testing?

Statistical usage testing is the testing process that is aimed at the fitness of the software system or application.

The test cases chosen for carrying out statistical usage testing mostly consist of the usage scenarios and so the testing has been named as statistical usage testing. Software quality is ensured by the extensive testing but that has to be quite efficient. Testing expenditures covers about 20 – 25 percent of the overall cost of the software project. In order to reduce the testing efforts, deploy the available testing tools since they can create automated tests. But usually what happens is that the important tests require manual intervention with the tester requiring thinking about the usage as well behavior of the software. This is just the repetition of the tasks that were done during the requirements analysis phase.

About Statistical Usage Testing

- A usage model forms the basis for the creation of tests in statistical usage testing.

- Usage model is actually a directed usage graph more like a state machine and it consists of various states and transitions. 

- Every transition state has a probability associated with it regarding the traversal of the transition when the system would be in a state that marks the beginning of the transition arc. 

- Therefore, the sum of the probabilities of outgoing transitions sum up to unity for every state.

- Every transition can be associated with an event and more with parameters that are known to trigger the particular transition. 

- Such event associated transitions can be further related to certain conditions called the guard conditions. 

- These conditions imply that the transition occurs only if the value of the event parameter satisfies the condition.

- For assigning probabilities to the transitions, 3 approaches have been defined as follows:

1. Uninformed approach: In this approach, same probability is assigned to the exit arcs of a state.
2. Informed approach: In this approach, a sample of user event sequences for calculating suitable properties. The sample is captured from either an earlier version of the software or its prototype.
3. Intended approach: This approach is used for shifting the focus of the test to certain state transitions and for modeling the hypothetical users.

- According to a property termed as the marcov property, the actual state is what on which the transition probabilities are dependent. 

- However, they are independent of the history again by the property. 

- This implies that the probabilities must be fixed numbers. 

- A system based up on this property is termed as a marcov chain and it requires conclusion of some analytical descriptions. 

- Usage distribution is one among such descriptions. 

- It gives for every state its steady–state probability i.e., appearance rate that is expected.

- All the states are associated to one or the other part of the software system or application and the part of the software that attracts more attention from the tests is shown by the usage distribution. 

- Some other important descriptions are:

1. Expected test case length
2. Number of test cases required for the verification of the desired reliability of the software system or application.

- The idea of the usage model generation can be extended by handling guard conditions and enabling the non–deterministic behavior of the system depending on the state of the system’s data. 

- All this helps towards the application of the statistical usage testing to systems over a wide range. 

- The use cases are defined by the top–level structure of the unified modeling language (UML). 

How does QTP recognize Objects in AUT?

The quick test professional comes with two types of object identification mechanisms unlike winrunner as mentioned below:

1. Usual or the normal object identification mechanism and
2. The smart identification mechanism

What is Usual or Normal Object Identification Mechanism

- In the usual object identification routine, the first step of the quick test professional is to learn the description of the object as provided by the user before the starting of the test. 

- The description of the object provided by the user consists of the properties of the object. 

- All the objects present in the software system or AUT (application under test) are matched one by one with this physical description.

- It is checked that how many properties of the object are matching with the properties mentioned in the description. 

- This method is the easiest  of the two above mentioned object identification routines. 

Now what if this usual object identification mechanism fails to identify the object? 

The alternative here is the second object identification mechanism i.e., the smart identification mechanism. 

Why does a the normal method fails?

- The normal method fails because the value of the object properties starts changing dynamically which makes it difficult for the quick test professional to track the object. 

- Another situation in which the normal identification mechanism can fail is when the quick test professional finds not one but more than one objects in the application under test matching with the properties mentioned in the description. 

- In such a case the quick test professional erases from its memory the learnt description of the object and calls up on the smart identification mechanism for the identification of that particular object. 

Let us see a comparison between the smart identification mechanism and the normal identification mechanism:

1. Smart identification is more complex then the usual one.
2. Smart identification is more flexible then the usual one.

What is Smart Identification Mechanism

- Smart identification mechanism is so reliable that it can work even if the currently provided description of the object fails.

- To get the best out of the smart identification mechanism, one needs to configure it properly in a logical way. 

- Smart identification mechanism is driven by two different sets of properties as described below:

1. Base filter properties: As the name suggests these are the base or the fundamental properties belonging to a particular test object class. The value of these fundamental properties cannot be changed until and unless you make changes in the properties of the original object.
2. Optional filter properties: The leftover properties i.e., all the properties leaving out the base filter properties are grouped under this category i.e., the optional filter properties. These properties unlike the other base filter properties do not change frequently. These properties can some times be ignored i.e., when they do not hold to be applicable and therefore have been named as optional filter properties.

How the smart identification process follows?

- The description given by the user is erased from the memory of the quick test professional and a list of objects called candidate list is created.

- Objects or candidates in this list match at least one property in the property list. 

- Now, the base filter properties are used for cutting down on the list of the object candidates. 

- The list is narrowed down to only one object which has most number of properties matching in the list of properties. 

- Some times it may happen even after reaching this stage the quick test professional may not find a matching object. 

- In such a case the quick test pro makes use of an ordinal identifier in addition with the learnt description. 

What is smart Identification in QTP?

Smart identification mechanism is one of the most important and effective mechanisms of the quick test professional. Usually for the identification of an object the quick test professional goes around the usual i.e., the normal identification process.

But what the quick test professional is supposed to do when this usual object identification routine fails? At this point the smart identification mechanism of the quick test professional comes to the rescue of the testers.

Smart Identification in QTP

- The smart identification mechanism has got many positive attributes.

- It is more flexible and works with more efficiency for the identification of the difficult objects present in the application that cannot be found with the normal identification mechanism. 

- But for the smart identification to work, it is required that you enable this option in the object identification settings. 

- Smart identification mechanism works up on two types of properties as mentioned below:

1. Base filter properties and
2. Optional filter properties.

- The first category of the properties i.e., the base filter properties constitute of the most fundamental properties pertaining to a particular class of test object. 
- The values of the properties falling in this category cannot be changed or altered without changing the same properties in the original object.
- The second category of the properties i.e., the optional filter properties consists of the other properties that contribute in the identification process of the objects. 
- In the process of smart identification of an object, the quick test professional first erases from its memory the learnt description or the description that was entered by the user in to the physical description field of the object. 
- Instead of this single object, a list of matching objects (in this case such objects are called candidates) is created by the quick test professional called the candidate list. 
- This list is created based up on the properties that have been defined in the category of base filter properties and thus all the objects match a little or lot (i.e., to say one or more properties) with the base filter properties. 
- Base filter properties as the name suggests are the major properties that help cut down or reduce the number of objects in the candidate so that an exact match can be found or we can say that the area of search is reduced. 
- The idea is to be left with only one object matching with most or all of the properties mentioned in the saved description. 
- In some cases it may happen that it may be required to invoke smart identification mechanism during the run session. 
- In such cases a warning message is generated by the test results tree which indicates about the invocation of the smart identification and insertion of a smart identification process. 
- It is said that the smart identification is applicable only for the web based applications. 
- The object is recorded from the AUT or application under testing and its properties are identified accordingly and finally the scripts are executed. 
- Some times you may receive a warning alert in the results of the tests. 
- You just need to navigate over this message in the file which stores the messages and read what it says and follow accordingly. 
- Usually the normal identification routine fails whenever there occurs a dynamic change in the properties of the object to be identified. 
- Because of such dynamic changes the value of the object properties keep on changing thus making it difficult for the quick test professional to track that particular object. 
- These dynamic changes are shown only in the result and are not stored in the local object repository and are accessible only for the run  time. 

How does QTP identifies objects?

Just like winrunner the process for identification of objects is same for quick test professional. In Winrunner, it is required that the winrunner learns about all the objects before it begins identifying them. After learning about the entire GUI objects present in the software system or application the quick test professional begins its search regarding the object or objects that seem to be matching with the properties as stated by the user in the description of the objects.

In many of the cases the description of the objects proves to be quite a useful and simplest method for the identification of the GUI objects residing in the software system or application. This mechanism of quick test professional of searching the objects is bound to work until and unless the properties of the GUI objects are changed. In some cases it happens that the quick test professional encounters two situations;

1. It does not proves successful in the identification of the object that is suppose to match with the description given by the user or
2. It finds more than two objects having the same description as stated by the user.

In the above two situations the quick test professional adopts another mechanism and rejects the usual way of object identification. This new mechanism is quite advance then the usual one and is thus known as the smart identification mechanism.

Smart Identification Mechanism for objects in QTP

- This smart identification mechanism being smarter than the normal mechanism is also more complex than that. 

- Even though it is more complex it is more flexible also.

- If the smart identification mechanism is configured according to proper logic then there are possibilities that the quick test professional may be able to identify the GUI objects even if the description provided by the user fails. 

- There are 2 basic properties that are unique to the smart identification mechanism of the quick test professional:

1. Basic filter properties: These properties represent the basic fundamental properties of the test object class. The values represented by these properties cannot be changed or replaced without making proper changes to the original object values.
2. Optional filter properties: These are the properties apart from the basic filter properties which also help in the identification of the GUI objects belonging to a particular test class. Unlike the basic filter properties, the optional filter properties do not change on a regular basis. Still these properties can be ignored if they no longer hold to be applicable.

When is Smart Identification Mechanism Activated by QTP?

The smart identification mechanism is activated by the quick test professional only when its unable to identify some object and the whole process is carried out through the following steps:

1. The quick test professional erases the learnt description of that particular object from its memory and creates a new list of the object candidates that fall under that particular parent object. The objects mentioned in this candidate list are the ones whose properties have a match with the properties in the list of the base filter properties.
2. The object in the candidates list is matched one by one with the properties listed in the base filter property list and if the no match is found that particular object candidate is simply rejected. The leftover object candidates in the list constitute a new candidate list.
3. When the list gets empty for one property, the previous candidate list is again adopted by the quick test professional and is considered for matching with the second property.
4. If only one object remains in the candidate list then this object is considered to be the required one. 

What are the different tips for writing test cases?

What is the foremost thing that should be taken care of to carry out effective testing? Its test cases of course! Writing test cases without thinking is not going to help because any amount of efforts and time you waste on them, you are not going to get effective results. Writing test cases forms a major component of any software testing methodology.

SOME IMPORTANT FACTS

1. Any change in the requirements tends to affect the concerned test case as well. Since the test cases are based upon the requirements, any change in requirements is reflected in the test cases. If you make any changes to the requirements, make sure you have altered the related test case also accordingly.

2. After you finish writing your test cases, revise them twice so that any error that you didn’t notice can be found out.

3. It rarely happens that all the written test cases are executed. So keep the number of test cases limited i.e., as many as you can execute. If you have less number of test cases, you will be able to focus on them properly.

4. Some test cases demand other test cases as their prerequisites. If that is the case, then you should go through the list of test cases and decide a sequence for executing the test cases.

5. Some test cases are dependent on each other and so it becomes difficult which one to execute first. So try to keep your test cases as much independent as possible.

Now keeping above points in mind, we define some tips for writing effective test cases that if followed will certainly reap you the expected results:

1. Keep your test cases as simple as possible. i.e., keep them to the point and precise. You should be able to integrate your test cases with the test plan.

2. Document your test cases.

3. Review all your test cases at least twice.

4. Include input test data along with the test cases because if the tester is to choose test data by himself/ herself, he/ she may omit the most important test data making the whole testing go in vain.

5. While writing test cases always keep in mind that the end user is the one who will use the AUT.

6. Pay due attention while writing the test cases because this will affect the whole testing process.

7. For every condition or assertion in the program, write a different test case so that you don’t mix up.

8. Make sure that your each test case tests only one aspect of the program. Otherwise you will have problems like over lapping and over complicated tests. If one test case focuses upon one aspect, then the code will get good coverage and tracking any error will be easy.

9. Prepare documentation for every test case. Your documentation should have assertions and description of 4 stages (set up, execution, verification, and output) of the test case.

10. Select a basis on which you want to organize your test cases like functionality or class, so that it becomes easy for you to add new test cases later. The order of your test cases should be consistent.

11. The test cases should be able to execute with a good speed, should be independent as discussed above, should be small and precise so that they can be repeated without much hassle is needed to and should be transparent. By transparency I mean that the test case should clearly state its purpose.

12. Your test cases should be reusable.

What is a bug and bug life cycle? What are guidelines for deciding severity of bugs?

A bug is defined as a defect or some abnormal behavior of software. Testing plays an important part in the removal of bug. Bug has to travel the whole bug life cycle until it is closed. The cycle includes following stages:
- New
When the bug is posted for first time and not yet approved.
- Open
When tester approves that bug is genuine.
- Assign
Bug is assigned to the developer.
- Test
After fixing the bug, it is assigned to testing team to re-test it.
- Deferred
When the bug is changed to deferred state, the bug is expected to be fixed in next releases.
- Rejected
If the developer feels that the bug is not genuine, he can reject the bug.
- Duplicate
If bug is repeated twice or two bugs gives the same concept, then one bug is labeled duplicate.
- Verified
Once the bug is fixed, it is verified that no bug is present and status is changed to verified.
- Reopened
In this stage, the bug traverses the bug cycle once again because the bug still exists.
- Closed
If the bug is fixed and does not exist, the tester changes the status to closed.

SEVERITY AND PRIORITY OF THE BUG HAS TO FOLLOW GUIDELINES:
- Critical bug prevents further testing of the product under test. No work around is possible for such bugs.
- Major bug is in which defect does not function as expected or cause other functionality to fail.
- Medium or average bug in which defects do not conform to standards and conventions.
- Minor or low bugs do not affect the functionality of the system.

To write bug description, follow these guidelines:
- Be specific.
- Use present tense.
- No unnecessary words.
- No exclamation points.
- Do not use all CAPS.
- Mention steps.

If the bug is found, what should be done ?

Test cases are written to detect if a feature of an application is working correctly. It is a document which consists of input, action and expected output. After a bug is found, the developers are informed about the bug and are asked to fix it. After it gets fixed, the module is re-tested and it is checked whether it has not created any problem.

- Information of the bug and its severity.
- Bug identifier.
- Bug status.
- Application name.
- The name of the module, function, object etc. where the bug occurred.
- Environment factors.
- Test case name and identifier.
- One line bug description.
- Full bug description.
- If the bug is not covered by test case, steps are described again.
- Names of file used in test.
- Severity level
- Can the bug be reproduced.
- Tester name and test date.
- Name of developer.
- Description of cause of the problem.
- Description of the fix.
- Date of fix.
- Application version that contains the fix.
- Description about tester.
- Date of retest.
- Results of retest.
- Requirements of regression testing.
- Tester who has done regression tests.
- Results of regression testing.

Sometimes, the software is so buggy that it becomes impossible to test it. In order to handle this type of situation, the testers should report whatever bugs they are coming across, focusing more on critical bugs. It depicts deeper problems in the software development process.