What are some of the formal approaches used for exploratory testing? Continued...

Charter states the goal and the tactics to be used.

A charter can be simple one to more descriptive giving the strategies and outlines for the testing process.
Charter summary contains
- Architectural the charters i.e. test planning.
- Brief information or guidelines on:
1.)Mission: Why do we test this?
2.)What should be tested?
3.)How to test?
4.)What problems to look for?
5.)tools to use
6.)Specific test techniques or tactics to use
7.)What risks are involved?
8.)Documents to examine
9.)Desired output from testing

Session Based Test Management(SBTM)

Session based test management is a formalized approach that uses the concept of charters and the sessions for performing the exploratory testing. A session is not a test case or bug report. It is the reviewable product produced by chartered and uninterrupted test effort. A session can last from 60 to 90 minutes but there is no hard and fast rule on the time spent for testing. If a session lasts closer to 45 minutes, we call it a short session. If it lasts closer to two hours, we call it a long session. Each session designed depends on the tester and the charter. After the session is completed, each session is de-briefed. The primary objective in the de-briefing is to understand and accept the session report, provide feedback and coaching to the tester. The de-briefings should help the manager to plan the sessions in future and also to estimate the time required for testing the similar functionality.
The de-briefing session is based on agenda called PROOF.
Past: What happened during the session?
Results: What was achieved during the session?
Outlook: What still needs to be done?
Obstacles: What got in the way of good testing?
Feeling: How does the tester feel about all this?

A session can be broadly classified into three tasks:
- Session test up: Time required in setting up the application under test.
- Test Design and execution: time required scanning the product and test.
- Bug investigation and reporting: time required finding the bugs and reporting to the concerned.

The entire session report consists of session charter, tester name, data and time started, task breakdown, data files, test notes, issues, bugs.

What are some of the formal approaches used for exploratory testing? Continued...

Approaches that are used for exploratory testing are:
- Record Failures
In exploratory testing, testing is done without having any documented test cases. If a bug is found, it is very difficult for us to test it after fix. This is because there are no documented steps too navigate to that particular scenario. Hence, we need to keep the track of the flow required to reach where a bug has been found. So while testing, it is important that at least the bugs that have been discovered are documented. By recording failures, we are able to keep track of work that has been done. This would also help even if the tester who was actually doing exploratory testing is not available. Since the document can be referred and list all the bugs that have been reported as well the flows for the same can be identified.

- Document issues and questions
The tester trying to test an application using exploratory testing methodology should feel comfortable to test. Hence, it is advisable that the tester navigates through the application once and notes any ambiguities or queries he might feel. He can even get the clarification on the work-flows he is not comfortable. Hence by documenting all the issues and questions that have been found while scanning or navigating the application can help the tester have testing done without any loss in time.

- Decompose the main task into smaller tasks. The smaller ones to still smaller activities
It is always easier to work with the smaller tasks when compared to large tasks. This is very useful in performing exploratory testing because lack of test cases might lead us to different routes. By having a smaller task, the scope as well as the boundary are confined which will help the tester to focus on his tetsing and plan accordingly.
If a big task is taken up for testing, as we explore the system, we might get deviated from our main goal or task. It might be hard to define boundaries if the application is a new one. With smaller tasks, the goal is known and hence the focus and the effort required can be properly planned.

What are some of the formal approaches used for exploratory testing? Continued...

Some of the formal approaches used for exploratory testing are:

- Identify the break points
Break points are the situations where the system starts behaving abnormally. It does not give the output it is supposed to give. So, by identifying such situations also, testing can be done. Use boundary values or invariance for finding the break points of the application. In most of the cases, it is observed that system would work for normal inputs and outputs. Try to give input that might be the ideal situation or the worse situation. By trying to identify the extreme conditions or the breakpoints would help the tester to uncover the hidden bugs. Such cases might not be covered in the normal scripted testing. hence, this helps in finding the bugs which might not be covered in normal testing.

- Check the UI against Windows Interface etc standards
The exploratory testing can be performed by identifying the user interface standards. There are set standards laid down for the user interfaces that need to be developed. These user standards are nothing but the look and feel aspects of the interfaces, the user interacts with. The user should be comfortable with any of the screens that he or she is working on. These aspects help the end user to accept the system faster. By identifying the user standards, define an approach to test because the application developed should be user friendly for the user's usage.

- Identify expected results
The tester should know what he is testing for and expected output for the given input. Until and unless, the aim of the testing is not known, there is no use of the testing that is done because the tester may not succeed in distinguishing the real error and normal work-flow. The tester needs to analyze what is the expected output for the scenario he is testing.

- Identify the interfaces with other interfaces/external applications
In the age of component development and maximum re-usability, developers try to pick up the already developed components and integrate them. In some cases, it would help the tester explore the areas where the components are coupled. The output of one component should be correctly sent to other component. Hence, such scenarios or work-flows need to be identified and explored more. There may be external interfaces, like the application is integrated with another application for the data. In such cases, focus should be more on the interface between the two applications.

What are some of the formal approaches used for exploratory testing?

Some of the formal approaches used for exploratory testing are:

- Identify the domain
The exploratory testing can be performed by identifying the application domain. If the tester has good knowledge of domain, the it would be easier to test the system without having any test cases. If the tester were well aware of the domain, it would help analyzing the system faster and better. His knowledge would help in identifying the various workflows that usually exist in the domain. He would also be able to decide what are the different scenarios and which are most critical for that system. Hence, he can focus his testing depending on the scenarios required. If a QA lead is trying to assign the tester to a task, it is advisable that the tester identifies the person who has the domain knowledge of that testing for exploratory testing.

- Identify the purpose
Another approach to exploratory testing is by identifying the purpose of the system i.e. What is that system used for. Thus, by identifying the primary and secondary functions for the system, testing can be done where more focus and effort can be given to primary functions as as compared to secondary functions.

- Identify the workflows
Identifying the workflows for testing any system without any scripted test cases can be considered as one of the best approaches used. The workflows are nothing but a visual representation of the scenarios as the system would behave for any given input. The workflows can be simple flow charts or data flow diagrams or something like state diagrams, use cases, models etc. The workflows will also help to identify the scope for that scenario. The workflows would help the tester to keep track of the scenarios for testing. It is suggested that the tester navigates through the application before he starts exploring. It helps the tester in identifying the various possible workflows and issues any found which he is comfortable can be discussed with the concerned team.

Introduction to Formalized Exploratory Testing ....

In normal testing style, the test process is planned well in advance before the actual testing begins. In this, the test design is separated from the test execution phase. Many times the test design and test execution is entrusted on different persons. Exploratory testing should not be confused with the dictionary meaning of ad hoc. Ad hoc testing normally refers to a process of improvised, impromptu bug searching.

Formalized Exploratory Testing

A structured and reasoned approach to exploratory testing is termed as formalized exploratory testing. This approach consists of specific tasks, objectives, and deliverables that make it a systematic process. Using the systematic approach, an outline of what to attack first, its scope, the time required to be spent etc is achieved. The approach might be using simple notes to more descriptive charters to some vague scripts. By using the systematic approach, the testing can be more organized focusing at the goal to be reached. Thus, solving the problem where the pure exploratory testing might drift away from the goal.

The formalized approach used for the exploratory testing can vary depending on the various criteria like resource, time, the knowledge of the application available etc. Depending o this criteria, the approach used to attack the system will also vary. It may involve creating the outlines on the notepad to more sophisticated way by using charters etc.

Exploratory Testing - Introduction and Overview

Exploratory testing is an interactive process of concurrent product exploration, test design and test execution. This testing is defined as simultaneous test design, test execution and bug reporting. In this approach, the tester explores the system without having any prior test cases or test scripts. It is also called as ad hoc testing, gorilla testing or intuitive testing. In operational terms, exploratory testing is an interactive process of concurrent product exploration, test design, and test execution. The outcome of an exploratory testing session is a set of notes about the product, failures found, and a concise record of how the product was tested. When practiced by trained testers, it yields consistently valuable and audit-able results. Every tester performs this type of testing at one point or the other. This testing totally depends on the skill and creativity of the tester. Different testers can explore the system in different ways depending on their skills. Thus, the tester has a very vital role to play in exploratory testing.

A systematic approach of exploratory testing can also be used where there is a plan to attack the system under test. This systematic approach of exploring the system is termed as formalized exploratory testing. It is a powerful approach but it has not got recognition and is often misunderstood and not gained the respect it needs. In many situations, it can be more productive than the scripted testing.
This testing believes in concurrent phases of product exploration, test design and test execution. It is categorized under black box testing. It is basically a free-style testing approach where you do not begin with the usual procedures of elaborate test plans and test steps. The test plan and strategy is very well in the tester's mind. The tester asks the right question and judges the outcome. During this phase, the tester is actually learning the product as he tests it. It is interactive and creative. A conscious plan by the tester gives good results.

Validation Phase - Beta Testing - Objectives

The beta testing is conducted at one or more customer sites by the end-user of the software. The beta test is a live application of the software in an environment that cannot be controlled by the developer. The software reaches beta stage when most of the functionalities are operating. The software is tested in customer's environment, giving the user an opportunity to exercise the software, find the errors so that they could be fixed before product release. Beta testing is a detailed testing and needs to cover all the functionalities of the product and also the dependent functionality testing. It also involves the user interface testing and documentation testing. Hence, it is essential that this is planned well and the task accomplished. The test plan document has to be prepared before the testing phase is started, which clearly lays down the objectives, scope of test, tasks to be performed and the test matrix which depicts the schedule of testing.

The objectives of beta testing is to:
- evaluate software technical content.
- evaluate software ease of use.
- evaluate user documentation draft.
- identify errors.
- report errors/findings.

The role of a test lead is to provide test instruction sheet that describes items such as testing objectives, steps to follow, data to enter, functions to invoke and to provide feedback forms and comments.
The role of a tester is to understand the software requirements and the testing objectives and carry out the test cases and report defects.