What is meant by peer review in software testing?

In the process of software development at every stage the software system or application under development has to be reviewed be it before its completion or after its completion. Many techniques have been designed to carry out an effective review of the software systems or applications.

What is Peer Review in Software Testing?

- Peer review is one such technique employed in software testing.

- Peer review is a type of software review technique which involves the examination of the software product or code or documentation by its author or developer as well as by his/ her colleagues for the evaluation of the quality of the product and its technical content.

- The number of peers for carrying out a peer review may vary from one to any number the author wishes.

- The process of peer review is aimed at providing an engineering practice which is well disciplined for the detection as well as the correction of the bugs and errors in the software product.

- It is also aimed at the prevention of the defects from leaking in to the operational field.

- These aims of the peer review have been defined on the basis of the CMM or capability maturity model.

- Peer reviews form an essential part of the software development cycle and proves to be very helpful in the detection of the bugs in the early stages of the software development.

- A requirements problem as identified by the peer review during the requirements testing is a lot easier and cheaper to rectify than what it would have been in the development stages of the software testing and architecture.

Now the question comes “how the peer review is different from the other kinds of reviews?”

- Peer reviews are somewhat different from their counterpart of management reviews.

- The management reviews are performed by the management representatives rather than by colleagues as in peer review.

- The management reviews are focussed up on the control and management purposes and not on the technical evaluation like the peer review.

- Peer reviews are also a way apart from the software audit reviews in the way that the software audit reviews are carried out by the personnel external for the evaluation of the level of compliance with the standards, specifications and agreements etc.

- Several formal and informal approaches have been designed for carrying out the peer reviews.

- Buddy checking is one informal approach.

- Some formal approaches are:

1. Technical peer reviews
2. Walk throughs and
3. Software inspections.

- The roles, processes and structures for the above mentioned formal processes are governed by the IEEE standards.

- Management representatives are chosen to carry out a peer review only very rarely when specific technical expertise is required or when the documentation to be reviewed is of management level.

- The third formal approach i.e., software inspection involves the assignment of specific roles to the participants, quantification of the stages by defining the entry and exit criteria, and capturing of the software metrics.

- The peer review carried out for the open sources is commonly known as open source review.

Advantages of Peer Reviews

- Peer reviews have always been observed as a powerful methodology for a great improvement in the software quality.

- A peer review can be performed in many forms like an inspection, team review, pair programming, peer desk check or pass around etc.

Which approach is to be followed is decided on the basis of the rigor, cost, spectrum of formality and of course effectiveness!

The cheapest review method should be selected that is more effective in reducing the project associated risks. But, the inspections are always the best approach and are fit for both high level and low level risk projects.

Defect Driven Exploratory Testing - Formalized Approach for Exploratory Testing

Defect driven exploratory testing is another formalized approach used for exploratory testing. It is a goal oriented approach focused on the critical areas identified on the defect analysis study based on procedural testing results.
In procedural testing, the tester executes readily available test cases, which are written based the requirement specifications. Although the test cases are executed completely, defects were found in the software while doing exploratory testing by just wandering through the product blindly. A reliable basis was needed for exploring the software. Thus, defect driven exploratory testing is an idea of exploring that part of the product based on the results obtained during procedural testing. After analyzing the defects found during defect driven exploratory testing process, it was found that these were the most critical bugs, which were camouflaged in the software and which if present could have made the software not fit for use.

There are some pre-requisites for defect driven exploratory testing:
- In-depth knowledge of the product.
- Procedural testing has to be carried out.
- Defect analysis based on scripted tests.

Advantages of defect driven exploratory testing:
- Tester has clear clues on the areas to be explored.
- Goal oriented approach, hence better results.
- No wastage of time.

What are some of the formal approaches used for exploratory testing? Continued...

Approaches that are used for exploratory testing are:
- Record Failures
In exploratory testing, testing is done without having any documented test cases. If a bug is found, it is very difficult for us to test it after fix. This is because there are no documented steps too navigate to that particular scenario. Hence, we need to keep the track of the flow required to reach where a bug has been found. So while testing, it is important that at least the bugs that have been discovered are documented. By recording failures, we are able to keep track of work that has been done. This would also help even if the tester who was actually doing exploratory testing is not available. Since the document can be referred and list all the bugs that have been reported as well the flows for the same can be identified.

- Document issues and questions
The tester trying to test an application using exploratory testing methodology should feel comfortable to test. Hence, it is advisable that the tester navigates through the application once and notes any ambiguities or queries he might feel. He can even get the clarification on the work-flows he is not comfortable. Hence by documenting all the issues and questions that have been found while scanning or navigating the application can help the tester have testing done without any loss in time.

- Decompose the main task into smaller tasks. The smaller ones to still smaller activities
It is always easier to work with the smaller tasks when compared to large tasks. This is very useful in performing exploratory testing because lack of test cases might lead us to different routes. By having a smaller task, the scope as well as the boundary are confined which will help the tester to focus on his tetsing and plan accordingly.
If a big task is taken up for testing, as we explore the system, we might get deviated from our main goal or task. It might be hard to define boundaries if the application is a new one. With smaller tasks, the goal is known and hence the focus and the effort required can be properly planned.

What are some of the formal approaches used for exploratory testing? Continued...

Some of the formal approaches used for exploratory testing are:

- Identify the break points
Break points are the situations where the system starts behaving abnormally. It does not give the output it is supposed to give. So, by identifying such situations also, testing can be done. Use boundary values or invariance for finding the break points of the application. In most of the cases, it is observed that system would work for normal inputs and outputs. Try to give input that might be the ideal situation or the worse situation. By trying to identify the extreme conditions or the breakpoints would help the tester to uncover the hidden bugs. Such cases might not be covered in the normal scripted testing. hence, this helps in finding the bugs which might not be covered in normal testing.

- Check the UI against Windows Interface etc standards
The exploratory testing can be performed by identifying the user interface standards. There are set standards laid down for the user interfaces that need to be developed. These user standards are nothing but the look and feel aspects of the interfaces, the user interacts with. The user should be comfortable with any of the screens that he or she is working on. These aspects help the end user to accept the system faster. By identifying the user standards, define an approach to test because the application developed should be user friendly for the user's usage.

- Identify expected results
The tester should know what he is testing for and expected output for the given input. Until and unless, the aim of the testing is not known, there is no use of the testing that is done because the tester may not succeed in distinguishing the real error and normal work-flow. The tester needs to analyze what is the expected output for the scenario he is testing.

- Identify the interfaces with other interfaces/external applications
In the age of component development and maximum re-usability, developers try to pick up the already developed components and integrate them. In some cases, it would help the tester explore the areas where the components are coupled. The output of one component should be correctly sent to other component. Hence, such scenarios or work-flows need to be identified and explored more. There may be external interfaces, like the application is integrated with another application for the data. In such cases, focus should be more on the interface between the two applications.

What are some of the formal approaches used for exploratory testing?

Some of the formal approaches used for exploratory testing are:

- Identify the domain
The exploratory testing can be performed by identifying the application domain. If the tester has good knowledge of domain, the it would be easier to test the system without having any test cases. If the tester were well aware of the domain, it would help analyzing the system faster and better. His knowledge would help in identifying the various workflows that usually exist in the domain. He would also be able to decide what are the different scenarios and which are most critical for that system. Hence, he can focus his testing depending on the scenarios required. If a QA lead is trying to assign the tester to a task, it is advisable that the tester identifies the person who has the domain knowledge of that testing for exploratory testing.

- Identify the purpose
Another approach to exploratory testing is by identifying the purpose of the system i.e. What is that system used for. Thus, by identifying the primary and secondary functions for the system, testing can be done where more focus and effort can be given to primary functions as as compared to secondary functions.

- Identify the workflows
Identifying the workflows for testing any system without any scripted test cases can be considered as one of the best approaches used. The workflows are nothing but a visual representation of the scenarios as the system would behave for any given input. The workflows can be simple flow charts or data flow diagrams or something like state diagrams, use cases, models etc. The workflows will also help to identify the scope for that scenario. The workflows would help the tester to keep track of the scenarios for testing. It is suggested that the tester navigates through the application before he starts exploring. It helps the tester in identifying the various possible workflows and issues any found which he is comfortable can be discussed with the concerned team.