What work does the testing team do at the start of a cycle - some answers

This post arose out of a question that a new manager of the team had for us, the middle management of the team. His basic question was that in the initial part of the cycle, where there were the requirement being framed by the product manager or the product management team (depending on how large the product was), and these requirements were being broken down into smaller features and tasks for the purpose of estimation. Even after this, there was the period where development of the features was happening, and this would be done by the development team who would finally deliver the required code to the testing team, and they could start testing. So, what was the role of the testing team. For the moment, this question was not meant to figure out what to do with the excess testing team at these periods of time, instead it was meant to understand the role of the testing team across the duration of the project cycle, from the beginning to the end. And there was a lot of understanding of what happened when the testing team got the features from the development team, but not in the initial parts of the project.

We had a detailed discussion with the manager about this, and helped him understand this from the perspective of the team. The first essential part was that the testing team did not just do testing, so we should call them them the QE team. Testing was what people saw them do, but there was a lot more work that they did. We finally walked him through a list such as this one below, and if you do not some do some of these in your team, maybe you can get some tips from here. And if your team does other work, then please share these in the comments.

- The testing team is an essential part of the breakdown of the features into tasks and also with the estimation. Members of the testing have an incredible knowledge of the features and the workflows because of their testing of the features on a regular basis; this knowledge can be more than the developers and the product management. So if there is a feature discussion, the testing team can help figure out the workflows that are impacted; in one case I saw the QE member causing embarrassment to the product manager when the QE pointed out that the workflow that the product manager was advocating could be done in much less change than what the product manager was advocating. After that, the product manager made sure that the QE team was involved in all such discussions.

- There are numerous cases when the previously released product needs a patch or a dot release. In such cases, the actual development change can be very small, but the testing of that patch or dot release can take much more time, and need many testing team members for one or less development team members.

- When the team is developing automation for testing, the initial time period of the project cycle helps give them a time required for developing automation scripts for features that are already released (and the automation ensures that the testing of these features takes less actual testing time and more regular automation runs).

- When the developer is busy doing the coding, that is the time when the QE team member finishes up writing or modifying the test plans and cases.

- During the design process of the feature, typically the QE team members plays an essential role, working with the product manager, the workflow / UI designer and the developers to work out the actual details of the feature in terms of implementation. This can take many iterations if the feature is complex, and in my experience, in many cases, the QE team member has pointed out several intricacies of current implementation that need to be taken care of in the design.

What is code coverage ?

People know what software testing is, and most people in the profession can differentiate between white box and black box testing. However, when you get into more details, and look to identify how testing can provide even greater value, the benefit of measures such as code coverage become apparent. At the same time, there would be a large number of software professionals who are not even aware of what code coverage is, and what are its key benefits. So the idea of this article is to try and articulate some of the benefits.
So what is Code Coverage ?
Well, Code coverage is a type of measure used in software testing which tries to answer the questions about the degree to which the entire source code of an application has been tested. Traditional black box testing, with its focus on functional testing cannot even come close to trying to answer this question, although White Box testing does come closer to trying to answer this question. In fact, if you examine code coverage practices in detail, it is possible to say that code coverage is a form of testing that inspects the code directly and is therefore a form of white box testing.
Code coverage techniques were amongst the first techniques invented for answering the question of systematic software testing. For those proponents of the text plan / case based method of testing, code coverage testing works on the principle that it is entirely possible that sections of a software application remain untouched by test data, and as a result, then it is not possible to say, with any degree of certainty, that these sections do not contain residual errors.
How do you go ahead with actually trying to do code coverage ?
Code Coverage requires support from engineering to proceed. Why is this so ? Getting code coverage in practise requires a different procedure from the normal software build - The target application / software is configured to be built with special options / libraries and/or run under a special environment such that every function that is exercised (executed) in the program(s) is mapped back to the function points in the source code. Doing this systematic process (although requires effort and the value does not seem immediately clear to individual developed and QE) allows developers and quality assurance personnel to look for parts of a system that are rarely or never accessed under normal conditions (error handling and the like) and helps reassure test engineers that the most important conditions (function points) have been tested. Once this exercise has been done, the output is further analysed to see what areas of code have not been exercised and the tests are updated to include these areas as necessary. Once this exercise has been completed (and it may need to be done on a regular basis as the code is in the process of being developed), it gives a much higher level of confidence about the overall quality of the code.

What's the role of documentation in QA?

Sometimes, for people who are new to the software line (and specifically new to the testing business), there are questions about how important documentation would be to regular Quality work. After all, the major work is about testing some software and getting the bugs resolved, how important could testing be ? Well, the role of documentation in enabling the success of QA activities is critical. Note that documentation can be in electronic form as well, no necessity that it should be only paper, and in fact, recent trends are more towards having electronic forms that can be placed in source control areas in their specific directory locations.
QA practices should be documented such that they are repeatable, and are not dependent on individual people. Software artifacts and processes such as requirement and design specifications, design documents such as architecture, HLD, LLD, business rules, inspection reports, configuration control design and operational documents, code changes, software test plans, test cases, bug reports and decision making on important bugs, user manuals, etc. should all be documented such that these can be referred later (and prove very useful in case the project personnel are changed or should be transitioned to other teams).
As a part of documentation, there needs to be a system for easily finding and obtaining documents and determining what documentation will have a particular piece of information. Change management for documentation should be used in all cases, else you will find later that it is hard to figure out why something changed and what were the reasons behind it.
One of the most common reasons for failures or overruns / delays in a complex software project is to have poorly documented requirements specifications. Requirement specifications are the details describing an application's externally-perceived functionality and properties. The condition for requirements should be clear, complete, reasonably detailed, cohesive, attainable, and testable. A non-testable requirement would be, for example, 'user-friendly' (too subjective). A testable requirement would be something like 'user needs to enter their date of birth while creating their profile'. Determining and organizing requirements details in a useful and efficient way can be a difficult effort; different methods are available depending on the particular project.
Care should be taken to involve ALL of a project's significant 'customers' and important stakeholders in the requirements process. 'Customers' could be in-house personnel or out, and could include end-users, customer acceptance testers, customer contract officers, customer management, future software maintenance engineers, salespeople, etc. Anyone who could later derail the project if their expectations aren't met should be included if possible. This also helps in ensuring that changes later are minimized (can never be eliminated).
Organizations vary considerably in their handling of requirements specifications. Ideally, the requirements are spelled out in a document with statements such as 'The product shall.....'. 'Design' specifications should not be confused with 'requirements'; design specifications should be traceable back to the requirements.
In some organizations requirements may end up in high level project plans, functional specification documents, in design documents, or in other documents at various levels of detail. No matter what they are called, some type of documentation with detailed requirements will be needed by testers in order to properly plan and execute tests. Without such documentation, there will be no clear-cut way to determine if a software application is performing correctly.
Test plans need to be documented properly with good change control, since a test plan forms the basis of determining the areas of testing, scope, responsibilities, etc. A test plan forms the first level of generation of confidence in the test strategy for the project.