Summary of testing needs during the SDLC

There is an often asked question about when testing should occur in the overall product development cycle?

Testing is sometimes incorrectly thought as an after-the-fact activity. Instead, testing should be performed at every development stage of the product. Test data sets must be derived and their correctness and consistency should be monitored throughout the development process. Testing should accompany each phase of “Software Development Life Cycle”. If testing is isolated as a single phase late in the cycle, errors in the problem statement or design may incur exorbitant costs. If this happens, not only must the original error be corrected, but the entire structure built upon it has to be changed. Therefore, testing should be involved throughout SDLC in order to bring out a quality product.

Testing Activities In Each Phase:

1. REQUIREMENTS ANALYSIS
• Determine correctness
• Generate functional test data.

2. Design
• Determine correctness and consistency
• Generate structural and functional test data.

3. PROGRAMMING
• Determine correctness and consistency.
• Generate structural and functional test data.
• Apply test data.
• Refine test data.

4. OPERATION & MAINTENANCE
• Retest

Of course, if you want to add examples of your own to substantiate this, please do so through the comments section.

Usability testing process

Usability testing can make a lot of difference to the ultimate success of the products of the company, and needs to be well planned. If you do the test wrong, or have some of the parameters wrong, then this could have negative results for the company (in addition, you could end up letting the key designers and engineers get wrong inputs). The key to successful usability testing is careful planning and preparation and making sure that the usability test is properly executed. The following are the steps that could be used in this process:

1. Draw up a list of things to test.

* You can't test everything. Decide what are the most important tasks you want users to be able to perform for your product or site, and prioritize them as to which are the ones to be done first.

2. Draw up the scenarios that you want the users to test.

* The scenarios should be based on some of the initial customers that you had designed, and should be realistic.
* Keep the tests to be a reasonable time frame. Remember that people take a certain amount of time to get comfortable with a topic, so anything less than 10 minutes could be tricky, One the other hand, if it gets too long, it could make the subjects weary.
* Test the scenarios on yourself before you use them on your test subjects.

3. Write a script for administering the test -- this is necessary for consistency. It is particularly important if more than one person will be conducting the test, or if you cannot do the test at some point of time. Also, finalizing a script is a good way of ensuring that key people have signed off on the tests.

* Include in the script an introduction to the test, an explanation of how the test will be conducted, the questions that will be asked, and suggestions for follow-up questions to encourage the subjects to discuss what they are doing and why.
* Allow for some free format answers
* You can ask test subjects to think out loud -- a behavior that is not always natural to everyone. You may need to drill this into the subjects periodically, but it is very important - it helps find out what they are thinking.

4. Test the test. Try it on some users -- find out if the scenarios are comprehensible; if there are some problems, then you may need to rework the tests.

5. Train the testers and recorders.

* First-time testers should each practice conducting a test with volunteers or on each other - this will help them get more comfortable and gain a bit of experience.
* First-time testers should then discuss the experience of conducting the test.
* Critique each other, appreciate good things, but learn from mistakes and highlight necessary improvements to be made.
* Practice and critique the recording of subjects, comments.

6. Gather volunteers to be tested, using the various sources that you would have (forums, existing customers, etc).

* Expect to provide some form of compensation to your volunteers, it could be something as simple as a copy of the last released version of the software, or maybe some sort of company branded gift item
* Volunteers should be selected so that they are representative for the general user population. Choose a mixture of age, year in school, occupation, etc.
* People who volunteered should be scheduled and sent several follow-up messages about the time and place for their test.
* Even with reminder messages, expect some people to drop out - there may be sudden personal issues or some other such reason.

7. Make sure you have a quiet place to do the testing. It's a good idea to conduct the practice test in the actual room that you will use for the real testing and that the subjects will not have much distractions in such testing. It will not help if people keep dropping in and out of the place.

8. Conduct the tests.

* Schedule enough time between tests for the tester and the recorder to debrief before the next test begins.
* If you discover that a test question or a design element is not working, it is unwise to continue testing.

Fix what is wrong before you continue testing. Make the change and test the correction. The computer industry calls this "rapid prototyping".

9. Record the test results as soon as possible after the test is completed. If possible, stream the actual process so that interested folks can see it easily as it is going on.

10. Analyze the results and determine how to correct the design problems. Redesign based upon usability evaluation results.

Brief steps that need to be taken:
- Recruiting (one month in advance - selecting locations than dates and participants
- Travel Planning (one month in advance - in tandem with the recruiting)
- Prototype development and refinement (2-4 weeks in advance depending on the fidelity)
- Task development (2-4 weeks in tandem with the prototype)
- Approvals (2-4 weeks in advance depending on the formality of the study)

Benefits of usability testing

Why is there such a lot of focus on doing proper usability testing ? Whenever there is any discussion of doing a product and plans are drawn, doing usability testing is deemed as a critical part of the overall plan. What are the benefits of doing usability testing and what can you learn about your product from this process ? Presented below is a list of queries / outcomes / benefits, many of which may seem relevant and others not so relevant:

Are the test participants able to complete the task scenarios successfully? Do the test participants understand the various test scenarios ? If they are not able to understand or frequently ask for directions or seem confused, it implies that there are problems that need to be addressed.

Do participants perform well enough to meet the usability objectives? Supposing the participants are able to move ahead in the flow, the next question is about their performance in the study. Their level of completion, the amount of assistance they required, whether they moved in the workflow the same way that the designers intended, all these are queries that need to be assessed.

How satisfied are participants with the site or application? It is very important to gauge the level of satisfaction of the users with the site / application. Preparing a series of questions during and after the study will allow determination of the satisfaction levels of the users with the overall site / application, as well as individual elements.

What changes are needed to make sure that the site / application will enable more users to perform more successfully? A usability study is an excellent time to find out which is the more user-friendly route for users. In fact, it is very much possible that during the course of a usability study, designs could be reworked in an iterative way to find the one that seems to be the most user-friendly.

Verification of user analysis data: Designs in a lot of cases have been made on the basis of previous user analysis data. Typically, such analysis leads to a lot of data about the workflows that users seem to prefer; however, it is also known that in many cases, users state something based on questions - but these can get modified when they actually see this in operation (also known as 'users finally get to know what they want when they see it'). Hence, usability studies are a good way of finally verifying the user analysis data.

Validate design guidelines: The usability testing also helps to confirm the various design guidelines that have been established for the project. These guidelines serve to set the basis of the user workflows. An example of some of these are:
* Make links predictable: Links should have the same type of UI and should work in the same way
* Make screens as simple as possible -- first screen should have only the most frequently used functions and secondary screens should have optional interactions
* Use graphics minimally -- only when necessary, and make graphics meaningful, to indicate where a link goes or what page it deals with; and
* Make the page easy to scan -- most people scan the page, picking out individual words or sentences and do not read word-by-word
Now these are general guidelines and may not be applicable for all cases.

Testing allows designers to settle disagreements and differences: Different designers in a team could have multiple opinions and getting usability tests determines which is the one most likely to work. This happens without hurting egos if decisions are made on the basis of user study.

Usability tests can help you understand how real users will use your product, and how the interface could be improved accordingly: Teams developing the product / site can have blinkered visions; getting target segment users brings you close to reality.

Results of testing give the designer ammunition to use in responding to outside attempts to change design based on whims or personal bias: Higher management or experts can have their own opinions and may want a workflow to happen in a certain way, and getting usability results helps defeat such attempts if they are based on personal whims.

Testing saves time and money. Rubin notes that "doing usability testing through the development project allows you to make user-centered changes as you go, before you have so much time and money invested in the project that it makes it difficult to make changes.

Testing is good public relations: It sends a message that the team/company is interested in what the user thinks and needs and can be used very easily for good coverage and to show that the company has done a best effort.

Testing allows the designer to cast a broad net: It helps ensure that the design is good for all users, not just those users who think like you do. Getting a cross section that is diverse is much easier if you are doing a bigger usability study.

In summary, the process can work like this - With the iterative approach, a site prototype can be created, viewed, and evaluated by a small subset of users who identify problems and suggest changes. Once immediate problems are corrected and suggestions are implemented, the site is released to a larger audience, who become part of the design and quality control team by using and evaluating the resources and sharing their feedback with the developers. Usability testing methods, by their very nature, keep the user in the forefront. User testing helped us, the designers, distance ourselves from the product. Rather than focusing on what we thought/liked/needed, we could focus on what the user thinks/likes/needs. What users say they need and want is often substantially disconnected from what they actually need and want when faced with using a product to perform a task. It is for these reasons that the only way to effectively determine what is best for users is to observe users performing tasks with the product of interest.