Getting feedback from users - having a process of early feedback gathering

In today's world, the field of software applications is increasingly complicated, since there are a number of competing products vying for the same set of customers. Margins are increasingly less and less, and even releasing a product with a killer feature only lasts for so long, before the competition copies the feature and incorporates it into their own product. Organizations have to have policies such that they have a very good understanding of the needs of the customer base, and keep on releasing / modifying features that have an impact on their customers. Such a policy is what is required to ensure that a software product remains ahead of its competitors, but it is not so easy to continue to do so. Here are some steps:
- Have an active set of pre-release customers and engage with them early. The team doing the software application needs to have a proper process to identify sets of users who represent their desired customers and engage them on a regular basis. Identifying such a customer set can be difficult by itself, since many of these people will not understand the need to sign legal documents in the nature of a Non Disclosure Agreement (when you want to ensure that your pre-release users do not release details of the feature in the software before it is released). Another problem is that it is an incredible challenge to get the profile of users who represent your customer base. In many cases, you will find users who want to get involved in the program to learn and try to influence what features you will implement (and these could be people such as those who write help books based on your software, those who provide training for your software, and those who use it for business). Need to understand that many of those volunteering have their own motives, and when you get feedback from them on features within the software, some of that feedback can be based on their own motivations rather than being a true portrayal of the desirability of the feature.
- You need to get more pre-release users that you need, since it is only a percentage of them who will actually be active; many of those who have volunteered do not do the active testing that you desire, or do testing only once in a while. A good ratio is that if you need around 100 active users, you should invite and accept into the program, around 250 people. And from time to time, you will need to remind those who are not participating to take part.

Product Development - Prerelease planning

During a product development cycle, getting inputs from users is of great importance. It is not so easy for a development team to get inputs so easily. One way is to find focus groups or do other usability studies where groups of users from the desired customer groups are quizzed about their needs and their workflows, and based on that, user workflow design is made. However, such studies are many times one-time only, and it is hard to do such usability studies on a continuous basis to get frequent guidance to the development team. One option for works fairly well in such scenarios is to get a pre-release / beta program where people from the target customer base interact with the team on a regular basis. They would do this under a NDA (Non Disclosure Agreement) where the fear of details of the product being revealed to the public are far reduced.
A pre-release program needs to be planned thoroughly, ideally when the project planning is happening. Some of the factors that needed to be planned as a part of this are:
1. At what stage should the pre-release program start ? Right at the start of requirements detailing, where pre-release participants can review snapshots of how the workflow would look like and provide details ? If this can be achieved, then it would make sense to do this through the development cycle. The team can get quick feedback of user reaction to proposed designs.
2. What is the number of participants that should be part of this program ? This is a question for the product team, but given that only a percentage of people signed up for a pre-release program actually provide useful feedback over a period of time, the team should plan to sign up more people than required.
3. What are the features that should be exposed to users through a pre-release program ? Features should be selected that are new, features where there are extensive usability issues, and features where there is a lot of doubt about the proposed implementation.
4. How should the users be exposed to features ? The normal process is that in the initial stages, pre-release users would be exposed to mockups that detail the workflow of the features, and as time goes by and builds start getting made with the features being implemented, the pre-release users would also start getting exposed to these builds and features and can test them with their own perspective and provide useful feedback.
5. Actual process for taking feedback: Typically when pre-release users get going, they can generate a significant amount of feedback over the cycle of the product development cycle, in terms of feedback on features, new feature requests, and bugs. All of them need to have their own channels for capturing such feedback; with people on the product team being deployed for monitoring the feedback, a separate process for making sure that bugs logged by users move into the bug tracking software used by the product team and the pre-release users are able to view the progress of these bugs.
6. How many ratings of the product: It is always useful to get users to rate the product along with allowing the pre-release users to provide open-ended feedback of the features in the product. If you do this a number of times in the cycle, it provides the product team with a view of which features most appeal to the users; it also give the product management team a list of features that can be taken for the next version.
There would be many other advantages as well, please provide these via comments when you know of other advantages of a robust pre-release system. For example, getting the pre-release system in place allows testing of the product across a broad range of devices and operating systems.

About usability testing and timing

Suppose you are in a tight development cycle. You have to deliver either a new product, or the next version of an existing product. Getting the features of a product right is always a touch task, given that there are a number of competing features that seem important, and prioritizing the features is something that is very important. This decides the priorities that the engineering team (the feature development team) will follow during the development cycle.
How is this priority actually decided ? If the company is in in the business of defining an absolutely new product that has not been conceptualized as yet, then getting some feedback from prospective customers is difficult; however, if there are already customers using an existing product (from the same company or a rival company), then it is absolutely essential that these users be polled for the features so that there is a good idea about the features that are most critical (it would also help to identify features that customers would be willing to pay a premium for).
Now consider that we are in the development phase of the project lifecycle, where the UI team works along with the engineering team to define the workflow for the feature. There is a lot of discussion around what the feature should be like (with a possibility of the discussion getting heated as a regular part of feature discussion), and eventually most people agree to what the feature should be like. The UI specs of the feature are drawn up and the feature implementation is based on the spec. At this point, everything may seem settled, but it is critically important that this final implementation be evaluated for usability issues. At this point, the team needs to find a set of people who would adequately represent the final set of users, and get them to see the feature working in the actual product. Such usability testing will help determine whether the determined final feature is actually something that the users can accept, or whether there are problems that need to be modified.
The timing of such user testing is most critical. Typically, such workflows reach a final form close to the end of the cycle, and this is the form in which users can actually exercise the workflows. However, in a contra effect, this time is also very late in the cycle, and the team will be hesitant to accept changes that are significant, since the amount of time required to make these changes may not be easily available.
What is the solution ? The solution that seems to work is to have a much more active involvement with users, starting with showing them mockups as the workflow gets more concrete, active question and answer sessions about what they may be looking for, till the time that they can review the actual product implementation. Further, if a workflow is very new and contentious, then it would make sense to try and complete it earlier. And finally, there needs to be time built into the schedule to take such changes.