Product Development - Prerelease planning

During a product development cycle, getting inputs from users is of great importance. It is not so easy for a development team to get inputs so easily. One way is to find focus groups or do other usability studies where groups of users from the desired customer groups are quizzed about their needs and their workflows, and based on that, user workflow design is made. However, such studies are many times one-time only, and it is hard to do such usability studies on a continuous basis to get frequent guidance to the development team. One option for works fairly well in such scenarios is to get a pre-release / beta program where people from the target customer base interact with the team on a regular basis. They would do this under a NDA (Non Disclosure Agreement) where the fear of details of the product being revealed to the public are far reduced.
A pre-release program needs to be planned thoroughly, ideally when the project planning is happening. Some of the factors that needed to be planned as a part of this are:
1. At what stage should the pre-release program start ? Right at the start of requirements detailing, where pre-release participants can review snapshots of how the workflow would look like and provide details ? If this can be achieved, then it would make sense to do this through the development cycle. The team can get quick feedback of user reaction to proposed designs.
2. What is the number of participants that should be part of this program ? This is a question for the product team, but given that only a percentage of people signed up for a pre-release program actually provide useful feedback over a period of time, the team should plan to sign up more people than required.
3. What are the features that should be exposed to users through a pre-release program ? Features should be selected that are new, features where there are extensive usability issues, and features where there is a lot of doubt about the proposed implementation.
4. How should the users be exposed to features ? The normal process is that in the initial stages, pre-release users would be exposed to mockups that detail the workflow of the features, and as time goes by and builds start getting made with the features being implemented, the pre-release users would also start getting exposed to these builds and features and can test them with their own perspective and provide useful feedback.
5. Actual process for taking feedback: Typically when pre-release users get going, they can generate a significant amount of feedback over the cycle of the product development cycle, in terms of feedback on features, new feature requests, and bugs. All of them need to have their own channels for capturing such feedback; with people on the product team being deployed for monitoring the feedback, a separate process for making sure that bugs logged by users move into the bug tracking software used by the product team and the pre-release users are able to view the progress of these bugs.
6. How many ratings of the product: It is always useful to get users to rate the product along with allowing the pre-release users to provide open-ended feedback of the features in the product. If you do this a number of times in the cycle, it provides the product team with a view of which features most appeal to the users; it also give the product management team a list of features that can be taken for the next version.
There would be many other advantages as well, please provide these via comments when you know of other advantages of a robust pre-release system. For example, getting the pre-release system in place allows testing of the product across a broad range of devices and operating systems.

Product Development - Actual Kickoff of Development Effort

During every product development cycle, there are 2 distinct stages that you reach in the cycle. There is an initial phase when the planning happens, and there is a later stage when the actual work happens with respect to the actual design and development effort. These are very broad level definitions, with many details and finer points.
- Planning stage: This is the stage where the team defines with broad strokes, the details of what the product will look like in terms of features. The general direction of the product, the features that will make it to the release, the schedule of the release, all of these are decided in this stage. In addition, if the company is structured in such a way that the configuration group, installer and release teams, internationalization teams, and other central technology groups are separate teams, then they will need to be signed up.
- Implementation stage: In this stage, the teams get working on actual implementation of features. By this time, the product team has already decided on the features, their priority, and schedule. The core of the development phase, which can be translated into requirements breakup and detailing, user design, engineering design, test case and test plans, coding, testing, and release all happen in this stage.
In between these 2 distinct phases of the project, there is a need to get an actual kickoff. If the product is such that it needs an approval from executive management, and also to get all the supporting team on the same discussion stage, a typical kickoff meeting is called. In this meeting, the project manager / program manager gets all the stakeholders (including the product team, management, executive sponsors, supporting teams, product management, etc) onto the same room and presents the schedule, final feature list, commitments of support, languages, pre-release plans, issues and risks and so on.
Such a meeting typically takes around a month to plan, since you need to get a meeting time right which is convenient for all the attendees, you need to prepare a presentation to run in the meeting, and so on. A kickoff meeting is the actual time when the sponsors say yes to the product development phase proceeding, at the same time, there is also a chance that the executive management team may need clarifications or may even ask the team to go back and review their plans.

Product Development - planning a minor / dot release - Challenges

In the last post, I had talked about why a minor (dot) release is needed, as well as some of the reasons as to why doing a dot release is an inconvenience. However, if the decision has been made to do a dot release, then it is necessary to understand the process of planning and executing a dot release, including some of the difficulties (challenges) that emerge in such a minor release.
What are some of the activities that need to be done ?
1. You need to finalize the changes that need to be a dot release. If this release is because of some known changes, then the changes need to be analysed, and the engineering response (and design) needs to be worked out.
2. If the changes are still unknown, for example, if your release is failing some security tests or some certification, then you need to figure out what can be done. If you take an example where the earlier release is failing some new certification norms (and for those who know how certification works, it can be a lot of effort to prepare the infrastructure and execute all the cases. As an example, you may need to take the help of tools for certification, and those tools may need an upgrade of memory. In other cases, the amount of testing required may be huge, and the actual calendar time needed may stretch the schedule of the dot release.
3. For companies where a lot of work is handled by support teams (configuration, build, release, internationalization, etc), the required overhead of handling all those teams and getting their support takes both budget allocation and time.
4. Too many minor releases causes cynicism in the market about the initial stability of the software release. For example, Microsoft releases many service packs for its software, and there are many people who do not migrate onto a newer version until they see 1 or 2 service packs because they would rather wait for some of the important bug fixes.
5. You run into issues where there are multiple releases for around the same version (with say versions 8.0, 8.1, 8.2, etc for a product). Once you have multiple versions, you get into support issues whereby issues are different for these versions, and support may be a nightmare.
6. Newer dot versions, in many cases need to be deployed to the retail channel (replacing the software already in the retail channel), and to the web store (including to many online software sellers who resell the software). All of these steps involve huge logistical nightmares and / or costs.
7. Branching strategies. Getting a dot (minor release) in process needs major configuration issues (getting branches in place, especially when work is also going for the next release), and takes a fair amount of explanation to both the Development and Quality teams.

Of course, if you folks can suggest more issues that make minor releases a challenge, please add comments.

Product Development - what is a minor / dot release

For those of you unacquainted with the concept of a minor release (I will also call it a Dot release from time to time), it is a release that you do after the main release has been done. If that did not make too much sense, let me try again ! Suppose you have released version 8.0 of Microsoft's Visual Studio to great fanfare, and with much expectation that this is 'the' release, perfect in all ways. Once you release 8.0, the overall grand plan would be to sell 8.0, and have the team working on developing 9.0 with no constraints.
Let me tell you a little secret. As you move closer to the release date of a product version, you (and in fact, most of the team members), start entering a time of partially restrained panic and impatience. It is also a normal decision that as you get closer to the release date, bugs that would have been fixed by the project team even a couple of months back will not be fixed now, unless they are deemed critical. The primary reason for this customer unfriendly move ? Simple Murphy's Law - 'If anything can go wrong, it will' ! Any bug fix has a cost - the fix needs to be reviewed, and then the quality team needs to test the impacted area of the bug. If something goes wrong with the bug fix earlier in the cycle, you have time to fix the impact. However, as you get closer to your release date, this buffer time is no longer available. As a result, almost every product team takes a call that as you get closer to the release date, bugs need to be more and more critical before they can be fixed. Thus, the product team may opt to not fix bugs that could potentially impact customers, but if the bug is found say a week before release, the bug would be classified based on whether you would stop the release for such a bug. Sadly, very few bugs make the criteria of being a release-stopper.
Once you have released version 8.0, there will be bugs that will come in from the field (including bugs that the product team had decided should not be fixed). Also, there may be other changes that are required in the release, and which cannot wait for version 9.0. And thus the stage is set for a generation of release of Visual Studio 8.1 (now you know where the term dot comes from, since this minor release is actually 8 'dot' 1). Such releases follow a few basic points:
1. Bugs (whether older bugs, or new bugs generated by customers) are evaluated as to whether they are deemed important enough to fix; these are the only bugs that are selected for inclusion in the dot
2. Typically, when a dot release is planned, the release supplants the earlier release. So, following the above example, new customers will get release 8.1 of Visual studio vs. release 8.0 that was released earlier
3. A dot will typically have all the activities of a full release, the only advantage being that the time frame is drastically reduced. A reverse is that the inter-group coordination that had more time available has to be compressed into a shorter time frame
4. A dot inconveniences everybody, since it takes much more attention to do everything in a shorter time period; further, it takes away resources (including team management attention) from working on the next release
5. Reaction time is much less than in a normal release