Defining single point responsibilities for decision making during a software cycle

It seems like a simple management issue, having single point responsibilities for different functions, but you would amazed at the number of teams that stumble on this issue when at a critical point in their schedule. Consider the case where a team is coming to a point where they have to declare whether they are now clear of all major discovered defects (no team can find and fix all known defects, it is an impossible task, the amount of effort involved in trying to detect all bugs starts increasing exponentially once you reach a certain point). At this point, many teams start working on adhoc testing, others start the process of release of the product to the consumer, and so on. It is a major milestone for the product.
But who takes a call that they are clear of all major defects. The key word here is 'major'. As long as testing is going on, there will always be issues coming up, and they have to be dealt. Depending on who see the defects, the classification of whether an issue is major or not can be dealt with differently, even with the best of defect classification criteria in place.
I remember an issue from a couple of decades back. Almost at the last stage, when the team was ready to close down the defect finding and defect fixing, a defect came up. It was an interesting defect since it was serious, but covered a small workflow that many considered a non-serious workflow, and some of the team members were fine delaying it for a later dot release (the team was in the process of releasing periodic dot releases, so such defects could go into the dot releases).
At that point, during the day, we realized were were going around in circles, trying to figure out whether it should be fixed and we take another build (with the subsequent testing of that build again) or we defect it and take it up later. There were strong opinion for both in the managers and leads in the team. We realized that we had never worked out the appropriate decision making process for cases such as this, and suddenly giving the decision making for this to one person could have caused tension within the team. Ultimately we had to setup a meeting of the senior leaders of the team to thrash through a decision, taking into account the costs and the impact (both for and against the decision).
The learning we had from this kind of case was we need to better refine the process of having decision makers for specific situations - in this case, for the next time, we made the testing manager as the decision maker about whether a defect that came up in the last minute was of a sufficient severity to be needed for fixing, with the concept that if the QA manager did recommend such a defect, they should also be able to justify the severity of such a defect later.

Software product localization - there may be base product changes

For somebody (people or teams) who have experience in releasing software products in multiple languages, they would typically have gone through a lot of learning in terms of how the nuances of different languages can cause changes in the base language product (in our case, and in most cases, the base language product is in English, and the product can be released in many other languages, for larger software products such as Operating Systems or MS Office or Photoshop, these can be many many languages).
However for a team that has so far been releasing software products in one base language and have now moved to try and release their product in other languages, it can be a fairly complex project. In simplistic terms, it is to make sure that all the strings used in the product (whether these be text on screens or on dialogs or error messages, etc) are all capable of being harvested, sent for translation and then reincorporated back into the product depending on the language in which the product is being released.
Based on this simple concept, things get more complicated as you proceed towards actually doing the project. There are additional schedule requirements, there is a lot more work for the developers since testing a product for localization reveals many changes that are required, there is the need to get external people who can do the testing of the product in the different languages (the language needs to be checked, as well as the functionality of the various parts of the product under different languages), and many other changes need to be planned (this post is not meant to be a full description of the process of getting a product localized for the first time - that is a massive endeavor that requires a lot of explanation). As an example, a simple text on an error message may turn out to be much longer in a language such as Russian or German, or reading from right to left in Arabic or Hebrew, and the error message may not display properly in such cases. Either the message needs to be re-written or the error message box needs to be re-sized, which also has implications for the help manuals that may need to be modified.
Ideally, a team planning to get their product localized for the first time needs to avail of the learning that other teams and products have gained over their cycles, and so either need to hire some people with the required experience for both development and testing, or atleast get a thorough discussion with teams that have done this. Getting a product localized for the first time is not that big a effort and can be done right, but it is also not something that you attempt without ensuring that you have done adequate preparation in terms of schedule and resources. Once you have done that level of planning, then you will still face challenges, but those should be fixable.

Taking the time to define and design metrics

I recall my initial years in software products where there was less of a focus on metrics. In fact, in some projects, there was a question of accounting for defects and handling them, and the daily defect count was held on the whiteboard, but that was about the extent of it. Over a period of time, however, this has come to change. Software development organizations have come to realize that there is a lot of optimization that can be done, such as in these areas:
1. Defect accounting - How many defects are generated by each team and team members, how many of these defects move towards being fixed vs. being withdrawn, how many people generate defects that are critical vs. trivial, and so on.
2. Coding work, efficiency, code review records, number of line of code being written, and so on.
You get an idea, there are a number of ways in which organizations are trying to determine information about the processes within a software cycle, so that this information can be used to determine optimization, as well as figure out the awards for the appraisal of employees. This kind of data helps in providing a quantitative part of the overall appraisal counselling and to some extent, being able to give the manager a comparison between the employees.
However, such information and metrics are not easy to come by and cannot be done on the fly. Trying to create metrics when the project is ongoing or expecting people to do it along with their regular jobs will lead to sub-standard metrics, or even getting some wrong data, and not enough effort to screen the resulting data to ensure that the data is as accurate as it can be.
Typically, a good starting point for ongoing project cycles is to do a review at regular periods so that the team can contribute as to what metrics would be useful, and why. Another comparison point would be about talking to other teams to see what metrics they have found useful. And during the starting period of a project, when the discussion stage is ongoing, there needs to be a small team or couple of people who can figure out the metrics that need to be created during the cycle.
There may be a metrics engine or some tool being used in the organization, and there may be a process for new metrics to be added to the engine, or even for getting existing metrics to be added for a new project, and the effort and coordination for that also needs to be planned.
Basic concept of this article is -> Get metrics for your project, and design for it rather than treating it as an after-thought. 

Regular sessions with Product Management and Customers

One of the most common experiences I had while interacting with people on product development teams, especially those who have been on 2-3 development cycles, from start to end is, that they seem to get a feeling for what the features in the product should be, how the workflow should happen and so on. It can get tricky when they have a strong feeling in this regard since they might incorporate their feelings in the workflow that they are implement and it would take some effort by the workflow designer or the product manager to get the design as they want (may not happen very often, but it is something to be watched out for).

But when you get the team members on the beta release program, or to monitor the user forums, or to attend sessions where actual product users meet the product development team, it can get interesting. Even when team members get to review the prerelease program (the beta program), they can seem surprised at the type of defects or feature requirements that come through these programs. I have seen cases where the team members almost dismiss these as defects that are worthy of deferring or not part of the general workflow and they have to be reminded that these are actual users, the ones who actually pay and use the software program.

The more perceptive of the team members welcome such interactions, since it gives them a great idea of how their customers are actually using the product (or atleast a section of the users) and the more they take part in their interactions, the more they are connected with how the users actually use the software and this in turn benefits the product since they are more attuned to the requirements of the user, and in fact, would actively hunt to figure out the requirements of the user.

At the same time, it is necessary that all members of the team are exposed to such interactions. Such interactions help team members understand what the users feel, that sometimes their perception of what users feel and what they themselves feel about how a feature should be like, or the severity of a defect can vary drastically, and such interactions help reduce these differences and give them a better understanding of the customer perception of what is important for features. The next when a new feature is being designed, it makes the process much smoother, and actually helps the product manager - for the case of teams that implement the Scrum development methodology, such a customer centric perception from the team members is absolutely essential and helps to drive the process of feature delivery. 

One example of this was where the feature requirement was detailed by a customer to the product management team who wrote feature specifications for the development team. However, the developer was a senior developer who had his own ideas of what the ideal feature should be like and proceeded to tweak the feature specification. It required some amount of rework to ensure that the feature was finally done as per the product management / customer requirement. 

Dedicated team for previous version releases

For software products that have had multiple releases over the years, there is a ticklish question about the resources that need to be dedicated for previous released product versions. This is even more critical for software products that have wide usage such as Microsoft Office, Photoshop, and Acrobat. Organizations have stated policies about providing support for previous versions, but there is a cost associated with doing the same. You need to have people who can do defect resolution and testing, you need people who can do all the entire build infrastructure including releasing the software fix. This can add upto a pretty packet in terms of resources, and this can be painful especially when there are time periods when there are no major fixes that are due.
Deployment in these previous support teams can also lead to tricky morale issues, given that people assigned to these teams would feel that they are put onto a team that is less important than those who are part of the team working on the current features. To avoid such morale issues, team can rotate people across these teams, but that comes with its own problems about people not being able to develop the expertise, something that comes only after gaining experience on the team.
A problem with not having these previous support teams - for some organizations and products, such an option is not possible; a full fledged support team is required. Consider somebody using a previous version of MS Office or Adobe Acrobat, and suddenly, there is news from a computer security researcher about a critical zero day or similar intensity defect in the product which could allow a hacker to get into the hole. These events can be a public relations disaster - such blowback cannot be countered just by Public Relations, but there is a need to have a team that can work on a full scale quick research and fix, such that the organization develops a reputation for responding quickly. And it's just not a fix in a previous version, but the same problem may be there in the ongoing version that is being developed and the team needs to get the research done by the previous version team and incorporate the fix in the version under development.
A lot of teams I know slightly under-budget a team for working on previous version defects, and if the problem requires a slightly enhanced amount of effort, then additional people are deployed for working on the defect fix, and are only there on a temporary basis (even though it might have some amount of impact for the current cycle, but there is no real solution for the same). People assigned to such teams are typically team members who are not very high in ambition, and are not necessarily the high rated people in the team. Further, the Product Management teams need to be involved in both sides, since many of the feedback in terms of defects or suggestions are actually coming from customers or end users and there is a possibility that some of them may have value for the next version to be released.   

Inter team - Pushing for your bug fixes

When you work in a slightly larger software development organization, you will find that there are numerous cases where teams have dependencies on external teams for getting defect fixes. A simple example can explain. Say, there are multiple teams that need a function for coding / decoding music - and there are so many different audio formats, some of which have free solutions, and others which are paid solutions (and even in the paid solutions, there will be some that are very cheap and others, for some specific audio formats, which are very expensive). To further complicate this, each external solution will have its own set of legal formalities and requirements which may or may not be easy fort the organization to follow (some of the open source solutions are almost non-touchable for typical software organizations because they have some stringent requirements on their own, like insisting that any software that uses them must be open source in its own way).
And there can be numerous examples such as this; we used to have a simple XML parser that almost every software would need, and as a result, there was one team that was mandated to write such a parser and own the solution. Net net, where there are multiple teams that need a common functionality, it makes sense for a central team to create and own this common functionality, to update it as and when needed and provide the required updates to all the teams that depend on them.
However, this dependency on a central team can be tricky. With every organization and every team working on the concept of limited resources, the question of priority comes in. Teams that are more important and critical to the organization realistically have more say in the release timeline of central components, and more importantly, the bug fixes that go into the central component.
For a team that ranks somewhat lower on the priority, it can be a strugle to get the component with your desired bug fixes as per your schedule, and realistically no amount of hollering or screaming is going to change that basic truth. However, you still do need those bug fixes, so what do you do ? It is not a simple solution to write your own code to replace the component - it may not be allowed, the resources or other costs may not be available to do this, or your team may not even have the capability to do this. Another solution is to align your schedule with some of the more higher priority teams, atleast you would get a rock solid component with some of the high priority bug fixes in it. If this is not really a solution, then another method is to ensure your communication is top notch. The relevant people in your team (both management and technical) are part of any mailing lists or discussion groups that talk about the component, its features and defects. Similarly, there is a need to setup regular checkin meetings with the component team to ensure that your relevant defects are passed on along with the required priority and severity. Further, you need to communicate regularly with the other team to ensure that your defects remain on their radar (including with the product management function who decide on features and defect fixes). All of these measures help to ensure that your required defects or features get highlighted; whether they make it or not is still not guaranteed though. It does help though if you are able to get customer inputs about the defects or features which tries to increase the importance of the defect or feature.

Not every defect should be fixed

During the course of a software development project, whether this be an updated version of a product or a one-off software project, the key currencies of the project are primarily features and defects (features and defects are what the teams end up doing most of the time during the development cycle, and in fact, at some intense points of the cycle, the team will be doing mostly defect finding, fixing, releasing fixes and re-test (in the Waterfall cycle), once the feature is released, it is bug fixing and more bug fixing.
In the software team, these are busy times. In a number of such development cycles, the number of defects that there are there in the product are such that it seems tough to think that all of these will be fixed in the given time cycle. This cycle of defect fixing primarily involves the developer and the tester. Some of these would be simple defects, where the feature as defined is not working and needs to be fixed by the developer. However, there will be other cases where there is some ambiguity in terms of how the feature is working and whether it is as per the definition - it could be that the feature definition is not complete to cover all the cases or that there is some disagreement regarding the way that the feature was supposed to be working vs. how it was written.
The biggest problem could be that the resolution of such issues happened in a path not intended to happen, something that the product manager had not intended. Now, it is not necessary that it happens this way, but one has to prepare for such an eventuality, or to be more clear, to define a process so that such a thing does not happen. The actual process is something that needs to be defined for each team, since what works for one team may not work for another team. For example, I once worked with a team that asked that all defects be triaged by a bug committee which decided whether it needed to be fixed or not, and if so, what would be the proper method of fixing (although the actual fix is something that the developer and tester could decide). Why this process may not work for all teams is because of the quantum of defects that may come in, overwhelming the defect review committee and causing a backlog. Other teams may find such a process too severe, trusting the developers and testers that they will not automatically make suspect fixes, checking with a defect committee or with the Product Manager before actually making a feature change or refinement.
However, it is essential that this be talked about and decided with the management and with the team before proceeding, else there is a chance that feature changes may happen just at the developer and tester level.