Getting feedback from customers - appointing a champion within the team

In the previous post (Exposing team to customer feedback), I talked about how the team benefits from exposure to customer feedback and the problems that customers face in their regular workflow. In this post, I will take a specific point about the logistics of doing this and how to ensure that the team does not drop this concept of interacting with customers for getting their workflows, primarily through interacting with the customer support team.
One of the ways of ensuring that the process remains ongoing, that the team does not get tired of seeing these problems with customers in their workflows (from experience, it has been that direct ongoing interaction with customer support to get information about problems can get depressing to the team, just hearing about problems and more problems), that there is a controlled system of working this feedback into the schedule of the team members. Managers of these software teams do get involved, but it cannot be a manager driven, command ruled interaction with the customer support team.
One way of ensuring that the team remains involved with the customer issues, including with the customer support team, in a controlled manner, is by deputizing somebody from the team to handle this interaction and figure out the parameters by which the team would operate. The person who does this works with the customer interaction team (primarily the product support team in most cases), works out the schedule of such interaction, works out the prioritization of the issues that are thrown up about customer workflows, and tries to work out a system whereby problems are highlighted, as well as positive customer workflows feedback or any direct appreciation from the customers about the product or the development team directly (this needs to be genuine feedback, not something that is created to keep the team in good spirits; you would not believe how many times positive news is massaged to make the team happy).
This entire plan of what the person is expected to do, as well as a slight reduction in the work schedule of the person in order to give them time to handle this additional work of customer support interaction is something that would emphasize to the team the importance of this work. In addition, the managers should atleast have some critical items from the customer feedback that are meant for incorporation into the product, again emphasizing the importance of feedback from the customers. It would be ideal if the person who is appointed the champion gets some prioritized items and presents this to the team and the product manager.
The role of a champion for this effort is something that should be offered to the team, to any member of the team who wants to work on this. Even though the managers might have somebody in mind for this, they should let somebody else do this work unless the mismatch is too high. The position can be rotated at a regular interval, say maybe every quarter so that more people get exposure to this role and also more exposure to feedback from customers. 

Preparing a plan for ensuring customer feedback reaches the software team

This post is not really going to lay out a plan for how to get customer feedback to the software development team. There are a large number of expert posts and articles that talk about different models of how to get customer feedback to the team, as well as different levels of interaction; this could be a high level of interaction or a summary level kind of detached level of information. The post is more about the imperative of ensuring that the software development team does get some kind of customer feedback, and in a structured way that continues even with changes in the team.
It is necessary for team members to be exposed to some amount of customer feedback, with the level of exposure being decided by the managers of the team; as well as the number of members of the team who are exposed to such feedback.
An important question is about why the team members need to be exposed to customer feedback; after all, in the classical definition of the processes of the team, the product manager is the one who is exposed to customer feedback, massages it to the appropriate functional change or addition and then puts this to the team in terms of the prioritization of this feature vs. other features; or classifies this as a defect which needs to be fixed, again as per prioritization.
However, the classical model needs to be changed. I have seen how exposure to customer feedback changes the way that the team works, their responses to defects, and the eagerness in some of the team members to get more involved with some amount of customer interaction. The biggest advantage comes in terms of getting more acquainted with the customer mind. In some cases, the shock when the team members realize that a defect that they had ignored as minor got some customers really hassled. In one case, the defect was raised to the company management through outside means and came down to the development team as a must fix; when the product manager saw the defect, there was some amount of discussion because this was a defect that had been raised earlier but had been dismissed since it was deemed as too minor and even more surprising, not really impacting customer workflows.
Even senior engineers on the development team have seen jarring signals, whereby the features they worked hard on and pushed a lot, were not really seen as important by the customer while the changes in some of the workflows that was prioritized as higher by the product management was appreciated in customer feedback mechanisms. What this did was ensured that the senior members of the team understood that they need to setup a regular structure of how to get customer workflow, as well as listen more to the product managers. This is even more important when you have development team members who have been there for a long time while there have been changes in the product management team.
Another advantage of getting customer feedback through some kind of mechanism is that the team will also get more acquainted with the people who actually interact with the customers, whether this be the customer support team or some similar kind of support mechanism, and this kind of interaction helps the team do question and answer where they learn even more of the kind of common problems that customers have in terms of workflows, and makes them more receptive to requests for changes in workflow rather than introducing new features.

Logistics - Deciding a common location for all document sharing

Some of these posts can seem very logical and obvious, but in reality, many of these items have come up based on experience learned from various projects, shortcomings and feedback provided by the teams with whom I have been working. And they can be very important, even though they may not seem much.
When you have a number of different teams working on the same set of tasks, there are a number of documents, instructions, examples, demos, and other artifacts that need to be shared between these different teams, shared real time and with the required access available to all these different teams and their team members.
And it was one such case that caused a delay of around a week in the actual schedule of one such task that was being coordinated between different teams. A delay of any kind has a ripple effect on the entire schedule, and when schedules are less than a year, a delay of a week in any such related task can cause significant problems to the schedule and lead to some fire-fighting in the team management. In this current case, we eventually ran into an issue where a rights / security caused a blockage, and the person tasked to ensure the coordination did not even know about this security problem.
During the course of a project, the project / program manager cannot take on each and every area, and in one particular case, the coordination between a team located in the main geography, another team located in a different geography and a vendor team located in another geography. The vendor team was new to the product and had to be brought up to speed on the processes and technical knowledge of the product. In the light of some of this coordination effort, a team lead with experience of some of the relevant functional area of the product was put in charge of the coordination effort, reporting every week to the overall manager of the product about the status.
The lead started out well, using previous experiences to set out some of the required documentation. However, when an outsider vendor is needed for the product work, they need to be granted permission for the documentation area, And this permission cannot be granted by the team, but by a central IT unit which is responsible for all server access.
The problem turned out in this case was that part of the documentation was placed in a server that was off-limits to any outside vendor (there were some specific security protocols for some of the more important servers, especially those where code is resident on the server); but this information was not known to the project lead responsible for the coordination. The project manager knew this information, but the multiple tasks being done by the manager and the leads ensured that a proper discussion did not happen for the next 4-5 days, at the end of which a meeting resulted in the lead learning this security information, It took another 2 days to make changes to the documentation location, and then get the required permission. This entire stand-off did need to some changes being made on some parts of the schedule, not something that the project manager welcomes.
What the learning from this was that we did not do the required leg-work before starting this part of the phase, especially about a common location for sharing of documents and the permissions for these.

Emotional stability - Somebody's got to remain in control during a heated discussion

This was a painful episode when seen in highlight, and something that has stayed with me the rest of my career and even intruded into my personal life. I did not do something outrageous, but the concept of making your way down a path where there is no easy way back has remained with me ever since, and everytime I get into a situation where it could become problematic, I recall the incident to see that I am not boxing myself into a corner.
The exact incident could vary, but the scenario is the same. You are Project Manager / Program Manager of a team with colleagues who have different roles, and their emphasis at different parts of the schedule is different. At a particular point in the schedule, even though both Development, Usability and Testing are supposed to be on the same wavelength, their focus and push could be different, and that is very natural. In this particular case, the situation happened at the time of the schedule when we were supposed to stop all new feature work, from that point on, all the focus would be on resolving defects in the features already implemented.
This is a critical moment for the testing team, since they essentially consider this the time when the entire product comes together, with all features working to some degree or the other (one is really considering a development process that is not ideal scrum). The integration testing happens more critically from this point onwards, and it can take a lot of time from this point in the schedule to thrash through most of the defects and make the product stable (of course, you cannot find all the defects in the product). The testing team is expected to not welcome any addition of new features from this point onwards.
This was a critical time. Some of the members of the development team had come in mid-cycle, so there was already some instability in the team, and it did take them some time to understand the product code and understand the team dynamics and work together as a team. At this time, the Product Management came in with a critical new feature (that had been introduced by the competition) which was expected to take another 2 weeks to develop, with the remaining testing time being around 1.5 months. Now, this was not an easy situation. The testing team was well justified in terms of not agreeing to take this feature, since in the end, it was their responsibility to ensure that the product was stable and had the least number of defects possible. The product manager was quite serious in terms of the need for this feature, and was pushing for it. The development team was ready to implement it.
The first meeting to discuss this was a disaster, with these stated positions being discussed, and getting heated. For me, unfortunately, being the Program Manager, it was my responsibility to ensure that the discussion remained in reasonable limits of discussion, without getting heated, and nobody getting into an inflexible position. However, in this case, at some point, the discussion went into a thread of policy and schedules, and it turned out that I started taking a more inflexible position, whereby such a change would not be possible. The discussion terminated soon after, and it got escalated, and during the midst of a discussion with my manager, I realized my mistake (while I was being chewed out). In every discussion, typically the managers need to ensure that discussions need to remain on a civil level, discussions can get heated, but nobody should reach a point where they get into a position from there was no roll-back, and certainly not the Project Manager / Program Manager. 

Getting a software team to figure out their own problems - Part 2

In the previous post in this series (Working with a software team to improve their processes - Part 1), I described a situation where a software team was working in a state of somewhat poor processes; where they had a lot of work and hence were working flat out. As a result, they believed that they were doing great work and were productive (or rather, some of them were worried about whether they were actually working effectively), but they also wanted suggestions about where they could be doing better. For a team that felt that they were productive and doing well, it would not be really useful to try and tell them a lot of stuff about processes and so on, instead it would be better to have a discussion with them and get them to raise such queries by themselves so that they would start driving improvements processes themselves. In this post, I will continue with some of the discussions that we had and post more of the way that the discussion happened:

- One of the initial points in the discussion was about whether they were aware of how defect free their code was ? They had defect metrics in terms of number of defects raised, closed and so on. However, when queried about whether they did some kind of review of the defects raised in the project so that some serious defects were analysed to prevent the occurrence of these defects in the next similar project, there was no answer. One of the team members then volunteered about how some of the more serious defects did occur from time to time, and this set the more senior members to evaluate as to how to make more time for doing such analysis. Even with the speed of their work, many of them did realize that it was serious to try and prevent these kind of high impact defects from recurring.
- It was necessary to keep on drilling further on this side, but in a discussion kind of mode. The next question was about their code practices, So, when the discussion moved to how robust the code was, and how the aim was to prevent defects from occurring in the code, the discussion finally moved onto the process of Code Reviews. There was a realization that Code Review was something that happened when the developer felt that there was time, and a reviewer was available. Members of the discussion team knew that Code Review could help in reduce the defects in the code, and that too right at the source (where the cost of the defect would be the minimum), but the speed and pressure ensured that this practice was not mandatory and did not happen in the hard pressure cases, where it was even more important.
- Discussions also happened in terms of coding guidelines and commenting inside the code. The team had a attrition rate that was increasing, and it was taking time for new team members to understand the code and learn why changes were made from time to time in the code. One of the developers who had joined 6 months back was asked about the code and what took time, and after some discussion with other developers, it turned out that sections of the code were not clear. It turned out that in many cases, previous defect fixes had caused changes in the code which were not clear for later developers, and since commenting in the code had not been drilled into them, there were not many references to the need to provide comments in the code with reference to why the changes were made and defect numbers.

This is proving to be a long post. Read the next section (TBD). If you have been in such a situation, please provide your comments. 

Getting a software team to figure out their own problems - Part 1

This is  my experience from a few months back. I was called to help a team that was handling a rough situation. They were part of a growing organization, and the amount of pressure on them was very high. The development team, the project manager and their product manager would get requests on an urgent basis, and their entire job over the next few days was about how to handle the project within the defined timeline, and the entire energy spent over the course of the project timeline (defined in terms of weeks) was about how to complete the project, run through issues and resolve them, and so on. And they were very confident of their abilities since they were completing projects on time, but their work time was highly stressed and they were starting to lose key people.
I was invited to help them in this, and one of my first interactions was with a junior team member. He was very confident of the team, its ability to complete projects on time, and since this had happened a number of times, he was not very confident of what some outside help would achieve ? And there was a mix of people within the project, some who shared his thinking, and some who wondered whether the team was indeed in the right in terms of processes. Wading into this project, it was clear that trying to tell the team that they were not doing things right would not work; trying to get them to make incremental improvements based on self-realization was the way to go, till they reached a point where they were ready for more serious changes. How would one go about this ?
Well, it was time to get started. A discussion started with the leads and managers within the team. Focus of discussion, trying to get them to open up. First item on the agenda, What did they think they did well on the projects ? A lot of comments about being on time, about ensuring that they did not cross the budgeted amount or the allocated resources. Good stuff, but not complete in terms of processes which they could replicate in other teams.
Next, what are the areas of improvements that they could identify. Things got more silent - nobody in their right mind would ever say that they could not make improvements. So, some steps, encourage them in this process, and so on; and then ask about review sessions with the team to figure out what things went wrong, what should they learn as improvements for next projects and so on. And now finally getting through with the team - this process happens in a very haphazard way, and there are no real such discussions which the leads / managers could emphatically quote.
Moving ahead with this approach, get them to start thinking about how do they really know that they are doing well. Just meeting the deadlines is not enough. Do they know that they could do the project with less people if they started following more processes (but not theoretical processes, more like improvements suggested through reviews or from other teams and which made sense to them) ? More benefits in terms of less tension to the team ?

This is proving to be a long post. Read the next section (Working with a software team to improve processes - Part 2). If you have been in such a situation, please provide your comments. 

External components: Sending one's own proposed schedule

This is something that happens in larger software development organizations. Typically, there would be multiple teams within the organization, responsible for specialized stuff. For example, a number of organizations have separate teams installer and build process since that is a function reasonable similar across multiple teams / software applications; there are many common components dealing with technical requirements that are common across applications - such as parsers, decoders, encoders, imaging libraries, document readers, and so on (the list of common technical stuff can be pretty large).
When an organization has multiple software development teams, there will be different schedules for the teams, with many of these schedules being widely different form the other teams. There will also be different priorities for these teams, with some products having a higher priority while other products may not be so important for the component team - this is also dependent on the money making capabilities and strategic importance of the various products within the organization. In addition, with different schedules for different teams, and the component team having a different schedule from our own team, reconciling the schedules in order to ensure that the product quality is good enough to take the component at the time of incorporating the schedule.

Consider an example:
Team XYZ - the team in which we are working
Schedule for team XYZ - Start in Feb, take component of high quality by July and release product by October

Component team - Need to take a component from this team
Schedule for component team: Previous component will release by January; next release will be in December

With these schedules being out of sync, taking a component where features have been incorporated and a good quality level being achieved looks difficult. There is no option to change the schedule of the component team, since they are in sync with another product, and that product has a higher priority. So what can be done ?
Well, this sort of schedule mismatches can happen fairly often, and there are solutions possible. Essentially, there is a need for having an ongoing communication between both the teams. The component team should be fully aware of our schedules, including the timeline by which an initial version of the component would be required, more versions to be taken with defect fixes, and a final version of the component.
In the end, there will be compromises needed to be made, with some features that could be taken, others that could not be taken for the current release, and if necessary, a fork made in the development of the component where a minimal set of features would be taken up and provided to our team.

Using the latest version of external components - Needs attention

Most software applications do use components that are not created by the software team; these could be components created by other teams within the organization, or by using components created by other organizations. For so many different functions, it is not required that the team try to do everything on their own; for example, you could consider many different types of parsers, or system access components, or image manipulation components, or video codecs / decoders, etc. In all of these, these are specialized components created by people far more competent in these areas; it would always make sense to use these components rather than try to replicate such software within our own application.
When your software is larger, there would be more such components that are in use which are created by other parties. Over a period of time, it gets to be a task to track all such components. I have experience where we used such components the first time many years back, pay for using the component at every new version of the application, and the number of such components that are in use is well over 50 within the application (some of those are free, others are paid with minimal charges, and some can be pretty expensive - some of the specialised video codecs can be pretty expensive, although one cannot do business in the video area without using these codecs in order to look professional).
Just like our software is updated with every version with new features and defects, even the components that we use also get new versions. In many cases, we could still continue using the older components since the newer version does not promise any new features that we need, and we can avoid the overhead of trying to incorporate a new version. However, it is still required to monitor these external components to ensure that we know the reasons why the new component has been released.
In many cases, there may have been some critical defects found in the component which required a new version to be released. In such a case, we should need to evaluate the defects that were found to determine whether it is required for us to take the new version. In some cases, the defects could be in a section of the component that we do not use, and hence we could still continue to use the old component. But in many cases, the defects are serious enough that it could pose a risk to the software application unless the new version of the component has been taken. However, taking the component will have an overhead, since other changes that have been implemented in the component would need to be evaluated and testing done to ensure that no other impact is caused due to these changes.
This whole process of ensuring a watch on these components is troublesome and takes time and effort, but it is essential. One would not want to release a software that integrates an older version of the component which has some serious risk. This can have further impact, and in some cases, the external component provider mandates that the newer version of the component has to be used.

Hallway discussions - need to be encouraged

Is this really a post worth writing about ? Talking about hallway discussions ? I believe so. Sometimes one should write about obvious items, but which do not happen - either do not happen at all, or happen in such low frequency that they need to be improved.
First of all, what are hallway discussions ? It can be a pretty broad term that can be tentatively be used to describe those conversations that happen in an unstructured way (outside of conference rooms), where people end up discussing an issue in the pantry, or getting in and out of the restroom, or even in the hallway when they run into each other. It can happen between people of very different seniority, and can be a short conversation or can be an extended conversation that could take many different directions.
First of all, how do you not encourage hallway conversations ? It would seem like a very natural thing to happen ? There are some reasons why such conversations can become rare, some by choice, and some due to the organization (and this is not a complete list, there could be many other reasons as well).
- Offices with high walls. I mean this in a physical sense. I once had visited an office where everybody had their own office, there was no common area where people could meet, and you had the entire picture of everybody being busy. In such an environment, it can be difficult to have an accidental conversation and almost impossible to have a planned hallway conversation (one where you see somebody leaving their seat and follow them to have a quick word with them outside of a planned formal discussion)
- Offices with metaphysical high walls. You know this kind, this could even be a sub-group within an organization. One is really not expected to try to have informal discussions on work related issues; those need to be held on email or through planned meetings (which could be noted down and notes sent). This may seem the right way of doing things, or it could just be the kind of culture that is being inculcated within the organization; no matter the reason, it makes discussions much more formal. To have any discussions with other people, one needs to set request meetings, which adds a lot more overhead to the entire process, and even though there are some advantages, in my view, it can be a lot more cumbersome.

So what can happen ? Well, the beauty of hallway conversations is that a lot of it happens naturally. You see a person, and suddenly something strikes you about some issue you have in your mind - this could be some technical problem that comes to mind when you see a more skilled computer engineer, it could be some part of the requirements that is not clear which comes to mind when you see the product manager, it could be some problem in a defect that comes to your mind when you see a tester - there can be many combinations. In a number of cases, the solution happens then and there, and is there no better and less cumbersome method of resolution.

How do you encourage hallway conversations ?
- Culture. Managers across the different teams need to ensure that people can approach other people if they meet them and have a conversation. Of course, this should only be till a reasonable level, and should not be used when a more detailed discussion is required with multiple people (and most people have an idea about where a hallway conversation will do, and where a formal meeting is required)
- Infrastructure. I also saw another office where there are a number of small meeting rooms, with only 3-4 seats in each (apart from the larger conference rooms). The idea being that if the hallway conversation lasts for more than a couple of minutes, both the people can pick up their coffee and have a slightly longer conversation in one of these rooms.

Letting participants of meetings get quick meeting notes right away

The concept of meeting notes is a much debated concept, with people having their own concepts of the timing, method and tracking of meeting notes. There are dedicated software for tracking and generating meeting notes, with one of the software being from Microsoft, which integrates well with Microsoft Outlook. And there are mobile and web based software solutions that increase the device range. One thing is for sure, if meeting notes are not available, one is condemned to have more meetings, as well as start losing people from these meetings (many would consider non-sending of such notes as an unprofessional practise and could also highlight this issue when they decline the invitation for a meeting).
So what is the practice that I have learnt works the best ? Now, this one may not be directly possible, and it depends on the people invited to the meeting; but it worked across many different sets of people, with varying work profiles.
Most people, after having attended a meeting where one or more issue has been discussed would have the issue fresh in their mind. It is also possible that even though during the discussion, people may not have proposed a solution, by the time that the meeting disperses and they go back to their offices, the solution or proposed paths to the solution would be coming to them. At such a time, if they are able to see an email which has the meeting notes along with the issue under discussion and action items, they will be able to quickly reply back.
How does this happen though ? Sending meeting notes almost at the same time as the meeting ends ? Well, for the person taking the meeting notes, it requires them to very attentive and taking down all the meaningful points of the discussion that is happening (but not required to record everything and anything - the idea is that meaningful points need to be covered, decisions notes, and action items recorded along with the person responsible for the action item and a end date by which the action item needs to be completed).
If this kind of note taking is happening, and the person taking down the notes has knowledge of the items being discussed, this kind of detail of the meeting can be put down fairly as and when the meeting is progressing, and where there is a decision that has been taken or an action item that has been put forward, it is fairly fine to actually get a clarification if there is any doubt. Once this kind of level of detail is happening, it takes very little time after the meeting to actually refine the meeting notes, add some presentation to the entire notes (in terms of bullet points, action items and due dates) and send this out.
What happens if this kind of note taking and confirmation is not possible just after the meeting (there may be another meeting happening or the matter may be very detailed and it would take time to get a precise detail), but it is still important to ensure that people remember stuff, a rough notes can be sent out while mentioning that these are rough notes and final notes will be sent out later. However, people should know / understand why it is not possible to send out the final version of the notes right now.

Need for doing a legal audit to detect all 3rd party instances

A lot of people do not even know about this concept ? What is a Legal Audit (or a similar name that may be followed by different software organizations). However, most project / program managers would know about using components from many different sources. And you would also have heard about patent disputes, where companies challenge each other about the software that they have written, and whether one of them was entitled to damages from the other for using a certain code over which the other claimed ownership (actually a patent is about the principle or concept or a specific feature, but you get the general idea).
How does this fit into the idea of something called a Legal Audit ? Patience.
Let me take a real life principle. In our team during the course of a product development cycle, the team is informed at the beginning of the cycle that they will not use any component from outside without speaking to their manager. However, during the middle of the cycle, I was speaking to the team about this (as a repeat) and later one of the team members approached me. It turned out that he was looking for an efficient XML parser and searched for an external component that would help him in this; he found something on the internet, downloaded and used it. Seems fine, after all, a lot of people might do this.
The problem was, we are living in a world where we need to respect the rights and copyrights of others, if we want others to respect our software. Our software has a global market of $40 million, and nobody would welcome a case against us for unlawful usage of an external component. It could be that we were fine with using this component, but nobody had done that kind of check. We looked at the component, and found that it had a license that was never going to be allowed for usage. The license wanted $1 for every customer usage of the software where this component was going to be installed, and if you think that we were ready to pay out tens of thousands of dollars for using such a component, I have nothing to say.
The Legal Audit is a way to do a scan of the software code to ensure that all the external components that are being used in the software are known, and the licenses are all approved towards this end. For product development where the product has been going through multiple versions, a lot of the components would have been in regular use over the years, and these can be quickly discounted. Most organizations would have a way to do this process in a way that minimizes the effort required.
Doing this process is essential, and in most cases, would require consultation with some software engineer or manager as well as with a legal expert (to sign off the final license agreements and to certify that the overall set of licenses used in the software are fine from the perspective of the organization).
And the Legal Audit can only be complete when the writing of new software is complete, since only then can it be sure that there is no further new code going to be written.

Getting statistics of build success rates and reasons for failure

During a regular product development cycle, the Daily Build is one of the most important items that have to successfully happen. If the Daily Build has failed, or is only partly successful, or even needs only some partial work to do before using, it can be a source of irritation for the development team. Even though they can use the previous build, the fixes from the previous day would not be there, which is an irritant. And if such build failures happen from time to time, the irritation can build up and cause doubts about whether the daily build can be depended upon to happen consistently.
There can always be reasons for the daily build to have failed - reasons could be some dependency problem that a developer did not verify before checking in their fix, or some hardware failure, or problems with some recent patch (once we had a case where a new Operating System patch was in turn causing the Virus Scan engine to not release one of the build process files, and it was a pain to diagnose that problem), or some other human failure on the part of the build engineer.
If the build failure happens on a periodic basis, status meetings are sure to resonate with worries about how the frequent build failures are impacting the ability of the team in terms of reduced productivity (and this worry can sometimes be enhanced from the actual levels, but it is not easy to make this statement in the status meeting). Further, it is an imperative that each and every build failure be monitored to ensure that the same reason for build failure does not happen again (or there better be a very good reason why the same reason could cause a build failure again).
The way to do this is a simple process. The build engineer can easily setup a process whereby the statistics for every day's build success are tabulated, and since it is an assumption that the reasons for the build failure or delay would have been monitored by the build engineer, these can be added to the stats for every case where there has been such an occurrence.
On a periodic basis, such a report should be sent out to all the people concerned with the build or who have been nominated by the team to track the build (we had created an email alias for the build issues and whoever wanted could become a member of the build alias, and this build report would be sent to this email alias). There were advantages of having such a status:
- It ensured that everybody had accurate figures about the number of times the build was broken or was delayed
- It pointed out issues that could be regular or cause repeated build failures and how they were handled. This one area of the report was a great confidence-builder
- We were able to monitor whether there were improvements in the build success rate (and that was a reasonable expectation)
- It also pointed out cases where the development team engineers caused failures and put pressure on the development manager to tighten the ship on the development side.

Build acceptance - Ensuring that one person certifies the build as accepted

During the regular product development cycle, the build (typically daily) is the most important input to ensure that the process of build testing, defect fixing and incorporation of the fix into the product, and testing again continues. If the build does not come on a regular basis, defects are not available for the testing team to fix on a regular basis; and it is essential that defects get into the product at a regular basis. If not, team developers start getting out of phase with respect to files being changed during defect fixes by multiple people, and more effort needs to be spent to resolve these dependencies.
Most teams typically like to get the daily product build early in the morning, before the developers and testers are in position. Such a technique ensures that when the team arrives, the build is already available and ensures the maximum utilization of time. However, as a part of this, it is also essential that the build be ready and be fully fit, not have major defects. For this purpose, it is essential that the build be verified before it is started to be used by the team.
A lot of teams have built automated smoke tests that take the build when it is available, launch it and do a quick major tests, and if everything goes right, an email could be configured to go out to a specially setup email list which would ensure that the right people knew that the build was correct. This works in most cases, but there can be cases where there are false negatives. For example, there is a small change in UI or some other feature which was not incorporated into the smoke test, then the smoke test would give a failure, but the build could have been successful (and one can think that such a case may not happen, where the information about a change would always be built into the smoke test, it can happen and has happened).
In other cases, a person needs to evaluate whether a partly successful smoke test and see whether the new build need to be built or the existing build can be used, with some assumptions. For example, one particular feature need not be fully working, but everything else could be working. Now why would the team want to use a build where not everything was working fine. Well, because there is a cost to make a new build. Building a new build could take time, many hours in some cases, and till that time, the developers and testers would have to be using an older build, which may be fine in the early stages, but gets more problematic as you move along the schedule, since such delays in getting defects fixes in the build may increase the cost and impact.
To take such calls needs to be done by a person on the team; and this needs to be specially assigned, since such a person would need to review the build and take a call before the other team members start working on the build, which would mean that the person would need to get on the job much earlier than normal.

Using videos to explain features to other team members

One of the most important processes during the entire product development process is to explain the features to the other members of the team. The product manager coordinates the requirements and the feature flow, the experience designer then converts these into a actual feature flow (screen by screen, and UI element by UI element), and then finally this is converted into an actual working application (whether this be a desktop or web or mobile application). Now, even though the different team members can be an integral part of all the discussions that have happened, and well aware of how the developer is delivering the software, there are still grounds where there can be discrepancies in the understanding between the developer and the tester (there was a case where I was deeply involved where the product requirements researcher and the developer had been interacting daily for the feature, and yet, after around a month, when the feature was developed and the researcher had a chance to look, the features were off by around 5%). I am not for a minute suggesting that such things can get down to 0% deviation, but we should all be doing steps that help in reducing the possibility of such deviations.
These were simple cases. There can be more complex features, where the feature work forms the background of another feature, or the feature needs to be tested by multiple testers, or the Help document writer has to write about the screen and element by element flow in the application, and needs to understand everything in detail (and not being involved since the beginning, I could understand why in many cases, the documents so produced led to frustration among the team and needed multiple iterations), and in the worst case, the testers are in a different time zone and different geography from the developer.
This is one idea that I saw a developer doing. He had figured out that these kind of problems above can be very frustrating, and even though he would be frustrated, and his development manager would back him up, there would be multiple times when he would have to explain the same screens (in a particular workflow that he done, there were many UI elements in the screen and some UI elements were dependent on the values entered in the other UI elements); he almost dreaded the explanations he would have to do to multiple people. What he ended up doing was, during the unit testing he was doing, he would have a screen recording application running and he would provide some running commentary while doing this. In the end, he would have videos that essentially ran through the entire gamut of options for the screen workflow and it required much less effort after that to make them run in an order that was relevant for somebody else trying to understand. He would also have explicitly budgeted for the time required for this, and was able to show the advantages of doing this (although there were always some developers who were against the idea altogether, maybe more of their personal feeling that they would end up spending too much time on the video part).

Ways to improve meetings held for conflict / issue resolution

What I write below may seem like common sense, the kind of stuff that you would get if you were to sit back and think about how to do a meeting. This is even more important if you were to read my previous post (Reducing the number of meetings held) about how to make an effort to reduce the number of meetings that are held, which can suck up the time available with the development and testing teams. In addition, with the number of meetings being held being high, the continuous time that the teams need when they are in the middle of something gets interrupted and this can have an incredibly bad effect on productivity.
During the tension and stress of an ongoing software development cycle, teams need to resolve issues fast. As a result, when there is some kind of issue that need resolution, teams forget all the basic concepts of how to setup a meeting, what are the preliminary steps that need to be done before starting a meeting, and the net result is that even though conflicts are resolved or atleast progress made, the teams end up spending a large amount of individual time on these meetings; and we know the impact this has on the team productivity.
So what is to be done in the case of some conflict or issue that needs to be resolved. Even though it may seem logical to quickly call a meeting to get a solution for the issue, this method should be retained only for those issues that are of a show-stopper nature (for example, a defect that is stopping the launching of the application, or something equally similar). In most other cases, if you spent a bit of time, you would realize that an hour or so before the meeting can help make the meeting more productive. What are some of the steps that could be done before the meeting ? In the below steps, there may be multiple people who may need to be consulted depending on the steps (these could be team leads, project manager, development manager, quality manager, etc)
- For the issue, need to identify the people who are most competent in the issue and who would be able to contribute the maximum to this issue
- For these people, need to identify whether they are available or are involved in some other critical issue. If they are so heavily involved that it would not be good to get them out of whatever they are doing, then somebody else who can provide a solution or inputs would be needed to be identified.
- Next, a brief write-up about what the issue is would need to be written up, especially focusing on what the problem and if there are any elements of a possible solution; and this would need to be sent out atleast an hour or more so before the meeting.
- Before sending out these items, the meeting that needs to be setup should also make it clear as to who all is important for the meeting, and who would be optional for the meeting. What this does is to give team members a say in whether they need to come for the meeting or not. In most cases, what would happen is that team members who are on the optional list would evaluate whether they can contribute or not and decide accordingly. This atleast helps in saving time for those who decide not to come, and even those who come for the meeting are more informed, which makes for a quicker meeting and hence more saving of time. 

Ensuring the minimum number of meetings necessary ...

In software companies, if there is one item that is necessary, it is the concept of meetings. Meetings are the life blood of companies, and sometimes, one feels that the number of meetings that are held are excessive. In my experience, I have had developers and testers coming and complaining that the number of meetings that they are being invited to is so excessive that they feel the time involved for work is getting impacted. Even further, when a person is getting into the core of their work - whether it being the design of a new feature, or the solution of a difficult defect, or the execution of a difficult test case, facing a break in between is guaranteed to rankle. And there are numerous meetings where the team members (developers / testers / others) could be called for meetings, where it would seem that they are necessary.
Some of these meetings could be:
- Feature discussion meetings (numerous)
- Schedule planning meetings
- Task definition meetings and estimation
- Issue resolution meetings
- Daily / Periodic status meetings
- Add your own set of meetings here
You get the idea. It can be pretty frustrating. I have seen weeks where there were atleast 4-5 meetings per day, and even though you would think that these meetings are important, they do cause other problems. And at some point, if the people invited for these meetings start feeling that the subject of the meeting is not important for them, then you will start getting resistance in ensuring that these meetings are held.

So what is the solution ? There is no magic bullet, no quick solution I can give that will reduce the number of meetings you need to do. It requires careful planning, discussions between the respective managers (such as the project / program manager, the development manager, and the testing manager). Some of the questions that need to be asked before a meeting is set are the following (and there would be other questions that are not in the list below, it is just an indicator of the thinking that needs to happen):
1. The invitee list for the meeting needs to be evaluated. Is each person really necessary for the meeting. Can some people be marked optional, so that they can determine for themselves whether it is necessary.
2. If there are some minor issues that could be settled in a meeting, but could also be settled through a quick phone call or a quick hallway discussion, then that makes sense. In many issues, one person really needs to make a call, and the others need to just be informed. In such cases, having a meeting is really not necessary.
3. Is the issue really necessary to be discussed urgently ? What this means is, if the meeting is pushed out by a couple of days or a week, would it still be fine ? If so, then an alternate would be to postpone the meeting and see whether the issue can be resolved through phone calls or an email discussion.
4. In some meetings, the discussion is about nobody willing to take a quick call, or because there are no real solutions. In such cases, the meeting in a number of cases will fail or another meeting will need to be held. It would be better to prepare a solution or a note that clarifies issues, and then send that out. In many cases, this would be accepted by most of the people who have the authority to accept it.

Keeping some capacity of the superstar developers free for quick resolution

In every development team, you will always have a mix of people of different technical capabilities. This may not be for lack of trying; a person may try real hard and put in a lot of effort but may still not be able to work as quickly and as efficiently as another developer who seems to have the knack of being much more skilled. The better developer will eventually get more rewards and sail through appraisals much quicker, just as long the person also puts in the hard work, besides the skill.
Because of the skill levels of the more skilled developer, in a lot of organizations, such a person would be used for more difficult technical work and yet not one that could end up to be boring or repetitive (sounds very obvious, but it can happen that the work that is assigned is not interesting enough and i have known such a skilled developer to want to join other teams when they start feeling that the work they are doing is not really interesting).
One area in which such a developer is used is for new work, or for some new feature where there are expected to be technical challenges and is also critical for the project / schedule. In such a case, one would want the best resources on the team to be on these investigations, since the actual work on these features depend on how well the initial prototype / technical research is done. Once the initial work is done, it might be feasible to hand over the actual detailed work to other members of the team and utilize the more skilled developer for other similar projects.
However, if you want to plan for amount of risk mitigation for the project, one risk that comes up again and again is about serious defects coming up during the cycle, especially near the middle to the end of the schedule. If there is a serious defect coming up in such a phase, it gets more problematic since such a defect needs to be evaluated with a lot of care. While fixing any defect, one needs to be very careful that the fix is done completely and no other problem is caused by the fixing of the defect, and it can and most likely will happen that fixing of a defect will cause some problem (the percentage of such events is low, but not zero, and will happen).
When such serious defects are found near the middle to the end of the project cycle, one really needs to be very careful that these defects are evaluated properly, risks enumerated, and any possible complications reduced or brought down to zero. And as one advances in the schedule, the time period available to do these evaluations reduces. In such a case, it is necessary that the most skilled person in the team either looks at such defects, or reviews the work done by other developers in order to reach the same conclusion.
For this to happen, some amount of time needs to remain free for the skilled developer so that he / she can be put on such defects. Else, if the developer is working full time for some other feature (and it would be an important feature), pulling aside for defect evaluation may not be possible, or may impact the other feature negatively, which has its own impact.

Trying to solve defects where the flow is not repetitive .. (contd..)

In the previous post (Trying to solve defects where the flow is not repetitive ..), we talked about defects that are not easily reproducible, and some points around that. In this post, we will talk about such defects with more points. Handling defects that are not reproducible is one of the biggest pain points of the development cycle, since you know that even though the defect is not easily reproducible, it will come back to bite you at some point or the other. So, you cannot ignore those defects, although after trying to reproduce a defect like this for hours, it is tempting to just forget those defects. It needs a strong policy to ensure that the team does not overlook such defects.

- The team needs to have a policy about handling defects that ensures that the team does not overlook some of these defects. The policy needs to define how the defect needs to be handled, how it is tracked, how many times does the team needs to re-test those scenarios, whether such defects need to be handed over to other testing personnel, and so on. These policies need to be ensured as a part of the system, so that people know exactly how to handle such defects.
- These kind of non-reproducible defects do not follow any easy pattern, but from time to time, groups of developers and testers do figure out steps on how to figure out how to reproduce some of these defects. I remember specific cases where a developer wrote an app that would be run along with the application during the testing process, and helped in around 10% of the cases of non-reproducible defects, and that was an amazing feat, since it helped save a lot of time and helped in resolving defects that were otherwise difficult to fix.
- Statistics for these kind of defects is very important, and during the development process rather than at the end. What these statistics for non-reproducible defects do is to determine for the team management about how many these kind of defects are being found, as well as how many are being resolved. Every team needs to specify a number beyond which action needs to be taken. For example, during a specific 3 week period, we found that there was a spike in the number of non-reproducible defects that were being logged. This in turn resulted in more time being spent by the testers on trying to reproducible and hence resolve these defects. A solution needed to be found for this - and we spent some more time on focusing superior development effort on these defects. Eventually, this extra effort helped, since it was a new version of an external component that was causing problems to the application and resulting in many such non-reproducible defects.

As a result, even though it is difficult to visualize handling such non-reproducible defects, they are a reality and teams need to have a process of handling such defects.

Trying to solve defects where the flow is not repetitive ..

If the title of the post did not make sense, then let me explain. What I refer to is the case where a defect has been logged (let's not discuss the severity of the defect right away), but the flow to replicate the defect is not easy. Anyhow, it is easy when a defect can be easily replicated, when the steps are easy. However, for anybody who has gone through the whole process of defect detection, analysis and resolution, finding an easy method of replicating the bug steps can almost be counted as lucky. There can be numerous cases where the steps for replicating a defect is not so easy.
Consider the case where a defect is caused due to a particular variable loading a specific value. This value has not been loaded during the current workflow, but gets loaded during a step in an unrelated workflow. Now, depending on whether the user has gone through this unrelated workflow previously, the defect will show up or will not show up. One might think that the person would be able to replicate this defect easily, but this is easy to think post-facto once the defect process has been understood. For the tester, there is no way of easily knowing that the defect is getting caused due to the operation of an unrelated workflow; it is only the skilled tester who tries the workflow inside the product multiple times who can try and figure that the defect is happening only in those cases where the unrelated workflow has been used. These kind of stories are common (not the majority of cases, but common enough that testers and developers need to be aware of these issues).
What does one do in these kind of cases ? Well, such cases are handled different by different teams, but let me suggest a few points for these:
- It is important to understand that every defect is important, and whether the defect is fixed or left unresolved needs to be decided in a determined manner. For a customer, the defect can be problematic, especially when the person used the unrelated workflow and wonders why the team left such a defect not fixed.
- Another variable that decides what needs to be done is the severity of the defect. For defects that were severe, it is not easy to decide to close the defect as unresolved. For defects that are minor, the bug decision committees would need to decide whether these minor defects need to be fixed, or can be deferred; that the team was comfortable with the minor inconvenience that such defects would cause to the customers.
- It is not easy for the developers to fix defects that the testing team cannot easily replicate, but that does not mean that they should not be fixed. There will be a larger number of customers than the testing cases within the development team, and one can be sure that some of the defects that could not be replicated will show up and cause inconvenience to the customer,
- There are methods that the development team can use to try and make it easier to find a solution to the defect. For example, it might be possible for the testing team to use special development versions of the software where the values of key variables can be recorded during the testing process, and this would help in trying to figure out exactly what is going on; and there are other development steps that can be done to try and get more detail on what exactly is causing these defects.
- Teams can setup processes on how to treat such defects; letting the tester spend some adhoc time on these defects; or give these defects to a different tester in order to get a fresh set of eyes; or ensure that the developer has some time for these defects in order to try and get some sort of solutions for atleast some of these defects.

Read more about this in the next post (Trying to solve defects where the flow is not repetitive .. (contd..))

Tracking platform usage and making decisions based on that

Even in today's data based world, if you are an analytics expert, you can't expect to be totally popular, or that people will welcome you with hands outstretched. There are many aspects of a product development cycle that would benefit from integration with a data analytics cycle - generating the questions, collecting the data, and the extremely important task of generating the output information and conclusions (and a wrong conclusion at this point can have negative implications for the product - putting in effort in wrong or unimportant areas). However, consider the case where there is support for using analytics and there are resources dedicated for ensuring that requisite changes to the product can be made (based on the analytics output).
One major area where the team needs to gather information is on the area of platform support. Platform could mean the Operating System (MacOS, Windows, others), Versions (Windows XP, Windows 7/8, etc) as well as browsers support (versions of Chrome, Internet Explorer, Firefox, Opera, etc). These can make a lot of different in terms of the amount of effort required for product development. For example, on different versions of Windows, the versions of systems files are different and it is a pain to support these different versions. There can be small changes to the functionality of the system based on these files, and in many cases, the product installer would actually detect the Windows version and install the required versions of these systems files. If you could find out that the number of people using these different versions, and find out that one of these versions is being used by a small number of consumers, then the decision to drop that particular operating system may be taken.
So, this is one of the areas in which analytics can work. The simplest way to do that is to make it part of the functionality of the product, where the product, on the user's computer, dials home and provides information such as the platform on which the product has been installed. Once this kind of data has been collected and a proper analysis is done, then the product team can look at this information and factor that into the feature list for the next version of the product. The team will need to decide on the time period for which the data would be captured, and also some of the benchmarks which will decide whether data from the analytics effort needed to be used for making decision (for example, the data seems to be such that people feel that the data is widely different from public perception).
However, this is also dependent on how much the team can depend on this data and the analysis; after all, even a small variation during the analysis can result in information that has levels of inaccuracies in it. But, it is necessary that the team spends effort in the analytics effort, since the payoff from using accurate data analysis and interpretation is very high.

Trying to get non-responsive members of the team be more schedule sensitive

We know this problem, it happens all the time. You have different members of the team, some more disciplined and some less disciplined. Actually discipline is the wrong word. When you have creative members of the team, or team members who are attached to multiple projects, then there can be problems with respect to scheduling of their deliverables. In the case of team members such as User Interface Designers or Visual Experts, or Visual Designers, they typically do not move to the same beat as that of the rest of the project teams, such as Engineers or Testing Engineers.
This can be problematic for the rest of the team, since schedules are interlinked to each other. For example, the User Interface Designer would prepare design specifications that are used by the team members to discuss and finalize the feature workflow and the technical architecture. These are then developed and passed onto the testing team which does the testing and then releases the feature. However, if the initial design does not come in time, then the rest of the schedule will get impacted.
One of the problems that I have experienced with User Designers or similar creative people is that they do not work in pieces; they would like to look at the overall workflow for the product and then release a completed design. But the team does not work like this, it would like workflow designs feature by feature, so that the work can be done feature by feature (and it makes logical sense).
Another option that could have been postulated is that the Workflow Designer could have a period of 2-3 months before the start of the cycle, so that the Designer gets enough time to make the design. This seems logical, but there are problems in this. The Workflow Designer does not work entirely on his / her own, but needs to work with the Product Manager and the team members (the team members  are involved so that the team could figure out the technical cost of doing the Workflow designs; some of these workflows may take more time and effort than other workflows and the contribution of the technical team in figuring out these is critical. This process can be iterative).
So how do you work out trying to get such more creative members as part of the process?
- First and foremost, it is necessary to ensure that you do not make the assumption that these resources understand the critical nature of meeting their schedule deliveries. It would be needed to spend much more time with these people and form a detailed plan for deliveries, doing this discussion multiple times till an understanding has been formed.
- In my experience, it was also necessary to have 2 dates in the schedule with a few dates gap between the 2 dates. It was necessary to push the delivery to happen for the first date, but there was also the understanding that the delivery happening on the 2nd date would also work fine without threatening the schedule.
- It was also realized that there was the need for a regular reminder along with checking about state of progress and updating the rest of the team on such progress. So, the Project Manager had setup a weekly meeting with the workflow designer to discuss the state of progress and the deliverable, and figure out alternatives if there was a delay.

Emergency defect fixing: Giving local fixes for quick verification

During the process of defect fixing and verification, there is a standard process whereby a build process is created and defect fixes checked into this build. What this does is to ensure that every day (the builds typically come every day) the defects which were fixed the previous day are available for testing in a proper installer which can be used by the testing team similar to the product that is available to the customer.
This process works pretty well, as long as everybody involved knows the process well, and there are people in the process who have responsibilities for the different parts of the process (for example, somebody who ensures that the build stability systems are in place, others who do a quick smoke testing to ensure that the build is usable), and so on.
However, such a system cannot protect against a case where a defect has not been fixed, either fully, or partially. In the normal case of operations, it is normal to have defects that are not fixed and are rejected by the testing team, or some part of the defect not being fixed well, and a new defect being filed for the same.
When does this process not work ? Consider the case where the product development process is nearing the end of the schedule. In such a case, the defects that are to be fixed are restricted, and only those defects that are being allocated for fixing are passed onto the development team and the list given to the testing team for verification. However, the cost of a failed defect fix can be fairly high.
A defect fix that has failed would mean that the build for that day is not ready for use, and this can be very expensive.
When such parts of the schedules have been reached, there is the need for much closer interaction between the specific developer and the tester(s) for that defect fix. When the developer has made the fix, he/she would work with the tester and provide the fix in a local build, made on the developer's machine, which could be quickly tested by the tester to ensure that the fix has been made to the satisfaction of both the developer and the tester. This goes a long way to help that the build that comes the next day is usable and important fixes are not failing.
There are some problems that can still happen in this process. The local build may not be incorporating changes made by the other developers, and this can cause a dependency problem that may still cause the defect fix to fail. However, the chance of this happening is low (or can be monitored by the developer to reduce the failure rate) and goes a long way to ensure that the development process has reduced risk near the end of the cycle.
However, this requires close collaboration between the tester and the developer, and is not really required to be done in the regular part of the development cycle, since there is an overhead involved in the process.