Using the latest version of external components - Needs attention

Most software applications do use components that are not created by the software team; these could be components created by other teams within the organization, or by using components created by other organizations. For so many different functions, it is not required that the team try to do everything on their own; for example, you could consider many different types of parsers, or system access components, or image manipulation components, or video codecs / decoders, etc. In all of these, these are specialized components created by people far more competent in these areas; it would always make sense to use these components rather than try to replicate such software within our own application.
When your software is larger, there would be more such components that are in use which are created by other parties. Over a period of time, it gets to be a task to track all such components. I have experience where we used such components the first time many years back, pay for using the component at every new version of the application, and the number of such components that are in use is well over 50 within the application (some of those are free, others are paid with minimal charges, and some can be pretty expensive - some of the specialised video codecs can be pretty expensive, although one cannot do business in the video area without using these codecs in order to look professional).
Just like our software is updated with every version with new features and defects, even the components that we use also get new versions. In many cases, we could still continue using the older components since the newer version does not promise any new features that we need, and we can avoid the overhead of trying to incorporate a new version. However, it is still required to monitor these external components to ensure that we know the reasons why the new component has been released.
In many cases, there may have been some critical defects found in the component which required a new version to be released. In such a case, we should need to evaluate the defects that were found to determine whether it is required for us to take the new version. In some cases, the defects could be in a section of the component that we do not use, and hence we could still continue to use the old component. But in many cases, the defects are serious enough that it could pose a risk to the software application unless the new version of the component has been taken. However, taking the component will have an overhead, since other changes that have been implemented in the component would need to be evaluated and testing done to ensure that no other impact is caused due to these changes.
This whole process of ensuring a watch on these components is troublesome and takes time and effort, but it is essential. One would not want to release a software that integrates an older version of the component which has some serious risk. This can have further impact, and in some cases, the external component provider mandates that the newer version of the component has to be used.

Hallway discussions - need to be encouraged

Is this really a post worth writing about ? Talking about hallway discussions ? I believe so. Sometimes one should write about obvious items, but which do not happen - either do not happen at all, or happen in such low frequency that they need to be improved.
First of all, what are hallway discussions ? It can be a pretty broad term that can be tentatively be used to describe those conversations that happen in an unstructured way (outside of conference rooms), where people end up discussing an issue in the pantry, or getting in and out of the restroom, or even in the hallway when they run into each other. It can happen between people of very different seniority, and can be a short conversation or can be an extended conversation that could take many different directions.
First of all, how do you not encourage hallway conversations ? It would seem like a very natural thing to happen ? There are some reasons why such conversations can become rare, some by choice, and some due to the organization (and this is not a complete list, there could be many other reasons as well).
- Offices with high walls. I mean this in a physical sense. I once had visited an office where everybody had their own office, there was no common area where people could meet, and you had the entire picture of everybody being busy. In such an environment, it can be difficult to have an accidental conversation and almost impossible to have a planned hallway conversation (one where you see somebody leaving their seat and follow them to have a quick word with them outside of a planned formal discussion)
- Offices with metaphysical high walls. You know this kind, this could even be a sub-group within an organization. One is really not expected to try to have informal discussions on work related issues; those need to be held on email or through planned meetings (which could be noted down and notes sent). This may seem the right way of doing things, or it could just be the kind of culture that is being inculcated within the organization; no matter the reason, it makes discussions much more formal. To have any discussions with other people, one needs to set request meetings, which adds a lot more overhead to the entire process, and even though there are some advantages, in my view, it can be a lot more cumbersome.

So what can happen ? Well, the beauty of hallway conversations is that a lot of it happens naturally. You see a person, and suddenly something strikes you about some issue you have in your mind - this could be some technical problem that comes to mind when you see a more skilled computer engineer, it could be some part of the requirements that is not clear which comes to mind when you see the product manager, it could be some problem in a defect that comes to your mind when you see a tester - there can be many combinations. In a number of cases, the solution happens then and there, and is there no better and less cumbersome method of resolution.

How do you encourage hallway conversations ?
- Culture. Managers across the different teams need to ensure that people can approach other people if they meet them and have a conversation. Of course, this should only be till a reasonable level, and should not be used when a more detailed discussion is required with multiple people (and most people have an idea about where a hallway conversation will do, and where a formal meeting is required)
- Infrastructure. I also saw another office where there are a number of small meeting rooms, with only 3-4 seats in each (apart from the larger conference rooms). The idea being that if the hallway conversation lasts for more than a couple of minutes, both the people can pick up their coffee and have a slightly longer conversation in one of these rooms.

Letting participants of meetings get quick meeting notes right away

The concept of meeting notes is a much debated concept, with people having their own concepts of the timing, method and tracking of meeting notes. There are dedicated software for tracking and generating meeting notes, with one of the software being from Microsoft, which integrates well with Microsoft Outlook. And there are mobile and web based software solutions that increase the device range. One thing is for sure, if meeting notes are not available, one is condemned to have more meetings, as well as start losing people from these meetings (many would consider non-sending of such notes as an unprofessional practise and could also highlight this issue when they decline the invitation for a meeting).
So what is the practice that I have learnt works the best ? Now, this one may not be directly possible, and it depends on the people invited to the meeting; but it worked across many different sets of people, with varying work profiles.
Most people, after having attended a meeting where one or more issue has been discussed would have the issue fresh in their mind. It is also possible that even though during the discussion, people may not have proposed a solution, by the time that the meeting disperses and they go back to their offices, the solution or proposed paths to the solution would be coming to them. At such a time, if they are able to see an email which has the meeting notes along with the issue under discussion and action items, they will be able to quickly reply back.
How does this happen though ? Sending meeting notes almost at the same time as the meeting ends ? Well, for the person taking the meeting notes, it requires them to very attentive and taking down all the meaningful points of the discussion that is happening (but not required to record everything and anything - the idea is that meaningful points need to be covered, decisions notes, and action items recorded along with the person responsible for the action item and a end date by which the action item needs to be completed).
If this kind of note taking is happening, and the person taking down the notes has knowledge of the items being discussed, this kind of detail of the meeting can be put down fairly as and when the meeting is progressing, and where there is a decision that has been taken or an action item that has been put forward, it is fairly fine to actually get a clarification if there is any doubt. Once this kind of level of detail is happening, it takes very little time after the meeting to actually refine the meeting notes, add some presentation to the entire notes (in terms of bullet points, action items and due dates) and send this out.
What happens if this kind of note taking and confirmation is not possible just after the meeting (there may be another meeting happening or the matter may be very detailed and it would take time to get a precise detail), but it is still important to ensure that people remember stuff, a rough notes can be sent out while mentioning that these are rough notes and final notes will be sent out later. However, people should know / understand why it is not possible to send out the final version of the notes right now.

Need for doing a legal audit to detect all 3rd party instances

A lot of people do not even know about this concept ? What is a Legal Audit (or a similar name that may be followed by different software organizations). However, most project / program managers would know about using components from many different sources. And you would also have heard about patent disputes, where companies challenge each other about the software that they have written, and whether one of them was entitled to damages from the other for using a certain code over which the other claimed ownership (actually a patent is about the principle or concept or a specific feature, but you get the general idea).
How does this fit into the idea of something called a Legal Audit ? Patience.
Let me take a real life principle. In our team during the course of a product development cycle, the team is informed at the beginning of the cycle that they will not use any component from outside without speaking to their manager. However, during the middle of the cycle, I was speaking to the team about this (as a repeat) and later one of the team members approached me. It turned out that he was looking for an efficient XML parser and searched for an external component that would help him in this; he found something on the internet, downloaded and used it. Seems fine, after all, a lot of people might do this.
The problem was, we are living in a world where we need to respect the rights and copyrights of others, if we want others to respect our software. Our software has a global market of $40 million, and nobody would welcome a case against us for unlawful usage of an external component. It could be that we were fine with using this component, but nobody had done that kind of check. We looked at the component, and found that it had a license that was never going to be allowed for usage. The license wanted $1 for every customer usage of the software where this component was going to be installed, and if you think that we were ready to pay out tens of thousands of dollars for using such a component, I have nothing to say.
The Legal Audit is a way to do a scan of the software code to ensure that all the external components that are being used in the software are known, and the licenses are all approved towards this end. For product development where the product has been going through multiple versions, a lot of the components would have been in regular use over the years, and these can be quickly discounted. Most organizations would have a way to do this process in a way that minimizes the effort required.
Doing this process is essential, and in most cases, would require consultation with some software engineer or manager as well as with a legal expert (to sign off the final license agreements and to certify that the overall set of licenses used in the software are fine from the perspective of the organization).
And the Legal Audit can only be complete when the writing of new software is complete, since only then can it be sure that there is no further new code going to be written.

Getting statistics of build success rates and reasons for failure

During a regular product development cycle, the Daily Build is one of the most important items that have to successfully happen. If the Daily Build has failed, or is only partly successful, or even needs only some partial work to do before using, it can be a source of irritation for the development team. Even though they can use the previous build, the fixes from the previous day would not be there, which is an irritant. And if such build failures happen from time to time, the irritation can build up and cause doubts about whether the daily build can be depended upon to happen consistently.
There can always be reasons for the daily build to have failed - reasons could be some dependency problem that a developer did not verify before checking in their fix, or some hardware failure, or problems with some recent patch (once we had a case where a new Operating System patch was in turn causing the Virus Scan engine to not release one of the build process files, and it was a pain to diagnose that problem), or some other human failure on the part of the build engineer.
If the build failure happens on a periodic basis, status meetings are sure to resonate with worries about how the frequent build failures are impacting the ability of the team in terms of reduced productivity (and this worry can sometimes be enhanced from the actual levels, but it is not easy to make this statement in the status meeting). Further, it is an imperative that each and every build failure be monitored to ensure that the same reason for build failure does not happen again (or there better be a very good reason why the same reason could cause a build failure again).
The way to do this is a simple process. The build engineer can easily setup a process whereby the statistics for every day's build success are tabulated, and since it is an assumption that the reasons for the build failure or delay would have been monitored by the build engineer, these can be added to the stats for every case where there has been such an occurrence.
On a periodic basis, such a report should be sent out to all the people concerned with the build or who have been nominated by the team to track the build (we had created an email alias for the build issues and whoever wanted could become a member of the build alias, and this build report would be sent to this email alias). There were advantages of having such a status:
- It ensured that everybody had accurate figures about the number of times the build was broken or was delayed
- It pointed out issues that could be regular or cause repeated build failures and how they were handled. This one area of the report was a great confidence-builder
- We were able to monitor whether there were improvements in the build success rate (and that was a reasonable expectation)
- It also pointed out cases where the development team engineers caused failures and put pressure on the development manager to tighten the ship on the development side.

Build acceptance - Ensuring that one person certifies the build as accepted

During the regular product development cycle, the build (typically daily) is the most important input to ensure that the process of build testing, defect fixing and incorporation of the fix into the product, and testing again continues. If the build does not come on a regular basis, defects are not available for the testing team to fix on a regular basis; and it is essential that defects get into the product at a regular basis. If not, team developers start getting out of phase with respect to files being changed during defect fixes by multiple people, and more effort needs to be spent to resolve these dependencies.
Most teams typically like to get the daily product build early in the morning, before the developers and testers are in position. Such a technique ensures that when the team arrives, the build is already available and ensures the maximum utilization of time. However, as a part of this, it is also essential that the build be ready and be fully fit, not have major defects. For this purpose, it is essential that the build be verified before it is started to be used by the team.
A lot of teams have built automated smoke tests that take the build when it is available, launch it and do a quick major tests, and if everything goes right, an email could be configured to go out to a specially setup email list which would ensure that the right people knew that the build was correct. This works in most cases, but there can be cases where there are false negatives. For example, there is a small change in UI or some other feature which was not incorporated into the smoke test, then the smoke test would give a failure, but the build could have been successful (and one can think that such a case may not happen, where the information about a change would always be built into the smoke test, it can happen and has happened).
In other cases, a person needs to evaluate whether a partly successful smoke test and see whether the new build need to be built or the existing build can be used, with some assumptions. For example, one particular feature need not be fully working, but everything else could be working. Now why would the team want to use a build where not everything was working fine. Well, because there is a cost to make a new build. Building a new build could take time, many hours in some cases, and till that time, the developers and testers would have to be using an older build, which may be fine in the early stages, but gets more problematic as you move along the schedule, since such delays in getting defects fixes in the build may increase the cost and impact.
To take such calls needs to be done by a person on the team; and this needs to be specially assigned, since such a person would need to review the build and take a call before the other team members start working on the build, which would mean that the person would need to get on the job much earlier than normal.

Using videos to explain features to other team members

One of the most important processes during the entire product development process is to explain the features to the other members of the team. The product manager coordinates the requirements and the feature flow, the experience designer then converts these into a actual feature flow (screen by screen, and UI element by UI element), and then finally this is converted into an actual working application (whether this be a desktop or web or mobile application). Now, even though the different team members can be an integral part of all the discussions that have happened, and well aware of how the developer is delivering the software, there are still grounds where there can be discrepancies in the understanding between the developer and the tester (there was a case where I was deeply involved where the product requirements researcher and the developer had been interacting daily for the feature, and yet, after around a month, when the feature was developed and the researcher had a chance to look, the features were off by around 5%). I am not for a minute suggesting that such things can get down to 0% deviation, but we should all be doing steps that help in reducing the possibility of such deviations.
These were simple cases. There can be more complex features, where the feature work forms the background of another feature, or the feature needs to be tested by multiple testers, or the Help document writer has to write about the screen and element by element flow in the application, and needs to understand everything in detail (and not being involved since the beginning, I could understand why in many cases, the documents so produced led to frustration among the team and needed multiple iterations), and in the worst case, the testers are in a different time zone and different geography from the developer.
This is one idea that I saw a developer doing. He had figured out that these kind of problems above can be very frustrating, and even though he would be frustrated, and his development manager would back him up, there would be multiple times when he would have to explain the same screens (in a particular workflow that he done, there were many UI elements in the screen and some UI elements were dependent on the values entered in the other UI elements); he almost dreaded the explanations he would have to do to multiple people. What he ended up doing was, during the unit testing he was doing, he would have a screen recording application running and he would provide some running commentary while doing this. In the end, he would have videos that essentially ran through the entire gamut of options for the screen workflow and it required much less effort after that to make them run in an order that was relevant for somebody else trying to understand. He would also have explicitly budgeted for the time required for this, and was able to show the advantages of doing this (although there were always some developers who were against the idea altogether, maybe more of their personal feeling that they would end up spending too much time on the video part).

Ways to improve meetings held for conflict / issue resolution

What I write below may seem like common sense, the kind of stuff that you would get if you were to sit back and think about how to do a meeting. This is even more important if you were to read my previous post (Reducing the number of meetings held) about how to make an effort to reduce the number of meetings that are held, which can suck up the time available with the development and testing teams. In addition, with the number of meetings being held being high, the continuous time that the teams need when they are in the middle of something gets interrupted and this can have an incredibly bad effect on productivity.
During the tension and stress of an ongoing software development cycle, teams need to resolve issues fast. As a result, when there is some kind of issue that need resolution, teams forget all the basic concepts of how to setup a meeting, what are the preliminary steps that need to be done before starting a meeting, and the net result is that even though conflicts are resolved or atleast progress made, the teams end up spending a large amount of individual time on these meetings; and we know the impact this has on the team productivity.
So what is to be done in the case of some conflict or issue that needs to be resolved. Even though it may seem logical to quickly call a meeting to get a solution for the issue, this method should be retained only for those issues that are of a show-stopper nature (for example, a defect that is stopping the launching of the application, or something equally similar). In most other cases, if you spent a bit of time, you would realize that an hour or so before the meeting can help make the meeting more productive. What are some of the steps that could be done before the meeting ? In the below steps, there may be multiple people who may need to be consulted depending on the steps (these could be team leads, project manager, development manager, quality manager, etc)
- For the issue, need to identify the people who are most competent in the issue and who would be able to contribute the maximum to this issue
- For these people, need to identify whether they are available or are involved in some other critical issue. If they are so heavily involved that it would not be good to get them out of whatever they are doing, then somebody else who can provide a solution or inputs would be needed to be identified.
- Next, a brief write-up about what the issue is would need to be written up, especially focusing on what the problem and if there are any elements of a possible solution; and this would need to be sent out atleast an hour or more so before the meeting.
- Before sending out these items, the meeting that needs to be setup should also make it clear as to who all is important for the meeting, and who would be optional for the meeting. What this does is to give team members a say in whether they need to come for the meeting or not. In most cases, what would happen is that team members who are on the optional list would evaluate whether they can contribute or not and decide accordingly. This atleast helps in saving time for those who decide not to come, and even those who come for the meeting are more informed, which makes for a quicker meeting and hence more saving of time. 

Ensuring the minimum number of meetings necessary ...

In software companies, if there is one item that is necessary, it is the concept of meetings. Meetings are the life blood of companies, and sometimes, one feels that the number of meetings that are held are excessive. In my experience, I have had developers and testers coming and complaining that the number of meetings that they are being invited to is so excessive that they feel the time involved for work is getting impacted. Even further, when a person is getting into the core of their work - whether it being the design of a new feature, or the solution of a difficult defect, or the execution of a difficult test case, facing a break in between is guaranteed to rankle. And there are numerous meetings where the team members (developers / testers / others) could be called for meetings, where it would seem that they are necessary.
Some of these meetings could be:
- Feature discussion meetings (numerous)
- Schedule planning meetings
- Task definition meetings and estimation
- Issue resolution meetings
- Daily / Periodic status meetings
- Add your own set of meetings here
You get the idea. It can be pretty frustrating. I have seen weeks where there were atleast 4-5 meetings per day, and even though you would think that these meetings are important, they do cause other problems. And at some point, if the people invited for these meetings start feeling that the subject of the meeting is not important for them, then you will start getting resistance in ensuring that these meetings are held.

So what is the solution ? There is no magic bullet, no quick solution I can give that will reduce the number of meetings you need to do. It requires careful planning, discussions between the respective managers (such as the project / program manager, the development manager, and the testing manager). Some of the questions that need to be asked before a meeting is set are the following (and there would be other questions that are not in the list below, it is just an indicator of the thinking that needs to happen):
1. The invitee list for the meeting needs to be evaluated. Is each person really necessary for the meeting. Can some people be marked optional, so that they can determine for themselves whether it is necessary.
2. If there are some minor issues that could be settled in a meeting, but could also be settled through a quick phone call or a quick hallway discussion, then that makes sense. In many issues, one person really needs to make a call, and the others need to just be informed. In such cases, having a meeting is really not necessary.
3. Is the issue really necessary to be discussed urgently ? What this means is, if the meeting is pushed out by a couple of days or a week, would it still be fine ? If so, then an alternate would be to postpone the meeting and see whether the issue can be resolved through phone calls or an email discussion.
4. In some meetings, the discussion is about nobody willing to take a quick call, or because there are no real solutions. In such cases, the meeting in a number of cases will fail or another meeting will need to be held. It would be better to prepare a solution or a note that clarifies issues, and then send that out. In many cases, this would be accepted by most of the people who have the authority to accept it.

Keeping some capacity of the superstar developers free for quick resolution

In every development team, you will always have a mix of people of different technical capabilities. This may not be for lack of trying; a person may try real hard and put in a lot of effort but may still not be able to work as quickly and as efficiently as another developer who seems to have the knack of being much more skilled. The better developer will eventually get more rewards and sail through appraisals much quicker, just as long the person also puts in the hard work, besides the skill.
Because of the skill levels of the more skilled developer, in a lot of organizations, such a person would be used for more difficult technical work and yet not one that could end up to be boring or repetitive (sounds very obvious, but it can happen that the work that is assigned is not interesting enough and i have known such a skilled developer to want to join other teams when they start feeling that the work they are doing is not really interesting).
One area in which such a developer is used is for new work, or for some new feature where there are expected to be technical challenges and is also critical for the project / schedule. In such a case, one would want the best resources on the team to be on these investigations, since the actual work on these features depend on how well the initial prototype / technical research is done. Once the initial work is done, it might be feasible to hand over the actual detailed work to other members of the team and utilize the more skilled developer for other similar projects.
However, if you want to plan for amount of risk mitigation for the project, one risk that comes up again and again is about serious defects coming up during the cycle, especially near the middle to the end of the schedule. If there is a serious defect coming up in such a phase, it gets more problematic since such a defect needs to be evaluated with a lot of care. While fixing any defect, one needs to be very careful that the fix is done completely and no other problem is caused by the fixing of the defect, and it can and most likely will happen that fixing of a defect will cause some problem (the percentage of such events is low, but not zero, and will happen).
When such serious defects are found near the middle to the end of the project cycle, one really needs to be very careful that these defects are evaluated properly, risks enumerated, and any possible complications reduced or brought down to zero. And as one advances in the schedule, the time period available to do these evaluations reduces. In such a case, it is necessary that the most skilled person in the team either looks at such defects, or reviews the work done by other developers in order to reach the same conclusion.
For this to happen, some amount of time needs to remain free for the skilled developer so that he / she can be put on such defects. Else, if the developer is working full time for some other feature (and it would be an important feature), pulling aside for defect evaluation may not be possible, or may impact the other feature negatively, which has its own impact.