The managers of the team looking at reviewing processes before the start of every project / product cycle

One of the most critical factors in making a project cycle successful is the processes that are used by the team. What do processes do ? Processes define what the team is supposed to do in different circumstances, whether this be part of the regular routine work of the company, or whether this be involved with exceptional circumstances. Some of the most critical and regular processes that a team deals with and which define whether the team will be successful or not are:
- The process used by the team members for the design phase (whether this be the high level design or the low level design)
- Processes during the coding phase (code re-use, code marking and modification processes (such as commenting with the code, etc), code review process, etc)
- Processes used during the testing phase (process of filing a defect, determining whether the defect has indeed been fixed and can be closed, etc)
Like these, there are numerous processes that are used during the development project, and they play a critical role in ensuring that the work gets done efficiently, and at the same time, as desired. A team may be using processes that have evolved after a lot of experience, or could have been designed by a small set of people (for example, when we wanted to incorporate a component from a new third party vendor, we had to define a process for testing, defect logging and communication with the vendor that worked for the vendor and for us, else things can go real haywire).
When the team is doing one release after another, the processes that they are following are typically those used in the previous release project, and since people are used to these processes, it is pretty easy to follow these processes. However, it is important that the management of the team spend some time in collecting information about how well these existing processes have worked, whether there is a need of modification of some of these processes, and whether some new processes need to be added.
One of the prime examples of collecting such information is the post-mortem meeting. One of the discussion points can be the processes being used in the team, and whether the team feels that there needs to be some modifications in the process being followed, or whether there needs to be any new processes to be created (and such a discussion cannot just start in the post-mortem meeting, but the team needs to be informed that the topic of the new processes will come up in the post-mortem meeting, and that gives them more time to bring the points for the meeting; in fact, the team managers should think through some seed points for such a discussion and include this in the earlier communication, and that will encourage the team to think through this in even more detail).
But this cannot be just restricted to one meeting. If required, specific agenda groups within the team need to be formed to explore some of these processes (for example, every cycle we set up a team to discuss improvements in the coding processes, including the code review process to see what are some of the improvements that are possible and whether some of the processes are no longer applicable and need to be dropped).
The managers can also play a critical role in such a discussion. Each manager can hold a similar discussion within their teams and figure out what are some of the improvements or changes that the team is looking for. When there are processes involved with the external teams, it is important to do a discussion with the external teams to figure out tweaks with such processes as well. There is not a single step to modify such processes, it can be incremental, taking several weeks or even months, but there has to be clarity that initiating such a process of review needs to happen.

Critical to do a smoke testing of the regular build - saves time and effort

For teams that have setup a smoke test of their regular builds, the benefits are such that they may not even understand about why such a post needs to be done; after all, it seems to be obvious that if you put a smoke test to do a quick check of the regular build (in our case, it was a daily build), there are multiple advantages. The most prominent advantage is that as soon as the build is available, the smoke test can be triggered to be run automatically, which in turn will take the build through a series of tests and determine whether there is a problem that needs to be resolved.
Sounds great, right ? After all, there are all advantages to having such a smoke testing of the regular build, but a number of teams do not use it, and at the same time, there are some additional effort that does need to be added in the calculation. Let me lay out the case which we used to have. Because of the size of our code and the amount of synchronizations that needed to be done with the source safe, we would have a daily build that would kick off in the night around 11 PM and typically get done around 7 AM. Now, the problem we used to have was that it was not possible to get anybody to be there to check whether, even if the build did get formed, whether the build would even launch and whether there were any major components or workflows that were broken.
When somebody from the testing team would typically get in around 9 AM, they would check the build and then provide their feedback. Our earnest hope used to be that there no problems revealed in this check. However, once in a while, the morning verification would reveal some problems that needed to be fixed, and it would mean that the build for that day would be late or would even have to be canceled, which had some impact on the defect find / fix rate.
The ideal solution was to have a smoke test run on the daily build. Now, the problem was that thought it sounded like an ideal solution, there were some incremental efforts that we had to calculate:
- Somebody had to write the smoke test cases and integrate that with the build system to provide an email output to several members of the team about the success or failure of the smoke test
- As and when we made some changes to any of the major UI or the workflows, we would need to update the smoke test to incorporate these changes. Sometimes these would get missed, and then you would find that the Smoke test stopped in the middle of some feature, and then it would take effort to do the diagnostic of what the problem was.
- The smoke test did yield some false positives as well. The team would then have to spend some time looking for the break in the build, and also examine whether this was a genuine catch, or whether the smoke test malfunctioned, and the problem could be ignored.
- The tools for running the smoke test could be expensive and the expenditure may need to be approved by somebody who does not understand as to why there is a need for such a software, so some amount of negotiation and discussion would be needed.
- Suppose the smoke test would start running at 7:30 AM and run for around 45 minutes, somebody from the team would need to be in place or looking at the report from the smoke testing at the completion time of the smoke test, and hence there would be a rotating chair where team members would be assigned this role for a specific period.

Ensuring automated scripts to test components that need to be integrated into the software product

Most modern software products use components as a part of their system. If you consider the case that we use as an example from time to time, of a software that generates greeting cards and use videos and images that the user can provide, then you can understand the need for components to make the software development process faster and more efficient. In the given product, the software would need to be able to read a large number of different video formats and use them in the greeting card to be generated. However, it would be very inefficient and would take a much longer time to develop a similar functionality that external components (free or paid) would be able to provide. Most applications use a similar kind of architecture, and hence the software development schedule incorporates the need to take ongoing versions of these components during the product schedule.
However, using such components also adds a large amount of risk to the software development project. External components typically do not follow the same schedule as the software application. In addition, external components can have a release for which the quality is suspect and unknown, and it is a problem taking such a component. We were incorporating some open source components for which we knew that quality could be a problem sometimes on the latest release (and hence we would always take a previous stable release), but for a couple of years, we were running into a problem even in a case where we were paying a vendor for the component delivery.
The component was having quality issues which indicated that a comprehensive quality scan was not happening, and this would only be clear only when some detailed testing was happening at our end (in one case, a significant problem only happened during the testing conducted by a Beta tester). Even while discussion was happening with the vendor to synchronize our testing policies, there was a risk associated with the component that needs to be handled.
In our case, given the number of components that we integrated and in some cases, our lack of control on the schedule of the external component, we needed to figure out the way to manage these risks. The only way we could do this was to do a much more comprehensive testing at our end. However, given that we had an iterative cycle where we would receive a component, report issues, get a new version of the component, and so on, it was very expensive in terms of testing time to keep on doing a comprehensive testing of these components every time we received a new version of this component.
We did have a budget for automation of some of our testing cases, and we decided to focus on these components. The execution plan was simple - prioritize the components that were received multiple times and automate the major testing cases for these components. Running these automation cases took significantly much less time time than the manual testing and found defects much earlier. This also gave a learning to use that we shared with other teams, since the benefits of automation of testing in these cases was much higher - after all, the cycle of testing-defect-fix-test was much more efficient when the code was written by the developers in the product rather than when this was related to a component written by an external team.

Trying to determine a risk index based on heuristics - would be a good start

Risk planning is a critical part of any project / program manager's job. Anytime during the course of a development cycle, the project may be exposed to a number of risks. If the team has not done as well a job of risk planning as it should have, they can land in a very unpleasant condition; and even if they do got help, the first question that anybody would fire at the manager would be about not properly anticipating these kind of problems and being ready for such problems as they occur.
So what can you do in terms of risk planning even before a project has started, and then continue doing through the course of a project ? There will be always be new risks or problems that will arise during the course of a project that have not happened previously or did not have the same gravity earlier, but there will always be risks that could have been anticipated in advance. And this is where a good project / program manager can excel, in terms of doing a good job at risk planning.
So what does this mean ? Well, the simplest case is if you are working on a new version of a product where there have been earlier versions. In such cases, if you have been around for previous versions, you typically have a good idea of what the risks are like and also know some ways of handling such risks. For example, you know the time period in which defects tend to peak, you know the time periods in which resourcing can become an issue, you know about components that you integrate which can cause the maximum damage to your schedule because they are either late or of bad quality. With all these, you can start to prepare such a list of problem areas.
Typically, risk planning is not something that comes of age in one version. What is required is to ensure that all problems and issues that you face in the previous version of the product have been recorded in some way, along with a note about the critical nature of the issue and the time period in which such an issue came out. With these as an input, an experience project / program manager can figure out which of these issues can become risks in the current versions of the product, and plan accordingly. If issues are not recorded, then you lose out some part of doing an effective risk planning for future versions of the product or project.
It is also necessary to find out more areas of risks that you would not know. There are risks that come to other people in the team, typically to the development or quality teams, which are known by their managers. For example, once we ran into a huge problem due to a different version of Visual Studio that we needed to upgrade to, and one of our components had a consistency problem with that version. This was something that our development managers knew would happen every other version, but we did not know from the project/program management side. It turned out to be a huge risk for the project, with the potential to need cutting out an important feature or add some more weeks to the schedule, and that did cause problems since it was not there in the list of risks, and yet was an old risk.

Learning - Setup groups within the company and share experiences - Part 1

The first question is about why such a topic would belong in a blog about software processes ? Seems a bit strange to have a blog about experience sharing, since it seems to something that is not specific to the software area. However, I have found that the busy nature of the software professional leads to a scarcity of time, leaving little time for reviewing ones own experience, what to talk about trying to do experience sharing with people from other teams. This being busy nature has only increased due to the economic problems of the last few years, with the same number of people expected to do more.
So what is this post about ? In any decent sized organization, whether it be a product development organization or an organization that works on projects, a number of teams do similar kind of work and hence are in a position to learn a lot from each other. Even though their products or projects may be somewhat different from each other, the level of difference with each other would not be significant enough. As a result, if there was coordination (or rather, let us not use the term of coordination, rather let us talk about how they can share their experiences), then the teams and their managers can learn a lot about how each team handled some situations which are common.
This is best served through an example. We had a situation where we were running into problems with an external vendor (not even a vendor, more like a open source component maker who would post new versions of their component with updates and notes on a regular basis, every 6 months or so). We did not have any clarity as to what were the contact details of the key people in the component maker, we did not know details about the kind of testing processes that they had, and so on. This was causing us problems, since the component was important to us (it saved us a lot of money that would otherwise be needed to buy a professional software that did the same function).
At that time, we recollected that another team in the company was doing something similar in terms of functionality. We did a quick engineering discussion with them, and realized that they were far ahead of us in terms of figuring out these kind of details that we were looking for. They had knowledge of the key people in that open source project, and they had discussions which provided them far more comfort in terms of the testing processes and the level of quality in the released version of that component.
Because of this particular discovery, we had a senior engineering person within our group interact with a similar person from their team on a regular basis for these kind of discussions. We also created an email group with the specific name of that component, and made it open for other people within the company to join. The resultant was that over a period of time, we discovered 2 smaller project teams within the company that were also looking to evaluate such a component, and they got a head start based on the discussions that they had with us and with the other teams.
Now, if you do such discussions, not only for such an example, but for other cases where you are running into problems, such as problems with the latest seeds of MS Windows or OS from Apple, or where you are running into other problems, such groups can provide a lot of help, since it allows people to share issues and share solutions. However, it takes time and effort to do this kind of sharing and coordination, even though the rewards that you get from such sharing can be well worth it.

I will add more about this in the next post in the series (Sharing experiences within the company - Part 2 - TBD)

What are Software Process Improvement resources?

A supportive and effective infrastructure is required for facilitating the coordination of various activities that place during the course of the whole program. In addition to these qualities the infrastructure should be quite flexible so as to be able to support the changing demands of the software process improvement with time. 

Resources for this program include:

1. Infrastructure and building support
2. Sponsorship
3. Commitment
4. Baseline activities
5. Technologies
6. Coordinate training resources
7. Planning expertise
8. Baseline action plan and so on.

- When this program is initiated, a primitive infrastructure is put in to place for the management of the activities that would be carried out by the organization under SPI. 

- The resources mentioned above are also the initial accomplishments that tell how well the infrastructure has been performing. 

- It is the purpose of the infrastructure to establish a link between the program’s vision and mission, to monitor it and guide it and obtaining resources and allocating them.

- Once the SPI program, a number of improvement activities will be taking place across the different units of the organization. 

- These improvement activities cannot be performed serially rather they take place in parallel. 

- The configuration management, project planning, requirements management and reviews etc. are addressed by the TWGs (technical working groups). 

- But all these activities are tracked by the infrastructure.

- Support for the following issues must be provided by the infrastructure:

1. For a technology that is to be introduced.
2. Providing sponsorship
3. Assessment of the organization impact

- As the program progresses, the functions to be performed by the infrastructure increase. 

- There are 3 major components of the SPI program:

1. SEPG or software engineering process group
2. MSG or management steering group
3. TWG or technical work group

- It is third component from which most of the resources are obtained including:

1. Human resources
2. Finance
3. Manufacturing
4. Development

- However, the most important is the first one and is often called the process group. 

- It provides sustaining support for the SPI and reinforcing the sponsorship. 

- The second component i.e., the MSG charters the SEPG.

- This is actually a contract between the SEPG and the management of the organization. 

- Its purpose is to outline the roles and the responsibilities and not to forget the authority of the SEPG. 

- The third component is also known as the process improvement team or process action team. 

- Different work groups created focus on different issues of the SPI program. 

- A software engineering domain is addressed by the technical work group. 

- It is not necessary for the TWGs to address the technical domains; they can address issues such as software standardization, purchasing, travel reimbursement and so on. 

- The team usually consists of the people who have both knowledge and experiencing regarding the area under improvement. 

- The life of TWGs is however finite and is defined in the charter. 

- Once they complete their duties, they return back to their normal work. 

- In the early stages of SPI program, the TWGs might tend to underestimate the time that would be required for the completion of the objectives assigned to them. 

- So the TWGs have to request to the MSG for allotting them more time. 

- Another important component could be the SPIAC or software process improvement advisory committee. 

- This is created in organizations where there are multiple functioning SEPGs. 

What is meant by Software Process Improvement?

About Software Process Improvement

- SPI or Software Process Improvement is a program that has been developed to provide guidance for the integrated long – range plan for the initiation and management of the SPI program. 

- SPI is based up on a model called the IDEAL model which has the following 5 major stages:

1. Initiating
2. Diagnosing
3. Establishing
4. Acting
5. Leveraging

- These 5 major steps form a continuous loop. 

- However, the time taken for the completion of one cycle varies from one organization to other. 

- Depending on the available resources an organization must be able to decide whether or not it would be able to commit to software process improvement. 

- SPI requires many activities to be carried out in parallel to each other. 

- Some part of the organization may take care of the activities in one phase while others take care of the other phase activities.

- Practically, the boundaries of the various stages in a software process improvement are not clearly defined. 

- The infrastructure also plays a great role in the success of the SPI. 

- The value added to SPI by infrastructure just cannot be underestimated. 

- It provides a great help in understanding its roles.

About Initiating Phase

- As the name indicates this is the starting point of the process. 

- This stage involves setting up of the improvement infrastructure. 

- Then the infrastructure’s roles and responsibilities are defined. 

- The resources are checked for availability and assigned.

- Finally, an SPI plan that will guide this initiating phase as well as the other higher stages. 

- It is during this stage that the goals of the software process improvement are defined and established based up on the organization’s business needs. 

- During the establishing phase these goals are further refined and specified.

- Two components are typically established namely:

Ø  A software engineering process group or SEPG

Ø  A management steering group or MSG 

About Diagnosing Phase

- In this stage, the organization as per the SPI plan starts. 

- This stage serves as foundation for the stages that will follow. 

- The plan is initiated keeping in view the vision of the organization along with its business strategy, past lessons, current business issues and long term goals. 

- Appraisal activities are carried out so that a baseline of the current state of the organization. 

- The results of these activities are reconciled with the existing efforts so as to be included in the main plan.

About Establishing Phase

 

- In this stage the issues to be addressed by the improvement activities are assigned priorities.

- Also, the strategies for obtaining a solution are also pursued. 

- The draft of the plan is completed as per the organization’s vision, plan, goals and issues. 

- From general goals, measurable goals are developed and put in to the final SPI plan. 

- Metrics essential to the process are also defined.

About Acting Phase

 

- Solutions addressing the improvement issues discovered in the previous stages are created and deployed in and out of the organization. 

- Other plans are developed for the evaluation of the improved processes.

About Leveraging Phase

 

- This stage is led by the objective of making the next pass through the process more effective. 

- By this time the organization has developed solutions and metrics concerning performance and achievement of the goals. 

- All this data obtained is stored in a process database that will later serve as source information for the next pass. 

- Also, this information would be used for the revaluation of the strategies and methods involved in the SPI program.

- Software process improvement activities work with two components namely, the tactical component and the strategic component. 

- The former is driven by the latter that is based up on the needs of the organization.