Risk planning - Accounting for when estimates are prepared by one person and task done by another

Let me admit it straight out, this is a pretty difficult topic to write about. In all my years of experience as a manager, trying to have a simple equation or factor for working out what happens when somebody prepares an estimate and somebody else has to do the actual development / coding for that work was not possible. There was no one size fits all for this kind of situation. In the previous post on this topic (Using a confidence level for estimates for later refinement), I put some initial thoughts on this subject. The idea was that estimates are prepared at the beginning of a cycle and can be different from the actual effort put in when the work happens on that features.
It could be argued about why estimates are needed to be prepared at the beginning of the cycle, but there can be many reasons for that. One of the biggest reasons was about wanting to have a good idea about the number of features that would be possible to be done in the given cycle. This is necessary for the team, and this is very important for the stakeholders and sponsors of the team. In a number of cases, the plan for the release will not be approved until there is an assurance that a Minimum Viable Product has not been planned. Given that the release of a product can have profound implications on the financial status of an organization, it can be very important to do planning about the number of features that can be fit into the release.
Doing this estimation also allows the team to do the required resource planning, and even though the number of resources may not change too much between releases, if there is a need to do a big bang release, then the team would want to know the number of extra resources that are required for this release. So, let us work with the assumption that the team needs to do the initial estimation for the features that are planned for the release.
Now, the biggest problem that occurs is that when the work for a release is being estimated, the people doing the estimation are the more senior developers in the team. Typically, the estimation exercise can only last for around 5-10% of the total time of the release, and it is really not possible to do the logical task of breaking down the features into detail deep enough that the estimate would be very accurate. So, a lesser number of people are drafted in to do this kind of work, and an equal assumption is that the estimation is primarily done by the person using their previous experience to determine the scope of work for the details of the feature and using this for the future.
But when it comes to doing the actual development work for the feature, you cannot have these senior people doing all the work. The work is shared among the team members and this is where discrepancies come in. You can have the senior person being very skilled in doing such work, but the person who is actually doing this work is not so skilled and you end up with more effort required for doing this work. So, whenever the work allocation is being done, there is a need to map the estimate with the person who did the estimate, and the person who is actually going to be doing the work. A skilled manager who knows both these people would be able to do this to a large extent and be pretty accurate in this. At the same time, the project manager should continue to track and compare the effort discrepancy mentioned by the engineering manager with the actual effort discrepancy.

Risk planning - Using a confidence level along with estimates so that these are refined later in the schedule

The normal cycle we would follow was for a 6 month period of software development. Every 6 months we would release the next version of our software, and towards that end, we had a list of features ready for the Product Manager to consider for prioritization and decide which features would make it in the next version of the software, and which would be taken up in future versions of the software. Once such a list was prepared, the next step was to get the Product Manager to spend time with the designer and break down the feature set into a slightly more detailed set, something like Use Cases. However, the level of detail that we could do in the time period that we had was of a level higher than Use Cases (the actual Use Cases would be generated only when work on the features was about to start).
Once this level of detail was generated, the next step was to get the senior engineering leads talking to the Product Manager and the designers to get some idea about the feature, look at some workflows and implementation, and then, start looking at how to estimate these. There are many methods of estimation, but in most cases we used the method called heuristics, which is a fancy word that we took to mean that these people knew how to do feature development, they had been doing it for quite some time, and as a result, they would be able to use their experience to determine the typical effort required for doing the feature.
Next, all the effort for the list of features was tabulated, and then assigned to the different members of the team while doing balancing, ensuring that everybody got a mix of interesting and boring work (wherever possible if there was exciting work, since that would help in terms of morale). There was some rationalization of the effort reported to take into account that some of the team members were more capable while others were not so capable. Finally the list of features would be generated to be presented to the stakeholders of the team and the organization.
However, once the team would start actually doing work on these features, then the actual effort would come into play. It was quite possible that the initial effort that was listed was the final effort once the team started implementation, but there was also a high chance that the effort would change once the team actually started implementation. There could be design issues that were not expected before actual implementation started, or there could be other setbacks that happened. It could be that the person doing the estimate had not done a correct estimate and this was clear when the team actually started the estimation.
This has an impact on risk planning. Part of the continuous effort to monitor the risk involved in the project is to determine the actual state of the project and also to determine whether the features that were expected to be done in the initial planning are on schedule or not. However, it should be part of the risk matrix to try and determine the level of inaccuracy present in these estimates (even though it can be pretty difficult) and to estimate the impact this could have on the feature schedule. It is critical that the Project / Program manager has been tracking the estimate accuracy of the features and is not majorly surprised if a feature is running late because of estimation issues.

When to re-estimate? What is a better way to estimate : Story points or Ideal Days?

When to Re-estimate?

Story points and ideal days are estimates of the size of a feature which helps you to know when to re-estimate. Re-estimate is done only when your opinion of the relative size of one or more stories has changed. One should not re-estimate just because progress is not coming as rapidly as expected.

Velocity should be allowed to take care of most estimation inaccuracies. Velocity is considered to be a great equalizer. The reason behind this is that the estimate for each feature is made relative to the estimates for other features, it does not matter if our estimates are correct, a little incorrect, or a lot incorrect. What matters is that they are consistent. As long as we are consistent with the estimates, measuring velocity over the first few iterations will allow us to have a reliable schedule.

At the end of an iteration, it is not recommended giving partial credit for partially finished user stories. The preference is for the team to count the entire estimate towards their velocity (if they completely finished and the feature has been accepted by the product owner) or for them to count nothing toward their story otherwise.

However, the team may choose to re-estimate partially complete user stories. Typically, this will mean estimating a user story representing the work that was completed during the iteration and one or more user stories that describe the remaining work. The sum of these estimates does not need to equal the initial estimate.

A team can choose to estimate either through story points or ideal days. Each has its advantages.


Benefits of Story Points
1. They help drive cross functional behavior.
2. The estimates derived by story points do not decay.
3. Story points are a pure measure of size.
4. Estimation through story points is faster.
5. Unlike ideal days, story points can be compared among team members. If one team member thinks that it will take him 4 ideal days, and another member thinks that it will take him 1 ideal day, both of them may be right yet there is no basis on which to argue and establish a single estimate.

Benefits of Ideal Days
1. They are more easily explained to those outside the team.
2. They are easier to get started with.

The advantages of story points are more compelling as compared to benefits of ideal days. one way is if a team is struggling with estimating the pure size, they can start off with estimating with ideal days and gradually switching to estimating by story points.