Dot / Patch release: Estimating the features where there is a change required

One of the previous posts I wrote about was that of a Dot / Patch release (Estimation of dot / patch release) where there was an effort to show what are some of the reasons for a dot release or a patch release and the broad level points about how to estimate the amount of overall effort needed as well as the generation of schedule needed for the release. This needs to be done even if you already have a hard date for the release of the patch / dot release, since if the generated schedule is going far beyond the required date, then there is something wrong which would need to be solved (possibly by adding more resources, although that cannot be done in every case).
This post talks about actually doing an estimation of the changes required by working on the different features to see the overall impact of the change. Consider a case where a dot release has to be made for the application, with a change made in one of the features for a major defect. Now, the dot release has to be made in a couple of months, with the exact release date being decided based on the estimates for the different areas of the release (development effort, testing, localization, etc). One of the starting points for this estimation effort is figuring the effort needed for development, and for that, there needs to more detailed investigation of what areas need development effort.
How do you do this ? The first and most important starting point is to ensure that you have an experienced developer to do the investigation and estimate. The actual effort can be done by somebody in the development team, but the investigation should be done by a senior and experienced developer. How do you start:
- First, make sure that the developer has been given details of the change needed in terms of requirements
- Give the developer some time to discuss with other team members about the areas which are impacted.
If this is a core area, then the developer would need to go across the application to determine the impact. For example, there may be a need in the user security framework, and that would spin across all the application. If the change is localized to a specific area, then it is easier to make the effort estimate of the change. There is no great complexity in this area, as long as the developer does a thorough job.
- The senior developer should spend time with the actual developer who is going to work on the release and make sure that he has enough understanding of the changes required, and depending on the skill of the developer assigned, make sure that the correct estimate is given (which may included a required buffer).
- The developer also needs to ensure that he has spent time with people from the localization development team and the installer / release teams so that he has an idea of the amount of time needed from their side and can also include these in the effort estimate.
- The developer needs to spend a fair amount of time with the testing team so that they have a good understanding of all the changes that are going to happen in the release as well as the actual impact of the change including all the areas of the application that are impacted. 

Being able to estimate effort needed for smaller releases (patch / dot release)

One of the most difficult tasks to estimate are the smaller projects that a team needs to do. Consider a long project, the typical new version release of a software (for a typical long example, consider the schedule for the release of a new version of Microsoft Office - is it always typically more than 1 year); now even when the team is busy doing this release, there will be always be the need for doing a dot release or a patch at the same time (or even multiple such releases). Why would this happen ?
- There could be a security issue for which an urgent patch needs to be released,
- There could be some new feature that is meant to be pushed through before the next release
- Because the release schedule is long, defects that are found during the next few months are collected and released in an interim release that is then released
- And this is one of the most interesting reasons for a release - I know several people who will not take a new version of a Microsoft until a service pack has been released, since that is when it would have stabilized - so this puts pressure on a company to do a release within a few months

With all this, who does these interim releases ? For products where there is already a definite schedule of interim releases, a small sub-team can be setup to take care of these releases. Such a team will soon gain expertise in doing these smaller releases, and can work on the main release if there are no interim releases ongoing. However, for teams where there is no definite schedule of release of these interim releases, it does not make sense for a team to put dedicated folks to work on the product. Even more, in my experience, when people are assigned to such interim releases, there is a need to rotate the people working on such releases, since these are seen as essential but maintenance, not with the excitement of working for something new.
Now, once people are assigned to do the project, there is a need to figure out the schedule for such a release. However, in most of these cases, the end date is already fixed - the Product Manager already has a date in mind about when these releases need to be released. But, the team still need to define the estimate and a probable schedule in mind:
- Whether this be a dot release or a patch (a patch is a small set of files that is downloaded and can be installed, a dot release is typically the full installer with a few changes in the files). The advantage of having a dot release is that it can replace the original installer
- Define the change that is happening (this typically means the files and features that are being changed, and the estimated impact of this change)
- The amount of time needed for the development team to make the required changes
- The amount of time needed for the testing team to test the area and surrounding areas (this would also include the installer created for the release)
- Typically most products have language versions, and those need to be created and tested when a dot release or a patch is made. So, the estimate for the amount of time needed for these activities also need to be incorporated into the schedule.

Together, all of these can be woven into a schedule for the interim release. If it turns out that the deadline given is less than the projected schedule end, then you need to push back. For a small release, it is next to impossible to meet the project with the required quality principles, unless sufficient time is given. We once had a situation where a particular variation in the installer made a small error in the registry, which then prevented those who installed from being able to install another update, and there was a lot of backlash over that - we had to release another patch for that one.

Read more about the estimation done by a senior member of the development team.

Ensuring that builds are posted in the locations where people are located

In today's world, applications are developed all over the world, in the sense that if you look at a product development team, it will have members located all over the world. So, I have worked on several products where some developers are based in a location in India, some are based in Germany, and some others are based in multiple locations in the United States. Now, one can argue that this would be confusing and not an ideal state, but the fact remains that because of reasons related to costs, acquisition of companies (I once had a colleague who was absorbed into the company as part of one product, and then assigned to another product based on a resourcing decision - with the associated issue being that the original product was being developed out of his location, but for the new product where he was now associated, he was now a remote worker), and the need to meet the geographic needs of talented people who would otherwise not join, teams can get geographically dispersed. So, let us take it for granted that such teams exist, and consider some of the problems that can happen with such teams. We will talk about coordination & communication issues in a separate topic, but one major issue with teams working on large products is the sheer size of getting the builds to them on a regular basis. Consider teams that work on applications such as MS Office, Photoshop, CorelDraw, and many other such products, the sheer size of the application builds can be upwards of 1GB+.
When you are working on an application development program, it is important to work on the newest version of the application. Typically for most product development applications, this means that a new build will be generated every day, available for the developers and testers. This allows the testers to test with all the fixes that have been made the previous days, and also ensures that developers have the same confidence that their fixes are in, and are being tested regularly. Now this sounds fine when everybody is located in the same geographic location (I mean there are other issues related to ensuring that the build is a solid build, that build failures are detected early, and so on), but gets more challenging when members of the development team are located in different locations. Given the size of some applications (as I mentioned above, they can be 1GB or more sometimes), even with very fast connections, it can take time to ensure that these builds are delivered to the remote locations within a short period that they are available in the location where they are generated. And when you consider locations such as those in India, the availability of fast broadband lines is not of the same assumed speed as those available in the United States (or which can actually be more expensive). So, there needs to be a lot of thought given to how these builds will reach the different geographic locations. All this rules out the possibility of transferring builds manually to the different geographic locations; instead what needs to be done is to add a sequence of commands in the build script near the end that check for whether the build has been made, that the sequence has not showed any obvious errors (and if possible, incorporate some smoke tests to ensure that the build launches and works fine) and then does an automatic ftp transfer to default directories on the other geographic locations. The reason for incorporating smoke tests is to ensure that you do not start sending out builds to the other locations that are not possible to use. Even in such cases, you will find that once in a while, there will be failures and build will not be available, but in most cases, you will find that you needs will be met.
However, there is a complication that can happen about setting the timing of the build delivery across multiple locations, and I will cover that in a forthcoming article (link to next article - TBD).

How is optimization of installation testing done?

It becomes very important to carry out installation testing towards the end of the release process. It is very important to subject a software system or application to installation testing so that it can be released without any hesitation to the customers. 

One can make sure without installation testing that the software system or application will successfully install on the user’s system in the first go itself! The user might need to install the software system or application again and again which in turn might frustrate the user. You will loose customers as a consequence of this. Therefore perform installation testing becomes absolutely necessary. 

In this article we have discussed about the need for optimizing the installation testing and how to optimize it. by the optimization of the installation testing we mean modifying it further so that  some aspects of it use less resources and work more efficiently. 

What is the need to optimize?

- When you optimize a testing process you can run it more quite rapidly and that too it uses very less memory then. 

- The optimization though has been named as “optimization”, it can never produce a truly optimal system. 

- The amount of time taken by a test case of the installation can be reduced by making it consume more memory.

- In cases where the memory is limited, slower algorithms can be used.

- While optimizing the installation testing one should keep in mind that there is no such design that fits all the test cases. 

On what levels installation testing is optimized?

The installation testing process much like the other processes is optimized through the following number of levels:

Design Level: 

- This is the highest level of the above mentioned process. 

- Design is the best aspect that can be optimized to utilize the resources in a much better way.

- The implementation of the optimized design can be made more efficient by using efficient algorithms which in turn have been benefited from code of good quality. 

- It is obvious for the architecture of the test design to have an overwhelming affect on its performance.

- In some cases optimization process may also involve the following:

(a)  More elaborate algorithms

(b)  Special tricks

(c)   Special cases

(d)  Complex trade offs

- A fully optimized installation testing process becomes difficult to comprehend and may also house more errors than the un- optimized processes.

Source code Level: 

- This step involves rooting out the bad quality code from the test cases and therefore it also improves performance.

- When the malfunctioning code is removed, the obvious slow downs are avoided. 

- However, it is possible that any further optimization made to that test case may decrease its maintainability. 

- But, today there are available what are called “optimizing compilers”.

Compile Level:

- At this stage, it is ensured by the optimizing compiler that the test case is optimized to such a level at which the test case can be compiled by the compiler.

Assembly Level:

- This is the lowest level of optimization process and involves taking advantage of the full repertoire of system instructions. 

- This is the reason why most of the operating systems use the embedded systems are written in assembly code.

- Many of the test cases are compiled from a high level language to assembly language and from there it is manually optimized.

Run time: 

- Run time optimization is needed and are performed by the assembler programmers and just in time compilers which exceed the capability of static compilers.

Two types of optimizations can be performed on the installation testing process namely:

1. Platform dependent optimization (uses specific properties of one platform) and
2. Platform independent optimization (effective on all platforms)

What are limitations and advantages of installation testing?

Imagine you have developed your software system or application properly and very carefully and it addresses all the requirements of your client. But what if your client tries to install your software product and it fails terribly? 

It means the installer of your software system or application was developed properly and was not tested also. Now imaging what effect will it have on your reputation in the eyes of your client? You will loose his confidence! Won’t you? 

The only way to escape such a horrible situation is that you carry out installation testing on the installer of your software product and see that it works with all the systems and with different configurations. Therefore the installation testing becomes very important since installation is the first thing with which the user interacts in a software application.

Limitations of Installation Testing

- The biggest limitation of the installation is the time factor since installation testing consumes a lot of time.

- To overcome this limitation one needs to automate the whole installation testing process.  

- Even a single test case of the installation testing takes at the minimum up to 2 hours.

Now you can imagine how much time it will take if you will have to test the installer of a big software system or application. 

- However, the time consumed for each test case also depends on whether the application is subjected to one of the following options:

1. Upgrading of installation and un- installation process.
2. Partial installation and un- installation process.
3. Full installation and un- installation process.

- The installation testing requires that both the software testing engineer and the configuration manager work in collaboration with each other.

- During installation, the software system or application is placed in to a pre- production environment or testing environment which may corrupt the code in some of the cases.

- From the testing environment there are 2 certainties as mentioned below:

1. The code may progress in to production process or
2. The code may not progress in to the production process.

- Since the code may get corrupted from other past as well as future releases, all these process need to be performed outside the software development environment which further aggravates the difficulty of the installation testing.

- There is just one simple approach to installation testing which involves running the install program commonly known as “package software”. 

- This package software makes use of a set up program which in some cases act as a multi- configuration wrapper and in other cases it does not! 

- Therefore, in the cases in which the set up program does not act as multi configuration wrapper, the software simultaneously cannot be installed on a variety of systems and configurations which again presents a headache. 

- Furthermore installation testing is limited to certain aspects regarding installation:

1. Dependency of the software to be installed on other software and patches.
2. Over installing of a new version over the older one.

Advantages of Installation Testing

- As for the advantages of the installation testing are concerned, the importance of the installation testing is the biggest advantage.
- Installation process is the first thing with which the user interacts and therefore it needs to be proper and efficient. 
- The installation testing keeps a check on this aspect whether it can be installed on a number of different systems and in different environments.