What are some common tools for installation testing?

Introducing a new student to the rest of the class gives us a practical example of the installation testing. The new student should be properly introduced to the rest of the class so that he/ she feel comfortable. Same holds true for software systems and applications. The effectiveness of any testing however also depends up on the type of the testing tools used. In this article we shall be discussing about some common tools for installation testing. 

If the installation of your software product fails, what impression it will make of you on the customer? Obviously it will cost you your reputation. How can you avoid such pathetic and embarrassing conditions? "Answer is by testing the software system’s or application’s installer with an appropriate combination of both the automated and manual processes on different machines and with different configurations". 

The factor of which you should be most concerned is nothing but “time” since installation testing eats up a lot of time, so you got to be wise and use testing tools that can save up your time. Execution of even a single test case takes a lot of time in installation testing. Now imagine if you are going to perform installation testing on a big software system or application? How much it is going to cost you? Before you start installation testing you need to decide on how many systems and configurations you are going to install the application software. Prepare a basic hard disk drive for this purpose with some other basic necessary components.

Role of Installation Testing Tools

- There are testing tools available that will help you in making sets of each of the configurations like file format and operating system that are used in the further installation testing.

- There are many other tools that can be used to automate the whole installation testing process.

- This involves setting out some systems for creating basic images of the base configuration. This can be done by using testing tools like Norton ghost which can create exact image of the operating systems very accurately and quickly.

- This step is guaranteed to save a huge amount of your time for each test case.

- Creating images takes only 5- 10 minutes whereas installing one Operating System with basic configuration takes up to one hour.

- There are some other tools that allow you to use one operating system with multiple attempts of installation.

- But in this case you will need to uninstall the application again and again and prepare the base state for the next installation.

- Apart from all the other testing tools, use flow diagrams also serve as a great aid for performing the installation testing.

- Use flow diagrams simplify the whole task so amicably.

- These flow diagrams can be further used for automating the test efforts since very easily these diagrams can be converted in to the automated scripts.

- Some test tools have been developed that test the installer scripts which are further used to check the disk space required for the installation.

- These tools make sure that only the required amount of space is utilized and not more than that otherwise an error is flagged. You should be careful with the fact that the space required is different for different file system formats.

- Another tool that can help you a big deal with installation testing is “distributed testing environment”.

- Adopting this methodology you can very effectively handle all the different test cases from just one machine and it saves you a lot of time.  

What is the relation between installation testing and configuration management?

Installation testing as we all know falls under the genre of quality assurance work in the field of software engineering and is all focused up on what is to be done by the customers in order to set up and install the new software system or application successfully. 

Installation testing process is constituted of the following processes:

1. Upgrading of installation and un- installation process.
2. Partial installation and un- installation process.
3. Full installation and un- installation process.

What is Configuration Management?

- Configuration management is somewhat self justifying and is looked up on as one of the best practices available for handling the changes that are made in the software system or application during the project development.  

- It is also used as a means for the identification of the physical as well as functional attributes of the software system or application at various points of time during the development process. 

- The configuration management takes up the systematic control of the changes to be made to the identified attributes with the following purposes:

1. Maintaining software trace-ability throughout the software development life cycle.
2. Maintaining software integrity throughout the software development life cycle.

Relation between Installation Testing and Configuration Management

Both the above mentioned processes i.e., installation testing and configuration management share a relation with each other.  

- Installation testing is performed by the software testing engineer in collaboration with the manager handling the configuration management process known as “configuration manager”. 

- The configuration management process shows up the need to trace the changes. 

- This process has the ability with the virtue of which it verifies whether or not the final developed software system or application has all the enhancements, features and functionality that are required to be in the release. 

- In a way we can say that the configuration management checks how correctly the installation testing has been implemented. 

- It is done by the means of four below mentioned procedures that need to be defined for all the software systems or applications in order to ensure that a proper configuration management process is implemented:

1. Configuration Identification: 

It involves identification of the attributes defining all the aspects of the configuration item.

2. Configuration Change Control:

It includes a set of approval stages and set of processes that are intended to change the attributes of a configuration item and re- baseline them.

3. Configuration Status Accounting:

By virtue of this ability the configuration baselines associated with all the configuration items at any instant of time are recorded and reported.

4. Configuration Audits:

It occurs at the moment of incorporating the change. These can be divided in to two categories:

(a)Functional configuration audits: ensures achievement of the performance and functional attributes of the configuration items and

(b)Physical configuration audits: ensures proper installation of the configuration item with the requirements mentioned in the design documentation.

- In installation testing a complied version of the code is placed in a pre- production environment which also acts as the configuration item for configuration management. 

- The configuration management is carried out outside the usual software development environment so that the corruption to the code is limited from the past as well as the future releases.

- The set up program used in the package software acts as a multi configuration wrapper and each and every configuration in it should be tested appropriately so that it can be released with confident. 

In a way we can say that these two processes i.e., installation testing and configuration testing are locked with each other to some extent and this is what that keeps them related to each other.

How is optimization of installation testing done?

It becomes very important to carry out installation testing towards the end of the release process. It is very important to subject a software system or application to installation testing so that it can be released without any hesitation to the customers. 

One can make sure without installation testing that the software system or application will successfully install on the user’s system in the first go itself! The user might need to install the software system or application again and again which in turn might frustrate the user. You will loose customers as a consequence of this. Therefore perform installation testing becomes absolutely necessary. 

In this article we have discussed about the need for optimizing the installation testing and how to optimize it. by the optimization of the installation testing we mean modifying it further so that  some aspects of it use less resources and work more efficiently. 

What is the need to optimize?

- When you optimize a testing process you can run it more quite rapidly and that too it uses very less memory then. 

- The optimization though has been named as “optimization”, it can never produce a truly optimal system. 

- The amount of time taken by a test case of the installation can be reduced by making it consume more memory.

- In cases where the memory is limited, slower algorithms can be used.

- While optimizing the installation testing one should keep in mind that there is no such design that fits all the test cases. 

On what levels installation testing is optimized?

The installation testing process much like the other processes is optimized through the following number of levels:

Design Level: 

- This is the highest level of the above mentioned process. 

- Design is the best aspect that can be optimized to utilize the resources in a much better way.

- The implementation of the optimized design can be made more efficient by using efficient algorithms which in turn have been benefited from code of good quality. 

- It is obvious for the architecture of the test design to have an overwhelming affect on its performance.

- In some cases optimization process may also involve the following:

(a)  More elaborate algorithms

(b)  Special tricks

(c)   Special cases

(d)  Complex trade offs

- A fully optimized installation testing process becomes difficult to comprehend and may also house more errors than the un- optimized processes.

Source code Level: 

- This step involves rooting out the bad quality code from the test cases and therefore it also improves performance.

- When the malfunctioning code is removed, the obvious slow downs are avoided. 

- However, it is possible that any further optimization made to that test case may decrease its maintainability. 

- But, today there are available what are called “optimizing compilers”.

Compile Level:

- At this stage, it is ensured by the optimizing compiler that the test case is optimized to such a level at which the test case can be compiled by the compiler.

Assembly Level:

- This is the lowest level of optimization process and involves taking advantage of the full repertoire of system instructions. 

- This is the reason why most of the operating systems use the embedded systems are written in assembly code.

- Many of the test cases are compiled from a high level language to assembly language and from there it is manually optimized.

Run time: 

- Run time optimization is needed and are performed by the assembler programmers and just in time compilers which exceed the capability of static compilers.

Two types of optimizations can be performed on the installation testing process namely:

1. Platform dependent optimization (uses specific properties of one platform) and
2. Platform independent optimization (effective on all platforms)

What are limitations and advantages of installation testing?

Imagine you have developed your software system or application properly and very carefully and it addresses all the requirements of your client. But what if your client tries to install your software product and it fails terribly? 

It means the installer of your software system or application was developed properly and was not tested also. Now imaging what effect will it have on your reputation in the eyes of your client? You will loose his confidence! Won’t you? 

The only way to escape such a horrible situation is that you carry out installation testing on the installer of your software product and see that it works with all the systems and with different configurations. Therefore the installation testing becomes very important since installation is the first thing with which the user interacts in a software application.

Limitations of Installation Testing

- The biggest limitation of the installation is the time factor since installation testing consumes a lot of time.

- To overcome this limitation one needs to automate the whole installation testing process.  

- Even a single test case of the installation testing takes at the minimum up to 2 hours.

Now you can imagine how much time it will take if you will have to test the installer of a big software system or application. 

- However, the time consumed for each test case also depends on whether the application is subjected to one of the following options:

1. Upgrading of installation and un- installation process.
2. Partial installation and un- installation process.
3. Full installation and un- installation process.

- The installation testing requires that both the software testing engineer and the configuration manager work in collaboration with each other.

- During installation, the software system or application is placed in to a pre- production environment or testing environment which may corrupt the code in some of the cases.

- From the testing environment there are 2 certainties as mentioned below:

1. The code may progress in to production process or
2. The code may not progress in to the production process.

- Since the code may get corrupted from other past as well as future releases, all these process need to be performed outside the software development environment which further aggravates the difficulty of the installation testing.

- There is just one simple approach to installation testing which involves running the install program commonly known as “package software”. 

- This package software makes use of a set up program which in some cases act as a multi- configuration wrapper and in other cases it does not! 

- Therefore, in the cases in which the set up program does not act as multi configuration wrapper, the software simultaneously cannot be installed on a variety of systems and configurations which again presents a headache. 

- Furthermore installation testing is limited to certain aspects regarding installation:

1. Dependency of the software to be installed on other software and patches.
2. Over installing of a new version over the older one.

Advantages of Installation Testing

- As for the advantages of the installation testing are concerned, the importance of the installation testing is the biggest advantage.
- Installation process is the first thing with which the user interacts and therefore it needs to be proper and efficient. 
- The installation testing keeps a check on this aspect whether it can be installed on a number of different systems and in different environments.

What are different characteristics of installation testing?

Which software testing methodology pertains to the quality assurance work and is focussed up on what is to be done by the customers in order to set up and install the software system or application successfully on their system?

The answer is “installation testing”. 

The installation testing may involve the below mentioned testing processes:

1. Full install/ uninstall process
2. Partial install/ uninstall process
3. Upgrade install/ uninstall process

About Installation Testing

- The installation testing is carried out by the software testing engineer in collaboration with the configuration manager. 

- Running an installation program is considered to be the simplest approach to installation testing.

- This approach is some times called “package software” and it typically deploys a set up program that is nothing but a multi- configuration wrapper using which the software can be easily installed on a variety of machines as well as many operating environments. 

- All the possible configurations should be given an appropriate level of installation testing so that they can be released to the customers without any hesitation. 

- There are some cases of the distributed system in which the software is to be released particularly in to a target environment that is live such as an operational web site. 

- In such cases the installation or you can say software deployment involves changes in the database schemas and installation of the new software system or application.

- Some deployment plans are created for these kinds of circumstances. 

- These deployment plans consist of back out procedures that are aimed at rolling back the target environment if the deployment does not show up as successful. 

- The deployment plan should be tested before deploying them in an environment that is an exact copy of the live environment. 

- Synchronizing the data in the test deployment environment with minimum disruption can increase the organizational requirements of such a plan. 

- This type of implementation plan includes:

1. Up grade of a multi tier application and
2. Testing of the processes which are carried out during installation.

- Such kind of testing is often compared with the “dry run”. 

- Implementation testing is nothing but another name for installation testing. 

- A successful installation of the software system or application will surely make the customer happy but an unsuccessful installation will disappoint the customer. 

The situation can get even worse; it can badly damage the user’s system. This is an example of how a lack of proper installation testing can ruin your impression of the customer. Now you must be wondering what you should do to avoid such embarrassing situations? 

- You should test the installer thoroughly and also with the combinations of manual and automated processes on various machines with several kind of different configurations. -“Time” is the factor that limits the scope of the installation testing. 

- Executing even a single test case takes a hell lot of time. 

Steps involved in Installation Testing

Below we are mentioning the steps involved in the installation testing:

1. Decide for how many different system configurations you want to test the installation.
2. Prepare a basic hard disk drive and format it with the default file system.
3. Install a common OS like windows on this hard drive.
4. Install some primary required components on this hard drive.
5. Create images of this HDD and this will enable you to create configurations on the drive.
6. Make different sets of each configuration like OS that can be used for further software system.

What are installation/ uninstallation testing? What are some tips for installation testing?

Installation testing comes under the category of quality assurance aspects. It is concerned with the procedure that the user has to follow in order to install that software. Installation may be of any type like partial, full or it might be an installation of an upgrade of the software.

SOME IMPORTANT FACTS ABOUT INSTALLATION TESTING
- Installation testing is carried out by the lead tester and configuration manager.
- Before carrying out installation testing, implementation testing is performed outside the testing environment.
- This is done to minimize the corruption of the software code.
- Usually for installing software, its package software or set up is run.
- The set up is actually a multi-configuration program.
- It plays a great role in the installation of software on a computer system.
- Every phase in the installation of software must be tested before it released to the public.
- In rare cases of distributed systems, it is some times required to change the database schemas and installation process of that particular software.
- For such situations a back up deployment plan is also prepared which can be implemented in case the release is unsuccessful.
- The deployment plan should also be tested as if it is a live plan.
- This kind of procedure for installation testing is called a “dry run”.
Installation testing and implementation testing are very closely related and in fact sometimes they are considered to be the same.
- The installation should be done as such that the installed program is able to cooperate with the other programs that are already installed on the computer system.
- If the program is installed successfully, it’s ok.
But, what if it doesn’t?
- It can cause considerable damage to the system.
- The user may need to format the whole system and reinstall everything again.
- The installation process should be tested both manually and automatically with all the possible configurations.
- Installation testing is a very time consuming process.
- Tests cases require much time to execute.

Un-installation testing also follows the same procedure.

TIPS FOR EFFECTIVE INSTALLATION AND UN-INSTALLATION TESTING

1. List all the configurations that you want to test.

2. Format one of the drives on the system and use it as the base for your testing.

3. Every time you have to execute a configuration, you can create a ghost of the basic images that will be required. This will save your time. This is the only automation that can be done in an installation or uninstallation testing.

4. You can attempt to install software on one operating system many times. Every time you uninstall the software, you can prepare the base for installation of the next configuration.

5. Always keep the flow chart of the whole testing process handy with you. With a flow chart you can how much you have progressed in the testing.

6. If previously you have installed the compact version of the software, then for the next test case install the full version.

7. If you are going according to the flow chart for installation process, then follow the same flow chart bottom up for the un-installation testing.

8. You can even use the flow chart to design automated test cases.

9. Check the amount of space that the software requires. Make sure that only that much is utilized and not more than that or less than that.

10. Always carry out installation and un-installation testing in a distributed testing environment. This helps you in managing many test cases simultaneously. You no need to manage different systems.

11. Try crashing the installation and un-installation process in between.

Validation phase - User Acceptance Testing and Installation Testing

User Acceptance Testing occurs just before the software is released to the customer. The end-users along with the developers perform the User Acceptance Testing with a certain set of test cases and typical scenarios.

Installation testing is often the most under tested area in testing. this type of testing is performed to ensure that all installed features and options function properly. It is also performed to verify that all necessary components of the application are, indeed, installed. Installation testing should take care of the following points:
- To check if while installing product checks for the dependent software/patches.
- The product should check for the version of the same product on the target machine, say the previous should not be over installed on the newer version.
- Installer should give a default installation path.
- Installation should allow user to install at location other than the default installation path.
- Check if the product can be installed "Over the Network".
- Installation should start automatically when the CD is inserted.
- Installer should give the Remove/Repair options.
- When uninstalling, check that all the registry keys, files, DLL, shortcuts, activeX components are removed from the system.
- Try to install the software without administrative privileges.
- Try installing on different operating system.
- Try installing on system having non-compliant configuration such as less memory/RAM/HDD.

Overview and Features of Installation Testing

Installation testing occurs outside the development environment. Installation testing can be compared as introducing a guest in your home. The new guest should be properly introduced to all the family members in order to feel him comfortable. Installation of new software is also quite like above example.
The customer will be happy if your installation is successful on the new system but if the installation fails then the program will not work on that system.
Testing application to make sure that it is working is a crucial step. You should check whether the application doing what it is suppose to do? Secondly, run three-four invocations of this application and check memory, CPU load, etc. Third, are other normal applications, hardware, etc working fine?

Installation testing should take care of the following points: -
- To check if while installing product checks for the dependent software / patches say Service pack3.
- The product should check for the version of the same product on the target machine.
- Installer should give a default installation path say “C:\programs\.”
- Installer should allow user to install at location other then the default installation path.
- Check if the product can be installed “Over the Network”.
- Installation should start automatically when the CD is inserted.
- Installer should give the remove / Repair options.
- When uninstalling, check that all the registry keys, files, Dll, shortcuts, active X components are removed from the system.
- Try to install the software without administrative privileges (login as guest).
- Try installing on different operating system.
- Try installing on system having non-compliant configuration such as less memory / RAM / HDD.