What are some common tools for installation testing?

Introducing a new student to the rest of the class gives us a practical example of the installation testing. The new student should be properly introduced to the rest of the class so that he/ she feel comfortable. Same holds true for software systems and applications. The effectiveness of any testing however also depends up on the type of the testing tools used. In this article we shall be discussing about some common tools for installation testing. 

If the installation of your software product fails, what impression it will make of you on the customer? Obviously it will cost you your reputation. How can you avoid such pathetic and embarrassing conditions? "Answer is by testing the software system’s or application’s installer with an appropriate combination of both the automated and manual processes on different machines and with different configurations". 

The factor of which you should be most concerned is nothing but “time” since installation testing eats up a lot of time, so you got to be wise and use testing tools that can save up your time. Execution of even a single test case takes a lot of time in installation testing. Now imagine if you are going to perform installation testing on a big software system or application? How much it is going to cost you? Before you start installation testing you need to decide on how many systems and configurations you are going to install the application software. Prepare a basic hard disk drive for this purpose with some other basic necessary components.

Role of Installation Testing Tools

- There are testing tools available that will help you in making sets of each of the configurations like file format and operating system that are used in the further installation testing.

- There are many other tools that can be used to automate the whole installation testing process.

- This involves setting out some systems for creating basic images of the base configuration. This can be done by using testing tools like Norton ghost which can create exact image of the operating systems very accurately and quickly.

- This step is guaranteed to save a huge amount of your time for each test case.

- Creating images takes only 5- 10 minutes whereas installing one Operating System with basic configuration takes up to one hour.

- There are some other tools that allow you to use one operating system with multiple attempts of installation.

- But in this case you will need to uninstall the application again and again and prepare the base state for the next installation.

- Apart from all the other testing tools, use flow diagrams also serve as a great aid for performing the installation testing.

- Use flow diagrams simplify the whole task so amicably.

- These flow diagrams can be further used for automating the test efforts since very easily these diagrams can be converted in to the automated scripts.

- Some test tools have been developed that test the installer scripts which are further used to check the disk space required for the installation.

- These tools make sure that only the required amount of space is utilized and not more than that otherwise an error is flagged. You should be careful with the fact that the space required is different for different file system formats.

- Another tool that can help you a big deal with installation testing is “distributed testing environment”.

- Adopting this methodology you can very effectively handle all the different test cases from just one machine and it saves you a lot of time.  

What is the relation between installation testing and configuration management?

Installation testing as we all know falls under the genre of quality assurance work in the field of software engineering and is all focused up on what is to be done by the customers in order to set up and install the new software system or application successfully. 

Installation testing process is constituted of the following processes:

1. Upgrading of installation and un- installation process.
2. Partial installation and un- installation process.
3. Full installation and un- installation process.

What is Configuration Management?

- Configuration management is somewhat self justifying and is looked up on as one of the best practices available for handling the changes that are made in the software system or application during the project development.  

- It is also used as a means for the identification of the physical as well as functional attributes of the software system or application at various points of time during the development process. 

- The configuration management takes up the systematic control of the changes to be made to the identified attributes with the following purposes:

1. Maintaining software trace-ability throughout the software development life cycle.
2. Maintaining software integrity throughout the software development life cycle.

Relation between Installation Testing and Configuration Management

Both the above mentioned processes i.e., installation testing and configuration management share a relation with each other.  

- Installation testing is performed by the software testing engineer in collaboration with the manager handling the configuration management process known as “configuration manager”. 

- The configuration management process shows up the need to trace the changes. 

- This process has the ability with the virtue of which it verifies whether or not the final developed software system or application has all the enhancements, features and functionality that are required to be in the release. 

- In a way we can say that the configuration management checks how correctly the installation testing has been implemented. 

- It is done by the means of four below mentioned procedures that need to be defined for all the software systems or applications in order to ensure that a proper configuration management process is implemented:

1. Configuration Identification: 

It involves identification of the attributes defining all the aspects of the configuration item.

2. Configuration Change Control:

It includes a set of approval stages and set of processes that are intended to change the attributes of a configuration item and re- baseline them.

3. Configuration Status Accounting:

By virtue of this ability the configuration baselines associated with all the configuration items at any instant of time are recorded and reported.

4. Configuration Audits:

It occurs at the moment of incorporating the change. These can be divided in to two categories:

(a)Functional configuration audits: ensures achievement of the performance and functional attributes of the configuration items and

(b)Physical configuration audits: ensures proper installation of the configuration item with the requirements mentioned in the design documentation.

- In installation testing a complied version of the code is placed in a pre- production environment which also acts as the configuration item for configuration management. 

- The configuration management is carried out outside the usual software development environment so that the corruption to the code is limited from the past as well as the future releases.

- The set up program used in the package software acts as a multi configuration wrapper and each and every configuration in it should be tested appropriately so that it can be released with confident. 

In a way we can say that these two processes i.e., installation testing and configuration testing are locked with each other to some extent and this is what that keeps them related to each other.

How is optimization of installation testing done?

It becomes very important to carry out installation testing towards the end of the release process. It is very important to subject a software system or application to installation testing so that it can be released without any hesitation to the customers. 

One can make sure without installation testing that the software system or application will successfully install on the user’s system in the first go itself! The user might need to install the software system or application again and again which in turn might frustrate the user. You will loose customers as a consequence of this. Therefore perform installation testing becomes absolutely necessary. 

In this article we have discussed about the need for optimizing the installation testing and how to optimize it. by the optimization of the installation testing we mean modifying it further so that  some aspects of it use less resources and work more efficiently. 

What is the need to optimize?

- When you optimize a testing process you can run it more quite rapidly and that too it uses very less memory then. 

- The optimization though has been named as “optimization”, it can never produce a truly optimal system. 

- The amount of time taken by a test case of the installation can be reduced by making it consume more memory.

- In cases where the memory is limited, slower algorithms can be used.

- While optimizing the installation testing one should keep in mind that there is no such design that fits all the test cases. 

On what levels installation testing is optimized?

The installation testing process much like the other processes is optimized through the following number of levels:

Design Level: 

- This is the highest level of the above mentioned process. 

- Design is the best aspect that can be optimized to utilize the resources in a much better way.

- The implementation of the optimized design can be made more efficient by using efficient algorithms which in turn have been benefited from code of good quality. 

- It is obvious for the architecture of the test design to have an overwhelming affect on its performance.

- In some cases optimization process may also involve the following:

(a)  More elaborate algorithms

(b)  Special tricks

(c)   Special cases

(d)  Complex trade offs

- A fully optimized installation testing process becomes difficult to comprehend and may also house more errors than the un- optimized processes.

Source code Level: 

- This step involves rooting out the bad quality code from the test cases and therefore it also improves performance.

- When the malfunctioning code is removed, the obvious slow downs are avoided. 

- However, it is possible that any further optimization made to that test case may decrease its maintainability. 

- But, today there are available what are called “optimizing compilers”.

Compile Level:

- At this stage, it is ensured by the optimizing compiler that the test case is optimized to such a level at which the test case can be compiled by the compiler.

Assembly Level:

- This is the lowest level of optimization process and involves taking advantage of the full repertoire of system instructions. 

- This is the reason why most of the operating systems use the embedded systems are written in assembly code.

- Many of the test cases are compiled from a high level language to assembly language and from there it is manually optimized.

Run time: 

- Run time optimization is needed and are performed by the assembler programmers and just in time compilers which exceed the capability of static compilers.

Two types of optimizations can be performed on the installation testing process namely:

1. Platform dependent optimization (uses specific properties of one platform) and
2. Platform independent optimization (effective on all platforms)

What are limitations and advantages of installation testing?

Imagine you have developed your software system or application properly and very carefully and it addresses all the requirements of your client. But what if your client tries to install your software product and it fails terribly? 

It means the installer of your software system or application was developed properly and was not tested also. Now imaging what effect will it have on your reputation in the eyes of your client? You will loose his confidence! Won’t you? 

The only way to escape such a horrible situation is that you carry out installation testing on the installer of your software product and see that it works with all the systems and with different configurations. Therefore the installation testing becomes very important since installation is the first thing with which the user interacts in a software application.

Limitations of Installation Testing

- The biggest limitation of the installation is the time factor since installation testing consumes a lot of time.

- To overcome this limitation one needs to automate the whole installation testing process.  

- Even a single test case of the installation testing takes at the minimum up to 2 hours.

Now you can imagine how much time it will take if you will have to test the installer of a big software system or application. 

- However, the time consumed for each test case also depends on whether the application is subjected to one of the following options:

1. Upgrading of installation and un- installation process.
2. Partial installation and un- installation process.
3. Full installation and un- installation process.

- The installation testing requires that both the software testing engineer and the configuration manager work in collaboration with each other.

- During installation, the software system or application is placed in to a pre- production environment or testing environment which may corrupt the code in some of the cases.

- From the testing environment there are 2 certainties as mentioned below:

1. The code may progress in to production process or
2. The code may not progress in to the production process.

- Since the code may get corrupted from other past as well as future releases, all these process need to be performed outside the software development environment which further aggravates the difficulty of the installation testing.

- There is just one simple approach to installation testing which involves running the install program commonly known as “package software”. 

- This package software makes use of a set up program which in some cases act as a multi- configuration wrapper and in other cases it does not! 

- Therefore, in the cases in which the set up program does not act as multi configuration wrapper, the software simultaneously cannot be installed on a variety of systems and configurations which again presents a headache. 

- Furthermore installation testing is limited to certain aspects regarding installation:

1. Dependency of the software to be installed on other software and patches.
2. Over installing of a new version over the older one.

Advantages of Installation Testing

- As for the advantages of the installation testing are concerned, the importance of the installation testing is the biggest advantage.
- Installation process is the first thing with which the user interacts and therefore it needs to be proper and efficient. 
- The installation testing keeps a check on this aspect whether it can be installed on a number of different systems and in different environments.

What is meant by storm worm?

Storm worm? You may not recognize this worm at the first instance since you might be knowing it by one of the following other names:

1. Small. Dam
2. Trojan- downloader. Win 32. Small. Dam
3. F secure as dubbed by the finnish company.
4. W32/ Numwar@MM
5. Downloader BAI (McAfee’s specific variant)
6. Trojan. DL. Tibs. Gen! Pact13
7. Trojan. Peacomm (Symantec)
8. Win32/ Nuwar (ESET)
9. W32/ Zhelatin (kaspersky, F secure)
10. Trojan. Peed (Bit Defender)
11. Trojan. Tibs (Bit Defender)
12. Win32/ Nuwar. N@MM! CME- 711 (windows live one care)
13. TROJ_SMALL. EDW (trend micro)
14. Trojan. Downloader – 647
15. Loland Mal/ Dorf (sophos)
16. CME- 711 (mitre)

Evolution of Storm Worm

- It was recognized as a back door Trojan horse that had most of its impact on the computer systems that use the Microsoft operating systems or applications or extensions.

- This worm was first observed on the date of 17th January in the year of 2007.

- The storm worm first took its affect in the countries of the United States and Europe infecting millions of computer systems starting on the date of 19th January 2007.

- It was usually sent to the users as an e-mail message having the subject as a headline about the recent weather disaster like “230 dead as storm batters Europe”.

- At the starting of this cyber epidemic, there were around 6 waves of attack subsequently.

- At the end of the January 2007, the storm worm was said to account for 8 percent of all the world wide malware infections.

- According to the PC world, the history or origin of the storm worm can be traced back to a Russian business network.

- Mostly the European wind storm “kyrill” was used as the subject of the infected e- mails.

- This email usually had an attachment accompanying it which when opened, automatically installed this malware on to the system of the users.

Steps involved in installing the Malware

The malware was installed via the following steps:

1. Installation of the wincom32 service
2. Injection of payload
3. Passing of the packets to destinations as mentioned in the malware code.
4. Download and run the W32. Mixor. Q@mm worm and Trojan. Abwiz. F Trojan.

These downloaded Trojans then attached themselves to spam like flashcard.exe, postcard.exe and so on. Other changes regarding the original attack wave were made as the mutation of the attack carried on. Below mentioned are some other prominent spam attachments:
1. Ecard.exe
2. Fullstory.exe
3. Read more. Exe
4. Greeting postcard.exe
5. Read more.exe
6. Full news.exe
7. Arcade world.exe
8. Fullvideo.exe
9. Video.exe
10. Full clip.exe
11. More here.exe
12. Click here.exe
13. Nfl stat tracker.exe
14. Arcade world game.exe

Later the storm worm came to be spread by subjects regarding love such as “touched by love”, “love birds” and so on. These e- mails had the links referring to the malicious web sites containing virus like:

1. With love.exe
2. With_love.exe
3. From me to you.exe
4. Fck2008.exe
5. Fck2009.exe
6. Love.exe
7. Iheart you.exe

The storm worm has an exceptional ability to stay resilient. The affected machine or system used to become a part of botnet networks which was controlled through a central server. A botnet is seeded by the storm worm that acts as a P2P network without any control. The connected systems then act as a host and share the list of other hosts. One peculiarity was observed in the working of these machines which is that none of them shared the whole list of botnets.

Validation phase - User Acceptance Testing and Installation Testing

User Acceptance Testing occurs just before the software is released to the customer. The end-users along with the developers perform the User Acceptance Testing with a certain set of test cases and typical scenarios.

Installation testing is often the most under tested area in testing. this type of testing is performed to ensure that all installed features and options function properly. It is also performed to verify that all necessary components of the application are, indeed, installed. Installation testing should take care of the following points:
- To check if while installing product checks for the dependent software/patches.
- The product should check for the version of the same product on the target machine, say the previous should not be over installed on the newer version.
- Installer should give a default installation path.
- Installation should allow user to install at location other than the default installation path.
- Check if the product can be installed "Over the Network".
- Installation should start automatically when the CD is inserted.
- Installer should give the Remove/Repair options.
- When uninstalling, check that all the registry keys, files, DLL, shortcuts, activeX components are removed from the system.
- Try to install the software without administrative privileges.
- Try installing on different operating system.
- Try installing on system having non-compliant configuration such as less memory/RAM/HDD.

Overview and Features of Installation Testing

Installation testing occurs outside the development environment. Installation testing can be compared as introducing a guest in your home. The new guest should be properly introduced to all the family members in order to feel him comfortable. Installation of new software is also quite like above example.
The customer will be happy if your installation is successful on the new system but if the installation fails then the program will not work on that system.
Testing application to make sure that it is working is a crucial step. You should check whether the application doing what it is suppose to do? Secondly, run three-four invocations of this application and check memory, CPU load, etc. Third, are other normal applications, hardware, etc working fine?

Installation testing should take care of the following points: -
- To check if while installing product checks for the dependent software / patches say Service pack3.
- The product should check for the version of the same product on the target machine.
- Installer should give a default installation path say “C:\programs\.”
- Installer should allow user to install at location other then the default installation path.
- Check if the product can be installed “Over the Network”.
- Installation should start automatically when the CD is inserted.
- Installer should give the remove / Repair options.
- When uninstalling, check that all the registry keys, files, Dll, shortcuts, active X components are removed from the system.
- Try to install the software without administrative privileges (login as guest).
- Try installing on different operating system.
- Try installing on system having non-compliant configuration such as less memory / RAM / HDD.