What are characteristics of fuzz technique? What are uses of fuzz technique?

Fuzz testing or fuzzing as it is commonly known as, is another type of very popular mixed software testing methodology i.e., it can be either implemented as a black box testing technique or white box testing technique or even as grey box testing technique, though the latter case being very rare. 

- It also provides the options for running it manually or by some semi automated or fully automated processes. 

- Unexpected, invalid or random data is what all is fed as input data values to the test cases created for the fuzz testing of a software system or application. 

- After the arrangement of the test input data values for the test cases, the whole software system or application is subjected to an inspection to determine if at all it is experiencing any problems like crashes, memory leaks or failure of the code assertions that are nothing but built in defects in the software system or application. 

- Fuzz testing or fuzzing is actually a testing methodology that is categorized under the category of the security testing and is often used for security checks of a software system or application. 

Fuzz testing if further divided results in to two sub categories as described below:

1. Generation based fuzz testing: 

This type involves creation of entirely new set of input data based on the model input data and

2. Mutation based fuzz testing: 

This type is concerned with the generation of new set of data by mutating the existing samples.

Fuzz testing lays more emphasis more on the network protocols and the file formats of the software system or application than the other aspects of the system. But this does not hold necessary that only the network protocols and file formats should undergo fuzz testing, other aspects of the input data for the system can also be subjected to the fuzz testing. Common input for the fuzz testing basically includes:

1. Key board events
2. Mouse events
3. Environment variables and
4. API calls sequence.

The unusual input forms are:

Such as those being mentioned below can also be subjected to fuzz testing:

1. Shared memory
2. Contents of the data base of the software system or application and
3. Inter leaving of threads (precise)

The input data which manages to cross the trust boundary of the software system or application catches the attention of the testers more than any other happening during the testing. The origination of the fuzz testing is traced back to the University of Wisconsin and discovered by the professor Barton Miller. 

Uses of Fuzz Testing

The fuzz testing has emerged as a software testing technique with more than one uses:

1. For the testing of large projects having a budget just enough for the development of the test tools it can be employed as black box testing technique.
2. It is a very feasible and affordable software testing technique and has a high benefit- to - cost ratio.
3. The fuzz testing proves helpful in providing a sample stating the behavior of the software system or application generated in a random manner.
4. It very effectively demonstrates the exception handling capability of a piece of the source code of the program without crashing.
5. Fuzz testing gives an assurance for the maintenance of the overall quality of the software system or application rather than just acting as a testing tool for finding bugs.
6. Fuzz testing can also be used as a substitute for the formal methods employed for exhaustive testing of the software system.
7. One can rely on fuzz testing for determining the reliability of the software system as an application of static analysis, code audits or partial rewrites.

What are common problems of test automation?

With the number of complex and advance applications increasing, the need for automation testing has also taken a strategic and critical point. In the past, the applications were quite simple and free of jargon. Assuming the level of testing to be sufficient at that time, how can we even think of testing today’s explosive software systems and applications with that level of software testing?

At this point of thinking, we come across two possibilities out of which first one is to increase the working staff and second one is to increase the level of testing. Following the first possibility seems like a bit ambiguous because the number of skilled testers is often less. So we are left with the second possibility. 

Over the years the testing has advanced and today we have what is called test automation process by virtue of which so many manual testing processes have been automated and precious human time has been saved. But this process too is not free of faults and problems. In this article we have focussed up on the common problems of the test automation.

Common Problems of Test Automation

- If you see the past cases you will come to know that the test automation did not reached the level of success at which it is today in one go itself, rather it faced many failures in the beginning. 

- So many test automation efforts were born which eventually died because of lack of efficiency in them. 

The First Problem:

- With test automation process, the vendors who develop and sell test automation tools do not provide any instantaneous demonstrations of the working of their tool and the simplicity and efficiency of the tools cannot be rightly judged. 

- What we see are some of the software systems and applications that the vendor has and up on which the testing tools work quite efficiently and think those tools will work in the same way for our applications also. 

- Consequently, what happens is that after doing many projects also we do not achieve same level of success. 

But we cannot blame the testing tools for the whole thing. 

- One possibility is also that the elements present in our software system or application might not be compatible with the testing tools that we are employing to test them.

- The only way to escape such pitfalls is that we try to develop some creative solutions using which we can make our software systems and applications compatible with the testing tools.

- This is perhaps the only way to make the testing tools work with our software systems and applications.

- There are so many commercial testing tools available that are marketed as solutions for testing the software products instead of being sold as solid aids for a wide test automation frame work. 

- One thing is common among all the software testing tools which is that they all contain one or the other scripting language that allows us to discover each tool’s failings. 

The Second Problem:

- Another problem with automation testing is that the testers carrying out the automation are not having any high development experience nor do they have been given ample training to exploit these testing processes and programming environments. 

- The automation testing requires both programming skills and testing skills, but what we have are only testers i.e., the testers who are not programmers. 

- Therefore for test automation to be effective, it is necessary that they should be expert in both testing as well as in programming. 

- Due to a lack of experience, the developers often make a simple solution far too complex to be maintained and implemented. 

Typically, what are the testing activities that are automated?

With the test automation use on the move, a lot of processes and testing activities are being automated nowadays as far as possible. Test automation has been widely accepted since it is quite an effective testing methodology that can help the software engineering industry to keep pace with this fast paced technology savvy world.  

Usually the processes that are automated include manual processes that make use of formalized testing processes. But besides these, there are many other software development processes and activities that are automated using the test automation and process and the best part is that they do not disappoint the testers.

In this article, we have taken up the discussion regarding the activities that are quite often subjected to automation. These activities are mostly concerned with the test automation. Usually test cases for automated software testing are exclusively written but the test cases for most of the manual testing processes that deploy formalized processes are subjected to test automation in order to save time and efforts both. However, before automating any testing activity it is made certain that the automated tests will be compatible with the system on which they will be run.

Today the developers are forced to develop software systems and applications in quite a small time frame which represents quite a big challenge. There is not only the need of testing the software system or application rigorously but also there is a need to do it as quickly as possible. 

What is automated Testing Life Cycle Methodology?

In order to make the development process quite systematic a methodology has been introduced which is commonly known as “automated testing life cycle methodology” or ATLM in short form. 

The ATLM lays down 6 processes or activities in the process of test automation and many of the sub activities are automate.

1. Decision to automate test: This includes:

(a)  Overcoming false expectations of automated testing.

(b)  Benefits of automated testing.

(c)  Acquiring management support.

     2. Test tool acquisition: 

    This is the second phase of ATLM and involves activities like tool evaluation and selection process. Here the activity tool evaluation can be automated to some extent. While selecting the testing tool it is required that the tester should keep in mind the system’s engineering environment.

    

     3. Automated testing introduction phase: This phase involves the following steps:

   

    (a)  Test process analysis: This analysis ensures that all the test strategies and processes are in one place. The test strategies, goals and objectives are all defined in this phase and are documented. In this phase only the testing techniques are defined and test plans are assessed.

    

    (b)  Test tool consideration: This step involves the investigation of the incorporated automated test tools and their viewing in the context of the automated project testing requirements. Also the mapping of the potential test utilities and tools to the test requirements is done. The compatibility factor of the testing tools with the software system or application and environment is verified and solutions are further investigated.

     4. Test planning, design and development

    5. Execution and management of tests: By this phase the test design and test development has been addressed by the testing team. The test procedures are now ready to be automated. The setting up of the test environment after every test case execution is also automated in accordance with the guidelines. Now the test plan is ready and test environment is also set up, the execution of the test cases is started. This whole process is automated in the favor of the exercising the software system or application under the test.

     6. Test program review and assessment

How do you plan test automation? Can test automation improve test effectiveness?

“How to plan test automation?” is perhaps the most frequently asked question of today’s software testing field. Another important question is, “can test automation improve test effectiveness?” 

In this article we have tried our level best to answer both of these questions! 

What is meant by Test Automation?

- In test automation, we actually develop a quick process that will take care of the execution of the tests by itself i.e., without any human intervention.

- It also compares the actual and predicted outcomes, sets up the pre- conditions for the tests.

- A common test automation process involves automation of a manual testing process that makes use of a formalized testing tool. 

- Although the manual testing can discover a whole lot of defects in the software system or application, it is quite laborious or you can say it involves a lot of drudgery and time.

- It is not possible for you to discover defects of all the classes using manual testing; it is a task that can only be accomplished using test automation. 

- Test automation takes time only once i.e., while you automate them and after this it saves a whole lot of time when compared to manual testing, tests can be run quickly and repeatedly. 

How to plan test automation?

There are two approaches to test automation and both of them are equally good and either of them can be followed. They are:

1. Code driven testing: This approach involves public interfaces to classes, libraries or modules to be tested with a whole variety of the input arguments for the validation of the results that are returned and are quite correct.

2. Graphical user interface testing: This approach involves a testing frame work that generates a user interface events like mouse clicks and key strokes for the validation of the observable behavior of the program i.e., whether it is correct or not?

There is one drawback which is that the test automation tools can cost you much and they cannot be employed without manual testing. Test automation is effective if you have to repeatedly test one thing. 

There is one more approach to the test automation which is commonly known as “model based testing”. 

- Using this approach you automatically generate tests through the use of a model of the software system or application for the generation of test case but this method of planning test automation is still under research and controversies. 

- One should keep the following questions in mind while planning the test automation:

1. What to automate?

2. When to automate?

3. Whether or not automation is really required?

Can Test Automation improve Test Effectiveness?

Yes, test automation can improve test effectiveness.

- The test cases generated by test automation are quite reliable as compared to those created by manual testing.

- In test automation, human error is eliminated.

- The tests generated are repeatable as well as programmable. 

- They also have the following characteristics:

* Reusable

* Comprehensive: The test suite that is built covers all the features of the software system or application.

* Cost reduction: Due to less resources, used cost is also reduced.

* Fast: Testing is quite fast.

* Better quality software: The resultant software system or application is of better quality since it is tested by a large number of tests with fewer resources and less time.

Describe some automation testing tools?

Automated testing has greatly reduced the drudgery of the developers who otherwise would have to carry out testing on big software systems and applications projects manually. Today there are many automation testing tools available to your advantage. Using these tools you can automate any of the test scripts which will make your work easier than ever consuming less time and efforts. In this article we have discussed about some of the automation testing tools. They are as follows:

1. TestComplete: 

- This testing tool has been designed for both the novice as well as advance testers.

- Using this tool, creating, managing and running tests for any system becomes so easy be it a desk top or web software. 

- It has got a quite easy to use interface and a whole lot of tools for the purpose of customizing it

- This tool supports cross browser web testing, html5 applications and distributed testing as well. 

- Using this tool, you can create tests using just one browser which can be run on almost any browser.

- You do not have to make any changes to the tests i.e., testing is completely browser independent. 

- Furthermore, this testing tool is independent of any other development tool or software application. 

- It supports all the languages.

2. Telerik by Test Studio: 

- This testing tool supports functional testing, load testing, performance testing, and mobile testing very well. 

- It also supports in depth desk top testing i.e., you can easily test AJAX, HTML5, WPF and Silverlight apps. 

- This tool is quite affordable.

- Using this tool, you can create as well as capture tests.

-Web performance tests can also be created which offer a detailed insight in to the performance metrics of your apps.

- This tool is quite advanced and covers most of the technologies and protocols. 

- Using this tool, you can even run a functional test which in turn can help you to gather data on server processing time, client rendering time and network latency etc. 

- Furthermore, this tool supports testing for iOS apps.

- It also provides support for iphone, ipad, ipod, test natives, web and hybrid apps etc.

3. Micro focus- SilkTest: 

- Using this tool, one can create robust, powerful and fast automated tests across a quite broad range of technologies and applications. 

- It can be used easily for the identification of the quality related problems that occur in the early stages of the SDLC or software development life cycle. 

- Till now two versions of this testing tool have proven to be quite worthy namely Silk4Net and Silk4J.

- These two testing tools provide you with the option of bringing the power of SilkTest to integrated development environment of your choice like eclipse or visual studio and so on. 

- This testing tool has showed quite lot robustness.

4. HP: Unified Functional Testing Software (quick professional tool): 

- This tool rules the software industry.

- This tool has got the ability of accelerating the functional testing by following a simplification process for simplifying the test design and maintaining both the non GUI based applications and GUI based applications.

- The integrated test scenarios can be validated quite easily.

- All these features result in a reduction in risk and improve the quality for the applications of the modern era. 

- This testing tool includes HP service test products and HP functional testing. 

- It also includes some of the other add-ins from HP. 

- Using this tool you can increase the efficiency of your testing by using the automated processes and re- using of the data. 

What are some common tools for installation testing?

Introducing a new student to the rest of the class gives us a practical example of the installation testing. The new student should be properly introduced to the rest of the class so that he/ she feel comfortable. Same holds true for software systems and applications. The effectiveness of any testing however also depends up on the type of the testing tools used. In this article we shall be discussing about some common tools for installation testing. 

If the installation of your software product fails, what impression it will make of you on the customer? Obviously it will cost you your reputation. How can you avoid such pathetic and embarrassing conditions? "Answer is by testing the software system’s or application’s installer with an appropriate combination of both the automated and manual processes on different machines and with different configurations". 

The factor of which you should be most concerned is nothing but “time” since installation testing eats up a lot of time, so you got to be wise and use testing tools that can save up your time. Execution of even a single test case takes a lot of time in installation testing. Now imagine if you are going to perform installation testing on a big software system or application? How much it is going to cost you? Before you start installation testing you need to decide on how many systems and configurations you are going to install the application software. Prepare a basic hard disk drive for this purpose with some other basic necessary components.

Role of Installation Testing Tools

- There are testing tools available that will help you in making sets of each of the configurations like file format and operating system that are used in the further installation testing.

- There are many other tools that can be used to automate the whole installation testing process.

- This involves setting out some systems for creating basic images of the base configuration. This can be done by using testing tools like Norton ghost which can create exact image of the operating systems very accurately and quickly.

- This step is guaranteed to save a huge amount of your time for each test case.

- Creating images takes only 5- 10 minutes whereas installing one Operating System with basic configuration takes up to one hour.

- There are some other tools that allow you to use one operating system with multiple attempts of installation.

- But in this case you will need to uninstall the application again and again and prepare the base state for the next installation.

- Apart from all the other testing tools, use flow diagrams also serve as a great aid for performing the installation testing.

- Use flow diagrams simplify the whole task so amicably.

- These flow diagrams can be further used for automating the test efforts since very easily these diagrams can be converted in to the automated scripts.

- Some test tools have been developed that test the installer scripts which are further used to check the disk space required for the installation.

- These tools make sure that only the required amount of space is utilized and not more than that otherwise an error is flagged. You should be careful with the fact that the space required is different for different file system formats.

- Another tool that can help you a big deal with installation testing is “distributed testing environment”.

- Adopting this methodology you can very effectively handle all the different test cases from just one machine and it saves you a lot of time.  

How is optimization of smoke testing done?

Smoke testing being one of the quick and dirty software testing methodologies, needs to be optimized. This article focuses on the need of optimizing the smoke tests and how to optimize them. 

Let us first see the scenario behind the need for optimization of the smoke tests that are carried out on the software system or application. 

- In most cases of the development of software systems or applications, the code can be executed either directly from the command prompt or via a sub- routine of the larger software systems or applications. This is called the command prompt approach. 

- The code is designed in such a way that it has the qualities like self awareness as well as it is autonomous. 

- By the code being self aware, we mean that if anything goes wrong during the execution, the code should explain it. 

- Commonly two types of problems are encountered during testing and have been mentioned below:

1. The path was compiled with too much of optimization and
2. The data directory is not pointed out properly by the path.

Steps for Optimization of Smoke tests

- In order to confirm that the code was compiled correctly one should run the test suite properly at least once. 

- The first step towards optimization of the smoke test is to run it and then examine the output. 

- There are two possibilities that either the code will pass the test or it won’t. 

- If it is the second case then there are two possibilities that where your smoke test went wrong:

1. Compiler bugs and errors or
2. The path has not been set properly.

Compiler Bugs and Errors

Let us take up the first possibility, i.e., the compiler bugs and errors. 

-  It is probable that the correct code might not have been produced by the compiler. 

- In some cases of the serious compiler bugs, it is possible that there might be some hardware exception and these kinds of errors are caused mainly by the compiler bugs. 

- In this case the optimization level of the code should be minimized. 

- After this the code should be recompiled and observed again. 

- Optimizing is good for code but if it is of aggressive kind then it will definitely cause problems.

If path is not set properly

- It is obvious that if the code is not able to trace its data files then it will definitely show up some error and this happens because the path has not been set properly. 

- In such cases you need to check which path was it, fix it and recompile the whole code and execute once again.

When does a system or an application crash?

Don’t think that the software system or application crashes only when there has aggressive optimization of the code! 

- Crashes also happens with those programs in which there is no optimization of the code. 

- But in such cases, only compiler errors can be blamed since it happens only if the compiler has not been set up properly on your system. 

- If no program executes, it means that your compiler is broken and you need to talk about this to your system administrator.

How to optimize smoke tests?

- A lot of help comes from MPGO (managed profile guided optimization). 

- The best way to optimize any kind of testing is to maintain a balance between the automated and manual testing.

- You need to run MPGO tool along with the necessary parameters for the test and then run the test. The test now will be optimized. 

- It is actually the internal binaries that are optimized either fully or partially. 

- Partially optimized binaries are deployed only in automated smoke testing. 

Data Restructuring - An activity involved in software re-engineering process model

Software re- engineering process model constitutes of the below mentioned 6 major  steps:

1. Inventory analysis
2. Documentation reconstruction
3. Reverse engineering
4. Code re- structuring
5. Data re- structuring
6. Forward engineering

In this article we have discussed about the 5^th stage of the software re- engineering process model i.e., the data re- structuring. 
Without a working code or an active flow of data a software system or application cannot be considered to be working. Therefore for the software systems and applications which have become outdated should be again brought up to the current acceptable software standards and data re- structuring is the one of the processes that can help in this regard. 


- This step of software re- engineering process model takes into consideration the restructuring of the data without making changes to the functionality of the software system or application. 
- With the re- structuring of data, the maintenance of the whole software system or application becomes so very easy. 
- The need to re- structure data arises when the data bases become too obsolete to be handled directly. 
- When the changes made to the data are confined only to the sub system, the data of the sub system needs to be re- structured. 
- The data re- structuring becomes even easier when the data re- structuring tools are easily available. 
- This step is carried out after the completion of the code re- structuring. 
- Unlike the previous re- structuring step, this step calls for a full cycle of reverse engineering. 
- The current structure of the data is dissected while defining the data models and the existing data models are run through a quality check or review test.
- So many of the legacy systems tend to make use of the global data structures and shared tables in order to save the memory space.
- Such shared data structures need to be converted in to objects like ADTs for restructuring. 

Benefits of Data Re-structuring

- One of the benefits of restructuring the data is that it certainly improves the quality of the software program or application. 
- Furthermore, it becomes easier to test and debug the software system or application along with a reduction in the efforts in the maintenance of the software and also the frustration of the developer is reduced. 
- As a bonus point, the productivity is also improved. 

Steps in Data Re-structuring

The re- structuring of the data involves the following steps:

1. Analysis of the source code of the application.
2. Redesigning of the data
3. Standardization of the data record.
4. Rationalization of the data name.
5. Translation of the file or data base.

Drawbacks of Data Restructuring

In spite of having so much plus points, the re- structuring does have some minus points which have been listed below:

1. Re- structuring does not improve the understandability of the data driven programs.
2. Re- structuring demands for heavy computations.
3. Re- structuring can lead to a loss of source code comments.
4. Restructuring does not helps with the problems with poor modularization i.e., the programs in which the components are wide spread through out the program.
5. Restructuring can lead to the loss of documentation.

From the discussion above, we can say the restructuring of data is a process that re –structures the source data in to the target data during the transformation process. 
Data re- structuring forms an integral part of the software re- engineering process model and therefore the above mentioned disadvantages of the data re- structuring are ignored. It is necessary that the data re- structuring is carried out after the code re- structuring since during the execution of the program, the code only handles the flow of the data. Therefore if the code itself has not been re- structured before re- structuring the data then the whole process does not makes any sense. 

Reverse Engineering - an activity involved in software re-engineering process model.

Software re-engineering process model is a very generic process meant to uplift the standard of the poor code that is currently not acceptable. This model is known to have 6 major stages namely:

1. Inventory analysis
2. Documentation reconstruction
3. Reverse engineering
4. Code re- structuring
5. Data re- structuring
6. Forward Engineering

This article is all about the third stage i.e., “reverse engineering” which is an important concept in itself. 

Re- engineering is usually required when the some of the sub systems of a larger software system or application need to frequently maintained. The re- engineered system is then restructured and re- documented. 

What is Reverse Engineering?

- The reverse engineering forms a very important factor in success of the re- engineering process. 

- The reverse engineering can be considered to be a process of recovering the design of the software system or application that is to be re- engineered. 

- This step involves analyzation of the software program in an effort to obtain an abstract level representation of the program that is higher than the level of the source code.  

- The software system or application is analyzed in such a way so as to understand its design and specifications and requirements. 

- Reverse engineering is an entirely individual process.

- In some cases the reverse re- engineering may be used  for specifying a software system before it is implemented again. 

- The reverse engineering process makes use of the program understanding tools like:

1. Browsers
2. Cross reference generators and so on.

Levels in Reverse Engineering

The reverse engineering process takes effect through the following levels:

1. Abstraction level: The design information of the software system or application is derived at the highest level possible.

2. Completeness: At the above abstraction level, the details of the system are obtained.

3. Interactivity: The degree of the human integration with 5the automated reverse engineering tools is measured.

4. Directionality: It can be either:

(a)  One way: All the extracted information is given to the sogftware engineering who is doing the maintenance.

(b)  Or two way: All the extracted information is fed to a re- engineering tool which then regenerates the old software program.

      5. Extract abstraction: From the old source the processing specifications are obtained.

     

Stages in Reverse Engineering Process

      Reverse engineering process from sommerville goes through the following stages:

      1. System to be re- engineered is subjected to automated analysis.

      2. Then it is manually annotated.

      3. With the system information obtained, a whole set of new documentation is generated containing:

       (a)  Program structure diagrams 

       (b)  Data structure diagrams and 

       (c)   Trace ability matrices.

     

Activities in Reverse Engineering Process

     There are 3 basic activities involved with the reverse engineering process:

   

    1. Understanding process: In order to understand the procedural abstractions as well as functionality analyze the source code on the level:

      (a)  System

      (b)  Program 

      (c)   Component 

      (d)  Statement and 

      (e)  Pattern

    

       2. Understanding data:  It analyzes internal data structures and data base structure.  

   3. User interfaces: It analyzes the basic actions processed by the interface, system’s behavioral response to those actions and equivalency of the interfaces.

When is Reverse Engineering Preferred?

- Reverse engineering is usually preferred when the specifications and designs of the system are required to carry out the program maintenance activities.

- The re- engineering process is preceded by the reverse engineering. 

- To put it simply, the reverse engineering can be thought of as a process that goes back through the whole development cycle.

- UML is a source that supports reverse engineering. 

- Different people have their own different perceptions of the reversion engineering. 

- It can also be thought of as an inversion of the water fall model of software development.