What are different aspects of stress testing?

Stress testing can be defined as a form of testing that is carried out to determine the stability and the stress handling capacity of a software system or module. Stress testing is all about testing the software system or application beyond the normal operational capacities. It is the testing of software system or application to its breaking or fatal point. This is done in order to observe the results.

Stress testing has a much broader meaning. What is basically understood by a stress test?
- It is referred to a test that mainly focuses upon the availability, error handling and robustness of a software system or application.
- In stress testing the software system or application is subjected heavy loads of tasks.
- It is not about considering the proper behavior under the normal operational conditions or user environment.

Typically the goal of stress testing is to test whether or not the software system or application crashes or fails in the case of catastrophic problems like unavailability of sufficient computing resources. These computational resources may include disk space or memory. It is also done to determine if the system crashes or fails under the situation of denial of service attacks and unusually high concurrency.

Stress testing, load testing, volume testing all seem like much similar kinds of testing.

A look at the following examples of stress testing will clear up the confusions regarding stress testing:


- Stress testing for web server:
A web server can be subjected to stress testing using bots, scripts and several denial of service tools to determine its performance and behavior during the peak data and tasks load.

- Stress testing can be studied in contrast with load testing.
Load testing is basically carried out to examine the entire testing environment and the huge database. It is also carried out to determine the response time of the software system or application whereas the stress testing exclusively focuses upon identifying the transactions and pushing them to a level at which a break occurs in the execution of the transaction software system.

Another point is that during the stress testing if the transactions are duly stressed, then the chance is that the database may not experience a huge data load. However if the transactions are not stressed, the data base may experience a heavy work load.

SOME IMPORTANT POINTS:
- Stress testing is another word for system stress testing.
- It can be defined as the loading of the concurrent users beyond and over the level that can be handled by the system.
- This leads to the breakage of the weakest link in the whole software system or application.
- While carrying out the stress testing the software engineers, developers or testers need to test the whole software system or application under the desired expected stress as well as under the accelerated stress.
- The goal here is to determine the working life of the software system.
- It is also aimed at determining the modes of failure for the software system or application.
- For the hardware counterpart of a complete system, the stress testing can be defined as the subjecting of the concerned hardware to the exaggerated levels of stress.
- This is basically done to determine the stability of the hardware system when used in a normal environment rather than a testing environment.
- Before modifying the CPU parameters during the processes of over clocking, over volting, under volting and under clocking, it is necessary to verify whether or not the new CPU parameters like frequency and core voltage are suitable for taking the heavy CPU loads.
- Stress testing for such parameters is usually carried out by executing a CPU intensive program for a prolonged significant period of time. It is observed if the system crashes or hangs.

What are the different fault injection methods?

In the context of software engineering, fault injection is a technique or methodology meant for improving the test coverage. This is usually done by introducing faults in the program source code in order to test the code paths. In particular it is done to test the error handling test paths that might otherwise be left un- followed and thus, untested. Fault injection technique is often used in combination with the stress testing.

Fault injection technique is considered to be one of the most important parts of developing robust software. To add to your knowledge, robustness testing or syntax testing or fuzz testing is also a kind of fault injection methodology which is commonly used to detect the potential vulnerabilities of the communication interfaces such as application programming interfaces, command line parameters and protocols.

The injected fault goes through a well defined cycle before it becomes an observable failure. After the fault is injected, the program or the application is executed. Upon execution, the injected fault may cause an error which is to be considered as an invalid state within the boundaries of the system. This error may cause many further errors within the boundaries of the system. In this way each error acts as an injected fault and propagates to the system boundaries and comes to observance. When the states of the errors are observed at the boundaries of the system they are known as failures. This whole mechanism is known as “fault – error – failure – cycle”.

This mechanism forms a key mechanism in the context of dependability. In 19s the fault injection technique was used to introduce faults at the level of hardware system. This type of fault injection method is known as HWIFI or hardware implemented fault injection. It tends to simulate failures within a hardware system. Soon after that it was found that faults could also be introduced in the software system and it could be helpful in accessing the software system more appropriately.

These fault injection techniques are collectively known as SWIFI or software implemented fault injection. Techniques for software implemented fault injection can be classified into two major categories namely:

Compile time injection:
This can be defined as an injection technique in which the source code is modified so that the simulated faults can be injected into the system. One popular method for doing this is “mutation testing” in which the existing code is changed so as to induce faults. The other technique is “code mutation” in which the faults produced are very similar to those added unintentionally by the programmers. Another technique which is modification of code mutation technique known as code insertion fault injection adds code rather than modifying the existing piece of code. This is done with the use of simple functions which take an existing value and perturb it using some logic into another value. Such functions are called perturbation functions.

Run time injection:
This technique can be defined as the technique which makes use of a software trigger to inject fault in the system which is executing. Using this technique faults can be injected in a variety of ways such as those listed below:

- Time based triggers
An interrupt is generated when the timer reaches a specified time. The interrupt handler associated with the timer will inject the fault.

- Interrupt based triggers
Software trap mechanisms and hardware exceptions are effectively used to generate a fault in the code of the system at a particular point. This gives instant access to a specific memory location.

- Corruption of memory space
- Syscall interposition techniques
- Network level fault injection

What are different characteristics of load testing?

Load testing can be defined as the process of putting demands on a software system or application or a device and measuring the response of the software system or application or that device.

- Usually load testing is carried out to determine the behavior of the software system, application or device under both kinds of conditions namely the normal load condition and anticipated peak load condition.

- Load testing helps to determine the maximum degree of the operating capacity of the software system or application along with any bottlenecks and to check which element or error and bug is causing degradation of the software system or the application.

- Sometimes the load levied on the software system or application is increased beyond the normal usage limits, in order to test the response and behavior of the software system at unusually high and peak loads. This kind of testing is known as stress testing.

- The load is usually kept so high that the expected result is nothing more than loads of errors and bugs.

- In load testing there is no clear boundary is known to exist when an operation ceases to be a load test and gets converted to a stress test.

Till date it is not very much clear that what are the specific goals of load testing?

The term load testing is often used synonymously with reliability testing, software performance testing and volume testing.

- Load testing is classified under the category of non-functional testing. The term load testing carries a wide number of meanings in the field of software testing.
- Load testing can be said to refer to the practice of determining the expected usage of a software system, program or application by simulating multiple user situation in which many users are accessing the program at the same time.

- Load testing holds more relevance for multi- user software systems or applications which are often built using a client or a server model i.e., using the web servers.
- This does not implies that only the software systems built using client or server model can only undergo the load testing. Also the other types of software systems or applications which are not built using server or client model are also eligible for load testing.

- The most accurate and efficient and effective load testing simulates the actual usage environment, as compared to testing the software system or application using theoretical modeling or the analytical modeling.
- Load testing allows us to measure the QOS performance quotient of any website based on the actual behavior of the customers.
- Most of the tools and frame works that are used to carry out load testing follow the procedure of the classical load testing paradigm.

How does stress test execute?

A stress test starts with a load test, and then additional activity is gradually increased until something breaks. An alternative type of stress test is a load test with sudden bursts of additional activity. The sudden bursts of activity generate substantial activity as sessions and connections are established, where as a gradual ramp-up in activity pushes various values past fixed system limitations.
Ideally, stress tests should incorporate two runs, one with burst type activity and the other with gradual ramp-up to ensure that the system under test will not fail catastrophically under excessive load. System reliability under severe load should not be negotiable and stress testing will identify reliability issues that arise under severe levels of load.
An alternative, or supplemental stress test is commonly referred to as a spike test, where a single short burst of concurrent activity is applied to a system. Such tests are typical of simulating extreme activity where a count down situation exists. For example, a system that will not take orders for a new product until a particular date and time. If demand is very strong, then many users will be poised to use the system the moment the count down ends, creating a spike of concurrent requests and load.

Overview of Stress Testing and its Focus..

Stress Tests determine the load under which a system fails, and how it fails. This is in contrast to load testing, which attempts to simulate anticipated load. It is important to know in advance if a stress situation will result in catastrophic system failure, or if everything just goes really slow. There are various varieties of stress tests, including spike, stepped and gradual ramp-up tests. Catastrophic failures require restarting various infrastructure and contribute to downtime, a stress-full environment to support staff and managers, as well as possible financial losses. If a major performance bottleneck is reached, then the system performance will usually degrade to a point that is unsatisfactory, but performance should return to normal when the excessive load is removed.
Before conducting a stress test, it is usually advisable to conduct targeted infrastructure tests on each of the key components in the system. A variation on targeted infrastructure tests would be to execute each one as a mini stress test.

What is the focus of stress tests?

In a stress event, it is most likely that many more connections will be requested per minute than under normal levels of expected peak activity. In many stress situations, the actions of each connected user will not be typical of actions observed under normal operating conditions. This is partly due to the slow response and partly due to the root cause of the stress event.

If we take an example of a large holiday resort web site, normal activity will be characterized by browsing, room searches and bookings. If a national online news service posted a sensational article about the resort and included a URL in the article, then the site may be subjected to a huge number of hits, but most of the visits would probably be a quick browse. It is unlikely that many of the additional visitors would search for rooms and it would be even less likely that they would make bookings. However, if instead of a news article, a national newspaper advertisement erroneously understand the price of accommodation, then there may well be an influx of visitors who cl amour to book a room, only to find that the price did not match their expectations.

In both of the above situations, the normal traffic would be increased with traffic of a different usage profile. So, a stress test design would incorporate a load test as well as additional virtual users running a special series of stress navigations and transactions.
For the sake of simplicity, one can just increase the number of users using the business processes and functions coded in the load test. However, one must then keep in mind that a system failure with that type of activity may be different to the type of failure that may occur if a special series of stress navigations were utilized for stress testing.

Validation phase - System Testing - Performance Testing - Capacity Planning - Stress Testing

Stress testing is another term that is used for performance testing. Though, load and stress testing are used synonymously for performance related efforts, their goal is different.
Unlike load testing, where testing is conducted for specified number of users, stress testing is conducted for the number of concurrent users beyond the specified limit. The objective is to identify the maximum number of users the system can handle before breaking down or degrading drastically. Since the aim is to put more stress on system. Think time of the user is ignored and the system is exposed to excess load.
The goals of stress testing are:
- It is the testing beyond the anticipated user base.
- It identifies the maximum load a system can handle.
- It checks whether the system degrades gracefully or crashes at a shot.

Stress testing also determines the behavior of the system as user base increases. Let us take an example of online shopping application to illustrate the objective of stress testing. It determines the maximum number of concurrent users an online system can service which can be beyond 1000 users. However there is a possibility that the maximum load that can be handled by the system may found to be same as anticipated limit.

The inference drawn from stress testing are:
- Whether the system is available or not?
- If yes, is the available system stable?
- If yes, is it moving towards unstable state?
- When is the system going to break down or degrade drastically?

Validation phase - System Testing - Security Testing, Stress Testing, Performance Testing

Security Testing

Security testing attempts to verify that protection mechanisms built into a system will, in fact, protect it from improper penetration. During security testing, password cracking, unauthorized entry into the software, network security are all taken into consideration. The six basic security concepts that need to be covered by security testing are: confidentiality, integrity, authentication, availability, authorization and non-repudiation.

Stress Testing

Stress testing executes a system in a manner that demands resources in abnormal quantity, frequency, or volume. The following types of tests may be conducted during stress testing are:
- Special tests may be designed that generate ten interrupts per second, when one or two is the average rate.
- Input data rates may increase by an order of magnitude to determine how input functions will respond.
- Test cases that require maximum memory or other resources.
- Test cases that may cause excessive hunting for disk resident data.
- Test cases that may cause thrashing in a virtual operating system.

Performance Testing

Performance testing of a web site is basically the process of understanding how the web application and its operating environment responds at various user load levels. In general, we want to measure the response time, throughput and utilization of the web site while simulating attempts by virtual users to simultaneously access the site. One of the main objectives of performance testing is to maintain a web site with low response time, high throughput, and low utilization.

Overview of Performance Testing and types of performance testing

Performance Testing finds out the speed and efficiency of the system, computer, product or device. Performance testing is testing that is performed, to determine how fast some aspect of a system performs under a particular workload. A software related performance problem can easily get identified through performance testing.
Performance testing demonstrate that the system meets performance criteria. It can compare two systems to find which performs better. It can also measure what parts of the system or workload causes the system to perform badly.

Types of Performance Testing

1.Load Testing : A load test is usually conducted to understand the behavior of the application under a specific expected load. Examples of load testing include:
- Downloading a series of large files from the Internet.
- Running multiple applications on a computer or server simultaneously.
- Assigning many jobs to a printer in a queue.

2.Stress Testing: It determines the load under which a system fails, and how it fails. There are various varieties of Stress Tests, including spike, stepped and gradual ramp-up tests.
3.Volume Testing: It test what happens if huge amounts of data are handled.
4.Soak Testing: The system is runm at high levels of load for prolonged periods of time.
5.Configuration Testing: The process of testing a system with each of the configurations of software and hardware that are supported.
7.Timing testing: It evaluates response times and time to perform an action.