What are different characteristics of endurance testing?

The term endurance defines the stamina, durability, resilience and the sufferance of something and its ability to perform actively and exert itself for a long period of time.

- Endurance also stands for the ability of the thing to resist, recover from and withstand.
- It also gives the degree of immunity of a thing towards trauma, fatigue or wounds.
- For software systems or applications, endurance testing has been developed to check the endurance of the software system or applications.
- Endurance testing is also known as soak testing.
- It is carried out to determine if the concerned software system or application can sustain the expected continuous load.
- The memory utilization is monitored during the endurance testing.
- This is done so as to check the potential memory leaks.
- The endurance is checked during the endurance testing but quite often, performance degradation is overlooked.
- The good response times and throughput are ensured throughout the life of the software system or application i.e., from the beginning of the usage of the software system or application to the last time of usage.

The endurance testing is aimed at testing the sustainability of a software system or application. Its goal is to check how the system behaves or responds under the prolonged significant pressure or load for an extended and significant period of time.

- It aims to test how the software system or application behaves under the sustained use.
- Endurance testing checks for the problems that may occur as a result of prolonged execution of the software program.
- It tests the application or the system under heavy loads of data for a desired period of time which is typically more than the normal usage time.
- Endurance testing is usually carried out to identify the problems that the software system or applications face under the prolonged execution.
- It also aims at identifying the buffer flows and memory leaks which are otherwise not identifiable without carrying out the endurance testing.
- The testing for the behavior of the program under significant load over a significant period of time is done with normal ramp down and ramp up time.

This can be illustrated with the following example:

- Always some memory is allocated to the objects of the program.
- In some cases it happens that the allocated memory is not de- allocated and thus remains occupied.
- This leads to a situation of over consumption of memory in which a chunk of memory is again and again taken away by the program whenever it is executed.

- Eventually the software program reaches a point when the leftover memory is not sufficient for the program to execute.
- At this point, the program crashes and is reported to have a breakdown. Such a situation is called a memory leak.
- Endurance testing is a part of performance testing.
- The other types of testings under performance testing include load testing, volume testing, and stress testing.
- The basic aim of the endurance is to check and correct the performance related problems being faced by the software system or application.
- Those problems appear only after the software system or application has been running for a long time under some significant load or stress.
- Some of the endurance test cases are available for free and have been checked into repository.
- It is recommended to include a report argument so that you visualize the usage of memory during the test and share it.
- The endurance tests create a real world model which employs a normal and high load pattern of usage for the software system or application to determine its potential and problems.

What is Long Session Soak Testing ?

When an application is used for long periods of time each day, the above approach should be modified, because the soak test driver is not logins and transactions per day, but transactions per active user for each user each day. This type of situation occurs in internal systems, such as ERP and CRM systems, where user logins and stay logged in for many hours, executing a number of business transactions during that time. A soak test for such a system should emulate multiple days of activity in a compacted time frame rather than just pump multiple days worth of transactions through the system.

Long session soak tests should run with realistic user concurrency, but the focus should be on the number of transactions processed. VUGen scripts used in long session soak testing may need to be more sophisticated than short session scripts, as they must be capable of running a long series of business transactions over a prolonged period of time.

The duration of most soak tests is often determined by the available time in the test lab. There are many applications that require extremely long soak tests. Any application that must run, uninterrupted for extended periods of time, may need a soak test to cover all of the activity for a period of time that is agreed to by the stakeholders. Most systems have a regular maintenance window, and the time between such windows is usually a key driver for determining the scope of soak test.

Overview of Soak testing.

Soak testing is running a system at high levels of load for prolonged periods of time. A soak test would normally execute several times more transactions in an entire day than would be expected in a busy day, to identify any performance problems that appear after a large number of transactions have been executed. Also, it is possible that a system may stop working after a certain number of transactions have been processed due to memory leaks or other defects. Soak tests provide an opportunity to identify such defects, whereas load tests and stress tests may not find such problems due to their relatively short duration.A soak test would run for as long as possible, given the limitations of the testing situation. For example, weekends are often an opportune time for a soak test.

There are some typical problems identified during soak tests are:
- Serious memory leaks that would eventually result in memory crisis.
- Failure to close connections between tiers of a multi-tiered system under some circumstances which could stall some or all modules of the system.
- Failure to close database cursors under some conditions which would eventually result in the entire system stalling.
- Gradual degradation of response time of some functions as internal data structures becomes less efficient during a long test.

Apart from monitoring response time, it is also important to measure CPU usage and available memory. If a server process needs to be available for the application to operate, it is often worthwhile to record its memory usage at the start and end of the soak test. It is also important to monitor internal memory usages of facilities such as Java virtual machines, if applicable.