What is meant by load shedding?

The network is monitored by the network monitoring systems. These systems need to be robust and must be capable of inevitably coping with the situations in which the overload occurs. The network gets overloaded because of the nodes generating large volumes of data at high rates. Overload might also occur because of the burstiness of the traffic in its normal course of operation. For reducing the load of the network, load shedding techniques are applied. 

- Load shedding techniques have to be followed if the network is under a lot of stress. 

- This has to be done while monitoring the network for avoiding the packet loss that otherwise might be uncontrollable. 

- Load shedding involves sampling the incoming traffic. 

- CoMo or continuous monitoring has been developed to serve this purpose. 

- It uses such a load shedding scheme which can infer the query’s cost using the relation between the set of features of the traffic and the actual resource usage without having any knowledge of the plug-ins. 

- Here, traffic feature can be defined as a counter describing the incoming traffic’s particular property. 

The property might be any of the following:

Ø  Number of packets

Ø  Number of bytes

Ø  Flows

Ø  Unique IP destination address and so on.

- The CoMo consists of a prediction and the load shedding sub-system for intercepting the packets prior to sending them to the plug-in from the filter.

- A traffic query is implemented by this plug-in. 

- The system completes the process in 4 phases. 

- In the first phase, it forms a batch of packets for each 100ms of the traffic. - It then processes each of these batches for extracting a predefined traffic features’ set that is quite large. 

- From these, the most relevant sets are selected by the feature selection sub-system based up on the present stats of the CPU usage of the query. 

- The selected subset is then supplied as input for the “multiple linear regression subsystem”. 

- This is done for the prediction of the CPU cycles that the query requires for processing the whole batch. 

- If the prediction is greater than the capacity of the system, the batch is pre-processed by the load shedding subsystem for discarding the packet’s portion. 

- The batch is discarded through packet or flow sampling. 

Load shedding is now being seen as an effective method for curbing the overload situations even in the real time systems. 

- It involves shedding excess of the load in such a way that the stability of the system is not disturbed and also the system buffers do not experience any overflows. 

- The idea for applying the technique of load shedding in the field of networking has been adopted from the concept of the electric power management.

- Here, the electric current is intentionally disconnected on particular lines when the demands for the power supply are higher than what is being supplied.

- CoMo is an open source system and can be quickly implemented and can be used for further deploying other network monitoring applications. 

- The system has been written using C language and uses a feature rich API. 

- The system works by predicting the CPU usage of the system and thus anticipates about the resource requirements bursts that might occur in future. 

- The load shedding scheme used by the CoMo has the capability of automatically identifying the features using which the resource usage can be best modeled for each monitoring application.

- This identification is made according to the previous resource usage measurements. 

- These measurements are then used for determining the system’s overall load and by what percentage the load must be shed. 

What is a Load Test Results Summary Report?

Load testing is one of the most important testing that is carried out under performance testing and involves putting demands or load on the software system or application in order to measure its response and also the quality of response. 

About Load Testing

1.Load testing forms a very convenient means for determining the behavior of the software system or application under question and under both normal as well as peak conditions. 

2.Load testing helps a lot regarding the identification of the maximum operating capacity of the software system or application plus the identification of elements which are causing degradation of the software system.

3.Load testing becomes stress testing when the load is raised above the normal usage patterns. 

4.The load is raised to such limits where the probability of expecting the errors is quite high.

5.There exists no boundary between the transition of the load testing in to stress testing. 

6.The goals of the load testing are still not so specific.

7.Load testing is actually categorized under non functional testing. 

8.Load testing involves the practice in which the expected usage of the software system or application is modeled by simulating the multiple users using the application concurrently.

About Load Test Result Summary Report

1. A good load testing report says much about the software system or application.

2. A good report is always presentable to the customer as well as the manager.

3. A load test results summary report is simply a cut short and modified version of the original load testing reports. 

4. The original load testing reports are quite lengthy and difficult to understand for a customer. Therefore, a summary of this report is prepared which is short enough and simple to understand. 

5. You cannot expect your customer to go through the whole bulky report rather you need to provide them with a glimpse of everything. 

6. A load test results summary report serves all these purposes. 

Elements of Load Test Summary Report

1. Load test goals: 

A load test differs from functional tests in the way that a load test cannot fail or pass without the inclusion of any extra conditions. We mean to say that you cannot test a software system or application stands a load in general rather you test it against some specific load having specific performance parameters.

2. Technical description of the test goals: 

Once you know what your goals are, the requirements need to be formalized and transformed in to a technical description that you can apply to the load testing tool. You need to define aspects like:

a)   Type of load to be given to the software system or application,

b)   Number of virtual users to be employed in every test phase.

c)   Acceptable response time and so on.

     3. Realistic emulation of the virtual users: 

    Most important element of the load test results summary report.  This section lays down the division of the all potential real users of the software system or application in to a number of different types based on their expected behavior. The special script developed for every user is also mentioned alongside. In addition to this the user sessions are also mentioned.

     4. Execution of the test: 

     It mentions what all tests were executed and with what load. 

   

     5. Analyzation of the test results:  

    It includes the result of the test whether pass or fail and valuable information obtained from the tests. It also mentions the following:

    a)   Number of errors

    b)   Response times 

    c)   Number of sessions or requests per second and so on.

    6. Problems and improvement suggestions. 

What is virtual user script? Why do you need to parametrize fields in your virtual user script?

Nowadays, the concept of the virtual user is quite common in the field of load testing. 

What is a Virtual User?

- Virtual user is one of the quite useful tools have been ever invented in the field of software engineering. 

- Virtual users can be used in a number of ways for testing load, stress or capacity of any software system or application you want. 

- It will be the easiest to define a virtual user as a virtualized representation of a real world user which has been specifically designed for simulating the same interactions and behaviors with the software system or application or web site to be tested in a way that a real world user would do exactly. 

Let us take an example, suppose at a peak your web site got 100 users over a particular hour. It becomes quite easy for you to simulate the same behavior using 100 virtual users making use of the scripts that invoke quite similar interaction and the navigation exactly as your real users do. Today such virtual users are also available that can playback the recorded scripts. 

In this article we discuss about what a virtual user script is and what is the need for parameterizing the fields in a virtual user script.  

What is a Virtual User Script & Need to Parametrize fields?

The fact that almost all of the scripts of the virtual user are automated is quite common. 

- For each and every automated script a particular entry point is marked by the script statement in virtual user. 

- From this point, the actions of the automated scripts are much similar to that of the real world users. 

- At any point these virtual user scripts can be made to click on any of the buttons or the windows, type some words, and move the mouse around and so on.

- Not only scripts, there are certain functions that are supported well by the virtual user and these functions can act as an extension to the virtual user scripts. 

- These functions have been given a general name called “tasks”. 

- These tasks consist of a procedure along with a list of parameters and in some cases they may also include an optional return value. 

- The information regarding the environment of the target computer is also collected by the virtual user with help of a statement called the match statement. 

- A specific environment element is searched by this match statement with the help of some descriptor traits like the following:

1. Static text
2. Edit text
3. Pictures
4. Icons
5. User items and so on.

- There is one more kind of statement that is used and is called the collect statement.

- It is used to collect all elements of a certain type in to the list.

- Afterwards the virtual user interacts with the software system or application environment using certain keywords like:

1. Select
2. Drag
3. Type
4. Close
5. Click and so on.

- Common objects of the operating system like windows, buttons, bars, menus and scrolls are accessed by the virtual users. 

- The virtual user software package comes with a log file feature using which one can write out all the information from within a script. 

- This feature also provides the scripter with information regarding the current run time state of the virtual user scripts. 

- These virtual user scripts can also be debugged whenever required by logging with the printh statement. 

Another fact about virtual user is that there exists no type checking which serves as a very good idea to log the parameters that serve as input to every task.

What are different aspects of stress testing?

Stress testing can be defined as a form of testing that is carried out to determine the stability and the stress handling capacity of a software system or module. Stress testing is all about testing the software system or application beyond the normal operational capacities. It is the testing of software system or application to its breaking or fatal point. This is done in order to observe the results.

Stress testing has a much broader meaning. What is basically understood by a stress test?
- It is referred to a test that mainly focuses upon the availability, error handling and robustness of a software system or application.
- In stress testing the software system or application is subjected heavy loads of tasks.
- It is not about considering the proper behavior under the normal operational conditions or user environment.

Typically the goal of stress testing is to test whether or not the software system or application crashes or fails in the case of catastrophic problems like unavailability of sufficient computing resources. These computational resources may include disk space or memory. It is also done to determine if the system crashes or fails under the situation of denial of service attacks and unusually high concurrency.

Stress testing, load testing, volume testing all seem like much similar kinds of testing.

A look at the following examples of stress testing will clear up the confusions regarding stress testing:


- Stress testing for web server:
A web server can be subjected to stress testing using bots, scripts and several denial of service tools to determine its performance and behavior during the peak data and tasks load.

- Stress testing can be studied in contrast with load testing.
Load testing is basically carried out to examine the entire testing environment and the huge database. It is also carried out to determine the response time of the software system or application whereas the stress testing exclusively focuses upon identifying the transactions and pushing them to a level at which a break occurs in the execution of the transaction software system.

Another point is that during the stress testing if the transactions are duly stressed, then the chance is that the database may not experience a huge data load. However if the transactions are not stressed, the data base may experience a heavy work load.

SOME IMPORTANT POINTS:
- Stress testing is another word for system stress testing.
- It can be defined as the loading of the concurrent users beyond and over the level that can be handled by the system.
- This leads to the breakage of the weakest link in the whole software system or application.
- While carrying out the stress testing the software engineers, developers or testers need to test the whole software system or application under the desired expected stress as well as under the accelerated stress.
- The goal here is to determine the working life of the software system.
- It is also aimed at determining the modes of failure for the software system or application.
- For the hardware counterpart of a complete system, the stress testing can be defined as the subjecting of the concerned hardware to the exaggerated levels of stress.
- This is basically done to determine the stability of the hardware system when used in a normal environment rather than a testing environment.
- Before modifying the CPU parameters during the processes of over clocking, over volting, under volting and under clocking, it is necessary to verify whether or not the new CPU parameters like frequency and core voltage are suitable for taking the heavy CPU loads.
- Stress testing for such parameters is usually carried out by executing a CPU intensive program for a prolonged significant period of time. It is observed if the system crashes or hangs.

What are different characteristics of endurance testing?

The term endurance defines the stamina, durability, resilience and the sufferance of something and its ability to perform actively and exert itself for a long period of time.

- Endurance also stands for the ability of the thing to resist, recover from and withstand.
- It also gives the degree of immunity of a thing towards trauma, fatigue or wounds.
- For software systems or applications, endurance testing has been developed to check the endurance of the software system or applications.
- Endurance testing is also known as soak testing.
- It is carried out to determine if the concerned software system or application can sustain the expected continuous load.
- The memory utilization is monitored during the endurance testing.
- This is done so as to check the potential memory leaks.
- The endurance is checked during the endurance testing but quite often, performance degradation is overlooked.
- The good response times and throughput are ensured throughout the life of the software system or application i.e., from the beginning of the usage of the software system or application to the last time of usage.

The endurance testing is aimed at testing the sustainability of a software system or application. Its goal is to check how the system behaves or responds under the prolonged significant pressure or load for an extended and significant period of time.

- It aims to test how the software system or application behaves under the sustained use.
- Endurance testing checks for the problems that may occur as a result of prolonged execution of the software program.
- It tests the application or the system under heavy loads of data for a desired period of time which is typically more than the normal usage time.
- Endurance testing is usually carried out to identify the problems that the software system or applications face under the prolonged execution.
- It also aims at identifying the buffer flows and memory leaks which are otherwise not identifiable without carrying out the endurance testing.
- The testing for the behavior of the program under significant load over a significant period of time is done with normal ramp down and ramp up time.

This can be illustrated with the following example:

- Always some memory is allocated to the objects of the program.
- In some cases it happens that the allocated memory is not de- allocated and thus remains occupied.
- This leads to a situation of over consumption of memory in which a chunk of memory is again and again taken away by the program whenever it is executed.

- Eventually the software program reaches a point when the leftover memory is not sufficient for the program to execute.
- At this point, the program crashes and is reported to have a breakdown. Such a situation is called a memory leak.
- Endurance testing is a part of performance testing.
- The other types of testings under performance testing include load testing, volume testing, and stress testing.
- The basic aim of the endurance is to check and correct the performance related problems being faced by the software system or application.
- Those problems appear only after the software system or application has been running for a long time under some significant load or stress.
- Some of the endurance test cases are available for free and have been checked into repository.
- It is recommended to include a report argument so that you visualize the usage of memory during the test and share it.
- The endurance tests create a real world model which employs a normal and high load pattern of usage for the software system or application to determine its potential and problems.

What are different characteristics of load testing?

Load testing can be defined as the process of putting demands on a software system or application or a device and measuring the response of the software system or application or that device.

- Usually load testing is carried out to determine the behavior of the software system, application or device under both kinds of conditions namely the normal load condition and anticipated peak load condition.

- Load testing helps to determine the maximum degree of the operating capacity of the software system or application along with any bottlenecks and to check which element or error and bug is causing degradation of the software system or the application.

- Sometimes the load levied on the software system or application is increased beyond the normal usage limits, in order to test the response and behavior of the software system at unusually high and peak loads. This kind of testing is known as stress testing.

- The load is usually kept so high that the expected result is nothing more than loads of errors and bugs.

- In load testing there is no clear boundary is known to exist when an operation ceases to be a load test and gets converted to a stress test.

Till date it is not very much clear that what are the specific goals of load testing?

The term load testing is often used synonymously with reliability testing, software performance testing and volume testing.

- Load testing is classified under the category of non-functional testing. The term load testing carries a wide number of meanings in the field of software testing.
- Load testing can be said to refer to the practice of determining the expected usage of a software system, program or application by simulating multiple user situation in which many users are accessing the program at the same time.

- Load testing holds more relevance for multi- user software systems or applications which are often built using a client or a server model i.e., using the web servers.
- This does not implies that only the software systems built using client or server model can only undergo the load testing. Also the other types of software systems or applications which are not built using server or client model are also eligible for load testing.

- The most accurate and efficient and effective load testing simulates the actual usage environment, as compared to testing the software system or application using theoretical modeling or the analytical modeling.
- Load testing allows us to measure the QOS performance quotient of any website based on the actual behavior of the customers.
- Most of the tools and frame works that are used to carry out load testing follow the procedure of the classical load testing paradigm.

What are different stages involved in web application testing?

Web application is a dynamic extension of an application server. Testing of a web application is done by a web application tester. Testing a web application involves different stages as follows:

Functionality testing:
In functionality testing, tests are made to check the links in web pages, database connection, forms used in web pages to submit or retrieve information from users.

Testing the links involves the following:
- Checking the outgoing links in all the pages from the specific domain under test.
- Internal links testing.
- Checking the links that jump on the same pages.
- The Links through which the email will be sent to the admin or to other users from different web pages must be checked.
- All the above mentioned links must be checked for broken links.

Testing the forms in all the web pages:
- Each field in a form must be checked for validity.
- Default values of each field must be checked.
- Options for creating, modifying, viewing or deleting a form must be verified during testing.

Testing of cookies:
The application must be tested by enabling or disabling cookies in browser options. The cookies must be tested for encryption before writing to user system. The session cookies must be tested for login sessions and user status after session ends.

Testing of database:
Data integrity must be monitored every time when data is created, modified, viewed or deleted within a database. Every database related functionality must be tested.

Usability testing:
Usability determines the ease with which a user can handle the application. The webpage designed must be user friendly with appropriate main menu in each page. It is very important to ensure that the application is readable and understandable by a user. It is recommended to use commonly accepted standards for font, frames and colors. Anchor text links must be analyzed.

Interface testing:
Web server, application server and database server are the main interfaces. Ensure whether each interaction between each of the interfaces is executed without any errors. Errors when occurred must be handled appropriately.

Compatibility testing:
Compatibility of a web application is a very important testing aspect. A web application must be compatible mainly with Browser and Operating System.

Browser compatibility:
A web application must be tested with different browsers to ensure its compatibility. Various browsers have different settings and configurations with which the application must be compatible. Web coding must be compatible with cross browser platform. If java scripts or AJAX calls are used for UI functionality and performing security checks or validations, browser compatibility must be given importance.

OS compatibility:
New technologies such as graphic designs, interface calls used in web development must be compatible with various browsers, which are popularly used.

Performance testing:
Performance of an application is measured by its ability to sustain load and stress. Load testing and stress testing are two main types of performance testing.

Load testing:
When large number of users requests the same page, the application must have the ability to sustain the load and perform appropriately. Performance of an application must be tested on different internet connections with varying speed.

Stress testing:
In stress testing the application is put beyond its limit of sustainability, to check the reaction of the application to stress or to check how the system recovers from crash.

Security testing:
Security testing must ensure that only authorized users can use the application and none of the internal pages must be opened for non registered users. CAPTCHA can be utilized to improve security in application.

How does stress test execute?

A stress test starts with a load test, and then additional activity is gradually increased until something breaks. An alternative type of stress test is a load test with sudden bursts of additional activity. The sudden bursts of activity generate substantial activity as sessions and connections are established, where as a gradual ramp-up in activity pushes various values past fixed system limitations.
Ideally, stress tests should incorporate two runs, one with burst type activity and the other with gradual ramp-up to ensure that the system under test will not fail catastrophically under excessive load. System reliability under severe load should not be negotiable and stress testing will identify reliability issues that arise under severe levels of load.
An alternative, or supplemental stress test is commonly referred to as a spike test, where a single short burst of concurrent activity is applied to a system. Such tests are typical of simulating extreme activity where a count down situation exists. For example, a system that will not take orders for a new product until a particular date and time. If demand is very strong, then many users will be poised to use the system the moment the count down ends, creating a spike of concurrent requests and load.

Overview of Stress Testing and its Focus..

Stress Tests determine the load under which a system fails, and how it fails. This is in contrast to load testing, which attempts to simulate anticipated load. It is important to know in advance if a stress situation will result in catastrophic system failure, or if everything just goes really slow. There are various varieties of stress tests, including spike, stepped and gradual ramp-up tests. Catastrophic failures require restarting various infrastructure and contribute to downtime, a stress-full environment to support staff and managers, as well as possible financial losses. If a major performance bottleneck is reached, then the system performance will usually degrade to a point that is unsatisfactory, but performance should return to normal when the excessive load is removed.
Before conducting a stress test, it is usually advisable to conduct targeted infrastructure tests on each of the key components in the system. A variation on targeted infrastructure tests would be to execute each one as a mini stress test.

What is the focus of stress tests?

In a stress event, it is most likely that many more connections will be requested per minute than under normal levels of expected peak activity. In many stress situations, the actions of each connected user will not be typical of actions observed under normal operating conditions. This is partly due to the slow response and partly due to the root cause of the stress event.

If we take an example of a large holiday resort web site, normal activity will be characterized by browsing, room searches and bookings. If a national online news service posted a sensational article about the resort and included a URL in the article, then the site may be subjected to a huge number of hits, but most of the visits would probably be a quick browse. It is unlikely that many of the additional visitors would search for rooms and it would be even less likely that they would make bookings. However, if instead of a news article, a national newspaper advertisement erroneously understand the price of accommodation, then there may well be an influx of visitors who cl amour to book a room, only to find that the price did not match their expectations.

In both of the above situations, the normal traffic would be increased with traffic of a different usage profile. So, a stress test design would incorporate a load test as well as additional virtual users running a special series of stress navigations and transactions.
For the sake of simplicity, one can just increase the number of users using the business processes and functions coded in the load test. However, one must then keep in mind that a system failure with that type of activity may be different to the type of failure that may occur if a special series of stress navigations were utilized for stress testing.

Overview of Reporting on response time at various levels of load, Fail-over Tests, Fail-back Testing

REPORTING ON RESPONSE TIME AT VARIOUS LEVELS OF LOAD
Expected output from a load test often includes a series of response time measures at various levels of load. It is important when determining the response time at any particular level of load, that the system has run in a stable manner for a significant amount of time before taking measurements.
For example, a ramp-up to 500 users may take ten minutes, but another ten minutes may be required to let the system activity stabilize. Taking measurements over the next ten minutes would then give a meaningful result. The next measurement can be taken after ramping up to the next level and waiting a further ten minutes for stabilization and ten minutes for the measurement period and so on for each level of load requiring detailed response time measures.

FAIL-OVER TESTS
Failover tests verify of redundancy mechanisms while the system is under load. This is in contrast to load tests which are conducted under anticipated load with no component failure during the course of a test. For example, in a web environment, failover testing determines what will happen if multiple web servers are being used under peak anticipated load, and one of them dies.
Failover testing allows technicians to address problems in advance, in the comfort of a testing situation, rather than in the heat of a production outrage. It also provides a baseline of failover capability so that a sick server can be shutdown with confidence, in the knowledge that the remaining infrastructure will cope with the surge of failover load.

FAIL-BACK TESTING
After verifying that a system can sustain a component outage, it is also important to verify that when the component is back up, that it is available to take load again, and that it can sustain the influx of activity when it comes back online.

What comprises Test Ware Development : Test Plan - System test Plan

The system test plan is the overall plan carrying out the system test level activities. In the system test, apart from testing the functional aspects of the system, there are some special testing activities carried out, such as stress testing etc. The following are the sections present in system test plan:

- What is to be tested?
This section defines the scope of system testing, very specific to the project. Normally, the system testing is based on the requirements. All the requirements are to be verified in the scope of the system testing. This covers the functionality of the product. Apart from this, what special testing is performed are also stated here.

- Functional groups and the sequence
The requirements can be grouped in terms of the functionality. Based on this, there may be priorities also among the functional groups. For example, in a banking application, anything related to customer accounts can be grouped into one area, anything related to inter-branch transactions may be grouped into one area etc.

- Special Testing Methods
This covers the different special tests like load/volume testing, stress testing, interoperability testing etc. These testing are to be done based on the nature of the product and it is not mandatory that every one of these special tests must be performed for every product.

Apart from above sections, the following sections are also addressed:
- System Testing Tools
- Priority of functional groups
- Naming Convention for test cases
- Status reporting mechanism
- Regression test approach
- ETVX criteria
- Build/Refresh Criteria

What are different types of black box testing ?

The base of the Black box testing strategy lies in the selection of appropriate data as per functionality and testing it against the functional specifications in order to check for normal and abnormal behavior of the system. These testing types are again divided in two groups:

Testing in which user plays a role of tester.

- Functional Testing : The testing of the software is done against the functional requirements.
- Load testing : It is the process of subjecting a computer, peripheral, server, network or application to a work level approaching the limits of its specifications.
- Stress Testing : The process of determining the ability of a computer, network, program or device to maintain a certain level of effectiveness under unfavorable conditions.
- Ad-hoc testing : Testing carried out informally; no formal test preparation takes place, no recognized test design technique is used, there are no expectations for results and randomness guides the test execution activity.
- Smoke Testing : It is done in order to check if the application is ready for further major testing and is working properly without failing up to least expected level.
- Recovery Testing : Testing aimed at verifying the system's ability to recover from varying degrees of failure.
- Volume Testing : Huge amount of data is processed through the application in order to check the extreme limitations of the system.
- Usability Testing : This testing is done if User Interface of the application stands an important consideration and needs to be specific for the specific type of user.

User is not required.

- Alpha Testing : Testing of a software product or system conducted at the developer's site by the end user.
- Beta Testing : The pre-testing of hardware or software products with selected typical customers to discover inadequate features or possible product enhancements before it is released to the general public. Testing of a rerelease of a software product conducted by customers.
- User Acceptance Testing : The end users who will be using the applications test the application before ‘accepting’ the application. This type of testing gives the end users the confidence that the application being delivered to them meets their requirements.